Confluent

MAY 21, 2019

When you build microservices architectures, one of the concerns you need to address is that of communication between the microservices. At first, you may think to use REST APIs—most programming languages have frameworks that make it very easy to implement REST APIs, so this is a common first choice. REST APIs define the HTTP methods that are used and the request and response payloads that are expected.

Kafka 110

Kafka Insurance Architecture Database 110

Data Engineering Podcast

MAY 19, 2019

Summary In recent years the traditional approach to building data warehouses has shifted from transforming records before loading, to transforming them afterwards. As a result, the tooling for those transformations needs to be reimagined. The data build tool (dbt) is designed to bring battle tested engineering practices to your analytics pipelines. By providing an opinionated set of best practices it simplifies collaboration and boosts confidence in your data teams.

Data Analytics 100

Data Analytics Building Engineering Data Warehouse 100

Teradata

MAY 23, 2019

Using Vantage, Air France-KLM Group performs cross-channel analytics of customer data to provide a seamless experience for their passengers.

Data 96

Data 96

Netflix Tech

MAY 21, 2019

Lerner?—?using RL agents for test case scheduling By: Stanislav Kirdey , Kevin Cureton , Scott Rick , Sankar Ramanathan Introduction Netflix brings delightful customer experiences to homes on a variety of devices that continues to grow each day. The device ecosystem is rich with partners ranging from Silicon-on-Chip (SoC) manufacturers, Original Design Manufacturer (ODM) and Original Equipment Manufacturer (OEM) vendors.

Amazon Web Services 65

Amazon Web Services Manufacturing Python AWS 65

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline

Confluent

MAY 23, 2019

Let us cut to the chase: Kafka Summit London session videos are available! If you were there, you know what a great time it was, and you know that you had to make sometimes-agonizing decisions about which sessions to attend and which to miss. Well, now you can make all those tradeoffs right by watching the whole catalog. And if you weren’t there? Well, dig in and start learning!

Kafka 109

Kafka Cloud Database Engineering 109

Cloudera

MAY 22, 2019

We are excited to be launching our first awards program together as the “New Cloudera.” Although the program is technically in its seventh year, as the first joint awards program, this year’s Data Impact Awards will span even more use cases, covering even more advances in IoT, data warehouse, machine learning, and more. The program recognizes organizations that are using Cloudera’s platform and services to unlock the power of data, with massive business and social impact.

Pharmaceutical 56

Pharmaceutical Recruitment Machine Learning Data Architecture 56

Rockset

MAY 24, 2019

Remote work affords organizations access to more talent and offers workers greater flexibility in their lives. With a vision for everyone to be able to work from anywhere, FULL Creative runs a contact center service using fully remote teams, tapping into the growing share of employees working remotely. FULL agents answer calls on behalf of 7,000 clients of all sizes, from plumbers to parking garages to legal and medical professionals.

Management 52

Management Medical SQL Utilities 52

Confluent

MAY 22, 2019

The internet is not just connecting people around the world. Through the Internet of Things (IoT), it is also connecting humans to the machines all around us and directly connecting machines to other machines. In light of this, we’ll share an emerging machine-to-machine (M2M) architecture pattern in which MQTT, Apache Kafka ® , and Scylla all work together to provide an end-to-end IoT solution.

Kafka 98

Kafka Google Cloud NoSQL Entertainment 98

Rockset

MAY 23, 2019

Adding an index to a database is one of those little joys in life. A query takes 10 seconds, you add a good index, and boom.10 milliseconds! Customers are happy, manager is happy, database is happy (according to its CPU graph at least). However, managing indexes gets old quickly. More indexes means writes are slower. There is always another query creeping up on the latency graph.

Database 52

Database Datasets Media Algorithm 52