Confluent
MARCH 19, 2019
When should you use an API? When should you use an event? Most contemporary software architectures are some mix of these two approaches. I will attempt to articulate in layman’s terms what an event-driven architecture (EDA) is and contrast it with service-oriented architecture (SOA). In essence, this is an attempt to differentiate and/or associate APIs with events.
Data Engineering Podcast
MARCH 18, 2019
Summary Delivering a data analytics project on time and with accurate information is critical to the success of any business. DataOps is a set of practices to increase the probability of success by creating value early and often, and using feedback loops to keep your project on course. In this episode Chris Bergh, head chef of Data Kitchen, explains how DataOps differs from DevOps, how the industry has begun adopting DataOps, and how to adopt an agile approach to building your data platform.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Teradata
MARCH 19, 2019
Teradata is thrilled to be named one the of the World’s Most Ethical Companies, for the tenth consecutive year.
Netflix Tech
MARCH 21, 2019
by Astha Singhal (Netflix Application Security) As Netflix continues to create entertainment people love, the security team continues to keep our members, partners, and employees secure. The security research community has partnered with us to improve the security of the Netflix service for the past few years through our responsible disclosure and bug bounty programs.
Advertisement
In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate
Confluent
MARCH 20, 2019
Back in May 2017, we laid out why we believe that Kafka Streams is better off without a concept of watermarks or triggers , and instead opts for a continuous refinement model. This article explains how we are fundamentally sticking with this model, while also opening the door for use cases that are incompatible with continuous refinement. By continuous refinement , I mean that Kafka Streams emits new results whenever records are updated.
Rockset
MARCH 21, 2019
When we started Rockset, we envisioned building a powerful cloud data management system that was really easy to use. Making the data stack simpler is fundamental to making data usable by developers and data scientists. Simplifying the Data Stack To that end, we incorporated user-friendly features that alleviate the pain we personally experienced as data practitioners.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
|
Cloudera
MARCH 19, 2019
Are you floundering in a Red Ocean of competition? Modern enterprises operate in an extremely competitive (red ocean) and turbulent, hyperconnected white water world. To better appreciate the competitive environment in which your company is operating, ask yourself the following questions: Are you confronted with increased competition both domestically and internationally?
Zalando Engineering
MARCH 21, 2019
Recently, I was developing a small stream processing application using Apache Flink. Zalando uses Kubernetes as the default deployment target, so naturally I wanted to deploy Flink and the developed job to our Kubernetes cluster. I learned a lot about Flink and Kubernetes along the way, which I want to share in this article. Challenges Compliance - At Zalando, all code running in production has to be reviewed by at least two people and all deployed artifacts have to be traceable to a git commit.
Rockset
MARCH 19, 2019
Think about the steel industry in the US, and you’ll likely think of Pittsburgh. Known as the “Steel City” for leading the nation in steel production in the first half of the 20th century, Pittsburgh also went by the moniker “the Smoky City,” due to the air pollution from steel and other heavy industries. With increased regulation and the decline of the steel industry, Pittsburgh has gotten much cleaner since its darkest, smokiest days in the 1940s, but it still hasn’t shed all the vestiges of s
Rockset
MARCH 19, 2019
Fynd is an online to offline (O2O) fashion e-commerce portal that brings in-store fashion products from retail brands to an online audience. Fynd pulls real-time streams of inventory data from over 9,000 stores in India to provide its 17 million customers up-to-date information on the latest offers and trends in fashion. Data and technology are at the heart of Fynd’s business.
Speaker: Tamara Fingerlin, Developer Advocate
Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.
Rockset
MARCH 19, 2019
A group of MIT students traveled to Instituto Alpha Lumen , a school in São José dos Campos, Brazil, in early 2019 to assist in the formation of a smart city initiative. The school offers a rigorous educational program for talented youth, providing students many opportunities to train in science and technology fields and preparing them for studies at top universities internationally and in Brazil.
Zalando Engineering
MARCH 16, 2019
Project Highlights Kube Metrics Adapter gained community attention as it was featured in a medium post 'Kubernetes autoscaling with Istio metrics'. Users provided very positive feedback on the project. Kube Metrics Adapter is currently maintained by Developer Productivity team at Zalando. It is a general purpose metrics adapter for Kubernetes that can collect and serve custom and external metrics for Horizontal Pod Autoscaling.
Expert insights. Personalized for you.
110 
Let's personalize your content