Start Data Engineering
MAY 9, 2020
Change data capture is a software design pattern used to capture changes to data and take corresponding action based on that change. The change to data is usually one of read, update or delete. The corresponding action usually is supposed to occur in another system in response to the change that was made in the source system.
Confluent
MAY 15, 2020
Currently, Apache Kafka® uses Apache ZooKeeper™ to store its metadata. Data such as the location of partitions and the configuration of topics are stored outside of Kafka itself, in a […].
Team Data Science
MAY 10, 2020
With careful consideration and learning about your market, the choices you need to make become narrower and more clear. I can now begin drafting my data ingestion/ streaming pipeline without being overwhelmed. For A Quick Recap You can find the first blog post here, where I learned which tech is most in demand in Toronto: [link] And the second blog post is here where I learn which Toronto industries need data engineers the most: [link] The Pipeline Proposal I'll be creating several pipelines in
Teradata
MAY 13, 2020
The COVID-19 pandemic has brought with it a Perfect Storm of disruption that impacts all of us -- from our health to the economy to the supply chain. Read more.
Advertisement
Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.
Data Engineering Podcast
MAY 11, 2020
Summary There have been several generations of platforms for managing streaming data, each with their own strengths and weaknesses, and different areas of focus. Pulsar is one of the recent entrants which has quickly gained adoption and an impressive set of capabilities. In this episode Sijie Guo discusses his motivations for spending so much of his time and energy on contributing to the project and growing the community.
Confluent
MAY 12, 2020
Imagine you’ve got a stream of data; it’s not “big data,” but it’s certainly a lot. Within the data, you’ve got some bits you’re interested in, and of those bits, […].
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Teradata
MAY 11, 2020
Learn how advanced analytics are being used in China amidst the COVID-19 pandemic to help combat the spread of coronavirus now and in the future.
Preset
MAY 10, 2020
Setting up Apache Superset for the first time can be difficult. Here's an easy way to accomplish the task!
Confluent
MAY 11, 2020
Everyone wants their infrastructure to be highly available, and ksqlDB is no different. But crucial properties like high availability don’t come without a thoughtful, rigorous design. We thought hard about […].
Rock the JVM
MAY 10, 2020
Explore how Akka Typed integrates good practices directly into the API
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Teradata
MAY 12, 2020
Teradata's top-ranked analytic software capabilities that market-leading companies have been using for years is available right now on Amazon Web Services (AWS). Learn more.
Confluent
MAY 13, 2020
We’re pleased to announce the release of ksqlDB 0.9.0! This version includes support for multi-join statements, enhanced LIKE expressions, and a host of usability improvements. We’ll go through a few […].
Expert insights. Personalized for you.
246 
Let's personalize your content