Data Engineering Podcast

DECEMBER 31, 2018

Preamble Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out Linode.

Lambda Architecture 100

Lambda Architecture Process Data Process Kafka 100

Data Engineering Podcast

NOVEMBER 20, 2021

In this episode Ori Rafael shares his experiences from Upsolver and building scalable stream processing for integrating and analyzing data, and what the tradeoffs are when coming from a batch oriented mindset. Can you start by giving an overview of the state of the market for data lakes today?

Data Lake 100

Data Lake Data Integration Lambda Architecture Process 100

Data Engineering Weekly

FEBRUARY 18, 2025

Fluss is a compelling new project in the realm of real-time data processing. Confluent Tableflow can bridge Kafka and Iceberg data, but that is just a data movement that data integration tools like Fivetran or Airbyte can also achieve.

Kafka 75

Kafka Lambda Architecture SQL Architecture 75

Ascend.io

JULY 28, 2023

This article aims to shed light on the essential components of successful data pipeline architectures and offer actionable advice for creating scalable, reliable, and resilient data pipelines. Ready to fortify your data management practice? Let’s dive into the world of data pipeline architecture.

Data Pipeline 52

Data Pipeline Architecture Lambda Architecture Data Architecture 52

Monte Carlo

MARCH 14, 2023

Data ingestion is the process of collecting data from various sources and moving it to your data warehouse or lake for processing and analysis. It is the first step in modern data management workflows. Some data teams will leverage micro-batch strategies for time sensitive use cases.

Data Ingestion 52

Data Ingestion Data Warehouse Lambda Architecture Raw Data 52