Data Engineering Podcast

NOVEMBER 20, 2021

In this episode Ori Rafael shares his experiences from Upsolver and building scalable stream processing for integrating and analyzing data, and what the tradeoffs are when coming from a batch oriented mindset. Can you start by giving an overview of the state of the market for data lakes today?

Data Lake 100

Data Lake Data Integration Lambda Architecture Process 100

Data Engineering Podcast

DECEMBER 31, 2018

Preamble Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out Linode.

Lambda Architecture 100

Lambda Architecture Process Data Process Kafka 100

Data Engineering Podcast

JULY 24, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. What are your goals with this book?

Data Engineer 100

Data Engineer Data Engineering Lambda Architecture Engineering 100

Data Engineering Podcast

MAY 11, 2020

Pulsar is a well engineered and robust platform for building the core of any system that relies on durable access to easily scalable streams of data. What is Pulsar’s role in the lifecycle of data and where does it fit in the overall ecosystem of data tools? Can you start by giving an overview of what Pulsar is?

Cloud 100

Cloud Lambda Architecture Kafka Hadoop 100

Data Engineering Podcast

AUGUST 21, 2022

In this episode Shruti Bhat gives her view on the state of the ecosystem for real-time data and the work that she and her team at Rockset is doing to make it easier for engineers to build those experiences. Can you describe what is driving the adoption of real-time analytics?

Lambda Architecture 100

Lambda Architecture MongoDB MySQL Scala 100

Data Engineering Weekly

FEBRUARY 18, 2025

Confluent Tableflow can bridge Kafka and Iceberg data, but that is just a data movement that data integration tools like Fivetran or Airbyte can also achieve. On the other hand, Fluss is a Kappa Architecture ; it stores one copy of data and presents it as a stream or a table, depending on the use case.

Kafka 73

Kafka Lambda Architecture SQL Architecture 73

Data Engineering Podcast

JUNE 1, 2020

What used to be entirely managed by the database engine is now a composition of multiple systems that need to be properly configured to work in concert. In order to bring the DBA into the new era of data management the team at Upsolver added a SQL interface to their data lake platform. We talked last in November of 2018.

Data Lake 100

Data Lake Database Building Lambda Architecture 100