Remove Data Management Remove Hadoop Remove Lambda Architecture
article thumbnail

Exploring Processing Patterns For Streaming Data Integration In Your Data Lake

Data Engineering Podcast

In this episode Ori Rafael shares his experiences from Upsolver and building scalable stream processing for integrating and analyzing data, and what the tradeoffs are when coming from a batch oriented mindset. Can you start by giving an overview of the state of the market for data lakes today?

Data Lake 100
article thumbnail

StreamNative Brings Streaming Data To The Cloud Native Landscape With Pulsar

Data Engineering Podcast

Pulsar is a well engineered and robust platform for building the core of any system that relies on durable access to easily scalable streams of data. What is Pulsar’s role in the lifecycle of data and where does it fit in the overall ecosystem of data tools? Can you start by giving an overview of what Pulsar is?

Cloud 100
article thumbnail

Maintaining Your Data Lake At Scale With Spark

Data Engineering Podcast

This conversation was useful for getting a better idea of the challenges that exist in large scale data analytics, and the current state of the tradeoffs between data lakes and data warehouses in the cloud. We have partnered with organizations such as O’Reilly Media, Dataversity, and the Open Data Science Conference.

Data Lake 100