Data Engineering Podcast

APRIL 14, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is brought to you by Datafold – a testing automation platform for data engineers that prevents data quality issues from entering every part of your data workflow, from migration to dbt deployment.

Non-relational Database 147

Non-relational Database Relational Database Database Designing 147

Data Engineering Podcast

FEBRUARY 18, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Data Engineering Podcast

FEBRUARY 11, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. How does that change as a function of the type of data? What are the requirements around governance and auditability of data access that need to be addressed when sharing data?

Data Lake 147

Data Lake High Quality Data Government Data Pipeline 147

Data Engineering Podcast

MARCH 31, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Datafold has recently launched data replication testing, providing ongoing validation for source-to-target replication. Your first 30 days are free! Your first 30 days are free!

Project 130

Project Data Lake High Quality Data Data Workflow 130

Data Engineering Podcast

MAY 12, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end-to-end data lakehouse platform built on Trino, the query engine Apache Iceberg was designed for, with complete support for all table formats including Apache Iceberg, Hive, and Delta Lake.

Management 147

Management Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

APRIL 21, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Data Lake 182

Data Lake High Quality Data Data Pipeline Machine Learning 182

Data Engineering Podcast

JUNE 9, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end-to-end data lakehouse platform built on Trino, the query engine Apache Iceberg was designed for, with complete support for all table formats including Apache Iceberg, Hive, and Delta Lake.

Process 147

Process Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

JANUARY 21, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. How have the architectural shifts changed the ways that organizations interact with their customer data? How have the responsibilities shifted across different roles?

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Data Engineering Podcast

MARCH 24, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Data Engineering Podcast

APRIL 28, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Building 147

Building Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

JANUARY 7, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data.

Data Process 147

Data Process Process Data Lake High Quality Data 147

Data Engineering Podcast

MARCH 17, 2024

Summary A significant portion of data workflows involve storing and processing information in database engines. In this episode Gleb Mezhanskiy, founder and CEO of Datafold, discusses the different error conditions and solutions that you need to know about to ensure the accuracy of your data. Data lakes are notoriously complex.

Database 147

Database Data Lake High Quality Data Data Workflow 147

Data Engineering Podcast

MAY 26, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex.

Systems 130

Systems Data Lake High Quality Data Google Cloud 130

Snowflake

JUNE 6, 2024

A look inside Snowflake Notebooks: A familiar notebook interface, integrated within Snowflake’s secure, scalable platform Keep all your data and development workflows within Snowflake’s security boundary, minimizing the need for data movement. Access Snowflake platform capabilities and data sets directly within your notebooks.

SQL 95

SQL Python Machine Learning Data Workflow 95

Data Engineering Podcast

JANUARY 28, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data.

Data Lake 147

Data Lake Building High Quality Data AWS 147

Data Engineering Podcast

DECEMBER 24, 2023

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Data Engineering Podcast

FEBRUARY 4, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data.

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Data Engineering Podcast

MAY 5, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Building 147

Building Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

NOVEMBER 26, 2023

Summary Building a data platform that is enjoyable and accessible for all of its end users is a substantial challenge. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. Data lakes are notoriously complex.

Architecture 147

Architecture Data Lake High Quality Data SQL 147

François Nguyen

MARCH 22, 2021

TL;DR After setting up and organizing the teams, we are describing 4 topics to make data mesh a reality. How do we build data products ? How can we interoperate between the data domains ? As you can see, this is in the code part where you are building your data pipelines, a misnomer because this is an over simplification.

Technology 130

Technology Architecture Google Cloud Metadata 130

Data Engineering Podcast

DECEMBER 31, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. How does that influence the architectural design/capabilities for data platforms in those organizations? Data governance is a notoriously challenging problem.

Designing 130

Designing Data Lake High Quality Data SQL 130

Data Engineering Podcast

DECEMBER 3, 2023

This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues for every part of your data workflow, from migration to deployment. Datafold has recently launched a 3-in-1 product experience to support accelerated data migrations. Datafold : ![Datafold]([link]

Systems 130

Systems Designing Data Lake SQL 130

Ascend.io

JULY 11, 2024

It enhances data quality, governance, and optimization, making data retrieval more efficient and enabling powerful automation in data engineering processes. As practitioners using metadata to fuel data teams, we at Ascend understand the critical role it plays in organizing, managing, and optimizing data workflows.

Metadata 52

Metadata IT Government High Quality Data 52

Data Engineering Podcast

MARCH 3, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Data Engineering Podcast

FEBRUARY 25, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Database 162

Database Technology Data Lake High Quality Data 162

Data Engineering Podcast

MARCH 10, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

Databand.ai

AUGUST 30, 2023

The core philosophy of DataOps is to treat data as a valuable asset that must be managed and processed efficiently. It emphasizes the importance of collaboration between different teams, such as data engineers, data scientists, and business analysts, to ensure that everyone has access to the right data at the right time.

Data Pipeline 52

Data Pipeline Data Governance Government Business Analyst 52

Data Engineering Podcast

DECEMBER 17, 2023

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Data Lake 130

Data Lake High Quality Data SQL Architecture 130

Data Engineering Podcast

NOVEMBER 19, 2023

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Project 147

Project Data Lake High Quality Data SQL 147

Data Engineering Podcast

NOVEMBER 5, 2023

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

PostgreSQL 147

PostgreSQL Data Lake High Quality Data SQL 147

Cloudera

AUGUST 22, 2024

Established in 2014, this center has become a cornerstone of Cloudera’s global strategy, playing a pivotal role in driving the company’s three growth pillars: accelerating enterprise AI, delivering a truly hybrid platform, and enabling modern data architectures.

Machine Learning 67

Machine Learning Manufacturing Cloud Computing Banking 67

Data Engineering Podcast

DECEMBER 10, 2023

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Data Lake 130

Data Lake High Quality Data SQL Architecture 130

Data Engineering Podcast

NOVEMBER 12, 2023

Learn more about Datafold by visiting dataengineeringpodcast.com/datafold Data lakes are notoriously complex. Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data.

Software Engineering 147

Software Engineering Software Engineer Engineering Data Lake 147

Databand.ai

AUGUST 30, 2023

Integrating these principles with data operation-specific requirements creates a more agile atmosphere that supports faster development cycles while maintaining high quality standards. This demands the implementation of advanced data integration techniques, such as real-time streaming ingestion, batch processing, and API-based access.

Data Governance 40

Data Governance Data Cleanse Government Data Science 40

Databand.ai

AUGUST 30, 2023

Data Orchestration Data orchestration refers to the coordination and management of data workflows, from data ingestion to data processing and analysis. DataOps tools should offer powerful data orchestration capabilities, allowing organizations to build, schedule, and monitor data workflows with ease.

Data Cleanse 52

Data Cleanse Data Pipeline Data Ingestion Data Validation 52

Monte Carlo

MAY 25, 2023

Automated data orchestration removes data bottlenecks by eliminating the need for manual data preparation, enabling analysts to both extract and activate data in real-time. Improved data governance. The organization phase , in which data orchestration tools gather and organize data pipelines.

Data Pipeline 52

Data Pipeline Data Workflow Data Data Governance 52

Monte Carlo

MAY 14, 2024

Depending on the quantity of data flowing through an organization’s pipeline — or the format the data typically takes — the right modern table format can help to make workflows more efficient, increase access, extend functionality, and even offer new opportunities to activate your unstructured data.

Data Lake 52

Data Lake Metadata Hadoop Data Governance 52