Data Engineering Podcast

NOVEMBER 26, 2019

Summary The team at Sentry has built a platform for anyone in the world to send software errors and events. As they scaled the volume of customers and data they began running into the limitations of their initial architecture. What are some of the downstream benefits of using Clickhouse for managing event data at Sentry?

Data Warehouse 100

Data Warehouse Building PostgreSQL Kafka 100

Data Engineering Podcast

FEBRUARY 18, 2024

In this episode Dain Sundstrom, CTO of Starburst, explains how the combination of the Trino query engine and the Iceberg table format offer the ease of use and execution speed of data warehouses with the infinite storage and scalability of data lakes. Join in with the event for the global data community, Data Council Austin.

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Knowledge Hut

NOVEMBER 19, 2023

With advances in technology, wearable devices do provide some trace of your health. How do they collect, process, and analyze data for you? Importance of Data Science in IoT IoT and Data Science are amongst the most defining technologies of our age. Data Science for IoT has its own share.

Data Science 52

Data Science Technology Machine Learning Big Data 52

Striim

NOVEMBER 8, 2023

Data Mesh is revolutionizing event streaming architecture by enabling organizations to quickly and easily integrate real-time data, streaming analytics, and more. In this article, we will explore the advantages and limitations of data mesh, while also providing best practices for building and optimizing a data mesh with Striim.

Architecture 52

Architecture Generalist Government Datasets 52

Cloudera

SEPTEMBER 14, 2020

When adopting cloud data management, there are some fundamental principles we need to embrace to be successful, or we risk security gaps, failure to maintain regulatory compliance or unexpected cost overruns. Fundamental principles to be successful with Cloud data management.

Data Management 111

Data Management Cloud Management Business Analyst 111

Data Engineering Podcast

JUNE 26, 2022

Summary The most complicated part of data engineering is the effort involved in making the raw data fit into the narrative of the business. Master Data Management (MDM) is the process of building consensus around what the information actually means in the context of the business and then shaping the data to match those semantics.

Data Management 100

Data Management Management MongoDB Scala 100

Data Engineering Podcast

JUNE 25, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Rudderstack]([link] RudderStack provides all your customer data pipelines in one platform.

Data Engineer 225

Data Engineer Data Engineering Python Engineering 225

Confluent

MARCH 19, 2019

When should you use an event? I will attempt to articulate in layman’s terms what an event-driven architecture (EDA) is and contrast it with service-oriented architecture (SOA). In essence, this is an attempt to differentiate and/or associate APIs with events. Is not life simply a series of events? If it can, should it?

Architecture 110

Architecture Telecommunication Transportation Database 110

Data Engineering Podcast

MARCH 3, 2024

Summary Artificial intelligence technologies promise to revolutionize business and produce new sources of value. Colleen Tartow has worked across all stages of the data lifecycle, and in this episode she shares her hard-earned wisdom about how to conduct an AI program for your organization.

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Data Engineering Podcast

MAY 21, 2023

In this episode David Yaffe and Johnny Graettinger share the story behind the business and technology and how you can start using it today to build a real-time data lake without all of the headache. Stream processing technologies have been around for around a decade. Can you describe what Estuary is and the story behind it?

Data Lake 162

Data Lake Machine Learning Kafka Data Warehouse 162

Data Engineering Podcast

MARCH 17, 2024

In this episode Gleb Mezhanskiy, founder and CEO of Datafold, discusses the different error conditions and solutions that you need to know about to ensure the accuracy of your data. Join us at the top event for the global data community, Data Council Austin. Don't miss out on our only event this year!

Database 147

Database Data Lake High Quality Data Data Workflow 147

Data Engineering Podcast

APRIL 9, 2023

Summary The customer data platform is a category of services that was developed early in the evolution of the current era of cloud services for data processing. Now that the data warehouse has taken center stage a new approach of composable customer data platforms is emerging.

Data Lake 147

Data Lake Data Warehouse Machine Learning Data 147

Data Engineering Podcast

MARCH 10, 2024

In this episode Alex Merced explains how the branching and merging functionality in Nessie allows you to use the same versioning semantics for your data lakehouse that you are used to from Git. Join us at the top event for the global data community, Data Council Austin. Don't miss out on our only event this year!

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

Data Engineering Podcast

MAY 14, 2023

Summary All of the advancements in our technology is based around the principles of abstraction. In this episode the host Tobias Macey shares his reflections on recent experiences where the abstractions leaked and some observances on how to deal with that situation in a data platform architecture.

Data Lake 147

Data Lake Machine Learning Data Warehouse AWS 147

Data Engineering Podcast

MAY 28, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Can you start by sharing your conception of the responsibilities of a data team?

Data Lake 162

Data Lake Machine Learning Data Warehouse Education 162

Data Engineering Podcast

MARCH 24, 2024

In this episode Pete Hunt, CEO of Dagster labs, outlines these new capabilities, how they reduce the burden on data teams, and the increased collaboration that they enable across teams and business units. Can you describe what the focus of Dagster+ is and the story behind it? What problems are you trying to solve with Dagster+?

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Data Engineering Podcast

MARCH 10, 2023

NetSpring is a warehouse-native product analytics service that allows you to gain powerful insights into your customers and their needs by combining your event streams with the rest of your business data. Don't miss out on their only event this year! Visit: dataengineeringpodcast.com/data-council today!

Data Warehouse 130

Data Warehouse Data Lake Machine Learning Data Science 130

Data Engineering Podcast

NOVEMBER 26, 2023

In this episode Tobias Macey shares his thoughts on the challenges that he is facing as he prepares to build the next set of architectural layers for his data platform to enable a larger audience to start accessing the data being managed by his team. Developing event-driven pipelines is going to be a lot easier - Meet Functions!

Architecture 147

Architecture Data Lake High Quality Data SQL 147

Data Engineering Podcast

MARCH 18, 2023

Summary As with all aspects of technology, security is a critical element of data applications, and the different controls can be at cross purposes with productivity. He also explains why data security is distinct from application security and some methods for reducing the challenge of working across different data systems.

Data Security 130

Data Security Machine Learning Data Science Metadata 130

Data Engineering Podcast

APRIL 16, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Rudderstack]([link] RudderStack provides all your customer data pipelines in one platform.

Business Intelligence 147

Business Intelligence Building Data Lake BI 147

Data Engineering Podcast

JUNE 17, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. How has the move to the cloud for data warehousing/data platforms influenced the practice of data modeling?

Data Lake 130

Data Lake Metadata Machine Learning Data Architecture 130

Data Engineering Podcast

JUNE 11, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Rudderstack]([link] RudderStack provides all your customer data pipelines in one platform. Sponsored By: Rudderstack : ![Rudderstack]([link]

Project 130

Project Building Data Lake Machine Learning 130

Data Engineering Podcast

JUNE 4, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Can you describe what Agile Data Engine is and the story behind it? RudderStack also supports real-time use cases.

Data Engineer 130

Data Engineer Data Engineering Engineering Data Lake 130

Data Engineering Podcast

MAY 7, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. What are some of the categories of attributes that need to be managed in a prototypical customer profile?

Pipeline-centric 130

Pipeline-centric Data Lake Machine Learning Data Warehouse 130

Data Engineering Podcast

APRIL 2, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Businesses that adapt well to change grow 3 times faster than the industry average. As your business adapts, so should your data. As your business adapts, so should your data.

Hadoop 130

Hadoop Machine Learning Python Architecture 130

Data Engineering Podcast

JANUARY 28, 2024

Summary Monitoring and auditing IT systems for security events requires the ability to quickly analyze massive volumes of unstructured log data. Cliff Crosland co-founded Scanner to provide fast querying of high scale log data for security auditing. Can you describe what Scanner is and the story behind it?

Data Lake 147

Data Lake Building High Quality Data AWS 147

Data Engineering Podcast

MARCH 25, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Businesses that adapt well to change grow 3 times faster than the industry average. As your business adapts, so should your data. If you're fed up with the 'Modern Data Stack', give TimeXtender a try.

MySQL 130

MySQL Python Architecture Machine Learning 130

Data Engineering Podcast

MARCH 5, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Hey there podcast listener, are you tired of dealing with the headache that is the 'Modern Data Stack'? It's supposed to make building smarter, faster, and more flexible data infrastructures a breeze.

Building 147

Building Database Design Machine Learning Metadata 147

Data Engineering Podcast

OCTOBER 15, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Process 182

Process Building SQL BI 182

Precisely

SEPTEMBER 5, 2024

HP Hood, Johnson & Johnson Vision, Loparex, Pactiv Evergreen, South Florida Water Management District, and Refresco share efficiencies and insights gained with Precisely Precisely hosted events during SAP Sapphire week in Orlando, FL – including an Automate User Group meeting, or “Inspiration Day.”

Finance 64

Finance Process Engineering Technology 64

Data Engineering Podcast

SEPTEMBER 17, 2023

Summary A significant amount of time in data engineering is dedicated to building connections and semantic meaning around pieces of information. Linked data technologies provide a means of tightly coupling metadata with raw information. What is the overlap between knowledge graphs and "linked data products"?

Building 189

Building SQL BI Python 189

Snowflake

MAY 7, 2024

Accelerate Healthcare and Life Sciences is a one-day virtual event, featuring technology and business leaders from Elevance Health, Ginkgo Bioworks, Datavant and more, to discover executive priorities, best practices and potential data and AI challenges that are top of mind for 2024. Executive panel.

Healthcare 90

Healthcare Cloud Data Technology 90

Data Engineering Podcast

OCTOBER 22, 2023

In this episode Ranjith Raghunath shares his thoughts on how to build a strategy for the development, delivery, and evolution of data products. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles.

BI 162

BI SQL Machine Learning Programming Language 162

Data Engineering Podcast

OCTOBER 8, 2023

In this episode he shares his journey of data collection and analysis and the challenges of automating an intentionally manual industry. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. Closing Announcements Thank you for listening!

Insurance 162

Insurance BI SQL Machine Learning 162

Data Engineering Podcast

OCTOBER 1, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Building 162

Building BI SQL Machine Learning 162

Snowflake

APRIL 22, 2024

By leveraging advanced technology and data analytics, Magnite offers a comprehensive suite of tools and services designed to maximize ad revenue for publishers while helping them effectively reach their target audiences.

AWS 73

AWS Cloud Storage Cloud Technology 73