Data Engineering Podcast

JUNE 11, 2023

Datafold has invested a lot of time into integrating with the workflow of dbt projects to add early verification that the changes you are making are correct. In this episode Gleb Mezhanskiy shares some valuable advice and insights into how you can build reliable and well-tested data assets with dbt and data-diff.

Project 130

Project Building Data Lake Machine Learning 130

U-Next

AUGUST 16, 2022

MBA in IT makes the most sense in today’s world as firms need a workforce that can understand IT and business management. Here, you will learn MBA in information technology fundamentals and everything else. Introduction to an MBA in Information Technology. What is an MBA in Information Technology?

Technology 52

Technology Finance Education Business Analyst 52

Data Engineering Podcast

OCTOBER 23, 2022

In this episode Shane Gibson shares practical advice and insights from his years of experience as a consultant and engineer working in data about how to adopt agile principles in your data work so that you can move faster and provide more value to the business, while building systems that are maintainable and adaptable.

Project 130

Project Metadata MongoDB MySQL 130

Data Engineering Podcast

JULY 17, 2022

In this episode Crux CTO Mark Etherington discusses the different costs involved in managing external data, how to think about the total return on investment for your data, and how the Crux platform is architected to reduce the toil involved in managing third party data. When is Crux the wrong choice?

Data Management 130

Data Management Management Metadata MongoDB 130

Data Engineering Podcast

JUNE 26, 2022

Summary The most complicated part of data engineering is the effort involved in making the raw data fit into the narrative of the business. Master Data Management (MDM) is the process of building consensus around what the information actually means in the context of the business and then shaping the data to match those semantics.

Data Management 100

Data Management Management MongoDB Scala 100

Data Engineering Podcast

JUNE 8, 2020

Summary The landscape of data management and processing is rapidly changing and evolving. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management What are the pieces of advice that you wish you had received early in your career of data engineering?

Data Management 100

Data Management Management Machine Learning Portfolio 100

Precisely

SEPTEMBER 5, 2023

With cloud computing, the capacity to extract value from data is greater than ever. As this realization grows, businesses are shifting their investments from hardware to technologies that optimize data assets. With data governance and the right technology tools, it is possible to achieve consistent, accurate results at scale.

Data Governance 52

Data Governance Government Data Management Management 52

Precisely

JANUARY 11, 2024

These challenges are increasingly being addressed by the adoption of business value engineering, a data- and insights-driven framework that aligns technology selection with purpose – strengthening your ability to demonstrate the value that those investments produce. If the return isn’t clear, the investments often falter.

Technology 59

Technology Data Validation Raw Data Portfolio 59

Data Engineering Podcast

MAY 18, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat. If you've learned something or tried out a project from the show then tell us about it! Closing Announcements Thank you for listening!

Building 278

Building Data Lake High Quality Data Business Intelligence 278

Data Engineering Podcast

NOVEMBER 16, 2020

In this episode CEO and co-founder of Isima Darshan Rawal explains how the biOS platform is architected to enable ease of use, the challenges that were involved in building an entirely new system from scratch, and how it can integrate with the rest of your data platform to allow for incremental adoption. What is the story behind the name?

Data Management 100

Data Management Management BI Business Intelligence 100

Data Engineering Podcast

JUNE 2, 2024

Nicola Askham found her way into data governance by accident, and stayed because of the benefit that she was able to provide by serving as a bridge between the technology and business. In this episode she shares the practical steps to implementing a data governance practice in your organization, and the pitfalls to avoid.

Data Governance 147

Data Governance Government Data Lake High Quality Data 147

Data Engineering Podcast

JUNE 16, 2024

In this episode Kevin Liu shares some of the interesting features that they have built by combining those technologies, as well as the challenges that they face in supporting the myriad workloads that are thrown at this layer of their data platform. Can you describe what role Trino and Iceberg play in Stripe's data architecture?

Data Lake 147

Data Lake High Quality Data Metadata Machine Learning 147

Pinterest Engineering

OCTOBER 2, 2024

The following picture depicts the situation: Figure 1: Example of four different services with no overlapping on common modules using two datastore technologies There are several problems with this approach: Maintenance-wise, each service is assembled differently, so there is a large cognitive overhead to become an expert in all of them.

Data Management 56

Data Management Management Metadata SQL 56

Data Engineering Podcast

MARCH 24, 2024

Summary A core differentiator of Dagster in the ecosystem of data orchestration is their focus on software defined assets as a means of building declarative workflows. With their launch of Dagster+ as the redesigned commercial companion to the open source project they are investing in that capability with a suite of new features.

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Data Engineering Podcast

MAY 26, 2024

Sriram Panyam has been involved in several projects that required migration of large volumes of data in high traffic environments. In this episode he shares some of the valuable lessons that he learned about how to make those projects successful. Can you start by sharing some of your experiences with data migration projects?

Systems 130

Systems Data Lake High Quality Data Google Cloud 130

Data Engineering Podcast

FEBRUARY 18, 2024

Summary A data lakehouse is intended to combine the benefits of data lakes (cost effective, scalable storage and compute) and data warehouses (user friendly SQL interface). Multiple open source projects and vendors have been working together to make this vision a reality. Closing Announcements Thank you for listening!

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Data Engineering Podcast

SEPTEMBER 3, 2018

Summary With the proliferation of data sources to give a more comprehensive view of the information critical to your business it is even more important to have a canonical view of the entities that you care about. Can you start by establishing a definition of data mastering that we can work from?

Data Management 100

Data Management Management Relational Database Business Intelligence 100

Data Engineering Podcast

JUNE 23, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Data lakes in various forms have been gaining significant popularity as a unified interface to an organization's analytics. Closing Announcements Thank you for listening!

Data Lake 162

Data Lake High Quality Data Hadoop Machine Learning 162

Knowledge Hut

OCTOBER 27, 2023

Its popularity and widespread adoption in the technology world make it an important tool for developers seeking to build robust and cutting-edge applications. In this article, we will look at some of the best Angular project ideas to help you practice your Angular skills and land a high-paying job. Source code: To-do List App 2.

Coding 52

Coding Project Media Portfolio 52

Data Engineering Podcast

JANUARY 21, 2024

In this episode Tasso Argyros, CEO of ActionIQ, gives a summary of the major epochs in database technologies and how he is applying the capabilities of cloud data warehouses to the challenge of building more comprehensive experiences for end-users through a modern customer data platform (CDP). Want to see Starburst in action?

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Knowledge Hut

DECEMBER 26, 2023

Project management that is intelligent and precise is necessary for complex client engagements. During marketing and pre-sales, during the actual sale, and after the consumer has made their purchase, project management assists in achieving the consistent brand messaging that is the hallmark of exceptional customer experiences.

Project 52

Project Management IT Utilities 52

Data Engineering Podcast

MARCH 31, 2019

Summary Analytics projects fail all the time, resulting in lost opportunities and wasted resources. However, many of them are and as data engineers we can help to keep our projects on the path to success. We have partnered with organizations such as O’Reilly Media, Dataversity, and the Open Data Science Conference.

Project 100

Project IT Media Data Warehouse 100

Data Engineering Podcast

JANUARY 7, 2024

Summary Data processing technologies have dramatically improved in their sophistication and raw throughput. Unfortunately, the volumes of data that are being generated continue to double, requiring further advancements in the platform capabilities to keep up. What do you have planned for the future of your academic research?

Data Process 147

Data Process Process Data Lake High Quality Data 147

Knowledge Hut

SEPTEMBER 29, 2023

AWS Solution Architect roles have significant applications in the realms of AI , Machine Learning, IoT, and numerous other cutting-edge technologies. As technology evolves, possessing this certification ensures a competitive edge in the industry. Why You Should Work on AWS Projects?

AWS 52

AWS Project Coding Cloud Computing 52

Knowledge Hut

DECEMBER 28, 2023

In this blog, we'll talk about intriguing and real-time sample Hadoop projects with source codes that can help you take your data analysis to the next level. Why Are Hadoop Projects So Important? The Apache Hadoop framework provides tools for efficient analysis, resource management, and parallel processing of datasets.

Hadoop 52

Hadoop Project Datasets Big Data 52

Data Engineering Podcast

JUNE 9, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat. Data observability has been gaining adoption for a number of years now, with a large focus on data warehouses.

Process 147

Process Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

OCTOBER 29, 2023

In this episode Tanya Bragin shares her experiences as a product manager for two major vendors and the lessons that she has learned about how teams should approach the process of tool selection. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles.

Database 189

Database BI SQL Machine Learning 189

Data Engineering Podcast

APRIL 21, 2024

Summary Generative AI has rapidly transformed everything in the technology sector. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Data Lake 182

Data Lake High Quality Data Data Pipeline Machine Learning 182

Zalando Engineering

JUNE 23, 2021

Introduction In our previous post about Technology Choices at Zalando we spoke about a few problems with scaling technology selection in Tech companies. Now, we'd like to reflect on our lessons learned, which you can use when designing technology selection processes. Such individuals are also essential for continued innovation.

Technology 52

Technology Data Lake PostgreSQL AWS 52

Data Engineering Podcast

JUNE 30, 2024

He highlights the role of data teams in modern organizations and how Synq is empowering them to achieve this. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Can you describe what Synq is and the story behind it?

Pipeline-centric 147

Pipeline-centric Engineering Data Lake High Quality Data 147

Data Engineering Podcast

JUNE 25, 2023

SQLMesh was designed as a unifying tool that is simple to work with but powerful enough for large-scale transformations and complex projects. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Engineering 225

Data Engineering Data Engineer Python Engineering 225

Data Engineering Podcast

FEBRUARY 4, 2024

In this episode Yingjun Wu explains how it is architected to power analytical workflows on continuous data flows, and the challenges of making it responsive and scalable. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex.

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Knowledge Hut

FEBRUARY 27, 2023

Due to rapid globalization, constant business competition, and the emergence of new markets, the role of a project manager is growing in every field of work. As per a report from PMI, on average, India alone will need more than 70K professional project managers annually in the coming decade to meet industry needs.

Project 52

Project Management Consulting Certification 52

Data Engineering Podcast

MARCH 10, 2024

The primary purpose of the catalog is to inform the query engine of what data exists and where, but the Nessie project aims to go beyond that simple utility. How have the design and goals of the project changed since it was first created? What is involved in integrating Nessie into a given data stack?

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

Data Engineering Podcast

AUGUST 6, 2023

Summary As businesses increasingly invest in technology and talent focused on data engineering and analytics, they want to know whether they are benefiting. So how do you calculate the return on investment for data? What are the typical motivations for measuring and tracking the ROI for a data team?

Machine Learning 130

Machine Learning SQL Data Engineering Data Engineer 130

Data Engineering Podcast

MARCH 3, 2024

Summary Artificial intelligence technologies promise to revolutionize business and produce new sources of value. Colleen Tartow has worked across all stages of the data lifecycle, and in this episode she shares her hard-earned wisdom about how to conduct an AI program for your organization.

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Zalando Engineering

JULY 28, 2021

The Master Data Management Challenge Master data management (MDM) is a technology-enabled discipline in which business and Information Technology work together to ensure the uniformity, accuracy, stewardship, semantic consistency and accountability of the enterprise's official shared master data assets.

Technology 52

Technology Data Systems Data Governance 52