Data Engineering Podcast

MAY 15, 2022

Summary Designing a data platform is a complex and iterative undertaking which requires accounting for many conflicting needs. Designing a platform that relies on a data lake as its central architectural tenet adds additional layers of difficulty. When is a data lake architecture the wrong choice?

Data Lake 100

Data Lake Building Architecture BI 100

Data Engineering Podcast

FEBRUARY 5, 2023

Sign up now for early access to Materialize and get started with the power of streaming data with the same simplicity and low implementation cost as batch cloud data warehouses. Go to [dataengineeringpodcast.com/materialize]([link] Support Data Engineering Podcast

Data Engineering 147

Data Engineering Data Engineer Engineering PostgreSQL 147

U-Next

SEPTEMBER 7, 2022

The terms “ Data Warehouse ” and “ Data Lake ” may have confused you, and you have some questions. Essentially, this is the difference between a lake and a warehouse. On the other hand, a data warehouse contains historical data that has been cleaned and arranged. .

Data Lake 52

Data Lake Data Warehouse Unstructured Data Amazon Web Services 52

Data Engineering Podcast

MAY 1, 2022

Datafold shows how a change in SQL code affects your data, both on a statistical level and down to individual rows and values before it gets merged to production. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows.

Data Warehouse 100

Data Warehouse Data Integration Cloud Google Cloud 100

Data Engineering Podcast

DECEMBER 18, 2022

Push information about data freshness and quality to your business intelligence, automatically scale up and down your warehouse based on usage patterns, and let the bots answer those questions in Slack so that the humans can focus on delivering real value.

Metadata 130

Metadata Business Intelligence Data Lake BI 130

Data Engineering Podcast

DECEMBER 29, 2022

Push information about data freshness and quality to your business intelligence, automatically scale up and down your warehouse based on usage patterns, and let the bots answer those questions in Slack so that the humans can focus on delivering real value.

Management 130

Management Metadata Business Intelligence Data Lake 130

Monte Carlo

AUGUST 25, 2023

Different vendors offering data warehouses, data lakes, and now data lakehouses all offer their own distinct advantages and disadvantages for data teams to consider. So let’s get to the bottom of the big question: what kind of data storage layer will provide the strongest foundation for your data platform?

Data Lake 52

Data Lake Data Warehouse Unstructured Data Raw Data 52

Data Engineering Podcast

JANUARY 8, 2023

Trusted by the data teams at Fox, JetBlue, and PagerDuty, Monte Carlo solves the costly problem of broken data pipelines. Monte Carlo also gives you a holistic picture of data health with automatic, end-to-end lineage from ingestion to the BI layer directly out of the box. Missing data? Missing data?

PostgreSQL 130

PostgreSQL Data Lake Data Warehouse BI 130

Data Engineering Podcast

JANUARY 15, 2023

Trusted by the data teams at Fox, JetBlue, and PagerDuty, Monte Carlo solves the costly problem of broken data pipelines. Monte Carlo also gives you a holistic picture of data health with automatic, end-to-end lineage from ingestion to the BI layer directly out of the box.

Coding 130

Coding Building PostgreSQL Data Lake 130

Data Engineering Weekly

FEBRUARY 2, 2025

[link] Get Your Guide: From Snowflake to Databricks: Our cost-effective journey to a unified data warehouse. GetYourGuide discusses migrating its Business Intelligence (BI) data source from Snowflake to Databricks, achieving a 20% cost reduction.

Data Engineering 57

Data Engineering Data Engineer Engineering Data Lake 57

Cloudera

JUNE 18, 2022

Cloudera customers run some of the biggest data lakes on earth. These lakes power mission critical large scale data analytics, business intelligence (BI), and machine learning use cases, including enterprise data warehouses. On data warehouses and data lakes.

Data Lake 94

Data Lake Data Warehouse BI SQL 94

Towards Data Science

DECEMBER 4, 2023

A robust data infrastructure is a must-have to compete in the F1 business. We’ll build a data architecture to support our racing team starting from the three canonical layers : Data Lake, Data Warehouse, and Data Mart. Data Marts There is a thin line between Data Warehouses and Data Marts.

Data Engineering 98

Data Engineering Data Engineer Engineering Data Lake 98

Data Engineering Podcast

NOVEMBER 25, 2018

Summary When your data lives in multiple locations, belonging to at least as many applications, it is exceedingly difficult to ask complex questions of it. The default way to manage this situation is by crafting pipelines that will extract the data from source systems and load it into a data lake or data warehouse.

Data Lake 100

Data Lake Data Warehouse Hadoop BI 100

Knowledge Hut

APRIL 25, 2024

However, with Business intelligence dashboards, knowledge is dispersed throughout the organization, enabling users to produce interactive reports, utilize data visualization, and disseminate the knowledge with internal and external stakeholders. What is a Business Intelligence Dashboard?

Business Intelligence 52

Business Intelligence BI Raw Data Data Warehouse 52

Cloudera

MARCH 2, 2021

The result is that streaming data tends to be “locked away” from everyone but a small few, and the data engineering team is highly overworked and backlogged. The declarative nature of the SQL language makes it a powerful paradigm for getting data to the people who need it. What do you mean by democratizing?

SQL 112

SQL Java Data Lake Scala 112

Advancing Analytics: Data Engineering

JULY 2, 2019

It’s not always the most accurate indicator, but a quick glance at google trends sees Data Engineer rocketing in popularity, compared to more traditional functions such as BI and ETL Developer: google trends Now, that’s not saying that the other roles are going away, not by a long stretch. or visit www.advancinganalytics.co.uk

Business Intelligence 52

Business Intelligence Data Engineering Data Engineer Engineering 52

Data Engineering Podcast

NOVEMBER 4, 2021

Summary The precursor to widespread adoption of cloud data warehouses was the creation of customer data platforms. Acting as a centralized repository of information about how your customers interact with your organization they drove a wave of analytics about how to improve products based on actual usage data.

Data Warehouse 100

Data Warehouse Business Intelligence ETL System Data Lake 100

Striim

MARCH 21, 2025

Since the value of data quickly drops over time, organizations need a way to analyze data as it is generated. To avoid disruptions to operational databases, companies typically replicate data to data warehouses for analysis. What is Change Data Capture?

IT 52

IT Data Lake Relational Database Data Warehouse 52

phData: Data Engineering

SEPTEMBER 19, 2023

With the amount of data companies are using growing to unprecedented levels, organizations are grappling with the challenge of efficiently managing and deriving insights from these vast volumes of structured and unstructured data. What is a Data Lake? Consistency of data throughout the data lake.

Data Lake 52

Data Lake Process Metadata Data Warehouse 52

AltexSoft

AUGUST 29, 2023

In 2010, a transformative concept took root in the realm of data storage and analytics — a data lake. The term was coined by James Dixon , Back-End Java, Data, and Business Intelligence Engineer, and it started a new era in how organizations could store, manage, and analyze their data.

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

Data Engineering Podcast

NOVEMBER 27, 2022

Push information about data freshness and quality to your business intelligence, automatically scale up and down your warehouse based on usage patterns, and let the bots answer those questions in Slack so that the humans can focus on delivering real value.

Data Process 100

Data Process Process Metadata Business Intelligence 100

Cloudera

JUNE 30, 2022

Over the past decade, Cloudera has enabled multi-function analytics on data lakes through the introduction of the Hive table format and Hive ACID. Companies, on the other hand, have continued to demand highly scalable and flexible analytic engines and services on the data lake, without vendor lock-in.

Data Lake 88

Data Lake Business Intelligence Metadata Data Warehouse 88

Cloudera

JANUARY 21, 2021

If so – you are likely one of the growing group of Line of Business (LoB) professionals forced into creating your own solution – creating your own Shadow IT. Cloudera Data Warehouse (CDW) is here to save the day! CDW is an integrated data warehouse service within Cloudera Data Platform (CDP).

IT 94

IT Data Lake Data Warehouse Cloud Storage 94

Data Engineering Podcast

MAY 19, 2019

Summary In recent years the traditional approach to building data warehouses has shifted from transforming records before loading, to transforming them afterwards. The data build tool (dbt) is designed to bring battle tested engineering practices to your analytics pipelines.

Data Analytics 100

Data Analytics Building Engineering Data Warehouse 100

Data Engineering Podcast

DECEMBER 19, 2021

In the same way that application performance monitoring ensures reliable software and keeps application downtime at bay, Monte Carlo solves the costly problem of broken data pipelines. Start trusting your data with Monte Carlo today! Hightouch is the easiest way to sync data into the platforms that your business teams rely on.

Systems 100

Systems Building Metadata Data Warehouse 100

Data Engineering Podcast

DECEMBER 21, 2020

Fortunately, there’s hope: in the same way that New Relic, DataDog, and other Application Performance Management solutions ensure reliable software and keep application downtime at bay, Monte Carlo solves the costly problem of broken data pipelines.

Data Governance 130

Data Governance Government Data Lake Banking 130

Striim

SEPTEMBER 11, 2024

In an ETL-based architecture, data is first extracted from source systems, then transformed into a structured format, and finally loaded into data stores, typically data warehouses. This method is advantageous when dealing with structured data that requires pre-processing before storage.

Data Pipeline 98

Data Pipeline Designing Data Lake Data Warehouse 98

Monte Carlo

JANUARY 23, 2023

When it comes to the data community, there’s always a debate broiling about something— and right now “data mesh vs data lake” is right at the top of that list. In this post we compare and contrast the data mesh vs data lake to illustrate the benefits of each and help discover what’s right for your data platform.

Data Lake 52

Data Lake Architecture Business Intelligence Unstructured Data 52

Data Engineering Podcast

OCTOBER 28, 2021

In the same way that application performance monitoring ensures reliable software and keeps application downtime at bay, Monte Carlo solves the costly problem of broken data pipelines. Start trusting your data with Monte Carlo today! Hightouch is the easiest way to sync data into the platforms that your business teams rely on.

Data Pipeline 100

Data Pipeline SQL Data Warehouse Data Lake 100

Data Engineering Podcast

DECEMBER 4, 2021

In the same way that application performance monitoring ensures reliable software and keeps application downtime at bay, Monte Carlo solves the costly problem of broken data pipelines. Start trusting your data with Monte Carlo today! Hightouch is the easiest way to sync data into the platforms that your business teams rely on.

Data Warehouse 100

Data Warehouse Data Lake Data BI 100

Data Engineering Podcast

DECEMBER 11, 2021

In the same way that application performance monitoring ensures reliable software and keeps application downtime at bay, Monte Carlo solves the costly problem of broken data pipelines. Start trusting your data with Monte Carlo today! Hightouch is the easiest way to sync data into the platforms that your business teams rely on.

Data Warehouse 100

Data Warehouse Raw Data Data Lake BI 100

Data Engineering Podcast

JANUARY 1, 2022

In the same way that application performance monitoring ensures reliable software and keeps application downtime at bay, Monte Carlo solves the costly problem of broken data pipelines. Start trusting your data with Monte Carlo today! Hightouch is the easiest way to sync data into the platforms that your business teams rely on.

Data Warehouse 100

Data Warehouse SQL Hadoop Data Lake 100

Data Engineering Podcast

DECEMBER 25, 2022

Push information about data freshness and quality to your business intelligence, automatically scale up and down your warehouse based on usage patterns, and let the bots answer those questions in Slack so that the humans can focus on delivering real value.

Building 100

Building Metadata Business Intelligence Data Lake 100

Data Engineering Podcast

NOVEMBER 27, 2021

In the same way that application performance monitoring ensures reliable software and keeps application downtime at bay, Monte Carlo solves the costly problem of broken data pipelines. Start trusting your data with Monte Carlo today! Hightouch is the easiest way to sync data into the platforms that your business teams rely on.

Data Warehouse 100

Data Warehouse Data Lake BI Business Intelligence 100

Data Engineering Podcast

SEPTEMBER 11, 2022

Push information about data freshness and quality to your business intelligence, automatically scale up and down your warehouse based on usage patterns, and let the bots answer those questions in Slack so that the humans can focus on delivering real value. Can you describe what Schemata is and the story behind it?

Systems 100

Systems Metadata Building MongoDB 100

Data Engineering Podcast

MAY 8, 2022

Acryl Data provides DataHub as an easy to consume SaaS product which has been adopted by several companies. Signup for the SaaS product at dataengineeringpodcast.com/acryl RudderStack helps you build a customer data platform on your warehouse or data lake.

Database 100

Database Data Lake BI Business Intelligence 100

Data Engineering Podcast

OCTOBER 20, 2021

Summary The focus of the past few years has been to consolidate all of the organization’s data into a cloud data warehouse. As a result there have been a number of trends in data that take advantage of the warehouse as a single focal point. Start trusting your data with Monte Carlo today!

Data Warehouse 100

Data Warehouse Data Lake Business Intelligence Data Engineering 100