Monte Carlo

FEBRUARY 11, 2025

Think of your data warehouse like a well-organized library. Thats where data warehouse schemas come in. A data warehouse schema is a blueprint for how your data is structured and linkedusually with fact tables (for measurable data) and dimension tables (for descriptive attributes).

Data Warehouse 52

Data Warehouse Electronics Retail Data 52

Seattle Data Guy

JANUARY 12, 2023

A data warehouse consultant plays an important role in companies looking to become data-driven. They help companies design and deploy centralized data sets that are easy to use and reliable. But in order to understand why you need a data warehouse consultant we should take a step back.

Consulting 130

Consulting Data Warehouse Data Designing 130

databricks

AUGUST 22, 2024

Today, we're excited to announce the launch of Data Warehouse Brickbuilder Migration Solutions. This is an expansion to the Brickbuilder Program , which.

Data Warehouse 132

Data Warehouse Data Programming 132

KDnuggets

OCTOBER 30, 2023

A comparative overview of data warehouses, data lakes, and data marts to help you make informed decisions on data storage solutions for your data architecture.

Data Lake 139

Data Lake Data Warehouse Data Storage Data 139

Start Data Engineering

APRIL 12, 2022

Introduction Data lakes and data warehouses Data lake Data warehouse Criteria to choose lake and warehouse tools Conclusion Further reading References Introduction With the data ecosystem growing fast, new terms are coming up every week.

Data Lake 130

Data Lake Data Warehouse Data 130

Precisely

JANUARY 9, 2025

The goal of this post is to understand how data integrity best practices have been embraced time and time again, no matter the technology underpinning. In the beginning, there was a data warehouse The data warehouse (DW) was an approach to data architecture and structured data management that really hit its stride in the early 1990s.

Data Integration 119

Data Integration Hadoop Data Warehouse Data Lake 119

Towards Data Science

JANUARY 28, 2024

How I organize everything in my new data warehouse Continue reading on Towards Data Science »

Data Warehouse 98

Data Warehouse Designing Data Science Data 98

databricks

FEBRUARY 4, 2025

Databricks welcomes BladeBridge, a proven provider of AI-powered migration solutions for enterprise data warehouses. Together, Databricks and BladeBridge will help enterprises accelerate the.

Data Warehouse 115

Data Warehouse Data 115

Seattle Data Guy

DECEMBER 19, 2022

Whether its helping increase revenue by finding new customers or reducing costs, all of it starts with data.

Data Lake 130

Data Lake Data Warehouse Database Data Storage 130

Data Engineering Podcast

MARCH 10, 2023

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. You can collect, transform, and route data across your entire stack with its event streaming, ETL, and reverse ETL pipelines.

Data Warehouse 130

Data Warehouse Data Lake Machine Learning Data Science 130

Seattle Data Guy

AUGUST 2, 2023

A few months ago, I uploaded a video where I discussed data warehouses, data lakes, and transactional databases. However, the world of data management is evolving rapidly, especially with the resurgence of AI and machine learning.

Data Lake 130

Data Lake Data Warehouse Data Machine Learning 130

Seattle Data Guy

APRIL 21, 2025

If you work in data, then youve likely used BigQuery and youve likely used it without really thinking about how it operates under the hood. On the surface BigQuery is Google Clouds fully-managed, serverless data warehouse. … Read more The post What Is BigQuery And How Do You Load Data Into It?

IT 130

IT Google Cloud Data Warehouse Data 130

Simon Späti

MARCH 11, 2025

But data volumes grow, analytical demands become more complex, and Postgres stops being enough. Therefore, you’ve probably come across terms like OLAP (Online Analytical Processing) systems, data warehouses, and, more recently, real-time analytical databases.

Database 130

Database Data Warehouse Data Engineering Data Engineer 130

Towards Data Science

JULY 30, 2024

Rethinking data warehousing: Why redefinition is necessary even beyond Modern Data Warehouse (MDW) and Lakehouse Models Continue reading on Towards Data Science »

Data Warehouse 75

Data Warehouse Data Science Data Data Architecture 75

Snowflake

NOVEMBER 2, 2023

Over the years, the technology landscape for data management has given rise to various architecture patterns, each thoughtfully designed to cater to specific use cases and requirements. These patterns include both centralized storage patterns like data warehouse , data lake and data lakehouse , and distributed patterns such as data mesh.

Data Lake 115

Data Lake Data Warehouse Cloud Unstructured Data 115

Towards Data Science

FEBRUARY 24, 2024

Best practice and advanced techniques for beginners Continue reading on Towards Data Science »

Data Warehouse 98

Data Warehouse Building Data Science Data 98

Christophe Blefari

MARCH 1, 2023

dbt Core is an open-source framework that helps you organise data warehouse SQL transformation. dbt was born out of the analysis that more and more companies were switching from on-premise Hadoop data infrastructure to cloud data warehouses. This switch has been lead by modern data stack vision.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Start Data Engineering

MAY 11, 2022

Data lake structure 5. Loading user purchase data into the data warehouse 5.2 Loading classified movie review data into the data warehouse 5.3 Prerequisite 4.2 AWS Infrastructure costs 4.3 Code walkthrough 5.1 Generating user behavior metric 5.4. Checking results 6. Tear down infra 7. Next steps 9.

Data Engineering 246

Data Engineering Data Engineer Project Data Lake 246

Hevo

SEPTEMBER 4, 2024

A data warehouse is a centralized system that stores, integrates, and analyzes large volumes of structured data from various sources. It is predicted that more than 200 zettabytes of data will be stored in the global cloud by 2025.

Data Warehouse 72

Data Warehouse Structured Data Data Cloud 72

Snowflake

APRIL 2, 2025

Data storage has been evolving, from databases to data warehouses and expansive data lakes, with each architecture responding to different business and data needs. Traditional databases excelled at structured data and transactional workloads but struggled with performance at scale as data volumes grew.

Data Lake 97

Data Lake Metadata Cloud Storage Data Warehouse 97

Christophe Blefari

JANUARY 11, 2025

He listed 4 things that are the most difficult data integration tasks: from mutable data to IT migrations, everything adds complexity to ingestion systems. The software development lifecycle within a modern data engineering framework — A great deep-dive about a data platform using dltHub, dbt and Dagster.

Data 130

Data Data Warehouse Coding Programming Language 130

Engineering at Meta

JANUARY 22, 2025

These stages propagate through various systems including function-based systems that load, process, and propagate data through stacks of function calls in different programming languages (e.g., For simplicity, we will demonstrate these for the web, the data warehouse, and AI, per the diagram below. Hack, C++, Python, etc.)

Data Warehouse 109

Data Warehouse SQL Programming Language Data 109

Analytics Vidhya

FEBRUARY 1, 2023

Introduction Nowadays, organizations are looking for multiple solutions to deal with big data and related challenges.

Data Cleanse 223

Data Cleanse Data Warehouse Big Data Cloud 223

Analytics Vidhya

FEBRUARY 8, 2023

Introduction Amazon Redshift is a fully managed, petabyte-scale data warehousing Amazon Web Services (AWS). It allows users to easily set up, operate, and scale a data warehouse in the cloud.

Amazon Web Services 227

Amazon Web Services AWS Data Warehouse Cloud 227

Data Engineering Podcast

FEBRUARY 18, 2024

Summary A data lakehouse is intended to combine the benefits of data lakes (cost effective, scalable storage and compute) and data warehouses (user friendly SQL interface). Multiple open source projects and vendors have been working together to make this vision a reality.

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Confessions of a Data Guy

MAY 30, 2024

Of all the duties that Data Engineers take on during the regular humdrum of business and work, it’s usually filled with the same old, same old. Build new pipeline, update pipeline, new data model, fix bug, etc, etc. It’s never-ending.

Building 184

Building Data Warehouse Data Data Engineering 184

Christophe Blefari

MARCH 15, 2024

Differential storage, a building block for a DuckDB-based data warehouse — It's MotherDuck vision, creating the next data warehouse on-top of DuckDB leveraging DuckDB morphing capacities between a single machine and a production ecosystem.

Metadata 272

Metadata Data Datasets Data Warehouse 272

Data Engineering Podcast

APRIL 23, 2023

You can collect, transform, and route data across your entire stack with its event streaming, ETL, and reverse ETL pipelines. You can Implement RudderStack SDKs once, then automatically send events to your warehouse and 150+ business tools, and you’ll never have to worry about API changes again.

Data Lake 277

Data Lake Kafka Machine Learning Data Warehouse 277

Seattle Data Guy

FEBRUARY 10, 2024

Photo by Tiger Lily Data warehouses and data lakes play a crucial role for many businesses. It gives businesses access to the data from all of their various systems. As well as often integrating data so that end-users can answer business critical questions.

Data Lake 162

Data Lake Data Warehouse Data Accessibility 162

Simon Späti

MARCH 9, 2021

This post focuses on practical data pipelines with examples from web-scraping real-estates, uploading them to S3 with MinIO, Spark and Delta Lake, adding some Data Science magic with Jupyter Notebooks, ingesting into Data Warehouse Apache Druid, visualising dashboards with Superset and managing everything with Dagster.

Data Engineering 189

Data Engineering Data Engineer Engineering Project 189

Snowflake

MARCH 12, 2025

While working together, they bonded over their shared passion for data. After experiencing numerous data quality challenges, they created Anomalo, a no-code platform for validating and documenting data warehouse information.

Unstructured Data 95

Unstructured Data High Quality Data Banking Machine Learning 95

Engineering at Meta

FEBRUARY 4, 2025

Meta joins the Data Transfer Project and has continuously led the development of shared technologies that enable users to port their data from one platform to another. 2024: Users can access data logs in Download Your Information. What are data logs?

Accessible 80

Accessible Accessibility Raw Data Data Warehouse 80

phData: Data Engineering

FEBRUARY 21, 2025

In this post, we will be particularly interested in the impact that cloud computing left on the modern data warehouse. We will explore the different options for data warehousing and how you can leverage this information to make the right decisions for your organization. Understanding the Basics What is a Data Warehouse?

Cloud 52

Cloud Data Warehouse Amazon Web Services Data Ingestion 52

Christophe Blefari

JUNE 21, 2024

Snowflake was founded in 2012 around its data warehouse product, which is still its core offering, and Databricks was founded in 2013 from academia with Spark co-creator researchers, becoming Apache Spark in 2014. Databricks is focusing on simplification (serverless, auto BI 2 , improved PySpark) while evolving into a data warehouse.

Metadata 147

Metadata Data Warehouse BI MySQL 147

Snowflake

DECEMBER 4, 2024

This remains important, of course, but the next step will be to make sure that the enterprise’s unified data is AI ready, able to be plugged into existing agents and applications. The trend to centralize data will accelerate, making sure that data is high-quality, accurate and well managed.

Unstructured Data 97

Unstructured Data Data Lake Deep Learning Structured Data 97

Data Engineering Podcast

APRIL 9, 2023

When it was difficult to wire together the event collection, data modeling, reporting, and activation it made sense to buy monolithic products that handled every stage of the customer data lifecycle. Now that the data warehouse has taken center stage a new approach of composable customer data platforms is emerging.

Data Lake 147

Data Lake Data Warehouse Machine Learning Data 147

Confessions of a Data Guy

JANUARY 12, 2023

Anyone who’s been roaming around the forest of Data Engineering has probably run into many of the newish tools that have been growing rapidly around the concepts of Data Warehouses, Data Lakes, and Lake Houses … the merging of the old relational database functionality with TB and PB level cloud-based file storage systems.

Data Lake 162

Data Lake Relational Database Data Warehouse Data Engineering 162