Precisely

JANUARY 9, 2025

The goal of this post is to understand how data integrity best practices have been embraced time and time again, no matter the technology underpinning. In the beginning, there was a data warehouse The data warehouse (DW) was an approach to data architecture and structured data management that really hit its stride in the early 1990s.

Data Integration 118

Data Integration Hadoop Data Warehouse Data Lake 118

Cloudera

JULY 16, 2021

Did you know Cloudera customers, such as SMG and Geisinger , offloaded their legacy DW environment to Cloudera Data Warehouse (CDW) to take advantage of CDW’s modern architecture and best-in-class performance? The Data Warehouse on Cloudera Data Platform provides easy to use self-service and advanced analytics use cases at scale.

Data Warehouse 120

Data Warehouse SQL PostgreSQL Database 120

Christophe Blefari

JANUARY 11, 2025

He listed 4 things that are the most difficult data integration tasks: from mutable data to IT migrations, everything adds complexity to ingestion systems. The software development lifecycle within a modern data engineering framework — A great deep-dive about a data platform using dltHub, dbt and Dagster.

Data 130

Data Data Warehouse Coding Programming Language 130

Engineering at Meta

JANUARY 22, 2025

In this blog, we will delve into an early stage in PAI implementation: data lineage. Data lineage refers to the process of tracing the journey of data as it moves through various systems, illustrating how data transitions from one data asset, such as a database table (the source asset), to another (the sink asset).

Data Warehouse 120

Data Warehouse SQL Programming Language Data 120

Seattle Data Guy

AUGUST 2, 2023

A few months ago, I uploaded a video where I discussed data warehouses, data lakes, and transactional databases. However, the world of data management is evolving rapidly, especially with the resurgence of AI and machine learning.

Data Lake 130

Data Lake Data Warehouse Data Machine Learning 130

Cloudera

SEPTEMBER 29, 2020

Performance is one of the key, if not the most important deciding criterion, in choosing a Cloud Data Warehouse service. In today’s fast changing world, enterprises have to make data driven decisions quickly and for that they rely heavily on their data warehouse service. . Cloudera Data Warehouse vs HDInsight.

Data Warehouse 120

Data Warehouse Cloud Storage Metadata Cloud 120

Towards Data Science

JANUARY 28, 2024

How I organize everything in my new data warehouse Continue reading on Towards Data Science »

Data Warehouse 98

Data Warehouse Designing Data Science Data 98

Snowflake

APRIL 2, 2025

Data storage has been evolving, from databases to data warehouses and expansive data lakes, with each architecture responding to different business and data needs. Traditional databases excelled at structured data and transactional workloads but struggled with performance at scale as data volumes grew.

Data Lake 94

Data Lake Cloud Storage Metadata Data Warehouse 94

Christophe Blefari

MARCH 1, 2023

dbt Core is an open-source framework that helps you organise data warehouse SQL transformation. dbt was born out of the analysis that more and more companies were switching from on-premise Hadoop data infrastructure to cloud data warehouses. This switch has been lead by modern data stack vision.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Towards Data Science

MAY 11, 2024

When architecting a transactional database or a data warehouse, it’s important not to forget about various types of technical columns… Continue reading on Towards Data Science »

Database Design 98

Database Design Database Designing Data Warehouse 98

Towards Data Science

JUNE 6, 2023

To start exploring I needed a good approach for performing data analysis over thousands of poorly documented JSON and CSV files … extra points for analysis that doesn’t require my data to leave my laptop. Let’s start looking at the variety of data available in the archive. Why DuckDB?

Data Warehouse 98

Data Warehouse Data SQL Data Analysis 98

Christophe Blefari

MARCH 15, 2024

Postgres creator launches DBOS, a transactional serverless computing platform — Mike sees DBOS like a cloud-native OS that runs on-top of the database in order to rethink application development and deployment. Arroyo, a stream-processing platform, rebuilt their engine using DataFusion.

Metadata 272

Metadata Data Software Engineer Software Engineering 272

Data Engineering Podcast

DECEMBER 11, 2022

Managing the operational concerns for your database can be complex and expensive, especially if you need to scale to large volumes of data, high traffic, or geographically distributed usage. No more shipping and praying, you can now know exactly what will change in your database! or any other destination you choose.

Database 100

Database MySQL Data Lake MongoDB 100

Analytics Vidhya

FEBRUARY 1, 2023

Introduction Nowadays, organizations are looking for multiple solutions to deal with big data and related challenges.

Data Cleanse 223

Data Cleanse Data Warehouse Big Data Cloud 223

Towards Data Science

FEBRUARY 24, 2024

Best practice and advanced techniques for beginners Continue reading on Towards Data Science »

Data Warehouse 98

Data Warehouse Building Data Science Data 98

Data Engineering Podcast

AUGUST 6, 2021

Summary Every organization needs to be able to use data to answer questions about their business. The trouble is that the data is usually spread across a wide and shifting array of systems, from databases to dashboards. Are you bored with writing scripts to move data into SaaS tools like Salesforce, Marketo, or Facebook Ads?

Database 100

Database SQL BI Business Intelligence 100

Data Engineering Podcast

MAY 1, 2022

Datafold shows how a change in SQL code affects your data, both on a statistical level and down to individual rows and values before it gets merged to production. No more shipping and praying, you can now know exactly what will change in your database! Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Warehouse 100

Data Warehouse Data Integration Cloud Google Cloud 100

Maxime Beauchemin

JANUARY 7, 2018

a simplified DAG of partitions While your database of choice may allow you to update, insert, and delete at will, it doesn’t mean you should. Results may vary depending on how smart your database optimizer is. But how do we model this in a functional data warehouse without mutating data?

Data Engineering 183

Data Engineering Data Engineer Data Process Process 183

Data Engineering Podcast

MAY 8, 2022

Summary Many of the events, ideas, and objects that we try to represent through data have a high degree of connectivity in the real world. TigerGraph is a leading database that offers a highly scalable and performant native graph engine for powering graph analytics and machine learning. Start trusting your data with Monte Carlo today!

Database 100

Database Data Lake BI Business Intelligence 100

Confessions of a Data Guy

JANUARY 12, 2023

Anyone who’s been roaming around the forest of Data Engineering has probably run into many of the newish tools that have been growing rapidly around the concepts of Data Warehouses, Data Lakes, and Lake Houses … the merging of the old relational database functionality with TB and PB level cloud-based file storage systems.

Data Lake 162

Data Lake Relational Database Data Warehouse Data Engineering 162

Data Engineering Podcast

AUGUST 6, 2022

Summary The optimal format for storage and retrieval of data is dependent on how it is going to be used. For analytical systems there are decades of investment in data warehouses and various modeling techniques. Who is the target audience for this database? Can you describe what Milvus is and the story behind it?

Machine Learning 100

Machine Learning Database MySQL MongoDB 100

Cloudera

APRIL 3, 2023

In this blog, we will share with you in detail how Cloudera integrates core compute engines including Apache Hive and Apache Impala in Cloudera Data Warehouse with Iceberg. We will publish follow up blogs for other data services. It allows us to independently upgrade the Virtual Warehouses and Database Catalogs.

Data Warehouse 82

Data Warehouse Java Metadata Data 82

Start Data Engineering

JULY 18, 2020

Given Source S3 path and file delimiter data warehouse connection details (endpoint, port, username, password and database name) data warehouse schema name and table name Run frequency Steps Log into your stitch account, here Click on the Destination tab and use the data warehouse connection details to establish a destination database.

Data Warehouse 130

Data Warehouse Database Data 130

Christophe Blefari

JUNE 21, 2024

Snowflake was founded in 2012 around its data warehouse product, which is still its core offering, and Databricks was founded in 2013 from academia with Spark co-creator researchers, becoming Apache Spark in 2014. Databricks is focusing on simplification (serverless, auto BI 2 , improved PySpark) while evolving into a data warehouse.

Metadata 147

Metadata Data Warehouse BI MySQL 147

Striim

MARCH 21, 2025

With instant elasticity, high-performance, and secure data sharing across multiple clouds , Snowflake has become highly in-demand for its cloud-based data warehouse offering. As organizations adopt Snowflake for business-critical workloads, they also need to look for a modern data integration approach.

Data Warehouse 52

Data Warehouse MongoDB MySQL Hadoop 52

Data Engineering Weekly

AUGUST 6, 2024

As a result, lakehouses support more dynamic and flexible data architectures, catering to a broader range of analytics and operational workloads. For instance, in a fast-paced retail environment, lakehouses can ensure that inventory data remains up-to-date and accurate in the data warehouse, optimizing supply chain efficiency.

Data Lake 125

Data Lake Data Warehouse Database Data Architecture 125

Data Engineering Podcast

FEBRUARY 5, 2023

Materialize]([link] Looking for the simplest way to get the freshest data possible to your teams? Look no further than Materialize, the streaming database you already know how to use. Materialize]([link] Looking for the simplest way to get the freshest data possible to your teams?

Data Engineering 147

Data Engineering Data Engineer Engineering PostgreSQL 147

Start Data Engineering

JULY 11, 2020

Intro A very common use case in data engineering is to build a ETL system for a data warehouse, to have data loaded in from multiple separate databases to enable data analysts/scientists to be able to run queries on this data, since the source databases are used by your applications and we do not want these analytic queries to affect our application (..)

Systems 130

Systems Designing ETL System Data Warehouse 130

Analytics Vidhya

MARCH 14, 2023

This results in the generation of so much data daily. This generated data is stored in the database and will maintain it. SQL is a structured query language used to read and write these databases. Introduction In today’s world, technology has increased tremendously, and many people are using the internet.

SQL 204

SQL Database Technology Data 204

Christophe Blefari

NOVEMBER 11, 2022

I'll speak about "How to build the data dream team" Let's jump onto the news. Ingredients of a Data Warehouse Going back to basics. Kovid wrote an article that tries to explain what are the ingredients of a data warehouse. Slowly, years after years, graph databases time is coming up.

BI 130

BI Data Warehouse Data Database 130

Data Engineering Podcast

FEBRUARY 4, 2024

RisingWave is a database engine that was created specifically for stream processing, with S3 as the storage layer. In this episode Yingjun Wu explains how it is architected to power analytical workflows on continuous data flows, and the challenges of making it responsive and scalable.

SQL 173

SQL Data Lake High Quality Data Machine Learning 173

Monte Carlo

FEBRUARY 12, 2024

Today, this first-party data mostly lives in two types of data repositories. If it is structured data then it’s often stored in a table within a modern database, data warehouse or lakehouse. If it’s unstructured data, then it’s often stored as a vector in a namespace within a vector database.

Database 72

Database Unstructured Data Data Pipeline Metadata 72

Monte Carlo

OCTOBER 31, 2024

The foundational skills are similar between traditional data engineers and AI data engineers are similar, with AI data engineers more heavily focused on machine learning data infrastructure, AI-specific tools, vector databases, and LLM pipelines. Let’s dive into the tools necessary to become an AI data engineer.

Data Engineering 52

Data Engineering Data Engineer Engineering Unstructured Data 52

Striim

MARCH 21, 2025

Business transactions captured in relational databases are critical to understanding the state of business operations. Since the value of data quickly drops over time, organizations need a way to analyze data as it is generated. Traditionally, businesses used batch-based approaches to move data once or several times a day.

IT 52

IT Data Lake Relational Database Data Warehouse 52

Data Engineering Podcast

JANUARY 21, 2024

Summary Databases and analytics architectures have gone through several generational shifts. A substantial amount of the data that is being managed in these systems is related to customers and their interactions with an organization. How has that changed the architectural approach to CDPs?

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Simon Späti

AUGUST 25, 2022

Didn’t know the difference between a Data Lakehouse and a Data Warehouse? Or you just wanted to govern your hundreds to thousands of files and have more database-like features but don’t know how? Asked yourself what components and features would that include.

Data Lake 130

Data Lake Data Warehouse Government Data 130

Simon Späti

SEPTEMBER 30, 2022

Didn’t know the difference between a Data Lakehouse and a Data Warehouse? Or you just wanted to govern your hundreds to thousands of files and have more database-like features but don’t know how? Asked yourself what components and features would that include.

Data Lake 130

Data Lake Data Warehouse Government Data 130