DataKitchen

MAY 10, 2024

The Five Use Cases in Data Observability: Effective Data Anomaly Monitoring (#2) Introduction Ensuring the accuracy and timeliness of data ingestion is a cornerstone for maintaining the integrity of data systems. Have all the source files/data arrived on time? Is the source data of expected quality?

Data Ingestion 53

Data Ingestion Transportation High Quality Data Data 53

Monte Carlo

JANUARY 25, 2023

There is, however, an added dimension to this relationship: data producers are often consumers of upstream data sources. Data warehouse producers wear both hats working with upstream producers so they can consume high-quality data and producing high-quality data to provide to their consumers.

Data Warehouse 52

Data Warehouse Data High Quality Data Metadata 52

DataKitchen

JUNE 19, 2023

Data Journeys run on software, on servers, and with code. Checking data at rest involves looking at syntactic attributes such as freshness, distribution, volume, schema, and lineage. Start checking data at rest with a strong data profile. The image above shows an example ‘’data at rest’ test result.

Data 40

Data Data Validation Utilities High Quality Data 40

Monte Carlo

APRIL 24, 2023

If streaming data is a priority for your platform, you might also choose to leverage a system like Confluent’s Apache Kafka along with some of the above mentioned technologies. That means less engineering time spent coding and maintaining pipelines—and less complexity down the road as you begin to invest in other layers of your data stack.

Data Pipeline 52

Data Pipeline Building Data Ingestion BI 52