Remove Data Programming Remove Kafka Remove Metadata
article thumbnail

Build Maintainable And Testable Data Applications With Dagster

Data Engineering Podcast

In this episode he explains his motivation for creating a product for data management, how the programming model simplifies the work of building testable and maintainable pipelines, and his vision for the future of data programming. Raghu Murthy, founder and CEO of Datacoral built data infrastructures at Yahoo!

Building 100
article thumbnail

Keeping Your Data Warehouse In Order With DataForm

Data Engineering Podcast

Dataform is a platform that helps you apply engineering principles to your data transformations and table definitions, including unit testing SQL scripts, defining repeatable pipelines, and adding metadata to your warehouse to improve your team’s communication. Visit Datacoral.com today to find out more.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Data Scientist vs Data Engineer: Differences and Why You Need Both

AltexSoft

They set up resources required by the model, create pipelines to connect them with data, manage computer resources, and monitor and configure the model’s performance. Managing data and metadata. There are different ways how data can be stored: a data warehouse, numerous data lakes and data hubs , etc.

article thumbnail

Scaling Data Trust: How AutoTrader UK Migrated to a Decentralized Data Platform with Monte Carlo

Monte Carlo

The challenge: building trust and empowering self-serve data “As we’re migrating trusted on-premises systems to the cloud, the users of those older systems need to have trust that the new cloud-based technologies are as reliable as the older systems they’ve used in the past,” said Edward. Today, Edward and his team have a robust data stack.

Data 40