Data Engineering Podcast

JUNE 2, 2024

Summary Modern businesses aspire to be data driven, and technologists enjoy working through the challenge of building data systems to support that goal. Data governance is the binding force between these two parts of the organization. Nicola Askham found her way into data governance by accident, and stayed because of the benefit that she was able to provide by serving as a bridge between the technology and business.

Data Governance 147

Data Governance Government Data Lake High Quality Data 147

databricks

JUNE 2, 2024

We are excited to announce a new data type called variant for semi-structured data. Variant provides an order of magnitude performance improvements compared.

Structured Data 136

Structured Data Data Data Engineer Data Engineering 136

Data Engineering Weekly

JUNE 2, 2024

Data Engineering Weekly is sponsored by Astronomer—Enterprise-Grade Apache Airflow. Deliver data on time with the speed and scale your application demands. Learn More → AI Verify Foundation: Model AI Governance Framework for Generative AI Several countries are working on building governance rules for Gen AI. Data sovereignty will play a vital role as countries formulate regulations.

Data Engineer 104

Data Engineer Data Engineering Engineering Pipeline-centric 104

Scott Logic

JUNE 2, 2024

It’s June! It’s Pride month! Rainbows! Love is love! We’re your ally! Buy stuff with rainbows on! Let’s come to your Pride parade, but make sure you tone it down a bit! More rainbows! Buy our products! Look, we’ve put a rainbow on it! We love everyone ! We love absolutely everyone, in a very non-specific way! We definitely love sparkly unicorn rainbows!

Consulting 98

Consulting AWS IT Management 98

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Zalando Engineering

JUNE 2, 2024

Introduction Our Data Science colleagues had been hosting an internal Data Science Days event for a few years. For our 2,000+ Engineers, we had been missing a similar community event. For several years we wanted to organize one, but got distracted by other priorities and external factors. Finally, in 2022 we decided to commit to hosting an internal Engineering Conference every year and included this commitment in our Engineering Strategy.

Engineering 62

Engineering Programming Data Science Data Engineer 62

Monte Carlo

JUNE 2, 2024

Every day, an uncountable amount of data flows through millions of businesses. Data Infrastructure Engineers are the professionals who ensure that this data flows smoothly and reliably. But what does a data infrastructure engineer do exactly? Table of Contents What is a Data Infrastructure Engineer? Difference Between a Data Infrastructure Engineer and a Data Science Engineer Skill Sets for Data Infrastructure Engineers Technical Expertise Day-to-Day Operations Collaboration with Cross-Functiona

Engineering 52

Engineering Amazon Web Services Data Science AWS 52