Data Engineering Podcast

AUGUST 13, 2022

In this episode founder Shayan Mohanty explains how he and his team are bringing software best practices and automation to the world of machine learning data preparation and how it allows data engineers to be involved in the process. Data stacks are becoming more and more complex. That’s where our friends at Ascend.io

Machine Learning 100

Machine Learning Pipeline-centric Database-centric MongoDB 100

Christophe Blefari

APRIL 8, 2023

At the same time Maxime Beauchemin wrote a post about Entity-Centric data modeling. In the recent years dbt simplified and revolutionised the tooling to create data models. This week I discovered SQLMesh , a all-in-one data pipelines tool. I hope he will fill the gaps. dbt, as of today, is the leading framework.

Pipeline-centric 130

Pipeline-centric Database-centric Algorithm Data 130

Christophe Blefari

APRIL 8, 2023

At the same time Maxime Beauchemin wrote a post about Entity-Centric data modeling. In the recent years dbt simplified and revolutionised the tooling to create data models. This week I discovered SQLMesh , a all-in-one data pipelines tool. I hope he will fill the gaps. dbt, as of today, is the leading framework.

Pipeline-centric 130

Pipeline-centric Database-centric Algorithm Data 130

Knowledge Hut

JUNE 20, 2023

Factors Data Engineer Machine Learning Definition Data engineers create, maintain, and optimize data infrastructure for data. In addition, they are responsible for developing pipelines that turn raw data into formats that data consumers can use easily. Assess the needs and goals of the business.

Machine Learning 52

Machine Learning Data Engineering Data Engineer Engineering 52

Snowflake

JUNE 28, 2023

Snowpark is our secure deployment and processing of non-SQL code, consisting of two layers: Familiar Client Side Libraries – Snowpark brings deeply integrated, DataFrame-style programming and OSS compatible APIs to the languages data practitioners like to use. Previously, tasks could be executed as quickly as 1-minute.

Python 52

Python Accessibility Accessible Pipeline-centric 52

DataKitchen

FEBRUARY 21, 2023

As a result, a less senior team member was made responsible for modifying a production pipeline. Create a Path To Production For Self-Service: “… business users explore data through self-service data preparation, few have established gatekeeping processes to deliver these workloads to production.”

Data Engineering 52

Data Engineering Data Engineer Engineering Pipeline-centric 52

Knowledge Hut

SEPTEMBER 26, 2023

Key Features of Azure Synapse Here are some of the key features of Azure Synapse: Cloud Data Service: Azure Synapse operates as a cloud-native service, residing within the Microsoft Azure cloud ecosystem. This cloud-centric approach ensures scalability, flexibility, and cost-efficiency for your data workloads.

Data Lake 52

Data Lake Database-centric Pipeline-centric Machine Learning 52