Silectis

JANUARY 22, 2021

For those using a robust analytics database, such as the Snowflake® Data Cloud , adding the power of a data engineering platform can help maximize the value you’re getting out of that database. Data Warehouses Have Boundaries Data warehouses do what they’re meant to, they provide a high-performance environment for data analytics.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

Christophe Blefari

OCTOBER 6, 2024

SAM 2, Segment Anything Model 2 can run on-device on Apple CoreML — A demo of image segmentation that run 100% offline and on-device. Web browser automation through agentic workflows — A Github repo with a demo using Gemini and Selenium to automate browser actions. Industrial application might easily follow out of this.

Data 130

Data SQL Hadoop Deep Learning 130

Monte Carlo

FEBRUARY 6, 2023

So, you’re planning a cloud data warehouse migration. But be warned, a warehouse migration isn’t for the faint of heart. As you probably already know if you’re reading this, a data warehouse migration is the process of moving data from one warehouse to another. A worthy quest to be sure.

Data Warehouse 52

Data Warehouse AWS Data Validation Data 52

Data Engineering Podcast

DECEMBER 28, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 130

Data Lake Data Warehouse Data Pipeline MongoDB 130

Data Engineering Podcast

DECEMBER 25, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Machine Learning 130

Machine Learning Systems Data Lake Data Warehouse 130

Monte Carlo

JULY 9, 2024

As more organizations race to adopt GenAI and build AI-powered data products, DuckDB provides emerging applications as the storage layer of RAG knowledge bases to streamline and expedite data management. MotherDuck turbocharged DuckDB’s efficiency with multiplayer cloud analytics, making it a lightweight but powerful data warehouse.

Data Warehouse 69

Data Warehouse Data Pipeline SQL Database 69

Data Engineering Podcast

DECEMBER 18, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 100

Data Lake Data Warehouse Data Pipeline MongoDB 100

Data Engineering Podcast

NOVEMBER 27, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 100

Data Lake Data Warehouse MongoDB Data Pipeline 100

Data Engineering Podcast

DECEMBER 4, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 100

Data Lake MongoDB MySQL Data Warehouse 100

Data Engineering Podcast

DECEMBER 11, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Database 100

Database MySQL Data Lake MongoDB 100

Data Engineering Podcast

OCTOBER 30, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Weekly

OCTOBER 6, 2024

The learning goes back to the fundamentals of pipeline design principles. Regularly review if pipelines are still required. Minimize the data used in pipelines, aka do incremental data pipeline design. Optimize pipeline schedules. Filter data effectively to make sure the query uses partition pruning.

Data Engineering 62

Data Engineering Data Engineer Engineering Data Warehouse 62

Ascend.io

APRIL 22, 2024

Plus, we’ll offer a teaser of what’s to come: an exclusive live demo that you simply cannot miss. The Evolution of Data Engineering Over the past decade, we’ve seen tremendous change in the data engineering landscape. Read More: The State of Data Engineering in 2023: Does Your Data Program Stack Up?

Project 52

Project Metadata Data Pipeline Data Engineering 52

Data Engineering Podcast

APRIL 24, 2022

If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. Struggling with broken pipelines?

Data Warehouse 100

Data Warehouse Data Lake Data Pipeline BI 100

Data Engineering Podcast

APRIL 3, 2022

Summary The flexibility of software oriented data workflows is useful for fulfilling complex requirements, but for simple and repetitious use cases it adds significant complexity. Coalesce is a platform designed to reduce repetitive work for common workflows by adopting a visual pipeline builder to support your data warehouse transformations.

Data Warehouse 100

Data Warehouse Data Workflow Data Architecture SQL 100

Data Engineering Podcast

NOVEMBER 20, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 130

Data Lake Data Ingestion MongoDB Google Cloud 130

Data Engineering Podcast

NOVEMBER 6, 2022

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Sign up free at dataengineeringpodcast.com/rudder Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

MongoDB 130

MongoDB Scala MySQL Machine Learning 130

Data Engineering Podcast

JULY 3, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Architecture 100

Architecture Metadata MongoDB MySQL 100

Data Engineering Podcast

JULY 17, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Management 130

Data Management Management Metadata MongoDB 130

Monte Carlo

MAY 26, 2023

Pro-tip : Don’t confuse data freshness with data latency. Data latency is the time between when the event occurs and when the data is available in the core data system (like a data warehouse) whereas data freshness is how recently the data within the final asset (table, BI report) has been updated.

Data Pipeline 98

Data Pipeline Data Data Warehouse Machine Learning 98

Data Engineering Podcast

JUNE 19, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Data Engineering Podcast

APRIL 17, 2022

If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Warehouse 100

Data Warehouse SQL Data Engineering Data Engineer 100

Data Engineering Podcast

MAY 18, 2021

Summary Data lineage is the common thread that ties together all of your data pipelines, workflows, and systems. In order to get a holistic understanding of your data quality, where errors are occurring, or how a report was constructed you need to track the lineage of the data from beginning to end.

Metadata 100

Metadata Kafka Data Warehouse Hadoop 100

Data Engineering Podcast

JULY 10, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Data Engineering Podcast

MARCH 16, 2020

Are you spending too much time maintaining your data pipeline? Snowplow empowers your business with a real-time event data pipeline running in your own cloud account without the hassle of maintenance. Set up a demo and mention you’re a listener for a special offer!

Building 100

Building Data Warehouse NoSQL Data Lake 100

Data Engineering Podcast

JUNE 26, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130

Data Engineering Podcast

OCTOBER 2, 2021

If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Building 100

Building BI Data Architecture Architecture 100

Monte Carlo

AUGUST 22, 2023

Slow running data pipelines cost data teams time, money, and goodwill. They utilize excess compute, cause data quality issues, and create a poor user experience for data consumers that must wait in exasperation for data to return, dashboards to load, and AI models to update. This could drive unwanted costs.

Data Pipeline 52

Data Pipeline Metadata Data Warehouse Utilities 52

Data Engineering Podcast

MAY 25, 2021

RudderStack’s smart customer data pipeline is warehouse-first. It builds your customer data warehouse and your identity graph on your data warehouse, with support for Snowflake, Google BigQuery, Amazon Redshift, and more. RudderStack’s smart customer data pipeline is warehouse-first.

Database 100

Database Building Data Warehouse Machine Learning 100

Monte Carlo

APRIL 28, 2023

Data Engineers : Data engineers who work with data pipelines and ETL processes use DuckDB to enhance the performance of their data processing tasks. They can leverage DuckDB’s efficient query execution and integration with data formats like Parquet to optimize their data workflows.

Data Warehouse 52

Data Warehouse Programming Language Datasets Database 52

Data Engineering Podcast

NOVEMBER 13, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

MongoDB 100

MongoDB Google Cloud Scala MySQL 100

Workfall

SEPTEMBER 18, 2023

Meet Airbyte, the data magician that turns integration complexities into child’s play. In this digital era, businesses thrive on data, and making this data dance harmoniously with your analytics tools is crucial. With its drag-and-drop interface, creating data pipelines becomes as easy as arranging blocks in a puzzle.

Data Pipeline 93

Data Pipeline Raw Data Data Schemas Healthcare 93

Grouparoo

AUGUST 24, 2021

What you really want is a unified view of your data using Customer Data Integration so you can take action on it. Customer data integration here might include creating a data warehouse where you can house your accurate and complete dataset. How do you know your warehouse is up to date?

Data Integration 52

Data Integration Data Consolidation Data Warehouse ETL Tools 52

Ascend.io

OCTOBER 28, 2024

Tools like Python’s requests library or ETL/ELT tools can facilitate data enrichment by automating the retrieval and merging of external data. Read More: Discover how to build a data pipeline in 6 steps Data Integration Data integration involves combining data from different sources into a single, unified view.

Raw Data 52

Raw Data Datasets Aggregated Data Data Pipeline 52

Data Engineering Podcast

MARCH 27, 2022

If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Relational Database 100

Relational Database Database SQL Data Warehouse 100

Data Engineering Podcast

FEBRUARY 24, 2020

Are you spending too much time maintaining your data pipeline? Snowplow empowers your business with a real-time event data pipeline running in your own cloud account without the hassle of maintenance. Set up a demo and mention you’re a listener for a special offer!

IT 100

IT Data Lake Data Pipeline Media 100

Data Engineering Podcast

JUNE 5, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Security 100

Data Security Metadata MongoDB Scala 100