Databand.ai

SEPTEMBER 9, 2022

How to analyze and resolve data pipeline incidents in Databand Niv Sluzki 2022-09-09 13:00:12 A data pipeline failure can cripple your downstream data flows. Whether it failed to start or quit unexpectedly, you need to know immediately if there is a pipeline incident.

Data Pipeline 52

Data Pipeline Datasets AWS Data 52

Data Engineering Podcast

JUNE 1, 2021

Summary Every part of the business relies on data, yet only a small team has the context and expertise to build and maintain workflows and data pipelines to transform, clean, and integrate it. RudderStack’s smart customer data pipeline is warehouse-first.

Data Pipeline 100

Data Pipeline Data Warehouse Data Engineer Data 100

Silectis

JANUARY 22, 2021

For those using a robust analytics database, such as the Snowflake® Data Cloud , adding the power of a data engineering platform can help maximize the value you’re getting out of that database. Magpie Fills in the Gaps for Better Data Engineering And we’re not talking about just ETL (extract, transform, load). Magpie can help.

Data Engineer 52

Data Engineer Data Engineering Engineering Data Warehouse 52

Databand.ai

SEPTEMBER 19, 2022

How to create data pipeline and data quality SLA alerts in Databand Helen Soloveichik 2022-09-20 01:49:30 Data engineers often get inundated by alerts from data issues. Databand helps fix this problem by breaking through noisy alerts with focused alerting and routing when a data pipeline and quality issues occur.

Data Pipeline 52

Data Pipeline Datasets Data Data Engineer 52

Christophe Blefari

OCTOBER 6, 2024

SAM 2, Segment Anything Model 2 can run on-device on Apple CoreML — A demo of image segmentation that run 100% offline and on-device. Web browser automation through agentic workflows — A Github repo with a demo using Gemini and Selenium to automate browser actions. Industrial application might easily follow out of this.

Data 130

Data SQL Hadoop Deep Learning 130

Snowflake

NOVEMBER 4, 2024

Register now and join thousands of fellow developers, data scientists and engineers to learn about the future of AI agents, how to effectively scale pandas, how to create retrieval-augmented generation (RAG) chatbots and much, much more. From Snowflake Native Apps to machine learning, there’s sure to be something fresh for everyone.

Unstructured Data 86

Unstructured Data Python Data Pipeline Machine Learning 86

Data Engineering Podcast

DECEMBER 28, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 130

Data Lake Data Warehouse Data Pipeline MongoDB 130

Snowflake

MARCH 2, 2023

Snowflake enables organizations to be data-driven by offering an expansive set of features for creating performant, scalable, and reliable data pipelines that feed dashboards, machine learning models, and applications. But before data can be transformed and served or shared, it must be ingested from source systems.

Kafka 125

Kafka Data Ingestion Data Pipeline Cloud Storage 125

Data Engineering Podcast

DECEMBER 25, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Machine Learning 130

Machine Learning Systems Data Lake Data Warehouse 130

Snowflake

JUNE 10, 2024

” —David Webb, Data Architect at Travelpass Build modern data pipelines with Snowflake Python APIs Snowflake’s latest suite of Python APIs (GA soon) simplifies the data pipeline development process with Python.

Python 106

Python Data Pipeline SQL Database 106

Christophe Blefari

JULY 13, 2024

kyutai released Moshi — Moshi is a "voice-enabled AI" The team as kyutai developed the model with an audio interface-first with an audio language model, which make the conversation with the AI more real (demo at 5:00 min) as it can interrupt you or kinda "think" (meaning for predict the next audio segment) while it speaks.

Kafka 130

Kafka AWS Data Database 130

Ascend.io

APRIL 22, 2024

Plus, we’ll offer a teaser of what’s to come: an exclusive live demo that you simply cannot miss. The Evolution of Data Engineering Over the past decade, we’ve seen tremendous change in the data engineering landscape. It’s time we stop Data Plumbing, and get back to Data Engineering. Let’s dive in!

Project 52

Project Metadata Data Pipeline Data Engineer 52

Confluent

NOVEMBER 20, 2020

Building data pipelines isn’t always straightforward. The gap between the shiny “hello world” examples of demos and the gritty reality of messy data and imperfect formats is sometimes all too […].

Cloud 139

Cloud Data Pipeline Data Building 139

Monte Carlo

JULY 9, 2024

Our platforms provide automated monitoring and alerting, as well as field-level lineage, to help data teams be the first to know when data pipelines break and speed up incident resolution. And it’s absolutely vital for MotherDuck users to know when data quality issues occur. If you’re new to data observability, welcome!

Data Warehouse 69

Data Warehouse Data Pipeline SQL Database 69

Cloudera

MARCH 14, 2023

Data leaders will be able to simplify and accelerate the development and deployment of data pipelines, saving time and money by enabling true self service. It is no secret that data leaders are under immense pressure. For more information or to see a demo, go to the DataFlow Product page.

Designing 79

Designing Data Pipeline Data Governance Government 79

Data Engineering Podcast

DECEMBER 18, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 100

Data Lake Data Warehouse Data Pipeline MongoDB 100

Grouparoo

MAY 11, 2021

We'll make the assumption that Dylan already has a server available, so I'm just going to demo this on my local machine for simplicity. The instructions and concepts easily transfer to wherever you want to run Grouparoo. Integration Steps First, we've got to do some setup.

Data Pipeline 52

Data Pipeline Data Project Utilities 52

Data Engineering Podcast

DECEMBER 4, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 100

Data Lake MongoDB MySQL Data Warehouse 100

Christophe Blefari

SEPTEMBER 13, 2024

NEO Beta, a humanoid company backed by OpenAI, released a first video demo. What are the key part of data engineering — Simple way to present what are the key part of data engineering. And it's impressive (🙃), the robot is able to handover a bag to a human! We hope next OpenAI model is not o7. /s

Pipeline-centric 100

Pipeline-centric Data Python Data Science 100

Data Engineering Podcast

DECEMBER 11, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Database 100

Database MySQL Data Lake MongoDB 100

DataKitchen

FEBRUARY 23, 2024

Webinar: Beyond Data Observability: Personalization DataKitchen DataOps Observability Problem Statement White Paper: ‘Taming Chaos’ Technical Product Overview Four-minute online demo Detailed Product: Documentation Webinar: Data Observability Demo Day DataKitchen DataOps TestGen Problem Statement White Paper: ‘Mystery Box Full Of Data Errors’ (..)

Data Validation 47

Data Validation Data Data Pipeline Data Solutions 47

Data Engineering Podcast

NOVEMBER 27, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 100

Data Lake Data Warehouse MongoDB Data Pipeline 100

Data Engineering Podcast

OCTOBER 30, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Engineering 100

Engineering MongoDB Scala MySQL 100

ThoughtSpot

JUNE 23, 2023

AI-assisted data modeling on shared data workloads Data sharing is critical to inform decision making across the organization. Snowflake eliminates the data sharing complexities of traditional data pipelines —making data secure, governed, and easily ready to query.

Cloud 104

Cloud Programming Language Government Data Security 104

Christophe Blefari

OCTOBER 14, 2023

Then Marc Lamberti gave a huge update about Airflow but done differently — It wasn't about slides with a list of new features but rather about how you can write, in 2023, a data pipeline with Airflow. He also demo a event-based DAG parsing that instantaneously display DAGs in the UI.

Python 130

Python Datasets Data Banking 130

Data Engineering Podcast

JULY 17, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Management 130

Data Management Management Metadata MongoDB 130

Monte Carlo

JUNE 6, 2023

But is there more to generative AI than a fancy demo on Twitter? And how will it impact data? How generative AI will disrupt data With the advent of OpenAI’s DALL-E and ChatGPT, large language models became much more useful to the vast majority of humans. Request a demo today. Let’s assess. Will you join us?

Data Engineer 84

Data Engineer Data Engineering Engineering Business Intelligence 84

Data Engineering Podcast

NOVEMBER 20, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 130

Data Lake Data Ingestion MongoDB Google Cloud 130

Data Engineering Podcast

NOVEMBER 6, 2022

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Sign up free at dataengineeringpodcast.com/rudder Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

MongoDB 130

MongoDB Scala MySQL Machine Learning 130

Ascend.io

JUNE 10, 2024

With our new integrated AI assistant, data engineering teams can capitalize on the power of a context aware AI agents to help build, scale, and optimize their data pipelines. Check out how we built an AI-integrated data pipeline using Snowflake’s Arctic LLM for sentiment analysis. #3

Data Engineer 52

Data Engineer Data Engineering Engineering High Quality Data 52

Data Engineering Podcast

MAY 1, 2022

If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. Struggling with broken pipelines?

Data Warehouse 100

Data Warehouse Data Integration Cloud Google Cloud 100

Data Engineering Weekly

OCTOBER 6, 2024

The learning goes back to the fundamentals of pipeline design principles. Regularly review if pipelines are still required. Minimize the data used in pipelines, aka do incremental data pipeline design. Optimize pipeline schedules. Filter data effectively to make sure the query uses partition pruning.

Data Engineer 62

Data Engineer Data Engineering Engineering Data Warehouse 62

Data Engineering Podcast

JULY 10, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Ascend.io

OCTOBER 28, 2024

Tools like Python’s requests library or ETL/ELT tools can facilitate data enrichment by automating the retrieval and merging of external data. Read More: Discover how to build a data pipeline in 6 steps Data Integration Data integration involves combining data from different sources into a single, unified view.

Raw Data 52

Raw Data Datasets Aggregated Data Data Pipeline 52

Databand.ai

AUGUST 30, 2023

Besides these categories, specialized solutions tailored specifically for particular domains or use cases also exist, such as extract, transform and load (ETL) tools for managing data pipelines, data integration tools for combining information from disparate sources or systems and more.

Data Cleanse 52

Data Cleanse Data Pipeline Datasets Data Validation 52

Data Engineering Podcast

JUNE 19, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Snowflake

APRIL 8, 2024

Many entries also used Snowpark , taking advantage of the ability to work in the code they prefer to develop data pipelines, ML models and apps, then execute in Snowflake. Our sincere thanks go out to everyone who participated in this year’s competition.

Pipeline-centric 128

Pipeline-centric Food Healthcare Unstructured Data 128