Datakin

AUGUST 10, 2021

Blog An instant demo of data lineage is worth a thousand words Written by Ross Turk on August 10, 2021 They say that a picture is worth a thousand words. If you’ve ever tried to describe how all the jobs in your data pipeline are interrelated using just words, I am sure it wasn’t easy.

Data Pipeline 52

Data Pipeline Metadata Data IT 52

Confluent

JUNE 22, 2022

How to use data pipelines, unlock the benefits of real-time data flow, and achieve seamless data streaming and analytics at scale with Confluent.

Data Pipeline 52

Data Pipeline Kafka Data 52

Silectis

JANUARY 22, 2021

For those using a robust analytics database, such as the Snowflake® Data Cloud , adding the power of a data engineering platform can help maximize the value you’re getting out of that database. Magpie Fills in the Gaps for Better Data Engineering And we’re not talking about just ETL (extract, transform, load). Magpie can help.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

Monte Carlo

NOVEMBER 21, 2024

Whether it’s customer transactions, IoT sensor readings, or just an endless stream of social media hot takes, you need a reliable way to get that data from point A to point B while doing something clever with it along the way. That’s where data pipeline design patterns come in. Data Mesh Pattern 8.

Data Pipeline 52

Data Pipeline Designing Lambda Architecture Kafka 52

Data Engineering Podcast

DECEMBER 28, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 130

Data Lake Data Warehouse Data Pipeline MongoDB 130

Data Engineering Podcast

DECEMBER 25, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Machine Learning 130

Machine Learning Systems Data Lake Data Warehouse 130

Striim

OCTOBER 11, 2024

A well-executed data pipeline can make or break your company’s ability to leverage real-time insights and stay competitive. Thriving in today’s world requires building modern data pipelines that make moving data and extracting valuable insights quick and simple. What is a Data Pipeline?

Data Pipeline 52

Data Pipeline MongoDB Unstructured Data Data Lake 52

Databand.ai

SEPTEMBER 9, 2022

How to analyze and resolve data pipeline incidents in Databand Niv Sluzki 2022-09-09 13:00:12 A data pipeline failure can cripple your downstream data flows. Whether it failed to start or quit unexpectedly, you need to know immediately if there is a pipeline incident.

Data Pipeline 52

Data Pipeline Datasets AWS Data 52

Snowflake

JUNE 10, 2024

” —David Webb, Data Architect at Travelpass Build modern data pipelines with Snowflake Python APIs Snowflake’s latest suite of Python APIs (GA soon) simplifies the data pipeline development process with Python.

Python 116

Python Data Pipeline SQL Database 116

Databand.ai

SEPTEMBER 19, 2022

How to create data pipeline and data quality SLA alerts in Databand Helen Soloveichik 2022-09-20 01:49:30 Data engineers often get inundated by alerts from data issues. Databand helps fix this problem by breaking through noisy alerts with focused alerting and routing when a data pipeline and quality issues occur.

Data Pipeline 52

Data Pipeline Datasets Data Data Engineering 52

Cloudera

OCTOBER 15, 2020

Provide a collection Name under Destination (in this example, we named it ‘solr-nifi-demo’). nifi-solr-demo. nifi-solr-demo. solr-nifi-demo-collection. On the Hue webUI select Indexes > + ‘Create index’ > from the Type drop down select ‘Manually’> Click Next. text_general. initial_release_date. Click Submit.

AWS 118

AWS Data Cloud Accessible 118

Christophe Blefari

JULY 13, 2024

kyutai released Moshi — Moshi is a "voice-enabled AI" The team as kyutai developed the model with an audio interface-first with an audio language model, which make the conversation with the AI more real (demo at 5:00 min) as it can interrupt you or kinda "think" (meaning for predict the next audio segment) while it speaks.

Kafka 130

Kafka AWS Data Database 130

Data Engineering Podcast

JULY 17, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Management 130

Data Management Management Metadata MongoDB 130

Data Engineering Podcast

DECEMBER 18, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 100

Data Lake Data Warehouse Data Pipeline MongoDB 100

Snowflake

NOVEMBER 4, 2024

Register now and join thousands of fellow developers, data scientists and engineers to learn about the future of AI agents, how to effectively scale pandas, how to create retrieval-augmented generation (RAG) chatbots and much, much more. From Snowflake Native Apps to machine learning, there’s sure to be something fresh for everyone.

Unstructured Data 99

Unstructured Data Python Data Pipeline Machine Learning 99

Data Engineering Podcast

JUNE 19, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Data Engineering Podcast

NOVEMBER 6, 2022

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Sign up free at dataengineeringpodcast.com/rudder Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

MongoDB 130

MongoDB MySQL Scala Machine Learning 130

Data Engineering Podcast

NOVEMBER 20, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 130

Data Lake Data Ingestion MongoDB Google Cloud 130

Snowflake

APRIL 8, 2024

Many entries also used Snowpark , taking advantage of the ability to work in the code they prefer to develop data pipelines, ML models and apps, then execute in Snowflake. Our sincere thanks go out to everyone who participated in this year’s competition.

Pipeline-centric 131

Pipeline-centric Food Healthcare Unstructured Data 131

Cloudera

MARCH 14, 2023

Data leaders will be able to simplify and accelerate the development and deployment of data pipelines, saving time and money by enabling true self service. It is no secret that data leaders are under immense pressure. For more information or to see a demo, go to the DataFlow Product page.

Designing 80

Designing Data Pipeline Data Governance Government 80

Data Engineering Podcast

DECEMBER 4, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 100

Data Lake MongoDB MySQL Data Warehouse 100

ThoughtSpot

JUNE 23, 2023

AI-assisted data modeling on shared data workloads Data sharing is critical to inform decision making across the organization. Snowflake eliminates the data sharing complexities of traditional data pipelines —making data secure, governed, and easily ready to query.

Cloud 104

Cloud Programming Language Government Data Security 104

Data Engineering Podcast

FEBRUARY 24, 2020

Are you spending too much time maintaining your data pipeline? Snowplow empowers your business with a real-time event data pipeline running in your own cloud account without the hassle of maintenance. Set up a demo and mention you’re a listener for a special offer!

IT 100

IT Data Lake Data Pipeline Media 100

Data Engineering Podcast

DECEMBER 11, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Database 100

Database MySQL Data Lake MongoDB 100

Data Engineering Podcast

NOVEMBER 27, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 100

Data Lake Data Warehouse MongoDB Data Pipeline 100

Data Engineering Podcast

DECEMBER 21, 2020

Fortunately, there’s hope: in the same way that New Relic, DataDog, and other Application Performance Management solutions ensure reliable software and keep application downtime at bay, Monte Carlo solves the costly problem of broken data pipelines. The first 25 will receive a free, limited edition Monte Carlo hat!

Data Governance 130

Data Governance Government Data Lake Banking 130

Data Engineering Podcast

JUNE 26, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130

Data Engineering Podcast

OCTOBER 30, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Engineering 100

Engineering MongoDB MySQL Scala 100

Christophe Blefari

OCTOBER 14, 2023

Then Marc Lamberti gave a huge update about Airflow but done differently — It wasn't about slides with a list of new features but rather about how you can write, in 2023, a data pipeline with Airflow. He also demo a event-based DAG parsing that instantaneously display DAGs in the UI.

Python 130

Python Datasets Data Banking 130

Data Engineering Podcast

MARCH 2, 2020

Are you spending too much time maintaining your data pipeline? Snowplow empowers your business with a real-time event data pipeline running in your own cloud account without the hassle of maintenance. Set up a demo and mention you’re a listener for a special offer!

Kafka 100

Kafka Process PostgreSQL MySQL 100

Data Engineering Podcast

MARCH 16, 2020

Are you spending too much time maintaining your data pipeline? Snowplow empowers your business with a real-time event data pipeline running in your own cloud account without the hassle of maintenance. Set up a demo and mention you’re a listener for a special offer!

Building 100

Building Data Warehouse NoSQL Data Lake 100

Data Engineering Podcast

MAY 1, 2022

If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. Struggling with broken pipelines?

Data Warehouse 100

Data Warehouse Data Integration Cloud Google Cloud 100

DataKitchen

DECEMBER 30, 2020

CRN’s The 10 Hottest Data Science & Machine Learning Startups of 2020 (So Far). In June of 2020, CRN featured DataKitchen’s DataOps Platform for its ability to manage the data pipeline end-to-end combining concepts from Agile development, DevOps, and statistical process control: DataKitchen.

Manufacturing 96

Manufacturing Big Data Data Pipeline Data Science 96

The Modern Data Company

JANUARY 31, 2023

This post highlights exactly how our founders taught us to think differently about data and why it matters. Here are the cornerstones of this new paradigm: Data ownership is a construct Data pipelines should be accessible to everyone Data products should adapt to the organization, not vice versa.

Retail 98

Retail Healthcare Data Architecture Data Pipeline 98

Snowflake

NOVEMBER 1, 2023

ML-based models: Forecasting (generally available soon): Train on historical time series data and forecast that time series into the future with automated handling of seasonality, scaling and more. Anomaly Detection (generally available soon): Identify outliers in your time series data for data pipeline monitoring and more.

Unstructured Data 120

Unstructured Data SQL Python Accessible 120

Data Engineering Podcast

MAY 18, 2021

Summary Data lineage is the common thread that ties together all of your data pipelines, workflows, and systems. In order to get a holistic understanding of your data quality, where errors are occurring, or how a report was constructed you need to track the lineage of the data from beginning to end.

Metadata 100

Metadata Kafka Data Warehouse Hadoop 100

Data Engineering Podcast

APRIL 24, 2022

If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. Struggling with broken pipelines?

Data Warehouse 100

Data Warehouse Data Lake Data Pipeline BI 100