Data Engineering Podcast

FEBRUARY 24, 2020

Are you spending too much time maintaining your data pipeline? Snowplow empowers your business with a real-time event data pipeline running in your own cloud account without the hassle of maintenance. Set up a demo and mention you’re a listener for a special offer!

IT 100

IT Data Lake Data Pipeline Media 100

Data Engineering Podcast

DECEMBER 18, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Lake 100

Data Lake Data Warehouse Data Pipeline MongoDB 100

The Modern Data Company

FEBRUARY 15, 2023

Limited resources: Data management has always been resource intensive, but not all organizations can maintain a full data team. Without suitable resources for company-wide data management, it’s easier to fall behind. Find out how DataOS can help you chip away at your data debt.

Government 98

Government Data Governance Raw Data Data 98

The Modern Data Company

JANUARY 22, 2024

This mission culminates in the esteemed recognition of honorable mention in Gartner’s 2023 Magic Quadrant for Data Integration, showcasing commitment to excellence and industry leadership in the data-driven era. Data engineering excellence Modern offers robust solutions for building, managing, and operationalizing data pipelines.

Data Integration 98

Data Integration Metadata Government Unstructured Data 98

Data Engineering Podcast

JUNE 5, 2022

Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Security 100

Data Security Metadata MongoDB MySQL 100

Data Engineering Podcast

JULY 10, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Metadata 100

Metadata MongoDB MySQL Machine Learning 100

The Modern Data Company

JULY 31, 2023

By omitting the complexity and debt of data pipelines resulting from multiple fragmented tools in the ‘MAD Ecosystem,’ these loops can be efficiently maintained Process Angle : Administering Data Products involves a combination of technology, culture, and process. Read more about our partnership.

Government 98

Government Data Governance Architecture Data Pipeline 98

The Modern Data Company

JANUARY 22, 2024

This mission culminates in the esteemed recognition of honorable mention in Gartner’s 2023 Magic Quadrant for Data Integration, showcasing commitment to excellence and industry leadership in the data-driven era. Data engineering excellence Modern offers robust solutions for building, managing, and operationalizing data pipelines.

Data Integration 87

Data Integration Metadata Government Unstructured Data 87

Precisely

JULY 15, 2024

The move to productize data also requires a way to package data products so they are easily and uniformly discoverable and consumed. Data products must be properly designed and organized to be reused across the organization. Then you need to know more about data mesh architecture. Does that align with your business goals?

Data Governance 71

Data Governance Data Integration Government Data 71

Monte Carlo

JUNE 6, 2023

But is there more to generative AI than a fancy demo on Twitter? And how will it impact data? How generative AI will disrupt data With the advent of OpenAI’s DALL-E and ChatGPT, large language models became much more useful to the vast majority of humans. Request a demo today. Let’s assess. Will you join us?

Data Engineering 84

Data Engineering Data Engineer Engineering Business Intelligence 84

Monte Carlo

MAY 2, 2023

Respondents averaged 642 tables across their data lake, lakehouse, or warehouse environments. Respondents averaged 290 manually-written tests across their data pipelines. Check out the full report , including commentary and reactions from nearly a dozen industry-leading data executives.

Data Lake 80

Data Lake Data Software Engineer Software Engineering 80

Monte Carlo

JUNE 27, 2024

Of course, there were plenty of flashy generative demos (like Shutterstock AI)—not to mention a couple of live snafus—but these were merely a light palette cleanser between keynotes as these events hurdled toward the real star of the shows—data enablement. Look, we’ve all been told there’s data in the LLM hills.

Data Governance 59

Data Governance Government Data Pipeline MySQL 59

Monte Carlo

SEPTEMBER 27, 2023

We’re also excited to announce Monte Carlo’s participation in the Alation Open Data Quality Initiative , a program designed to give Alation customers the freedom to choose the data quality or observability solution of their choice to expand data trust in the Alation platform. See you there!

Data Governance 69

Data Governance Government Data Pipeline Data 69

The Modern Data Company

JUNE 26, 2023

Strong data governance is essential Risks and challenges associated with using customer data–such as concerns around privacy and security–can derail insights. ABAC governance enables flexible and scalable policies that adapt to changing or new compliance regulations.

Retail 52

Retail Government Data Governance Unstructured Data 52

The Modern Data Company

JUNE 26, 2023

Strong data governance is essential Risks and challenges associated with using customer data–such as concerns around privacy and security–can derail insights. ABAC governance enables flexible and scalable policies that adapt to changing or new compliance regulations.

Retail 52

Retail Government Data Governance Unstructured Data 52

The Modern Data Company

JUNE 26, 2023

Strong data governance is essential Risks and challenges associated with using customer data–such as concerns around privacy and security–can derail insights. ABAC governance enables flexible and scalable policies that adapt to changing or new compliance regulations.

Retail 52

Retail Government Data Governance Unstructured Data 52

Monte Carlo

APRIL 29, 2021

Simultaneously, Monte Carlo provides CDOs and other data stakeholders with a holistic view of their company’s data health and reliability across critical business use cases. We are delighted to partner with Monte Carlo for their data observability capabilities,” said Tarik Dwiek, Director of Technology Alliances at Snowflake.

Insurance 52

Insurance Business Intelligence Data Pipeline Cloud 52

Silectis

DECEMBER 15, 2020

Integrated Security & Governance Features Ensuring that your data is protected and secure is one of our top priorities. We’ve designed Magpie to keep you from putting out fires like broken data pipelines with the built-in features mentioned above. The time to identify more insights out of huge sets of data.

Data Engineering 52

Data Engineering Data Engineer Engineering ETL Tools 52

Data Engineering Weekly

OCTOBER 6, 2024

The learning goes back to the fundamentals of pipeline design principles. Regularly review if pipelines are still required. Minimize the data used in pipelines, aka do incremental data pipeline design. Optimize pipeline schedules. Filter data effectively to make sure the query uses partition pruning.

Data Engineering 62

Data Engineering Data Engineer Engineering Data Warehouse 62

Monte Carlo

MARCH 18, 2025

Long-term usability Data should be structured in a way that makes sense not just for todays teams but for future employees, too. Your data pipelines need regular monitoring, too. Lets look into building responsible and long-lasting data practices. Just clean, accurate, and sustainable data that you can trust.

Data Pipeline 52

Data Pipeline Data Data Governance Government 52

Databand.ai

AUGUST 30, 2023

Besides these categories, specialized solutions tailored specifically for particular domains or use cases also exist, such as extract, transform and load (ETL) tools for managing data pipelines, data integration tools for combining information from disparate sources or systems and more.

Data Cleanse 52

Data Cleanse Data Pipeline Datasets Data Validation 52

Monte Carlo

JULY 15, 2024

Key Responsibilities of Data Owners The primary responsibility of a data owner is ensuring the quality, accuracy, and integrity of their assigned data assets. Career advancement: As organizations become more data-centric, your role as a data owner offers opportunities for career growth.

Pipeline-centric 52

Pipeline-centric Government Data Governance Retail 52

Data Engineering Weekly

SEPTEMBER 8, 2024

Take Astro (the fully managed Airflow solution) for a test drive today and unlock a suite of features designed to simplify, optimize, and scale your data pipelines. Join Monte Carlo for a deep dive session into how data governance leaders can take their data quality strategies to the next level with end-to-end data observability.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Monte Carlo

JULY 22, 2024

Common Challenges in Data Versioning While data versioning offers many benefits, it also comes with its own set of challenges: Versioning Large Datasets : Managing multiple versions of large datasets can be challenging due to additional storage and performance requirements. Don’t just version your data – observe it.

Metadata 52

Metadata Datasets ETL Tools Data 52

The Modern Data Company

JULY 31, 2023

By omitting the complexity and debt of data pipelines resulting from multiple fragmented tools in the ‘MAD Ecosystem,’ these loops can be efficiently maintained. Process Angle : Administering Data Products involves a combination of technology, culture, and process.

Government 52

Government Data Governance Architecture Data Pipeline 52

The Modern Data Company

JULY 31, 2023

By omitting the complexity and debt of data pipelines resulting from multiple fragmented tools in the ‘MAD Ecosystem,’ these loops can be efficiently maintained. Process Angle : Administering Data Products involves a combination of technology, culture, and process.

Government 52

Government Data Governance Architecture Data Pipeline 52

Ascend.io

APRIL 7, 2023

With the monolithic architectures most organizations have today, business users are stuck, constantly waiting for new data pipelines to be built or amended based on their requests. Data engineers aren’t huge fans of this paradigm either. But they’re living in what is essentially the opposite of their fantasy.

Metadata 52

Metadata Datasets Data Governance Government 52

The Modern Data Company

JANUARY 17, 2023

In addition, a CDP owned and controlled by the marketing department may not get the full attention of already overwhelmed IT engineers, becoming an afterthought regarding data governance. As a result, CDPs may be less secure than other data stores even though they hold a lot of customer PII.

Metadata 59

Metadata Government Data Lake Data 59

Monte Carlo

JUNE 16, 2023

You may even want to set data SLAs for your most important tables with specific data freshness and overall uptime targets. Data governance frameworks can be implemented to define roles, responsibilities, and processes for managing data quality across the organization. Request a demo of Monte Carlo now.

Data Integration 52

Data Integration Hospitality Banking Healthcare 52

Ascend.io

JULY 17, 2023

Our intelligent data pipelines detect and respond to changes, ensure data quality, and efficiently use cloud resources—so you can spend less time on implementing your data mesh architecture and more time reaping the benefits. Ready to take a look?

Architecture 52

Architecture Datasets Telecommunication Data Pipeline 52

AltexSoft

MARCH 30, 2023

Source: The Data Team’s Guide to the Databricks Lakehouse Platform Integrating with Apache Spark and other analytics engines, Delta Lake supports both batch and stream data processing. Besides that, it’s fully compatible with various data ingestion and ETL tools.

Scala 64

Scala Data Lake Machine Learning BI 64

Monte Carlo

JANUARY 23, 2023

Data lineage , an automated visualization of the relationships for how data flows across tables and other data assets, is a must-have in the data engineering toolbox. Not only is it helpful for data governance and compliance use cases, it also plays a starring role as one of the 5 pillars of data observability.

IT 52

IT BI Government Data Governance 52

AltexSoft

OCTOBER 8, 2021

To make a more informed decision, enquire about demos and do your own in-depth research. Oracle Data Integrator, IBM InfoSphere, Snaplogic, Xplenty, and. The platform’s main capabilities comprise data integration, data quality assurance, and data governance. Here, we’ll be comparing such vendors as.

Data Integration 52

Data Integration Hadoop Data Warehouse Data Lake 52

Monte Carlo

APRIL 20, 2023

Data lineage helps ensure you have clear lines of accountability that data team structure–centralized, center of excellence , data mesh– you use or however your data pipelines evolve. Data Governance and Compliance The only things propagating faster than data are data regulations.

Data Warehouse 52

Data Warehouse BI Data Government 52

Monte Carlo

FEBRUARY 6, 2023

Data Warehouse Security Best Practices Data governance is critical to the protection of both internal and customer data. Before you begin any data warehouse migration, take the time to thoroughly review your data security protocols. Send us a note or check out our demo below!

Data Warehouse 52

Data Warehouse AWS Data Data Validation 52

Christophe Blefari

APRIL 12, 2024

In the same conference a lot of other talks took place here a few selection you should check out: How to pivot your data team from a service team to a value-generator — Very often data teams struggle in delivering value or in finding what's their real identity.

BI 130

BI SQL Data Coding 130

Monte Carlo

JUNE 20, 2024

The quality of the input affects the quality of the output – and in order for data teams to produce high-quality data products, they need high-quality data from the very start. But, in reality, a data pipeline is more nuanced than just the quality of your internal or external data sources.

High Quality Data 52

High Quality Data Machine Learning Algorithm Datasets 52