databricks

SEPTEMBER 30, 2024

This blog explores how new technologies such as Databricks Data Intelligence Platform can pave the way for more effective and efficient multi-omics data management.

Data Management 118

Data Management Management Data Technology 118

Precisely

OCTOBER 23, 2023

When most people think of master data management, they first think of customers and products. But master data encompasses so much more than data about customers and products. Challenges of Master Data Management A decade ago, master data management (MDM) was a much simpler proposition than it is today.

Data Management 110

Data Management Management Data Data Integration 110

KDnuggets

NOVEMBER 13, 2023

This week, we delve into the vital world of Databases, SQL, Data Management, and Statistical Concepts in Data Science. Welcome back to Week 2 of KDnuggets’ "Back to Basics" series.

Database 135

Database SQL Data Management Management 135

Engineering at Meta

MAY 22, 2024

In recent years, Meta’s data management systems have evolved into a composable architecture that creates interoperability, promotes reusability, and improves engineering efficiency. Data is at the core of every product and service at Meta. Data is at the core of every product and service at Meta.

Data Management 91

Data Management Management Data SQL 91

KDnuggets

DECEMBER 12, 2023

This article provides an overview of two new data preparation techniques that enable data democratization while minimizing transformation burdens.

Data Management 149

Data Management Management Data Preparation Data 149

KDnuggets

AUGUST 31, 2023

These are my predictions based on my personal experiences, recent research and reports from leading platforms.

Data Management 137

Data Management Management Data Data Engineering 137

Engineering at Meta

FEBRUARY 20, 2024

This new convergence helps Meta and the larger community build data management systems that are unified, more efficient, and composable. Meta’s Data Infrastructure teams have been rethinking how data management systems are designed.

Data Management 100

Data Management Bytes Management Datasets 100

ArcGIS

FEBRUARY 3, 2025

Streamline data management, reduce duplication, enhance data quality, and provide a single source of truth for address and road data.

Data Management 76

Data Management Management Data Transportation 76

databricks

FEBRUARY 22, 2024

For many organizations, log data that security professionals need for effective. In today's environment, proactive cybersecurity is crucial to any public sector agency.

Data Management 122

Data Management Management Data 122

ArcGIS

APRIL 21, 2025

Learn how to save geoprocessing tool outputs to the memory workspace, and about some updates in ArcGIS Pro 3.5!

Data Management 98

Data Management Management Data 98

databricks

JUNE 28, 2024

Introduction Financial institutions face a demanding environment with complex regulatory examinations and a pressing need for flexible and comprehensive risk management solutions.

Data Management 98

Data Management Management Data 98

Precisely

OCTOBER 31, 2024

Key Takeaways: Data mesh is a decentralized approach to data management, designed to shift creation and ownership of data products to domain-specific teams. Data fabric is a unified approach to data management, creating a consistent way to manage, access, and share data across distributed environments.

Data Architecture 69

Data Architecture Architecture Metadata Government 69

Data Engineering Podcast

APRIL 23, 2023

In this episode DeVaris Brown discusses the types of applications that are possible when teams don't have to manage the complex infrastructure necessary to support continuous data flows. Can you describe what Meroxa is and the story behind it? How have the focus and goals of the platform and company evolved over the past 2 years?

Data Lake 277

Data Lake Kafka Machine Learning Data Warehouse 277

Precisely

JANUARY 9, 2025

The goal of this post is to understand how data integrity best practices have been embraced time and time again, no matter the technology underpinning. In the beginning, there was a data warehouse The data warehouse (DW) was an approach to data architecture and structured data management that really hit its stride in the early 1990s.

Data Integration 119

Data Integration Hadoop Data Warehouse Data Lake 119

Data Engineering Podcast

FEBRUARY 18, 2024

In this episode Dain Sundstrom, CTO of Starburst, explains how the combination of the Trino query engine and the Iceberg table format offer the ease of use and execution speed of data warehouses with the infinite storage and scalability of data lakes. What do you have planned for the future of Trino/Starburst?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Data Engineering Podcast

JUNE 25, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Can you describe what SQLMesh is and the story behind it? DataOps is a term that has been co-opted and overloaded.

Data Engineering 225

Data Engineering Data Engineer Python Engineering 225

Data Engineering Podcast

DECEMBER 24, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Precisely

JANUARY 30, 2025

Integrate data governance and data quality practices to create a seamless user experience and build trust in your data. When planning your data governance approach, start small, iterate purposefully, and foster data literacy to drive meaningful business outcomes.

Data Governance 103

Data Governance Government Metadata Data 103

Precisely

NOVEMBER 18, 2024

However, they require a strong data foundation to be effective. With the rise of cloud-based data management, many organizations face the challenge of accessing both on-premises and cloud-based data. Without a unified, clean data structure, leveraging these diverse data sources is often problematic.

Data Analytics 72

Data Analytics Data Governance Data Integration Government 72

Data Engineering Podcast

SEPTEMBER 10, 2023

In this episode Nick Schrock, creator of Dagster, shares his perspective on the state of data orchestration technology and its application to help inform its implementation in your environment. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles.

BI 208

BI SQL Machine Learning Data 208

Data Engineering Podcast

JUNE 2, 2024

In this episode she shares the practical steps to implementing a data governance practice in your organization, and the pitfalls to avoid. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Are there any trends that concern you?

Data Governance 147

Data Governance Government Data Lake High Quality Data 147

Data Engineering Podcast

JUNE 23, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Data lakes in various forms have been gaining significant popularity as a unified interface to an organization's analytics. Closing Announcements Thank you for listening!

Data Lake 162

Data Lake High Quality Data Hadoop Machine Learning 162

Data Engineering Podcast

APRIL 21, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Can you describe what Shortwave is and the story behind it? What is the core problem that you are addressing with Shortwave?

Data Lake 182

Data Lake High Quality Data Machine Learning Data Pipeline 182

Edureka

APRIL 22, 2025

It lets you describe data more complexly and make predictions. AI-powered data engineering solutions make it easier to streamline the data management process, which helps businesses find useful insights with little to no manual work. This will help make better analytics predictions and improve data management.

Data Engineering 52

Data Engineering Data Engineer Engineering Consulting 52

Data Engineering Weekly

JANUARY 8, 2025

Together, we discussed how Hudi drives innovation, the state of open standards, and what lies ahead for data lakehouses in 2025 and beyond. This foundational concept addresses a key challenge for enterprises: building scalable, high-performing data platforms that can support the complexity of modern data ecosystems.

Data Lake 64

Data Lake Datasets Retail Data Ingestion 64

Data Engineering Podcast

JUNE 16, 2024

In this episode Kevin Liu shares some of the interesting features that they have built by combining those technologies, as well as the challenges that they face in supporting the myriad workloads that are thrown at this layer of their data platform. Can you describe what role Trino and Iceberg play in Stripe's data architecture?

Data Lake 147

Data Lake High Quality Data Metadata Machine Learning 147

Data Engineering Podcast

OCTOBER 29, 2023

In this episode Tanya Bragin shares her experiences as a product manager for two major vendors and the lessons that she has learned about how teams should approach the process of tool selection. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles.

Database 189

Database SQL BI Machine Learning 189

Data Engineering Podcast

SEPTEMBER 17, 2023

In this episode Brian Platz explains how JSON-LD can be used as a shared representation of linked data for building semantic data products. Can you describe what the term "linked data product" means and some examples of when you might build one? Closing Announcements Thank you for listening!

Building 189

Building SQL BI Python 189

Data Engineering Podcast

MAY 28, 2023

Ghalib Suleiman has been on both sides of this equation and joins the show to share his hard-won wisdom about how to start and grow a data team in the early days of company growth. Can you start by sharing your conception of the responsibilities of a data team? When is it more practical to outsource the data work?

Data Lake 162

Data Lake Machine Learning Data Warehouse Education 162

Data Engineering Podcast

FEBRUARY 4, 2024

In this episode Yingjun Wu explains how it is architected to power analytical workflows on continuous data flows, and the challenges of making it responsive and scalable. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex.

SQL 173

SQL Data Lake High Quality Data Machine Learning 173

Data Engineering Podcast

JUNE 30, 2024

He highlights the role of data teams in modern organizations and how Synq is empowering them to achieve this. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Can you describe what Synq is and the story behind it?

Pipeline-centric 147

Pipeline-centric Engineering Data Lake High Quality Data 147

Data Engineering Podcast

MAY 21, 2023

In this episode David Yaffe and Johnny Graettinger share the story behind the business and technology and how you can start using it today to build a real-time data lake without all of the headache. Can you describe what Estuary is and the story behind it? Stream processing technologies have been around for around a decade.

Data Lake 162

Data Lake Machine Learning Kafka Data Warehouse 162

databricks

JUNE 3, 2024

We are excited to announce that we have agreed to acquire Tabular, Inc, a data management company founded by Ryan Blue, Daniel Weeks.

Data Management 145

Data Management Management Data 145

KDnuggets

MAY 19, 2022

Read the best books on Programming, Statistics, Data Engineering, Web Scraping, Data Analytics, Business Intelligence, Data Applications, Data Management, Big Data, and Cloud Architecture.

Data Science 160

Data Science Business Intelligence Big Data Architecture 160

Data Engineering Podcast

JUNE 9, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat. Data observability has been gaining adoption for a number of years now, with a large focus on data warehouses.

Process 147

Process Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

OCTOBER 15, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Process 182

Process Building SQL BI 182

Data Engineering Podcast

JANUARY 7, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. What are the open questions today in technical scalability of data engines? What are the open questions today in technical scalability of data engines?

Data Process 147

Data Process Process Data Lake High Quality Data 147