Precisely

JANUARY 9, 2025

Disclaimer: Throughout this post, I discuss a variety of complex technologies but avoid trying to explain how these technologies work. The goal of this post is to understand how data integrity best practices have been embraced time and time again, no matter the technology underpinning. Then came Big Data and Hadoop!

Data Integration 119

Data Integration Hadoop Data Lake Data Warehouse 119

databricks

SEPTEMBER 30, 2024

This blog explores how new technologies such as Databricks Data Intelligence Platform can pave the way for more effective and efficient multi-omics data management.

Data Management 114

Data Management Management Data Technology 114

ProjectPro

JUNE 6, 2025

Explore the advanced features of this powerful cloud-based solution and take your data management to the next level with this comprehensive guide. To gain a competitive edge in today's fast-growing big data industry, it's crucial to have hands-on experience with this cutting-edge technology.

Database 40

Database Technology Cloud PostgreSQL 40

databricks

JUNE 12, 2025

160 Spear Street, 15th Floor San Francisco, CA 94105 1-866-330-0121 See Careers at Databricks © Databricks 2025. 160 Spear Street, 15th Floor San Francisco, CA 94105 1-866-330-0121 See Careers at Databricks © Databricks 2025. 160 Spear Street, 15th Floor San Francisco, CA 94105 1-866-330-0121 See Careers at Databricks © Databricks 2025.

BI 90

BI Entertainment Manufacturing Consulting 90

Data Engineering Podcast

APRIL 23, 2023

In this episode DeVaris Brown discusses the types of applications that are possible when teams don't have to manage the complex infrastructure necessary to support continuous data flows. Can you describe what Meroxa is and the story behind it? How have the focus and goals of the platform and company evolved over the past 2 years?

Data Lake 277

Data Lake Kafka Machine Learning Data Warehouse 277

databricks

JUNE 11, 2025

Openness Most technologies have some degree of lock-in, but nothing has more lock-in than traditional OLTP databases. At its core, a lakebase is grounded in battle-tested, open source technologies. As a result, there has been very little innovation in this space for decades.

Entertainment 81

Entertainment Data Lake Manufacturing Consulting 81

Precisely

JANUARY 30, 2025

Integrate data governance and data quality practices to create a seamless user experience and build trust in your data. When planning your data governance approach, start small, iterate purposefully, and foster data literacy to drive meaningful business outcomes.

Data Governance 103

Data Governance Government Metadata Data 103

Data Engineering Podcast

FEBRUARY 18, 2024

In this episode Dain Sundstrom, CTO of Starburst, explains how the combination of the Trino query engine and the Iceberg table format offer the ease of use and execution speed of data warehouses with the infinite storage and scalability of data lakes. What do you have planned for the future of Trino/Starburst?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Precisely

NOVEMBER 18, 2024

Data quality and data governance are the top data integrity challenges, and priorities. A long-term approach to your data strategy is key to success as business environments and technologies continue to evolve. However, they require a strong data foundation to be effective. Take a proactive approach.

Data Analytics 72

Data Analytics Data Governance Government Data Integration 72

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. Can you describe your experiences with Kafka?

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Snowflake

JUNE 1, 2025

Today, Snowflake advances our vision to be the ultimate platform for data-driven innovation with our announcement that we have agreed to acquire Crunchy Data, a leading provider of trusted, open source PostgreSQL technology. Crunchy Data is also a proven innovator when it comes to creating a great experience for developers.

PostgreSQL 94

PostgreSQL Database Cloud Government 94

databricks

JUNE 11, 2025

160 Spear Street, 15th Floor San Francisco, CA 94105 1-866-330-0121 See Careers at Databricks © Databricks 2025. 160 Spear Street, 15th Floor San Francisco, CA 94105 1-866-330-0121 See Careers at Databricks © Databricks 2025.

Entertainment 75

Entertainment Manufacturing Consulting Retail 75

Precisely

OCTOBER 31, 2024

Key Takeaways: Data mesh is a decentralized approach to data management, designed to shift creation and ownership of data products to domain-specific teams. Data fabric is a unified approach to data management, creating a consistent way to manage, access, and share data across distributed environments.

Data Architecture 69

Data Architecture Architecture Metadata Government 69

Data Engineering Podcast

SEPTEMBER 10, 2023

Summary Data systems are inherently complex and often require integration of multiple technologies. This offers a single location for managing visibility and error handling so that data platform engineers can manage complexity. container orchestration, generalized workflow orchestration, etc.)

BI 208

BI SQL Data Machine Learning 208

Data Engineering Podcast

JUNE 25, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Can you describe what SQLMesh is and the story behind it? DataOps is a term that has been co-opted and overloaded.

Data Engineering 225

Data Engineering Data Engineer Python Engineering 225

Christophe Blefari

MARCH 15, 2024

AI News 🤖 Mira Murati answers the Wall Street Journal about OpenAI Sora — OpenAI CTO has been asked a few questions about the underlying technology in Sora. The technology under this, is, Cityvision. Pandera, a data validation library for dataframes, now supports Polars. She revealed a few insights.

Metadata 272

Metadata Software Engineer Software Engineering Data Warehouse 272

Data Engineering Weekly

JANUARY 8, 2025

Together, we discussed how Hudi drives innovation, the state of open standards, and what lies ahead for data lakehouses in 2025 and beyond. This foundational concept addresses a key challenge for enterprises: building scalable, high-performing data platforms that can support the complexity of modern data ecosystems.

Data Lake 62

Data Lake Datasets Retail Data Ingestion 62

Data Engineering Podcast

APRIL 21, 2024

Summary Generative AI has rapidly transformed everything in the technology sector. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Data Lake 182

Data Lake High Quality Data Government Data Pipeline 182

Data Engineering Podcast

JUNE 23, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Data lakes in various forms have been gaining significant popularity as a unified interface to an organization's analytics. Closing Announcements Thank you for listening!

Data Lake 162

Data Lake High Quality Data Hadoop Government 162

phData: Data Engineering

NOVEMBER 1, 2024

Quotes It's extremely important because many of the Gen AI and LLM applications take an unstructured data approach, meaning many of the tools require you to give the tools full access to your data in an unrestricted way and let it crawl and parse it completely. Data governance is the only way to ensure those requirements are met.

Government 52

Government Data Governance Finance Metadata 52

Snowflake

NOVEMBER 11, 2024

Internally, banks are using AI to reduce the burden of data management, including data lineage and data quality controls, or drive efficiencies with business intelligence particularly in call centers. Those requirements can be fulfilled by leveraging cloud infrastructure and services.

Banking 98

Banking Finance Retail Food 98

Data Engineering Podcast

JUNE 16, 2024

In this episode Kevin Liu shares some of the interesting features that they have built by combining those technologies, as well as the challenges that they face in supporting the myriad workloads that are thrown at this layer of their data platform. Can you describe what role Trino and Iceberg play in Stripe's data architecture?

Data Lake 147

Data Lake High Quality Data Metadata Government 147

Cloudera

DECEMBER 16, 2024

However, I also expect a new, reverse trend to take shape: IT teams and data scientists will start to glean even greater business acumen to plug into the broader needs of the enterprise. With so much data being fed into AI model services, security and governance will also come to the fore.

Government 84

Government Finance Healthcare Cloud 84

Data Engineering Podcast

JUNE 2, 2024

Nicola Askham found her way into data governance by accident, and stayed because of the benefit that she was able to provide by serving as a bridge between the technology and business. In this episode she shares the practical steps to implementing a data governance practice in your organization, and the pitfalls to avoid.

Data Governance 147

Data Governance Government Data Lake High Quality Data 147

Data Engineering Podcast

MAY 21, 2023

In this episode David Yaffe and Johnny Graettinger share the story behind the business and technology and how you can start using it today to build a real-time data lake without all of the headache. Stream processing technologies have been around for around a decade. Can you describe what Estuary is and the story behind it?

Data Lake 162

Data Lake Kafka Machine Learning Data Warehouse 162

Data Engineering Podcast

OCTOBER 15, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Process 182

Process Building SQL BI 182

Data Engineering Podcast

OCTOBER 29, 2023

In this episode Tanya Bragin shares her experiences as a product manager for two major vendors and the lessons that she has learned about how teams should approach the process of tool selection. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles.

Database 189

Database BI SQL Machine Learning 189

Data Engineering Podcast

SEPTEMBER 17, 2023

Summary A significant amount of time in data engineering is dedicated to building connections and semantic meaning around pieces of information. Linked data technologies provide a means of tightly coupling metadata with raw information. What is the overlap between knowledge graphs and "linked data products"?

Building 189

Building BI SQL Python 189

Data Engineering Podcast

MARCH 24, 2024

In this episode Pete Hunt, CEO of Dagster labs, outlines these new capabilities, how they reduce the burden on data teams, and the increased collaboration that they enable across teams and business units. Can you describe what the focus of Dagster+ is and the story behind it? What problems are you trying to solve with Dagster+?

Data Lake 162

Data Lake High Quality Data Hadoop Government 162

Precisely

NOVEMBER 11, 2024

For successful personalization, you need to unify your communication technology. This involves integrating customer data across various channels – like your CRM systems, data warehouses, and more – so that the most relevant and up-to-date information is used consistently in your customer interactions. Focus on high-quality data.

High Quality Data 52

High Quality Data Data Data Warehouse Technology 52

Data Engineering Podcast

FEBRUARY 4, 2024

In this episode Yingjun Wu explains how it is architected to power analytical workflows on continuous data flows, and the challenges of making it responsive and scalable. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex.

SQL 173

SQL Data Lake High Quality Data Kafka 173

Precisely

NOVEMBER 18, 2024

Data quality and data governance are the top data integrity challenges, and priorities. A long-term approach to your data strategy is key to success as business environments and technologies continue to evolve. However, they require a strong data foundation to be effective. Take a proactive approach.

Data Analytics 59

Data Analytics Data Governance Government Data Integration 59

Data Engineering Podcast

JUNE 30, 2024

He highlights the role of data teams in modern organizations and how Synq is empowering them to achieve this. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Can you describe what Synq is and the story behind it?

Pipeline-centric 147

Pipeline-centric Engineering Data Lake High Quality Data 147

Edureka

APRIL 22, 2025

This blog will explore the significant advancements, challenges, and opportunities impacting data engineering in 2025, highlighting the increasing importance for companies to stay updated. Key Trends in Data Engineering for 2025 In the fast-paced world of technology, data engineering services keep companies that focus on data running.

Data Engineering 52

Data Engineering Data Engineer Engineering Consulting 52

Engineering at Meta

APRIL 28, 2025

To address these challenges, we made substantial investments in advanced data understanding technologies, as part of our Privacy Aware Infrastructure (PAI). Specifically, we have adopted a “shift-left” approach, integrating data schematization and annotations early in the product development process.

Metadata 83

Metadata Data Utilities Data Warehouse 83

Data Engineering Podcast

APRIL 7, 2024

Different roles and tasks in the business need their own ways to access and analyze the data in the organization. In order to enable this use case, while maintaining a single point of access, the semantic layer has evolved as a technological solution to the problem. What do you have planned for the future of Cube?

Data Lake 162

Data Lake High Quality Data BI Data Workflow 162

Data Engineering Podcast

JUNE 9, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat. Data observability has been gaining adoption for a number of years now, with a large focus on data warehouses.

Process 147

Process Data Lake High Quality Data Government 147