Data Engineering Podcast

DECEMBER 21, 2020

Summary Data governance is a term that encompasses a wide range of responsibilities, both technical and process oriented. One of the more complex aspects is that of access control to the data assets that an organization is responsible for managing. What is data governance?

Data Governance 130

Data Governance Government Data Lake Banking 130

Data Engineering Podcast

DECEMBER 28, 2020

The team at Satori has a background in cybersecurity and they are using the lessons that they learned in that field to address the challenge of access control and auditing for data governance. This is an interesting conversation about the intersection of data and security and the lessons that can be learned in each direction.

Data Governance 130

Data Governance Government Transportation Data 130

Cloudera

NOVEMBER 19, 2020

November 15-21 marks International Fraud Awareness Week – but for many in government, that’s every week. From bogus benefits claims to fraudulent network activity, fraud in all its forms represents a significant threat to government at all levels. The Public Sector data challenge. Modernization has been a boon to government.

Government 119

Government Machine Learning Algorithm Raw Data 119

Data Engineering Podcast

MARCH 27, 2022

Summary Data governance is a practice that requires a high degree of flexibility and collaboration at the organizational and technical levels. The growing prominence of cloud and hybrid environments in data management adds additional stress to an already complex endeavor. Closing Announcements Thank you for listening!

Data Governance 130

Data Governance Government Cloud Building 130

Precisely

OCTOBER 31, 2024

Key Takeaways: Data mesh is a decentralized approach to data management, designed to shift creation and ownership of data products to domain-specific teams. Data fabric is a unified approach to data management, creating a consistent way to manage, access, and share data across distributed environments.

Data Architecture 68

Data Architecture Architecture Metadata Government 68

Precisely

OCTOBER 23, 2023

When most people think of master data management, they first think of customers and products. But master data encompasses so much more than data about customers and products. Challenges of Master Data Management A decade ago, master data management (MDM) was a much simpler proposition than it is today.

Data Management 109

Data Management Management Data Data Integration 109

Precisely

NOVEMBER 18, 2024

Key Takeaways: Data integrity is required for AI initiatives, better decision-making, and more – but data trust is on the decline. Data quality and data governance are the top data integrity challenges, and priorities. However, they require a strong data foundation to be effective.

Data Analytics 71

Data Analytics Data Governance Data Integration Government 71

Cloudera

MARCH 29, 2022

State and local governments generate and store enormous amounts of data essential to their ability to deliver citizen services. But how can they capitalize on all of their data to become engines of growth and innovation, empowering and enhancing their ability to provide services and better serve their communities?

Government 97

Government Cloud Cloud Computing Data Science 97

Data Engineering Podcast

JULY 17, 2022

In this episode Crux CTO Mark Etherington discusses the different costs involved in managing external data, how to think about the total return on investment for your data, and how the Crux platform is architected to reduce the toil involved in managing third party data.

Data Management 130

Data Management Management Metadata MongoDB 130

Precisely

SEPTEMBER 19, 2024

Key Takeaways: Data integrity is essential for AI success and reliability – helping you prevent harmful biases and inaccuracies in AI models. Robust data governance for AI ensures data privacy, compliance, and ethical AI use. Proactive data quality measures are critical, especially in AI applications.

Data Governance 77

Data Governance Government High Quality Data Datasets 77

Data Engineering Podcast

FEBRUARY 18, 2024

In this episode Dain Sundstrom, CTO of Starburst, explains how the combination of the Trino query engine and the Iceberg table format offer the ease of use and execution speed of data warehouses with the infinite storage and scalability of data lakes. What do you have planned for the future of Trino/Starburst?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Cloudera

JUNE 14, 2024

But for all the excitement and movement happening within hybrid cloud infrastructure and its potential with AI, there are still risks and challenges that need to be appropriately managed—specifically when it comes to the issue of data governance. The need for effective data governance itself is not a new phenomenon.

Data Governance 74

Data Governance Government Cloud Insurance 74

Data Engineering Podcast

DECEMBER 24, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

ArcGIS

JUNE 26, 2023

Tax Parcel Data Management helps to inventory tax parcels from record information and share this info with internal and external stakeholders.

Data Management 98

Data Management Management Data Government 98

Snowflake

NOVEMBER 19, 2024

In these conversations, there are a number of questions that I hear time and time again: Will my data platform be scalable and reliable enough? How will my data stay secure and governed? Will it be easy to use for my entire team? What will costs look like?

Data Governance 98

Data Governance Government Healthcare Building 98

Cloudera

DECEMBER 3, 2021

The most effective systems are ones that intelligently bring together both human judgement and AI, these take into account model drift, confidence intervals and impact, as well as level of governance. Governance. Where and how data scientists and engineers fit into an organisational structure may vary. Model Drift. Regulation.

Government 102

Government Hospitality Education Designing 102

ArcGIS

MARCH 30, 2023

ROW and Easement Data Management improves infrastructure planning, utility maintenance, and other functions that require access to land.

Data Management 104

Data Management Management Utilities Data 104

Precisely

MARCH 18, 2025

There were many Gartner keynotes and analyst-led sessions that had titles like: Scale Data and Analytics on Your AI Journeys” What Everyone in D&A Needs to Know About (Generative) AI: The Foundations AI Governance: Design an Effective AI Governance Operating Model The advice offered during the event was relevant, valuable, and actionable.

Data Analytics 63

Data Analytics Data Governance Government Consulting 63

Data Engineering Podcast

JUNE 30, 2024

He highlights the role of data teams in modern organizations and how Synq is empowering them to achieve this. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Can you describe what Synq is and the story behind it?

Pipeline-centric 147

Pipeline-centric Engineering Data Lake High Quality Data 147

Data Engineering Podcast

JUNE 23, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Data lakes in various forms have been gaining significant popularity as a unified interface to an organization's analytics. Closing Announcements Thank you for listening!

Data Lake 162

Data Lake High Quality Data Hadoop Machine Learning 162

Snowflake

NOVEMBER 11, 2024

Snowflake is committed to doing just that by continually adding features to help our customers simplify how they architect their data infrastructure. Snowflake Unistore consolidates both into a single database so users get a drastically simplified architecture with less data movement and consistent security and governance controls.

Data Architecture 105

Data Architecture Architecture Data Lake Kafka 105

Precisely

NOVEMBER 18, 2024

Key Takeaways: Data integrity is required for AI initiatives, better decision-making, and more – but data trust is on the decline. Data quality and data governance are the top data integrity challenges, and priorities. However, they require a strong data foundation to be effective.

Data Analytics 58

Data Analytics Data Governance Government Data Integration 58

Data Engineering Podcast

APRIL 21, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Can you describe what Shortwave is and the story behind it? What is the core problem that you are addressing with Shortwave?

Data Lake 182

Data Lake High Quality Data Machine Learning Data Pipeline 182

Cloudera

JANUARY 26, 2024

In this article, we will walk you through the process of implementing fine grained access control for the data governance framework within the Cloudera platform. In a good data governance strategy, it is important to define roles that allow the business to limit the level of access that users can have to their strategic data assets.

Metadata 73

Metadata Data Governance Government Management 73

Data Engineering Podcast

JUNE 16, 2024

In this episode Kevin Liu shares some of the interesting features that they have built by combining those technologies, as well as the challenges that they face in supporting the myriad workloads that are thrown at this layer of their data platform. Can you describe what role Trino and Iceberg play in Stripe's data architecture?

Data Lake 147

Data Lake High Quality Data Metadata Machine Learning 147

Snowflake

FEBRUARY 13, 2025

Despite most having security and governance features, these may require the customer to integrate multiple services to provide a comprehensive solution, or worse yet, may not be available in earlier versions of the product, forcing upgrades. As a result, data often went underutilized.

Management 85

Management Government Cloud Unstructured Data 85

ArcGIS

FEBRUARY 3, 2025

Streamline data management, reduce duplication, enhance data quality, and provide a single source of truth for address and road data.

Data Management 78

Data Management Management Data Transportation 78

Data Engineering Podcast

FEBRUARY 4, 2024

In this episode Yingjun Wu explains how it is architected to power analytical workflows on continuous data flows, and the challenges of making it responsive and scalable. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex.

SQL 173

SQL Data Lake High Quality Data Machine Learning 173

Data Engineering Podcast

JUNE 9, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat. Data observability has been gaining adoption for a number of years now, with a large focus on data warehouses.

Process 147

Process Data Lake High Quality Data Machine Learning 147

Snowflake

NOVEMBER 11, 2024

.” — Paul Chang, Head of Payment Networks, AWS “Data warehouses are gaining a lot of momentum right now, and Snowflake is at the forefront of this trend. This is not surprising when you consider all the benefits, such as reducing complexity [and] costs and enabling zero-copy data access (ideal for centralizing data governance).

Banking 96

Banking Finance Retail Food 96

Cloudera

OCTOBER 23, 2024

In August, we wrote about how in a future where distributed data architectures are inevitable, unifying and managing operational and business metadata is critical to successfully maximizing the value of data, analytics, and AI. They are free to choose the infrastructure best suited for each workload.

Metadata 84

Metadata BI Data Lake Business Intelligence 84

Precisely

JANUARY 27, 2025

What are the primary data challenges blocking the path to AI success? Data governance tops the list, according to 62% of survey respondents. In our survey, 42% of respondents say that a shortage of skills and resources is one of the biggest challenges facing the success of their data programs, an increase from 37% in 2023.

Data Integration 59

Data Integration Data Governance Government Data Programming 59

Data Engineering Podcast

MARCH 24, 2024

In this episode Pete Hunt, CEO of Dagster labs, outlines these new capabilities, how they reduce the burden on data teams, and the increased collaboration that they enable across teams and business units. Can you describe what the focus of Dagster+ is and the story behind it? What problems are you trying to solve with Dagster+?

Data Lake 162

Data Lake High Quality Data Hadoop Machine Learning 162

Edureka

APRIL 22, 2025

It lets you describe data more complexly and make predictions. AI-powered data engineering solutions make it easier to streamline the data management process, which helps businesses find useful insights with little to no manual work. This will help make better analytics predictions and improve data management.

Data Engineer 52

Data Engineer Data Engineering Engineering Consulting 52

LinkedIn Engineering

JULY 19, 2023

This unification is crucial for simplifying lakehouse management, organizing data for optimal query performance, instituting governance, and declarative metadata management - all to provide an enhanced developer experience. Tables are governed as per agreed upon company standards. Tables are maintained regularly.

Big Data 78

Big Data Data Management Management Metadata 78

Data Engineering Weekly

APRIL 24, 2025

Challenges in Data Readiness Avinash openly discussed challenges that many enterprises face concerning data readiness, including fragmented data ecosystems, legacy systems, and inadequate data governance.

Government 52

Government Data Governance High Quality Data Machine Learning 52

Data Engineering Podcast

FEBRUARY 11, 2024

In this episode Andrew Jefferson explains the complexities of building a robust system for data sharing, the techno-social considerations, and how the Bobsled platform that he is building aims to simplify the process. What is the current state of the ecosystem for data sharing protocols/practices/platforms? tabular, image, etc.)

Data Lake 147

Data Lake High Quality Data Government Machine Learning 147