Hevo

JULY 15, 2024

So many companies need help in managing and analyzing enterprise data efficiently. Introducing Snowflake Horizon, the game-changing solution that will revolutionize data management and analysis. In this blog post, I will walk you […]

Data Management 52

Data Management Management Data Process 52

Edureka

APRIL 22, 2025

To remain competitive in the current digital environment, businesses must effectively gather, handle, and manage it. Data engineering can help with it. It is the force behind seamless data flow, enabling everything from AI-driven automation to real-time analytics. It lets you describe data more complexly and make predictions.

Data Engineer 52

Data Engineer Data Engineering Engineering Consulting 52

LinkedIn Engineering

JULY 19, 2023

To provide an experience designed to reduce toil for product engineering and take charge of tables, we built and deployed OpenHouse, a control plane that allows our developers to interface with managed tables in our open source data lakehouse.

Big Data 78

Big Data Data Management Management Metadata 78

Data Engineering Podcast

APRIL 21, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Can you describe what Shortwave is and the story behind it? What is the core problem that you are addressing with Shortwave?

Data Lake 182

Data Lake High Quality Data Machine Learning Data Pipeline 182

Cloudera

OCTOBER 23, 2024

In August, we wrote about how in a future where distributed data architectures are inevitable, unifying and managing operational and business metadata is critical to successfully maximizing the value of data, analytics, and AI.

Metadata 84

Metadata BI Data Lake Business Intelligence 84

Pinterest Engineering

OCTOBER 2, 2024

Tooling (libraries, user interface, etc.) , which provides ways to facilitate the use of the platform to the clients Due to length constraints, this blog will specifically focus on the Online Query Serving component, which can also be referred to interchangeably as “SDS” or “SDS Online Query Serving.”

Data Management 64

Data Management Management Metadata SQL 64

Data Engineering Podcast

FEBRUARY 26, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Are you tired of dealing with the headache that is the 'Modern Data Stack'? It's supposed to make building smarter, faster, and more flexible data infrastructures a breeze. We feel your pain.

Building 147

Building Machine Learning Metadata Data Integration 147

Jesse Anderson

NOVEMBER 14, 2023

A Guest Post by Ole Olesen-Bagneux In this blog post I would like to describe a new data team, that I call ‘the data discovery team’. First of all, in data science, data discovery means finding patterns in data using database query languages to test hypotheses.

Metadata 147

Metadata Data Science Big Data Data 147

Cloudera

DECEMBER 17, 2024

Since 5G networks began rolling out commercially in 2019, telecom carriers have faced a wide range of new challenges: managing high-velocity workloads, reducing infrastructure costs, and adopting AI and automation. The post Telco Enterprise Data Platforms: Key Success Factors in Building for an AI Future appeared first on Cloudera Blog.

Building 62

Building Telecommunication Data Architecture Architecture 62

Precisely

DECEMBER 9, 2024

To name a few: privacy and security considerations compliance demands interest in emerging data management architectures like data mesh and data fabric increased AI adoption The findings show that data governance is the most-cited data challenge inhibiting progress toward AI initiatives (62%).

Data Governance 64

Data Governance Government Data Integration Architecture 64

Data Engineering Podcast

FEBRUARY 25, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. This was the core of your recent re-write of the InfluxDB engine. Closing Announcements Thank you for listening!

Database 162

Database Technology Data Lake High Quality Data 162

Cloudera

OCTOBER 12, 2020

And, like all CDP data management and analytic cloud services, DataFlow will offer a consistent user experience on public and private clouds – for real hybrid cloud data streaming. . The post Cloudera acquires Eventador to accelerate Stream Processing in Public & Hybrid Clouds appeared first on Cloudera Blog.

Cloud 132

Cloud Process Scala Kafka 132

Data Engineering Podcast

JULY 9, 2023

In this episode Maxime Beauchemin of Airflow and Superset fame shares his vision for the entity-centric data model and how you can incorporate it into your own warehouse design. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles.

Database-centric 130

Database-centric Machine Learning SQL Data Engineer 130

Data Engineering Podcast

APRIL 14, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is brought to you by Datafold – a testing automation platform for data engineers that prevents data quality issues from entering every part of your data workflow, from migration to dbt deployment.

Non-relational Database 147

Non-relational Database Relational Database Database Designing 147

phData: Data Engineering

NOVEMBER 1, 2024

In an effort to better understand where data governance is heading, we spoke with top executives from IT, healthcare, and finance to hear their thoughts on the biggest trends, key challenges, and what insights they would recommend. This blog is a collection of those insights, but for the full trendbook, we recommend downloading the PDF.

Government 52

Government Data Governance Finance Metadata 52

DataKitchen

DECEMBER 9, 2024

A Drug Launch Case Study in the Amazing Efficiency of a Data Team Using DataOps How a Small Team Powered the Multi-Billion Dollar Acquisition of a Pharma Startup When launching a groundbreaking pharmaceutical product, the stakes and the rewards couldnt be higher.

Pharmaceutical 64

Pharmaceutical Data Lake Cloud Storage Project 64

Cloudera

OCTOBER 7, 2021

In this blog, I will demonstrate the value of Cloudera DataFlow (CDF) , the edge-to-cloud streaming data platform available on the Cloudera Data Platform (CDP) , as a Data integration and Democratization fabric. need to integrate multiple “point solutions” used in a data ecosystem) and organization reasons (e.g.,

Architecture 124

Architecture Metadata Kafka Government 124

Data Engineering Podcast

MARCH 10, 2024

In this episode Alex Merced explains how the branching and merging functionality in Nessie allows you to use the same versioning semantics for your data lakehouse that you are used to from Git. Can you describe what Nessie is and the story behind it? What are the core problems/complexities that Nessie is designed to solve?

Data Lake 147

Data Lake High Quality Data Architecture Machine Learning 147

Data Engineering Podcast

DECEMBER 12, 2021

In this episode Gokul Prabagaren shares his use for it in calculating your rewards points, including the auditing requirements and how he designed his pipeline to maintain all of the necessary information through a pattern of data enrichment. Closing Announcements Thank you for listening!

Building 130

Building Data Pipeline BI Data Warehouse 130

Data Engineering Podcast

MAY 7, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. What are some of the categories of attributes that need to be managed in a prototypical customer profile?

Pipeline-centric 130

Pipeline-centric Data Lake Machine Learning Data Warehouse 130

Ascend.io

DECEMBER 4, 2024

Automation, AI, DataOps, and strategic alignment are no longer optional —they are essential components of a successful data strategy. As we look towards 2025, it’s clear that data teams must evolve to meet the demands of evolving technology and opportunities. Further Exloration: What is data automation?

Data Pipeline 52

Data Pipeline Metadata Data Workflow Data 52

Cloudera

JUNE 7, 2024

A constant flow of breaking news from the data lakehouse space is making notable tech headlines this week. On Tuesday, Databricks announced that it will acquire Tabular, a data management company founded by the creators of Apache Iceberg, Ryan Blue, Daniel Weeks, and Jason Reidfor.

AWS 109

AWS Cloud Architecture Data Analytics 109

The Modern Data Company

APRIL 7, 2023

It is a big enough issue that several of our recent blog posts ( Lessons in Technical Debt from Southwest Airlines , Start Paying Down Your Technical Debt Today , and A Better Way to Plan the Payoff of Technical Debt) discussed it at length. This blog post will explain what data debt is, the impacts it can have, and how to handle it.

Management 98

Management Government Data Technology 98

Cloudera

APRIL 2, 2024

I am pleased to announce that Cloudera has achieved FedRAMP “In Process”, a significant milestone that underscores our commitment to providing the public sector with secure and reliable data management solutions across on-prem, hybrid and multi-cloud environments.

Designing 101

Designing Process Government Metadata 101

Data Engineering Podcast

FEBRUARY 6, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

SQL 100

SQL Building IT Google Cloud 100

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 130

Metadata MongoDB MySQL Scala 130

ArcGIS

DECEMBER 31, 2024

This blog provides step-by-step guidance to determine and use a silent install when configuring a driver to use Excel files in ArcGIS Pro. Learn More.

Data Management 117

Data Management Management Data 117

ArcGIS

NOVEMBER 17, 2023

This blog introduces the Geodatabase Resources Hub, a one-stop shop for all content offered by Esri's Geodatabase Team.

Data Management 128

Data Management Management Data 128

Cloudera

JULY 11, 2022

The need for data fabric. As Cloudera CMO David Moxey outlined in his blog , we live in a hybrid data world. Data is growing and continues to accelerate its growth. Especially since Cloudera’s platform can be deployed not only as a data fabric; it is capable of end-to-end multi-functional analytics.

Data Architecture 107

Data Architecture Architecture Data Government 107

Data Engineering Weekly

JUNE 2, 2024

Data plays a critical role in helping the founders ’s vision to iterate faster and grow the business. How should one think about a data strategy if you’re a startup? The author highlights the structured approach to building data infrastructure, data management, and metrics.

Data Engineer 104

Data Engineer Data Engineering Engineering Pipeline-centric 104

Data Engineering Weekly

AUGUST 6, 2024

Essential Features The core functionalities that make the CDC tool robust and effective include: - Low-latency data capture and delivery. Reliable change data capture mechanisms (e.g., Support for schema evolution and data type transformations. Ensure data consistency and integrity during capture and delivery.

Data Lake 126

Data Lake Data Warehouse Database Data Architecture 126

Data Engineering Podcast

JULY 2, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Integration 100

Data Integration SQL BI Data Warehouse 100

Cloudera

MAY 14, 2024

To attain that level of data quality, a majority of business and IT leaders have opted to take a hybrid approach to data management, moving data between cloud, on-premises -or a combination of the two – to where they can best use it for analytics or feeding AI models. appeared first on Cloudera Blog.

Cloud 99

Cloud Data Governance Unstructured Data Data Architecture 99

Data Engineering Podcast

MARCH 25, 2023

In this episode they explain why streaming architectures are so challenging, how they have designed Grainite to be robust and scalable, and how you can start using it today to build your streaming data applications without all of the operational headache. As your business adapts, so should your data.

MySQL 130

MySQL Python Architecture Machine Learning 130

DataKitchen

DECEMBER 30, 2020

Full disclosure: some images have been edited to remove ads or to shorten the scrolling in this blog post. DBTA’s 100 Companies That Matter Most in Data. DMI Awards 2020 Best Data Ops Solution Provider. Congrats on making it to the end of this blog post! SD Times’s Companies to Watch in 2021. DataKitchen.

Manufacturing 96

Manufacturing Big Data Data Pipeline Data Science 96

Data Engineering Podcast

AUGUST 24, 2020

In this episode Mars Lan and Pardhu Gunnam explain how they designed the platform, how it integrates into their data platforms, and how it is being used to power data discovery and analytics at LinkedIn. If you hand a book to a new data engineer, what wisdom would you add to it? Closing Announcements Thank you for listening!

Metadata 100

Metadata Management Kafka Data Engineer 100

Ascend.io

DECEMBER 4, 2024

Automation, AI, DataOps, and strategic alignment are no longer optional —they are essential components of a successful data strategy. As we look towards 2025, it’s clear that data teams must evolve to meet the demands of evolving technology and opportunities. Further Exloration: What is data automation?

Data Pipeline 52

Data Pipeline Metadata Data Workflow Data 52