DataKitchen

NOVEMBER 5, 2024

The Race For Data Quality In A Medallion Architecture The Medallion architecture pattern is gaining traction among data teams. The Medallion architecture is a design pattern that helps data teams organize data processing and storage into three distinct layers, often called Bronze, Silver, and Gold.

Architecture 81

Architecture Raw Data Pipeline-centric Data Ingestion 81

Simon Späti

MAY 26, 2023

In this third part, we’ll delve into data architecture patterns and their influence on data modeling. .” If you’ve journeyed with us from Part 1, where we dove into the importance and history of data modeling, or joined us in Part 2 to explore various approaches and techniques, I’m delighted you’ve stuck around.

Architecture 130

Architecture Data Engineer Data Engineering Engineering 130

Simon Späti

MAY 26, 2023

In this third part, we’ll delve into data architecture patterns and their influence on data modeling. .” If you’ve journeyed with us from Part 1, where we dove into the importance and history of data modeling, or joined us in Part 2 to explore various approaches and techniques, I’m delighted you’ve stuck around.

Architecture 130

Architecture Data Engineer Data Engineering Engineering 130

Data Engineering Podcast

FEBRUARY 18, 2024

And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. What are the technical/architectural/UX challenges that have hindered the progression of lakehouses? Want to see Starburst in action? Want to see Starburst in action?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Snowflake

NOVEMBER 11, 2024

What if you could streamline your efforts while still building an architecture that best fits your business and technology needs? At BUILD 2024, we announced several enhancements and innovations designed to help you build and manage your data architecture on your terms. Here’s a closer look.

Data Architecture 85

Data Architecture Architecture Data Lake Kafka 85

Data Engineering Podcast

DECEMBER 24, 2023

You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! Want to see Starburst in action? Want to see Starburst in action?

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Start Data Engineering

SEPTEMBER 18, 2024

Data flow architecture 3. Define SLAs so stakeholders know what to expect 3.1.4. Define checks to ensure the output dataset is usable 3.2. Identify what tool to use to process data 3.3.

Project 240

Project Building Datasets Architecture 240

databricks

MAY 1, 2024

Modern data stacks provide the necessary flexibility and efficiency for analytics and AI. Learn how the Databricks Data Intelligence Platform makes use of them.

Data Architecture 125

Data Architecture Architecture Data 125

KDnuggets

JULY 26, 2024

When it comes to AI, every expert in an MoE model specializes in a much larger problem—just like every doctor specializes in their medical field. This improves efficiency and increases system efficacy and accuracy.

Architecture 100

Architecture Medical Systems IT 100

The Pragmatic Engineer

OCTOBER 10, 2023

We originally built M3 to support Uber’s scale on a microservices/container (cloud native) architecture. It was that very architecture that created the scale, reliability, performance and cost-efficiency challenges. None of the existing vendors had built and optimized for a cloud-native architecture.

Software Engineer 268

Software Engineer Software Engineering Architecture Media 268

Data Engineering Podcast

MAY 18, 2024

What other technical architectures have you had to develop to support the use of AI in Zenlytic? What other technical architectures have you had to develop to support the use of AI in Zenlytic? Firstly, what are data agents and why do you think they're important? How are data agents different from chatbots?

Building 278

Building Data Lake High Quality Data Business Intelligence 278

Data Engineering Podcast

FEBRUARY 25, 2024

And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Can you start by describing the FDAP stack and how the components combine to provide a foundational architecture for database engines? Want to see Starburst in action?

Database 162

Database Technology Data Lake High Quality Data 162

Data Engineering Podcast

JUNE 23, 2024

Summary Data lakehouse architectures have been gaining significant adoption. To accelerate adoption in the enterprise Microsoft has created the Fabric platform, based on their OneLake architecture. To accelerate adoption in the enterprise Microsoft has created the Fabric platform, based on their OneLake architecture.

Data Lake 162

Data Lake High Quality Data Hadoop Machine Learning 162

Data Engineering Podcast

FEBRUARY 4, 2024

And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. What are some of the platforms/architectures that teams are replacing with RisingWave? Want to see Starburst in action? Want to see Starburst in action?

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Data Engineering Podcast

JANUARY 21, 2024

Summary Databases and analytics architectures have gone through several generational shifts. And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. How has that changed the architectural approach to CDPs?

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Data Engineering Podcast

APRIL 21, 2024

And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data.

Data Lake 182

Data Lake High Quality Data Data Pipeline Machine Learning 182

Data Engineering Podcast

SEPTEMBER 10, 2023

You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! With Materialize, you can!

BI 208

BI SQL Data Machine Learning 208

Data Engineering Podcast

JANUARY 7, 2024

And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data.

Data Process 147

Data Process Process Data Lake High Quality Data 147

Snowflake

APRIL 24, 2024

Table 1 Model architecture and training compute for Arctic, Llama-2 70B, DBRX and Mixtral 8x22B. Training efficiency To achieve this level of training efficiency, Arctic uses a unique Dense-MoE Hybrid transformer architecture. For a detailed breakdown of results by individual benchmark, see the Metrics section.

Amazon Web Services 145

Amazon Web Services SQL AWS Architecture 145

KDnuggets

FEBRUARY 5, 2024

How and why the data lake architecture often fails to meet its promises. And how better governance helps mitigate such challenges.

Data Lake 145

Data Lake IT Government Data 145

KDnuggets

MAY 19, 2022

Read the best books on Programming, Statistics, Data Engineering, Web Scraping, Data Analytics, Business Intelligence, Data Applications, Data Management, Big Data, and Cloud Architecture.

Data Science 160

Data Science Business Intelligence Big Data Architecture 160

Data Engineering Podcast

JUNE 9, 2024

How much of the complexity is due to the nature of streaming data vs. the architectural realities of Flink? What role does Flink play in the architecture of generative AI systems? What role does Flink play in the architecture of generative AI systems? What role does Flink play in the architecture of generative AI systems?

Process 147

Process Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

JANUARY 28, 2024

And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Can you describe the architecture of the Scanner platform? Can you describe the architecture of the Scanner platform? Want to see Starburst in action?

Data Lake 147

Data Lake Building High Quality Data AWS 147

Data Engineering Podcast

MARCH 10, 2024

Summary Data lakehouse architectures are gaining popularity due to the flexibility and cost effectiveness that they offer. And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Can you describe the architecture of Nessie?

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

KDnuggets

SEPTEMBER 27, 2023

From the basics of Python to clean architecture and more, here are five free books to level up your Python skills.

Python 157

Python Architecture 157

Data Engineering Podcast

APRIL 28, 2024

And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data.

Building 147

Building Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

MAY 5, 2024

And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. What are the practical/architectural methods necessary to build more cognitive AI systems? Want to see Starburst in action? Want to see Starburst in action?

Building 147

Building Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

MARCH 24, 2024

And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data.

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Data Engineering Podcast

DECEMBER 31, 2023

And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. How does that influence the architectural design/capabilities for data platforms in those organizations? Want to see Starburst in action? With Materialize, you can!

Designing 130

Designing Data Lake High Quality Data SQL 130

ArcGIS

MARCH 11, 2024

The ArcGIS Well-Architected Framework and ArcGIS Architecture Center provides guidance for implementing systems with ArcGIS.

Architecture 110

Architecture Systems Designing Technology 110

Data Engineering Podcast

NOVEMBER 12, 2023

And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. What are the opportunities for specialization of the model architecture given the highly structured nature of the problem domain? Want to see Starburst in action?

Software Engineer 147

Software Engineer Software Engineering Engineering Data Lake 147

Data Engineering Podcast

OCTOBER 29, 2023

You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! With Materialize, you can!

Database 189

Database BI SQL Machine Learning 189

Data Engineering Podcast

SEPTEMBER 17, 2023

You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! With Materialize, you can!

Building 189

Building SQL BI Python 189

KDnuggets

OCTOBER 30, 2023

A comparative overview of data warehouses, data lakes, and data marts to help you make informed decisions on data storage solutions for your data architecture.

Data Lake 145

Data Lake Data Warehouse Data Storage Data 145