Data Engineering Podcast

MAY 18, 2024

Summary The purpose of business intelligence systems is to allow anyone in the business to access and decode data to help them make informed decisions. Contact Info Ryan LinkedIn Paul LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Building 278

Building Data Lake High Quality Data Business Intelligence 278

Snowflake

MARCH 19, 2025

Hear from technology and industry experts about the ways in which leading retail and consumer goods companies are building connected consumer experiences with Snowflakes AI Data Cloud and maximizing the potential of AI. Media measurement: Explore how VideoAmp and Warner Bros.

Retail 89

Retail Media Entertainment Unstructured Data 89

Monte Carlo

JANUARY 3, 2024

It gives the LLM access to that enterprise proprietary data. Here’s an oversimplification of RAG application architecture: RAG architecture combines information retrieval with a text generator model, so it has access to your database while trying to answer a question from the user. Let me give you a hint: high-quality proprietary data.

Technology 113

Technology Database Data Engineering Data Engineer 113

Engineering at Meta

JANUARY 22, 2025

Data lineage is an instrumental part of Metas Privacy Aware Infrastructure (PAI) initiative, a suite of technologies that efficiently protect user privacy. However, these tools are limited by their lack of access to runtime data, which can lead to false positives from unexecuted code.

Data Warehouse 109

Data Warehouse SQL Programming Language Data 109

Snowflake

NOVEMBER 6, 2024

But as technology speeds forward, organizations of all sizes are realizing that generative AI isn’t just aspirational: It’s accessible and applicable now. You can begin to recognize that this technology, built with Cortex AI, can even start driving care, including suggesting and creating orders.

Hospitality 94

Hospitality Medical Government Software Engineer 94

Snowflake

DECEMBER 4, 2024

In the enterprise technology space, both the greatest certainties and the most significant potential surprises come from one area: the rapidly advancing field of artificial intelligence. But businesses will continue to hesitate to put in front of customers a technology that may display bias or provide inaccurate responses.

Unstructured Data 94

Unstructured Data Data Lake Deep Learning Structured Data 94

Snowflake

NOVEMBER 11, 2024

Customers such as Avios, CHG Healthcare and Keysight Technologies are already developing container-based models in Snowflake ML. Additionally, we launched cross-region inference , allowing you to access preferred LLMs even if they aren’t available in your primary region.

Unstructured Data 97

Unstructured Data SQL AWS Healthcare 97

Snowflake

APRIL 2, 2025

Without adopting Iceberg tables, data teams are forced to spend significant time and resources managing migrations and governance before being able to capture the opportunities new technologies and solutions offer. As technology continues to evolve, Snowflake continues to prioritize its customers by supporting open source initiatives.

Data Lake 88

Data Lake Cloud Storage Metadata Data Warehouse 88

Engineering at Meta

MARCH 4, 2025

QLC technology addresses these challenges by forming a middle tier between HDDs and TLC SSDs. This has been forcing data center engineers to meet their storage performance needs by shifting hot (frequently accessed) data to a TLC flash tier or by overprovisioning storage. QLC flash as a technology has been around since 2009.

Bytes 101

Bytes Media Data Technology 101

Data Engineering Podcast

FEBRUARY 18, 2024

What are the differences in terms of pipeline design/access and usage patterns when using a Trino/Iceberg lakehouse as compared to other popular warehouse/lakehouse structures? Contact Info LinkedIn dain on GitHub Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Precisely

OCTOBER 31, 2024

Data fabric is a unified approach to data management, creating a consistent way to manage, access, and share data across distributed environments. As data management grows increasingly complex, you need modern solutions that allow you to integrate and access your data seamlessly.

Data Architecture 69

Data Architecture Architecture Metadata Government 69

Snowflake

FEBRUARY 13, 2025

Furthermore, most vendors require valuable time and resources for cluster spin-up and spin-down, disruptive upgrades, code refactoring or even migrations to new editions to access features such as serverless capabilities and performance improvements.

Management 82

Management Government Cloud Unstructured Data 82

Snowflake

NOVEMBER 26, 2024

In essence, that was the story of WHOOP , the Boston-based wearable technology company aimed at enhancing human performance and endorsed by superstar athletes such as LeBron James and Cristiano Ronaldo. Data becomes distributed across multiple platforms; different teams end up using different tools. million in cost savings annually.

Data Warehouse 103

Data Warehouse Cloud PostgreSQL Hadoop 103

Precisely

NOVEMBER 18, 2024

A long-term approach to your data strategy is key to success as business environments and technologies continue to evolve. The rapid pace of technological change has made data-driven initiatives more crucial than ever within modern business strategies. However, they require a strong data foundation to be effective.

Data Analytics 72

Data Analytics Data Governance Data Integration Government 72

The Pragmatic Engineer

APRIL 21, 2024

Sometimes the best explanations of how a technology solution works come from the software engineers who built it. ChatGPT works impressively well with human language, and has access to more information than any one human could handle. See a longer version of this article here: Scaling ChatGPT: Five Real-World Engineering Challenges.

Engineering 358

Engineering Software Engineer Software Engineering Programming 358

The Pragmatic Engineer

SEPTEMBER 26, 2023

Since then, all Cloud providers have expanded their availability, and the price of Cloud compute has dropped due to competition and technological progress. With full-remote work, the risk is higher that someone other than the employee accesses the codebase. Full subscribers can access a list with links here.

Cloud 297

Cloud Software Engineer Software Engineering Cloud Computing 297

Snowflake

NOVEMBER 11, 2024

What if you could streamline your efforts while still building an architecture that best fits your business and technology needs? Ingest data more efficiently and manage costs For data managed by Snowflake, we are introducing features that help you access data easily and cost-effectively.

Data Architecture 105

Data Architecture Architecture Data Lake Kafka 105

phData: Data Engineering

NOVEMBER 1, 2024

It serves as a vital protective measure, ensuring proper data access while managing risks like data breaches and unauthorized use. Chief Technology Officer, Information Technology Industry The impact on data governance due to GenAI/LLM is that these technologies can spot trends much faster than humans or other applications.

Government 52

Government Data Governance Finance Metadata 52

Snowflake

FEBRUARY 3, 2025

High-quality, accessible and well-governed data enables organizations to realize the efficiency and productivity gains executives seek. By establishing data standardization, accessibility, and integration, partners help clients overcome the barriers that often derail AI initiatives. Ready to lead?

Government 84

Government Data Governance Accessible Accessibility 84

Precisely

MARCH 28, 2025

However, they faced a growing challenge: integrating and accessing data across a complex environment. They realized they needed a more automated, streamlined way to access the data. It continues to sustain its operational excellence and customer-centric focus all while improving data access for teams company-wide.

Data Integration 69

Data Integration Pipeline-centric Database-centric Kafka 69

The Pragmatic Engineer

MARCH 19, 2024

Today, full subscribers got access to a comprehensive Senior-and-above tech compensation research. Source: Cognition So far, all we have is video demos, and accounts of those with access to this tool. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers.

Software Engineer 308

Software Engineer Software Engineering Programming Language Media 308

Engineering at Meta

FEBRUARY 13, 2025

Mixed reality hardware products: As Meta continues to roll out mixed reality products , we work to encourage security research into these hardware and AI-driven technologies to help us find and fix potential bugs as quickly as possible. More details here.

Programming 109

Programming Designing Accessible Accessibility 109

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Operating it at scale, however, is notoriously challenging.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Data Engineering Podcast

APRIL 21, 2024

Summary Generative AI has rapidly transformed everything in the technology sector. Contact Info LinkedIn Blog Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? When Andrew Lee started work on Shortwave he was focused on making email more productive.

Data Lake 182

Data Lake High Quality Data Machine Learning Data Pipeline 182

Precisely

FEBRUARY 20, 2025

It enables faster decision-making, boosts efficiency, and reduces costs by providing self-service access to data for AI models. Artificial intelligence (AI): its the ultimate example of garbage in/garbage out technology. Not to mention, if internal trust of these technologies is damaged, it can obstruct further adoption.

Data Integration 69

Data Integration Government Datasets Data Pipeline 69

Snowflake

JUNE 27, 2024

By bringing workloads closer to the data, Snowflake Native Apps integrated with Snowpark Container Services makes it easier for RAI’s customers to adopt its technology. Snowflake Marketplace access Aref says RelationalAI saw new customer interest and demand grow exponentially, even when their product was available only in private preview.

Data Governance 124

Data Governance Algorithm Government Cloud 124

Snowflake

JUNE 5, 2024

At the same time, organizations must ensure the right people have access to the right content, while also protecting sensitive and/or Personally Identifiable Information (PII) and fulfilling a growing list of regulatory requirements.

Government 115

Government Accessibility Accessible Cloud 115

Data Engineering Podcast

APRIL 7, 2024

Different roles and tasks in the business need their own ways to access and analyze the data in the organization. In order to enable this use case, while maintaining a single point of access, the semantic layer has evolved as a technological solution to the problem. What do you have planned for the future of Cube?

Data Lake 162

Data Lake High Quality Data BI Data Workflow 162

Precisely

DECEMBER 16, 2024

As technology continues its rapid ongoing evolution, IT environments have become increasingly complex which leaves businesses needing to adapt at unprecedented speeds. For example: privileged users access to failures customer data What will my alert categories be? Scalable solutions are key for future-ready IT operations.

Systems 59

Systems IT Machine Learning Insurance 59

Edureka

MARCH 15, 2025

Compared to large language models (LLMs), which are limited in size, speed, and ease of customization, small language models (SLMs) would be a more economical, efficient, and space-saving AI technology for users with limited resources. Let’s now talk about the differences between SLM and LLM technologies. billion parameters.

Entertainment 74

Entertainment Retail Education Datasets 74

Snowflake

JANUARY 16, 2025

Customers can access these in Cortex AI via the complete function. SwiftKV on Snowflake Cortex AI SwiftKVs introduction comes at a critical moment for enterprises embracing LLM technologies. SwiftKV-optimized Llama 3.3 70B and Llama 3.1 405B models, referred to as Snowflake-LLama-3.3-70B 70B and Snowflake-Llama-3.1-405B,

Algorithm 93

Algorithm Data Analysis Building Process 93

Data Engineering Podcast

JUNE 23, 2024

What are the benefits of layering on top of existing technologies rather than building a fully custom solution? Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? What are the elements of Fabric that were engineered specifically for the service?

Data Lake 162

Data Lake High Quality Data Hadoop Machine Learning 162

Striim

OCTOBER 31, 2024

A nonprofit educational healthcare organization is faced with the challenge of modernizing its critical systems while ensuring uninterrupted access to essential services. This was crucial for guaranteeing uninterrupted access to student records, class schedules, and other key services.

Education 52

Education Healthcare Data Integration Cloud 52

Data Engineering Podcast

JUNE 16, 2024

In this episode Kevin Liu shares some of the interesting features that they have built by combining those technologies, as well as the challenges that they face in supporting the myriad workloads that are thrown at this layer of their data platform. What are the other systems that feed into and rely on the Trino/Iceberg service?

Data Lake 147

Data Lake High Quality Data Metadata Machine Learning 147

Snowflake

JANUARY 15, 2024

“Informatica’s Snowflake Native App is really about meeting our customers where they are,” says Rik Tamm-Daniels, Global VP of Ecosystems and Technology at Informatica. Now, they can access the full set of capabilities of Informatica’s Intelligent Data Management Cloud (IDMC) platform through a single drag-and-drop interface.

Data Integration 120

Data Integration BI Cloud Data 120

Data Engineering Podcast

MAY 26, 2024

In Code Comments, host Jamie Parker, Red Hatter and experienced engineer, shares the journey of technologists from across the industry and their hard-won lessons in implementing new technologies. What are the characteristics or features of data technologies and the overall ecosystem that can reduce the burden of data migration in the future?

Systems 130

Systems Data Lake High Quality Data Google Cloud 130