Data Engineering Podcast

FEBRUARY 25, 2024

What are some of the other tools/technologies that can benefit from some or all of the pieces of the FDAP stack? Contact Info LinkedIn pauldix on GitHub Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening!

Database 162

Database Technology Data Lake High Quality Data 162

The Pragmatic Engineer

NOVEMBER 21, 2023

Joshua is currently VP of Product & Strategy at VMware, a cloud computing and virtualization technology company. Joshua also writes an excellent Substack newsletter about how to design products which customers love, how to operate live services at scale, grow and optimize your technology orgs, and the history of the tech industry.

Engineering 345

Engineering Bytes Cloud Computing AWS 345

Snowflake

MARCH 31, 2025

All customer accounts are automatically provisioned to have access to default CPU and GPU compute pools that are only in use during an active notebook session and automatically suspended when inactive. Secure access to open source repositories via pip and the ability to bring in any model from hubs such as Hugging Face (see example here ).

Healthcare 96

Healthcare Medical Government Food 96

Snowflake

NOVEMBER 6, 2024

But as technology speeds forward, organizations of all sizes are realizing that generative AI isn’t just aspirational: It’s accessible and applicable now. You can begin to recognize that this technology, built with Cortex AI, can even start driving care, including suggesting and creating orders.

Hospitality 97

Hospitality Medical Government Software Engineer 97

Engineering at Meta

JANUARY 22, 2025

Data lineage is an instrumental part of Metas Privacy Aware Infrastructure (PAI) initiative, a suite of technologies that efficiently protect user privacy. It is a critical and powerful tool for scalable discovery of relevant data and data flows, which supports privacy controls across Metas systems.

Data Warehouse 109

Data Warehouse SQL Programming Language Data 109

Snowflake

DECEMBER 4, 2024

In the enterprise technology space, both the greatest certainties and the most significant potential surprises come from one area: the rapidly advancing field of artificial intelligence. But businesses will continue to hesitate to put in front of customers a technology that may display bias or provide inaccurate responses.

Unstructured Data 97

Unstructured Data Data Lake Deep Learning Structured Data 97

Snowflake

JANUARY 21, 2025

Data and technology (yes, AI) can now deeply impact the relevance of advertising creative, but that data needs to be secured and democratized across all levels and all departments within the agency landscape. Now, more than ever, the siloes of art and science need to be realigned.

Media 98

Media Unstructured Data Data Retail 98

Snowflake

APRIL 2, 2025

Without adopting Iceberg tables, data teams are forced to spend significant time and resources managing migrations and governance before being able to capture the opportunities new technologies and solutions offer. As technology continues to evolve, Snowflake continues to prioritize its customers by supporting open source initiatives.

Data Lake 97

Data Lake Cloud Storage Metadata Data Warehouse 97

Snowflake

MARCH 19, 2025

Hear from technology and industry experts about the ways in which leading retail and consumer goods companies are building connected consumer experiences with Snowflakes AI Data Cloud and maximizing the potential of AI. But 84% of the IT practitioners surveyed spend at least one hour a day fixing data problems.

Retail 96

Retail Media Entertainment Unstructured Data 96

Data Engineering Podcast

FEBRUARY 19, 2023

Your host is Tobias Macey and today I'm interviewing Ryan Blue about the evolution and applications of the Iceberg table format and how he is making it more accessible at Tabular Interview Introduction How did you get involved in the area of data management? We feel your pain. It ends up being anything but that.

IT 147

IT Data Lake Metadata Data Warehouse 147

Snowflake

MAY 14, 2024

The pharmaceutical industry generates a great deal of identifiable data (such as clinical trial data, patient engagement data) that has guardrails around “use and access.” To counter these hurdles, life sciences companies are seeking technological interventions to bring scalability and security to the data they gather for secondary analysis.

Pharmaceutical 108

Pharmaceutical IT Hospitality Data 108

Snowflake

APRIL 7, 2025

We are inspired by the transformative potential of technology to solve persistent challenges in product quality and compliance that we experienced firsthand. With advanced encryption, strict access controls and strong data governance, Snowflake helps us ensure the confidentiality and protection of our clients information.

Food 90

Food Data Transparency Software Engineer Software Engineering 90

Snowflake

FEBRUARY 13, 2025

Furthermore, most vendors require valuable time and resources for cluster spin-up and spin-down, disruptive upgrades, code refactoring or even migrations to new editions to access features such as serverless capabilities and performance improvements. It further shows how the value of a Snowflake credit changes over time.

Management 93

Management Government Cloud Unstructured Data 93

Snowflake

NOVEMBER 11, 2024

This is not surprising when you consider all the benefits, such as reducing complexity [and] costs and enabling zero-copy data access (ideal for centralizing data governance). Those requirements can be fulfilled by leveraging cloud infrastructure and services.

Banking 98

Banking Finance Retail Food 98

The Pragmatic Engineer

SEPTEMBER 26, 2023

Since then, all Cloud providers have expanded their availability, and the price of Cloud compute has dropped due to competition and technological progress. No wonder compute time was so valuable! The input/output area of the Atlas computer (right) and the computer itself, occupying a large room with its circuit boards inside closets.

Cloud 316

Cloud Software Engineer Software Engineering Cloud Computing 316

Snowflake

MARCH 14, 2024

Changes to SAP’s warehousing strategy over time mean many customers may need to patch together and manage multiple technologies to extract data in a useful format. Getting direct access to SAP data is critical because it holds such a breadth of ERP information. Even then, companies can struggle with latency and merge issues.

IT 121

IT Data Ingestion Data AWS 121

Robinhood

MARCH 21, 2024

Her decades of experience driving economic inclusion, social empowerment, and access to the markets makes her an invaluable voice as we look to grow globally. “Susan is widely recognized for her work and extensive contributions in Latin America as well as in the financial sector,” said Vlad Tenev, CEO and Co-Founder of Robinhood.

IT 119

IT Banking Finance Accessible 119

Snowflake

NOVEMBER 26, 2024

In essence, that was the story of WHOOP , the Boston-based wearable technology company aimed at enhancing human performance and endorsed by superstar athletes such as LeBron James and Cristiano Ronaldo. That’s why we’ve collected these migration success stories to help you get started on your migration to Snowflake.

Data Warehouse 105

Data Warehouse Cloud PostgreSQL Hadoop 105

Snowflake

NOVEMBER 11, 2024

Customers such as Avios, CHG Healthcare and Keysight Technologies are already developing container-based models in Snowflake ML. Additionally, we launched cross-region inference , allowing you to access preferred LLMs even if they aren’t available in your primary region. Learn more.

Unstructured Data 98

Unstructured Data SQL AWS Healthcare 98

Snowflake

JANUARY 30, 2025

Gen AI makes this all easy and accessible because anyone in an enterprise can simply interact with data by using natural language. What if our app doesnt have access to the right data and generates inaccurate results for stakeholders? Sales teams are usually boxed into dashboards to get insights.

Unstructured Data 97

Unstructured Data Entertainment Healthcare Telecommunication 97

The Pragmatic Engineer

APRIL 21, 2024

Sometimes the best explanations of how a technology solution works come from the software engineers who built it. See a longer version of this article here: Scaling ChatGPT: Five Real-World Engineering Challenges. To explain how ChatGPT (and other large language models) operate, I turned to the ChatGPT engineering team. "How

Engineering 363

Engineering Software Engineering Software Engineer Programming 363

Data Engineering Podcast

APRIL 23, 2023

What are the shifts that have made them more accessible to a wider variety of teams? Contact Info LinkedIn @devarispbrown on Twitter Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Summary Real-time capabilities have quickly become an expectation for consumers.

Data Lake 277

Data Lake Kafka Machine Learning Data Warehouse 277

The Pragmatic Engineer

MARCH 19, 2024

Today, full subscribers got access to a comprehensive Senior-and-above tech compensation research. Source: Cognition So far, all we have is video demos, and accounts of those with access to this tool. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers.

Software Engineer 327

Software Engineer Software Engineering Programming Language Media 327

Data Engineering Podcast

FEBRUARY 18, 2024

What are the differences in terms of pipeline design/access and usage patterns when using a Trino/Iceberg lakehouse as compared to other popular warehouse/lakehouse structures? Contact Info LinkedIn dain on GitHub Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Precisely

OCTOBER 31, 2024

Data fabric is a unified approach to data management, creating a consistent way to manage, access, and share data across distributed environments. As data management grows increasingly complex, you need modern solutions that allow you to integrate and access your data seamlessly.

Data Architecture 69

Data Architecture Architecture Metadata Government 69

Precisely

JANUARY 30, 2025

These architectures have both emerged to accelerate the delivery of trusted data to users so that its actionable and accessible for informed decision-making. Align people, processes, and technology Successful data governance requires a holistic approach. But as Woods noted, AI isnt a replacement for people its an augmentation tool.

Data Governance 103

Data Governance Government Metadata Data 103

The Pragmatic Engineer

SEPTEMBER 19, 2024

Without the backing of management, a large-scale rewrite is likely to fail. With this, it’s over to Lou. Use tech debt payments to get into the flow and stay in it A good reason to add new comments to old code before you change it is to speed up a code review. Adding comments is also a good way to reduce cognitive load.

Recruitment 297

Recruitment Java Coding Project 297

Precisely

NOVEMBER 18, 2024

A long-term approach to your data strategy is key to success as business environments and technologies continue to evolve. The rapid pace of technological change has made data-driven initiatives more crucial than ever within modern business strategies. However, they require a strong data foundation to be effective.

Data Analytics 72

Data Analytics Data Governance Data Integration Government 72

Snowflake

APRIL 5, 2025

The Llama 4 Maverick and Llama 4 Scout models can be accessed within the secure Snowflake perimeter on Cortex AI. Our AI Research team has been actively developing cutting-edge technologies on top of these Llama models. The model is well suited for precise image understanding and creative writing. 70B and Snowflake-Llama-3.1-405B,

Architecture 74

Architecture SQL Accessibility Accessible 74

Snowflake

FEBRUARY 3, 2025

High-quality, accessible and well-governed data enables organizations to realize the efficiency and productivity gains executives seek. By establishing data standardization, accessibility, and integration, partners help clients overcome the barriers that often derail AI initiatives.

Government 94

Government Data Governance Accessibility Accessible 94

Snowflake

APRIL 13, 2025

Managing and utilizing data effectively is crucial for organizational success in today's fast-paced technological landscape. As the Snowflake CTO at Deloitte, I have seen the powerful impact of these technologies, especially when leveraging the combined experience of the Deloitte and Snowflake alliance.

Data Management 80

Data Management Management Consulting Unstructured Data 80

Netflix Tech

DECEMBER 17, 2024

This fragmentation leads to inconsistencies and wastes valuable time as teams end up reinventing metrics or seeking clarification on definitions that should be standardized and readily accessible. Enter DataJunction (DJ). DJ acts as a central store where metric definitions can live and evolve.

Engineering 84

Engineering Entertainment Amazon Web Services Utilities 84

Snowflake

NOVEMBER 11, 2024

What if you could streamline your efforts while still building an architecture that best fits your business and technology needs? Ingest data more efficiently and manage costs For data managed by Snowflake, we are introducing features that help you access data easily and cost-effectively.

Data Architecture 105

Data Architecture Architecture Data Lake Kafka 105

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Operating it at scale, however, is notoriously challenging.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

The Pragmatic Engineer

MAY 11, 2023

A quick summary of these technologies: Prometheus : a time series database. The internet has been speculating the past few days on which crypto company spent $65M on Datadog in 2022. I confirmed it was Coinbase, and here are the details of what happened. Originally published on 11 May 2023. Can you possibly shed a little more light?“

AWS 347

AWS Software Engineering Software Engineer Google Cloud 347

phData: Data Engineering

NOVEMBER 1, 2024

It serves as a vital protective measure, ensuring proper data access while managing risks like data breaches and unauthorized use. It serves as a vital protective measure, ensuring proper data access while managing risks like data breaches and unauthorized use. Just click this button and fill out the form to download it.

Government 52

Government Data Governance Finance Metadata 52