Data Engineering Podcast

JANUARY 28, 2024

For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics. When is Scanner the wrong choice?

Data Lake 147

Data Lake Building High Quality Data AWS 147

Knowledge Hut

DECEMBER 26, 2023

Technology has come a long way. With UPI, even a vegetable vendor or a small business owner can access high-end new technology (Newest tech). Top 10 Latest Technology Trends in 2024 It is no surprise that the demand for tech-savvy professionals is increasing as the world gets digital.

Technology 98

Technology Cloud Computing Healthcare Data Science 98

Engineering at Meta

DECEMBER 6, 2023

Our aim is to ensure that everyone’s personal messages on Messenger can only be accessed by the sender and the intended recipients, and that everyone can be sure the messages they receive are from an authentic sender. End-to-end encryption isn’t about the technology at its core.

Building 145

Building Designing Consulting Accessible 145

Data Engineering Podcast

MAY 5, 2024

In this episode he explains his approach to building AI in a more human-like fashion and the emphasis on learning rather than statistical prediction. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Building 147

Building Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

APRIL 16, 2023

As the capabilities of these systems has improved and become more accessible, the target of what self-serve means changes. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Who are the target users?

Business Intelligence 147

Business Intelligence Building Data Lake BI 147

The Pragmatic Engineer

APRIL 21, 2024

Sometimes the best explanations of how a technology solution works come from the software engineers who built it. A refresher on OpenAI, and on Evan Evan: how did you join OpenAI, and end up heading the Applied engineering group – which also builds ChatGPT? "How does ChatGPT work, under the hood? My questions are in italic.

Engineering 351

Engineering Software Engineer Software Engineering Programming 351

Data Engineering Podcast

JULY 23, 2023

Summary Real-time data processing has steadily been gaining adoption due to advances in the accessibility of the technologies involved. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Building 130

Building Machine Learning SQL Python 130

Monte Carlo

JANUARY 3, 2024

But building a generative AI model that actually drives business value is hard. It gives the LLM access to that enterprise proprietary data. Snowflake Cortex now enables organizations to quickly analyze data and build AI apps directly in Snowflake. Data teams are scrambling to answer the call. It’s GenAI, but where’s the moat?

Technology 113

Technology Database Data Engineering Data Engineer 113

Data Engineering Podcast

NOVEMBER 13, 2022

Summary Building data products is an undertaking that has historically required substantial investments of time and talent. With the rise in cloud platforms and self-serve data technologies the barrier of entry is dropping. Shane Gibson co-founded AgileData to make analytics accessible to companies of all sizes.

Building 130

Building Metadata MongoDB MySQL 130

Data Engineering Podcast

JANUARY 15, 2023

Summary The modern data stack has made it more economical to use enterprise grade technologies to power analytics at organizations of every scale. Go to dataengineeringpodcast.com/materialize today and sign up for early access to get started. Struggling with broken pipelines? Stale dashboards? Missing data?

Coding 130

Coding Building PostgreSQL Data Lake 130

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Data Engineering Podcast

JANUARY 29, 2023

The team at Omni have taken a new approach by automatically building models based on the queries that are executed. Go to dataengineeringpodcast.com/materialize today and sign up for early access to get started. What are the core goals that you are trying to achieve with building Omni?

Business Intelligence 130

Business Intelligence PostgreSQL Building BI 130

Snowflake

DECEMBER 18, 2023

Data clean rooms have emerged as the technology to meet this need, enabling interoperability where multiple parties can collaborate on and analyze sensitive data in a governed way without exposing direct access to the underlying data and business logic. Snowflake’s acquisition of Samooha is subject to customary closing conditions.

Cloud 120

Cloud Building Entertainment Government 120

The Pragmatic Engineer

MARCH 19, 2024

Today, full subscribers got access to a comprehensive Senior-and-above tech compensation research. Devin has dominated tech news for the past few days, with major media outlets running sensational headlines, including the usually reserved Bloomberg’s, Gold-Medalist Coders Build an AI That Can Do Their Job for Them.

Software Engineer 276

Software Engineer Software Engineering Programming Language Media 276

The Pragmatic Engineer

SEPTEMBER 26, 2023

This means more repositories are needed, which are fast enough to build and work with, but which increase fragmentation. Since then, all Cloud providers have expanded their availability, and the price of Cloud compute has dropped due to competition and technological progress. Executing a build is much slower while on a call.

Cloud 258

Cloud Software Engineer Software Engineering Cloud Computing 258

Cloudera

DECEMBER 1, 2023

Recent Government Initiatives on Public Sector AI Solutions In recent years, governments across the globe have recognized the transformative potential of artificial intelligence (AI) and have embarked on initiatives to harness this technology to drive innovation and serve their citizens more effectively. million), among others.

Building 115

Building Government Transportation Data Governance 115

Data Engineering Podcast

APRIL 21, 2024

Summary Generative AI has rapidly transformed everything in the technology sector. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Go to dataengineeringpodcast.com/dagster today to get started.

Data Lake 182

Data Lake High Quality Data Data Pipeline Machine Learning 182

Snowflake

OCTOBER 2, 2023

To help customers overcome these challenges, RudderStack and Snowflake recently launched Profiles , a new product that allows every data team to build a customer 360 directly in their Snowflake Data Cloud environment. Now teams can leverage their existing data engineering tools and workflows to build their customer 360.

Cloud 108

Cloud Building Insurance Data Engineering 108

The Pragmatic Engineer

MAY 11, 2023

A quick summary of these technologies: Prometheus : a time series database. Prometheus is part of the Cloud Native Foundation, membership of which indicates that it’s safe to build on top of Prometheus, as it’s actively maintained and will continue to be. But why is this? What happens next?

AWS 318

AWS Software Engineer Software Engineering Google Cloud 318

Snowflake

APRIL 17, 2024

In today’s data-driven world, developer productivity is essential for organizations to build effective and reliable products, accelerate time to value, and fuel ongoing innovation. We’re excited to share more innovations soon, making data even more accessible for all.

Data Pipeline 113

Data Pipeline Python Data Engineering Data Engineer 113

Data Engineering Podcast

APRIL 7, 2024

Different roles and tasks in the business need their own ways to access and analyze the data in the organization. In order to enable this use case, while maintaining a single point of access, the semantic layer has evolved as a technological solution to the problem. Go to dataengineeringpodcast.com/dagster today to get started.

Data Lake 162

Data Lake High Quality Data BI Data Workflow 162

Data Engineering Podcast

FEBRUARY 11, 2024

In this episode Andrew Jefferson explains the complexities of building a robust system for data sharing, the techno-social considerations, and how the Bobsled platform that he is building aims to simplify the process. Dagster offers a new approach to building and running data platforms and data pipelines. tabular, image, etc.)

Data Lake 147

Data Lake High Quality Data Government Data Pipeline 147

Snowflake

NOVEMBER 1, 2023

And because the Notebook is natively integrated into Snowflake’s role-based access controls (RBAC), it’s easy to securely share and collaborate on your code and results without compromising on any enterprise data. What’s Next? Check out the Snowpark ML demo from Snowday to see the latest launches in action.

Building 111

Building Python SQL Programming Language 111

Data Engineering Podcast

FEBRUARY 4, 2024

For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics. Your first 30 days are free!

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Snowflake

JUNE 5, 2024

Enterprises can utilize gen AI to extract more value from their data and build conversational interfaces for customer and employee applications. Snowflake Cortex AI is a fully managed service designed to unlock the potential of the technology for everyone within an organization, regardless of their technical expertise.

Coding 93

Coding Building Management Government 93

Data Engineering Podcast

JUNE 23, 2024

For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end-to-end data lakehouse platform built on Trino, the query engine Apache Iceberg was designed for, with complete support for all table formats including Apache Iceberg, Hive, and Delta Lake. When is Fabric the wrong choice?

Data Lake 162

Data Lake High Quality Data Hadoop Machine Learning 162

The Pragmatic Engineer

OCTOBER 10, 2023

He’s solved interesting engineering challenges along the way, too – like building observability for Amazon’s EC2 offering, and being one of the first engineers on Uber’s observability platform. The focus seemed to shift to: invent something new → build a service for it → ship it.

Software Engineer 268

Software Engineer Software Engineering Architecture Media 268

Data Engineering Podcast

MARCH 24, 2024

Summary A core differentiator of Dagster in the ecosystem of data orchestration is their focus on software defined assets as a means of building declarative workflows. Contact Info Twitter LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Data Engineering Podcast

JUNE 2, 2024

Summary Modern businesses aspire to be data driven, and technologists enjoy working through the challenge of building data systems to support that goal. Nicola Askham found her way into data governance by accident, and stayed because of the benefit that she was able to provide by serving as a bridge between the technology and business.

Data Governance 147

Data Governance Government Data Lake High Quality Data 147

Data Engineering Podcast

MARCH 3, 2024

Summary Artificial intelligence technologies promise to revolutionize business and produce new sources of value. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Your first 30 days are free!

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Data Engineering Podcast

JUNE 16, 2024

In this episode Kevin Liu shares some of the interesting features that they have built by combining those technologies, as well as the challenges that they face in supporting the myriad workloads that are thrown at this layer of their data platform. What are the other systems that feed into and rely on the Trino/Iceberg service?

Data Lake 147

Data Lake High Quality Data Metadata Machine Learning 147

Data Engineering Podcast

JANUARY 21, 2024

In this episode Tasso Argyros, CEO of ActionIQ, gives a summary of the major epochs in database technologies and how he is applying the capabilities of cloud data warehouses to the challenge of building more comprehensive experiences for end-users through a modern customer data platform (CDP).

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Data Engineering Podcast

MAY 12, 2024

Summary Building a data platform is a substrantial engineering endeavor. In Code Comments, host Jamie Parker, Red Hatter and experienced engineer, shares the journey of technologists from across the industry and their hard-won lessons in implementing new technologies. My thanks to the team at Code Comments for their support.

Management 147

Management Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

JUNE 9, 2024

In Code Comments, host Jamie Parker, Red Hatter and experienced engineer, shares the journey of technologists from across the industry and their hard-won lessons in implementing new technologies. Can you describe how Datorios is used in a day-to-day workflow for someone building streaming applications on Flink?

Process 147

Process Data Lake High Quality Data Machine Learning 147

Rockset

JUNE 21, 2024

We are thrilled to join the OpenAI team and bring our technology and expertise to building safe and beneficial AGI. With this acquisition, what we’ve developed over the years will help make AI accessible to all in a safe and beneficial way. I’m excited to share that OpenAI has completed the acquisition of Rockset.

Database 145

Database Cloud Accessible Accessibility 145

Data Engineering Podcast

JANUARY 7, 2024

Summary Data processing technologies have dramatically improved in their sophistication and raw throughput. When performing research and building prototypes of the projects, what is your process for incorporating user experience into the implementation of the product? What do you have planned for the future of your academic research?

Data Process 147

Data Process Process Data Lake High Quality Data 147

Data Engineering Podcast

MAY 26, 2024

For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end-to-end data lakehouse platform built on Trino, the query engine Apache Iceberg was designed for, with complete support for all table formats including Apache Iceberg, Hive, and Delta Lake. When is a migration the wrong choice?

Systems 130

Systems Data Lake High Quality Data Google Cloud 130