Data Engineering Podcast

NOVEMBER 26, 2019

Summary The team at Sentry has built a platform for anyone in the world to send software errors and events. Go to dataengineeringpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today. What did the previous system look like?

Data Warehouse 100

Data Warehouse Building PostgreSQL Kafka 100

Snowflake

OCTOBER 30, 2023

We are excited to announce the general availability of Snowflake Event Tables for logging and tracing, an essential feature to boost application observability and supportability for Snowflake developers. Developers can then query Event Tables to troubleshoot their applications or gain insights into performance and code behavior.

Java 98

Java Scala Hadoop Data Ingestion 98

Confluent

SEPTEMBER 19, 2023

Discover the top 5 best practices for building event-driven architectures using Confluent and AWS Lambda. Learn how to optimize your architecture for scalability, reliability, and performance.

AWS 69

AWS Architecture Building 69

Data Engineering Podcast

SEPTEMBER 17, 2023

Summary A significant amount of time in data engineering is dedicated to building connections and semantic meaning around pieces of information. In this episode Brian Platz explains how JSON-LD can be used as a shared representation of linked data for building semantic data products. Hex brings everything together.

Building 189

Building SQL BI Python 189

The Pragmatic Engineer

JULY 20, 2023

Venture funding is on a downward trend , and we seem to be at the start – or the middle – of a “startup purge” event. This news is hot off the press, publicly announced by Postman and by Akita yesterday, and you are among the early ones to hear about this event. We pivoted to API observability in 2020.

Building 204

Building Programming Language Programming Python 204

Data Engineering Podcast

OCTOBER 1, 2023

Now that AI has reached the level of sophistication seen in the various generative models it is being used to build new ETL workflows. In this episode Jay Mishra shares his experiences and insights building ETL pipelines with the help of generative AI. How can you get the best results for your use case?

Building 162

Building BI SQL Machine Learning 162

Snowflake

SEPTEMBER 12, 2023

Data Cloud Industry Day 2023 is a free virtual event on September 28, 2023, dedicated to what’s possible for you and your industry in the world of data. The event will feature industry-specific sessions hosted by Snowflake’s industry leaders along with exciting customer guest speakers. Attendance at this event is totally FREE.

Cloud 94

Cloud Manufacturing Retail Healthcare 94

Knowledge Hut

APRIL 25, 2023

In this article, we will look at how a scrum master facilitates events such as daily scrum meetings, sprint planning, sprint review, and sprint retrospective meetings. They lead the team through the Scrum process, making sure that everyone is involved and that all events are carried out successfully.

Utilities 98

Utilities Project Certification Process 98

LinkedIn Engineering

OCTOBER 19, 2023

Authors: Bingfeng Xia and Xinyu Liu Background At LinkedIn, Apache Beam plays a pivotal role in stream processing infrastructures that process over 4 trillion events daily through more than 3,000 pipelines across multiple production data centers.

Process 119

Process Lambda Architecture Kafka Machine Learning 119

Confluent

NOVEMBER 30, 2022

Putting Apache Kafka To Use: A Practical Guide to Building an Event Streaming Platform.

Kafka 105

Kafka Building Process 105

Confluent

DECEMBER 7, 2022

Learn how to build a real-time streaming application using Apache Kafka® and.NET producer and consumer clients.

Building 59

Building Kafka Process 59

Data Engineering Podcast

APRIL 16, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. How have the recent breakthroughs in large language models (LLMs) improved your ability to build features in Zenlytic? Who are the target users?

Business Intelligence 147

Business Intelligence Building Data Lake BI 147

Data Engineering Podcast

MARCH 5, 2023

It's supposed to make building smarter, faster, and more flexible data infrastructures a breeze. By bringing all the layers of the data stack together, TimeXtender helps you build data solutions up to 10 times faster and saves you 70-80% on costs. What do you have planned for the future of this topic at Data Council events?

Building 147

Building Database Design Machine Learning Metadata 147

Knowledge Hut

FEBRUARY 29, 2024

Event handling essentially allows the user to interact with a webpage and do something specific when a certain event like a click or a hover happens. When the user interacts with the application, events are fired, for example, mouseover, key press, change event, and so on. React events are written in camelCase.

Python 52

Python Coding Certification IT 52

Knowledge Hut

JUNE 4, 2024

If you've always wanted to learn how to use event binding in Angular, you've come to the right place. This article will discuss Angular's event binding and how to apply it to our Angular project. Events are triggered after these actions are completed. How Does Event Binding Works in Angular ?

Metadata 52

Metadata Coding Project Python 52

Data Engineering Podcast

JANUARY 28, 2024

Summary Monitoring and auditing IT systems for security events requires the ability to quickly analyze massive volumes of unstructured log data. The majority of products that are available either require too much effort to structure the logs, or aren't fast enough for interactive use cases.

Data Lake 147

Data Lake Building High Quality Data AWS 147

Data Engineering Podcast

JUNE 11, 2023

Summary Data engineering is all about building workflows, pipelines, systems, and interfaces to provide stable and reliable data. In this episode Gleb Mezhanskiy shares some valuable advice and insights into how you can build reliable and well-tested data assets with dbt and data-diff. RudderStack also supports real-time use cases.

Project 130

Project Building Data Lake Machine Learning 130

dbt Developer Hub

NOVEMBER 3, 2022

Let’s discuss how to convert events from an event-driven microservice architecture into relational tables in a warehouse like Snowflake. We use Snowflake as our data warehouse where we build dashboards both for internal use and for customers. So our solution was to start using an intentional contract: Events.

Kafka 52

Kafka ETL Tools BI Database 52

Confluent

SEPTEMBER 21, 2022

A deep dive into how microservices work, why it’s the backbone of real-time applications, and how to build event-driven microservices applications with Python and Kafka.

Kafka 98

Kafka Python Building 98

Data Engineering Podcast

OCTOBER 15, 2023

Summary Building streaming applications has gotten substantially easier over the past several years. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. How can you get the best results for your use case?

Process 182

Process Building SQL BI 182

Snowflake

NOVEMBER 15, 2023

BUILD 2023 is where AI gets real. Join our two-day virtual global conference and learn how to build with the app dev innovations you heard about at Snowflake Summit and Snowday. As always, BUILD is created by builders, for builders, so you’ll find plenty to spark your interest and boost your skill set.

Building 114

Building Accessible Accessibility IT 114

Data Engineering Podcast

AUGUST 10, 2020

Summary Event based data is a rich source of information for analytics, unless none of the event structures are consistent. The team at Iteratively are building a platform to manage the end to end flow of collaboration around what events are needed, how to structure the attributes, and how they are captured.

Data Collection 100

Data Collection Data Engineering Data Engineer Education 100

Workfall

SEPTEMBER 26, 2023

Reading Time: 9 minutes In this blog, we will cover: What are Server-Sent Events? Why Stream Data Using Server-Sent Events (SSE)? Hands-On Conclusion What are Server-Sent Events? These events can carry information like notifications, updates, or alerts. What is FastAPI?

Python 85

Python Bytes Coding Project 85

Confluent

FEBRUARY 27, 2019

What is more, as the world adopts the event-driven streaming architecture, how does it fit with serverless? Serverless functions provide a synergistic relationship with event streaming applications; they behave differently with respect to streaming workloads but are both event driven. Event-first FaaS? What is FaaS?

Kafka 109

Kafka AWS Architecture Cloud 109

Workfall

JUNE 27, 2023

In this blog, we’ll explore how to harness the power of Kafka to streamline event streaming within a microservices architecture and unlock its full potential for building scalable and responsive systems. In the event of server failure, Kafka can autonomously restart itself, ensuring uninterrupted data flow. Let’s get started!

Kafka 75

Kafka Architecture AWS Transportation 75

Confluent

DECEMBER 14, 2021

This oft-talked-about architecture has no shortage of blog posts, conference talks, podcasts, and discussions. One thing that you may have found lacking is a concrete guide on precisely […].

Building 129

Building Architecture Data 129

Confluent

NOVEMBER 25, 2020

The event provided an opportunity for participants to learn, build, and showcase […]. As part of this year’s Kafka Summit, the Confluent Community team hosted a community hackathon named Kafkathon 2020.

Kafka 105

Kafka Building 105

Confluent

MAY 9, 2019

Event-first thinking enables us to build a new atomic unit: the event. This model is completely free form, we can build anything provided that we apply mechanical sympathy with the underlying system behavior. Four pillars of event streaming. Building the KPay payment system. Putting it all together.

Kafka 94

Kafka Pipeline-centric Architecture Database-centric 94

Data Engineering Podcast

NOVEMBER 6, 2022

In this episode Nick King discusses how you can be intentional about data creation in your applications and services to reduce the friction and errors involved in building data products and ML applications. What are the organizational patterns that are required to support effective workflows for building data generation capabilities?

Building 130

Building IT Metadata MongoDB 130

Confluent

JUNE 27, 2024

You’re not alone—GenAI apps can be challenging to build. Luckily, taking an event-driven approach can make the process more manageable. Struggling to get your GenAI app off the ground? Learn how.

Building 80

Building Process Management 80

Striim

NOVEMBER 8, 2023

Data Mesh is revolutionizing event streaming architecture by enabling organizations to quickly and easily integrate real-time data, streaming analytics, and more. In this article, we will explore the advantages and limitations of data mesh, while also providing best practices for building and optimizing a data mesh with Striim.

Architecture 52

Architecture Generalist Government Datasets 52

Confluent

AUGUST 2, 2023

Learn how to combine Lambda and Confluent to build event-driven serverless applications.

Building 52

Building AWS Cloud 52

Confluent

SEPTEMBER 11, 2023

See how you can leverage a data streaming platform to build event-driven microservices and do stream processing for faster, better customer onboarding.

Cloud 69

Cloud Building Process Data 69

Pinterest Engineering

JANUARY 4, 2024

In order to build a distributed and replicated service using RocksDB, we built a real time replicator library: Rocksplicator. Maintaining these disparate systems and building common functionality among them was adding a huge overhead to the teams. RocksDB is a single node key value store.

Database 93

Database Building Datasets Relational Database 93

Confluent

NOVEMBER 30, 2022

This guide will contain specific advice on how to go about building an event streaming platform in your organization. This is the second part of our guide on streaming data and Apache Kafka.

Kafka 52

Kafka Building Data Process 52

KDnuggets

NOVEMBER 30, 2021

Learn best practice guidelines for building AI solutions responsibly. Join AI experts from Microsoft and BCG at Put Responsible AI into Practice—a free Azure digital event on December 7.

Building 115

Building 115

Lyft Engineering

JUNE 28, 2023

On the flip side, there was a substantial appetite to build real-time ML systems from developers at Lyft. The Event Driven Decisions capability in particular turned out to be general enough as to be applicable to a wide range of use cases. One key component is the Analytics Event Abstraction layer.

Machine Learning 116

Machine Learning Building Metadata Kafka 116