Data Engineering Podcast

FEBRUARY 18, 2024

Summary A data lakehouse is intended to combine the benefits of data lakes (cost effective, scalable storage and compute) and data warehouses (user friendly SQL interface). Multiple open source projects and vendors have been working together to make this vision a reality. In this episode Dain Sundstrom, CTO of Starburst, explains how the combination of the Trino query engine and the Iceberg table format offer the ease of use and execution speed of data warehouses with the infinite storage and sc

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Rock the JVM

FEBRUARY 18, 2024

by Herbert Kateu 1. Introduction The WebSocket protocol enables persistent two-way communication between a client and a server where packets can be passed in both directions without the need for additional HTTP requests. The specification for this protocol is outlined in RFC 6455. WebSockets are used in applications such as Instant Messaging, Gaming, Simultaneous editing, and stock tickers to mention but a few.

Scala 94

Scala Coding Programming Accessibility 94

Towards Data Science

FEBRUARY 18, 2024

How to Stream and Apply Real-Time Prediction Models on High-Throughput Time-Series Data Photo by JJ Ying on Unsplash Most of the stream processing libraries are not python friendly while the majority of machine learning and data mining libraries are python based. Although the Faust library aims to bring Kafka Streaming ideas into the Python ecosystem, it may pose challenges in terms of ease of use.

Kafka 74

Kafka Python Process Google Cloud 74

Data Engineering Weekly

FEBRUARY 18, 2024

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Visit rudderstack.com to learn more. Editor’s Note: DEWCon Next? Aswin and I started DEWCon as a fun experiment. Midway through organizing the conference, fear washed over us.

Data Engineer 64

Data Engineer Data Engineering Engineering Data 64

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Rock the JVM

FEBRUARY 18, 2024

Learn how to implement WebSockets in Scala with http4s to enable seamless two-way communication between your frontend and backend

Scala 52

Scala 52

Knowledge Hut

FEBRUARY 18, 2024

In the evolving landscape of professional development, the year 2024 marks a pivotal moment as we delve into the significance and purpose of outcome-based upskilling. Traditional training models are undergoing a transformation, emphasizing the need for a more dynamic and responsive approach that focuses on measurable results and real-world applications.

Education 52

Education Technology Designing Data Analytics 52