Thu.Dec 19, 2024

article thumbnail

Queues in Apache Kafka®: Enhancing Message Processing and Scalability

Confluent

Queue support in Apache Kafka 4.0, enabled by share groups, lets you accommodate traditional queue-type workloads through cooperative consumption.

Kafka 129
article thumbnail

Introducing Configurable Metaflow

Netflix Tech

David J. Berg * , David Casler ^, Romain Cledat * , Qian Huang * , Rui Lin * , Nissan Pow * , Nurcan Sonmez * , Shashank Srikanth * , Chaoying Wang * , Regina Wang * , Darin Yu * *: Model Development Team, Machine Learning Platform ^: Content Demand ModelingTeam A month ago at QConSF, we showcased how Netflix utilizes Metaflow to power a diverse set of ML and AI use cases , managing thousands of unique Metaflow flows.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

How to Use Docker for Local Development Environments

KDnuggets

Using Docker for local development brings stability, flexibility, and ease of management of the environment. No matter what operating system you're using. Learn how to use Docker on Windows, Linux, and macOS to simplify your development setup, from creating your first container to managing complex environments with Docker Compose.

Systems 94
article thumbnail

Philadelphia Union: Streamlining MLS Roster Planning with GenAI

databricks

Staying competitive in Major League Soccer (MLS) demands building and maintaining a strong squad through strategic roster planning and smart, effective navigation of.

article thumbnail

Apache Airflow® Best Practices for ETL and ELT Pipelines

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

article thumbnail

6 Ways to Improve Your Predictive Models in Data Science

KDnuggets

Whether you aim for building the perfect image classifier, sales predictor, or price estimator, these six pracitcal tips and insights will help you get there!

article thumbnail

Databricks Assistant Year in Review

databricks

Since its launch in 2023, Databricks Assistant has grown to hundreds of thousands of monthly users, including developers at major enterprises like Rivian.

IT 71

More Trending

article thumbnail

Key Takeaways from AWS re:Invent 2024

Cloudera

AWS re:Invent is one of my favorite trade shows. It is one of the biggest technology conferences of the year and is an opportunity to have hundreds of conversations with customers and prospects, listen to their priorities and challenges, hopes, and give them a Cloudera tote bag or a pair of orange sunglasses. What follows is a collection of just a few things I learned and observed during my week in Las Vegas.

AWS 55
article thumbnail

Elevating Global Health with Databricks and The Virtue Foundation

databricks

Introduction Databricks has joined forces with the Virtue Foundation through Databricks for Good, a grassroots initiative providing pro bono professional services to drive.

article thumbnail

December 2024 Top Ten (by Dagster Labs)

Data Council

Hey Data Council-ers! I'm Pedram Navid, Chief Dashboard Officer at Dagster Labs, the modern data orchestrator for data engineers building data platforms. I'm excited to share some recent articles I've had my eye on these past few weeks.

article thumbnail

OLTP vs OLAP: What are the Major Differences Between the Two?

Hevo

Data is utilized in almost all domains today, from designing products/services to delivering top-notch customer services. There are two approaches to processing data for business operations: OLAP and OLTP. OLAP is used for complex querying using a structured language, while OLTP facilitates real-time online transactions at scale.

article thumbnail

Apache Airflow®: The Ultimate Guide to DAG Writing

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!