KDnuggets

FEBRUARY 27, 2024

Want to become a data analyst? This free beginner-friendly data analyst bootcamp is all you need.

Data 158

Data Data Science 158

Confluent

FEBRUARY 27, 2024

Apache Kafka 3.7 introduces updates to the Consumer rebalance protocol, an official Apache Kafka Docker image, JBOD support in Kraft-based clusters, and more!

Kafka 140

Kafka 140

KDnuggets

FEBRUARY 27, 2024

Your Ultimate Learning Companion.

Python 144

Python 144

DoorDash Engineering

FEBRUARY 27, 2024

We reviewed the architecture of our global search at DoorDash in early 2022 and concluded that our rapid growth meant within three years we wouldn’t be able to scale the system efficiently, particularly as global search shifted from store-only to a hybrid item-and-store search experience. Our analysis identified Elasticsearch as our architecture’s primary bottleneck.

Engineering 130

Engineering Systems Designing Architecture 130

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Waitingforcode

FEBRUARY 27, 2024

Apache Spark Structured Streaming relies on the micro-batch pattern which evaluates the same query in each execution. That's only a high level vision, though. Under-the-hood, there are many other interesting things that happen.

130

130

databricks

FEBRUARY 27, 2024

Introduction Apache Spark™ Structured Streaming is a popular open-source stream processing platform that provides scalability and fault tolerance, built on top of the S.

Process 128

Process Data Engineer Data Engineering Engineering 128

Snowflake

FEBRUARY 27, 2024

Snowflake Marketplace is a vibrant resource, with hundreds of providers offering thousands of ready-to-try or ready-to-buy third-party data sets, applications and services. Many of these providers make their products available on Snowflake Marketplace for Snowflake customers to purchase — and they use our integrated Marketplace Monetization capabilities to simplify the process and speed up procurement and sales cycles.

Bytes 104

Bytes Electronics Banking Data 104

databricks

FEBRUARY 27, 2024

This blog was written in collaboration with Tim Sedlak, Senior Solutions Architect at Stardog In healthcare and life sciences, accuracy is everything. That's.

Healthcare 97

Healthcare 97

Confluent

FEBRUARY 27, 2024

Learn about key new features added to our serverless Flink offering this year including Topic Actions, Terraform support, and expansion into GCP and Azure.

Cloud 75

Cloud Process 75

databricks

FEBRUARY 27, 2024

In this blog post, we discuss Fine-Grained RLHF, a framework that enables training and learning from reward functions that are fine-grained in two.

85

85

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Monte Carlo

FEBRUARY 27, 2024

If you’re a data engineer experiencing GenAI-induced whiplash, you’re not alone. On one hand, everyone’s talking about whether GenAI’s not-insignificant data engineering skills are going to automate away their jobs. On the other, business leaders are realizing that slapping a quick ChatGPT integration into their product isn’t going to cut it — and they’re looking to data engineers for a better solution.

Data Engineer 59

Data Engineer Data Engineering Engineering High Quality Data 59

Cloudyard

FEBRUARY 27, 2024

Read Time: 2 Minute, 18 Second In the ever-evolving world of data management, streamlining processes and ensuring data freshness are crucial. Snowflake, a cloud-based data platform, provides a revolutionary feature known as Dynamic Tables. This blog post showcases a novel approach combining Snowflake’s Event Logging and Dynamic Tables to automate the creation and population of dynamic tables based on Copy operations.

Data Warehouse 52

Data Warehouse Utilities Architecture Data Management 52

Confluent

FEBRUARY 27, 2024

Learn more about Sergio’s experience at Confluent as a solutions engineer and what he appreciates most about his role in customer success and his team.

Engineering 52

Engineering 52

Knowledge Hut

FEBRUARY 27, 2024

Introduction No other industry perhaps is characterized by a change as much as the software industry. While every segment of society and more so the industry, goes through change, the pace and magnitude of change in the software industry are leaps and bounds ahead of all other segments. This magnitude of change can be unsettling as just when one thinks that one has cracked the secret of success, the rug is pulled from under the feet by the change and success formulae have to be reinvented all ov

Project 52

Project Utilities Designing Technology 52

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

DataKitchen

FEBRUARY 27, 2024

Your LLM Needs a Data Journey: A Comprehensive Guide for Data Engineers The rise of Large Language Models (LLMs) such as GPT-4 marks a transformative era in artificial intelligence, heralding new possibilities and challenges in equal measure. LLMs have the potential to revolutionize how we interact with data, automate processes, and extract insights.

Data Engineer 49

Data Engineer Data Engineering Engineering Unstructured Data 49