Confluent

JULY 24, 2019

Using Jaeger tracing, I’ve been able to answer an important question that nearly every Apache Kafka ® project that I’ve worked on posed: how is data flowing through my distributed system? Distributed tracing with Apache Kafka and Jaeger. Example of a Kafka project with Jaeger tracing. What does this all mean?

Kafka 54

Kafka Systems Bytes Project 54

Confluent

APRIL 30, 2019

With the release of Apache Kafka ® 2.1.0, Kafka Streams introduced the processor topology optimization framework at the Kafka Streams DSL layer. In what follows, we provide some context around how a processor topology was generated inside Kafka Streams before 2.1, Kafka Streams topology generation 101.

Kafka 91

Kafka Coding Process Software Engineering 91

Confluent

SEPTEMBER 27, 2019

Today, nearly everyone uses standard data formats like Avro, JSON, and Protobuf to define how they will communicate information between services within an organization, either synchronously through RPC calls or asynchronously through Apache Kafka ® messages. To allow Schema Validation on write, Confluent Server must be schema aware.

Kafka 16

Kafka Data Governance Bytes Government 16

Pinterest Engineering

NOVEMBER 28, 2023

When there is a full GC, it leads to full halt to the data processing pipeline and causes both back-pressure for upstream kafka clusters and cascading failure for downstream TSDB. Pyoung = Seden / Ralloc where Pyoung is the period between young GC, Seden is the size of Eden and Ralloc is the rate of memory allocations (bytes per second).

Kafka 106

Kafka Bytes Architecture Software Engineering 106

Pinterest Engineering

SEPTEMBER 17, 2024

Jeff Xiang | Senior Software Engineer, Logging Platform; Vahid Hashemian | Staff Software Engineer, LoggingPlatform When it comes to PubSub solutions, few have achieved higher degrees of ubiquity, community support, and adoption than Apache Kafka, which has become the industry standard for data transportation at large scale.

Kafka 49

Kafka Bytes Transportation Metadata 49

ProjectPro

JUNE 29, 2021

Your search for Apache Kafka interview questions ends right here! Let us now dive directly into the Apache Kafka interview questions and answers and help you get started with your Big Data interview preparation! How to study for Kafka interview? What is Kafka used for? What are main APIs of Kafka?

Kafka 40

Kafka Big Data Bytes Java 40

Pinterest Engineering

SEPTEMBER 9, 2024

Gokus ingestor component consumes from this Kafka topic and then produces into another kafka topic (partition corresponds to GokuSshard). GokuS consumes from this second Kafka topic and backs up the data intoS3. The GokuS cluster consumes data points from all the kafka topics (i.e. from every namespace).

Database 44

Database Bytes Kafka Software Engineering 44