Remove Bytes Remove Java Remove Kafka
article thumbnail

Fault Tolerance in Distributed Systems: Tracing with Apache Kafka and Jaeger

Confluent

Using Jaeger tracing, I’ve been able to answer an important question that nearly every Apache Kafka ® project that I’ve worked on posed: how is data flowing through my distributed system? Distributed tracing with Apache Kafka and Jaeger. Example of a Kafka project with Jaeger tracing. What does this all mean?

Kafka 54
article thumbnail

Kafka Connect Deep Dive – JDBC Source Connector

Confluent

One of the most common integrations that people want to do with Apache Kafka ® is getting data in from a database. The existing data in a database, and any changes to that data, can be streamed into a Kafka topic. Here, I’m going to dig into one of the options available—the JDBC connector for Kafka Connect. Introduction.

Kafka 89
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Getting Started with Rust and Apache Kafka

Confluent

I’ve written an event sourcing bank simulation in Clojure (a lisp build for Java virtual machines or JVMs) called open-bank-mark , which you are welcome to read about in my previous blog post explaining the story behind this open source example. The schemas are also useful for generating specific Java classes. The bank application.

Kafka 18
article thumbnail

Deploying Kafka Streams and KSQL with Gradle – Part 3: KSQL User-Defined Functions and Kafka Streams

Confluent

As discussed in part 2, I created a GitHub repository with Docker Compose functionality for starting a Kafka and Confluent Platform environment, as well as the code samples mentioned below. We used Groovy instead of Java to write our UDFs, so we’ve applied the groovy plugin. jar Zip file size: 5849 bytes, number of entries: 5.

Kafka 89
article thumbnail

Deploying Kafka Streams and KSQL with Gradle – Part 2: Managing KSQL Implementations

Confluent

In part 1 , we discussed an event streaming architecture that we implemented for a customer using Apache Kafka ® , KSQL from Confluent, and Kafka Streams. In part 3, we’ll explore using Gradle to build and deploy KSQL user-defined functions (UDFs) and Kafka Streams microservices. gradlew composeUp. The KSQL pipeline flow.

Kafka 96
article thumbnail

Top 50 Java Interview Questions for Hadoop Developers

ProjectPro

Hiring managers agree that “Java is one of the most in-demand and essential skill for Hadoop jobs. But how do you get one of those hot java hadoop jobs ? You have to ace those pesky java hadoop job interviews artfully. To demonstrate your java and hadoop skills at an interview, preparation is vital.

Java 40
article thumbnail

A Glimpse into the Redesigned Goku-Ingestor vNext at Pinterest

Pinterest Engineering

When there is a full GC, it leads to full halt to the data processing pipeline and causes both back-pressure for upstream kafka clusters and cascading failure for downstream TSDB. Pyoung = Seden / Ralloc where Pyoung is the period between young GC, Seden is the size of Eden and Ralloc is the rate of memory allocations (bytes per second).

Kafka 106