Streaming Data from the Universe with Apache Kafka
Confluent
JUNE 13, 2019
For alert rates of millions per night, scientists need a more structured data format for automated analysis pipelines. After researching formats—and reading about Confluent’s suggestion of using Avro with Kafka —we settled on using Avro, an open source, JSON-based binary format, for serializing the data in the alert messages.
Let's personalize your content