AltexSoft

OCTOBER 21, 2022

Kafka can continue the list of brand names that became generic terms for the entire type of technology. In this article, we’ll explain why businesses choose Kafka and what problems they face when using it. In this article, we’ll explain why businesses choose Kafka and what problems they face when using it. What is Kafka?

Kafka 93

Kafka Hadoop Big Data ETL Tools 93

Cloudera

NOVEMBER 3, 2023

Cloudera was named as IBM’s preferred partner of choice for Real-Time Data Movement using Kafka. As a technology partner in their Cloud Innovation Center in Rome, Italy, we work with Accenture to bring clients tested, cutting-edge, and tailored IT solutions. They have been a Cloudera partner for more than 5 years.

AWS 98

AWS Technology Amazon Web Services Insurance 98

Snowflake

MARCH 2, 2023

As part of this, we are also supporting Snowpipe Streaming as an ingestion method for our Snowflake Connector for Kafka. Now we are able to ingest our data in near real time directly from Kafka topics to a Snowflake table, drastically reducing the cost of ingestion and improving our SLA from 15 minutes to within 60 seconds.

Kafka 126

Kafka Data Ingestion Data Pipeline Cloud Storage 126

Confluent

SEPTEMBER 28, 2023

See the highlights on how the data streaming platform is evolving, why Kafka and Flink are like PB&J, why generative AI needs fresh data, and lots more from the Current 2023 premiere data streaming technology event.

Kafka 75

Kafka Technology Data 75

Confluent

FEBRUARY 27, 2019

What is more, as the world adopts the event-driven streaming architecture, how does it fit with serverless? When it comes to the emerging serverless world, It makes sense to validate how Apache Kafka ® fits in considering that it is mission critical in 90 percent of companies. Event-first FaaS? Event-first FaaS.

Kafka 109

Kafka AWS Architecture Cloud 109

ProjectPro

FEBRUARY 8, 2023

Did you know thousands of businesses, including over 80% of the Fortune 100, use Apache Kafka to modernize their data strategies? Apache Kafka is the most widely used open-source stream-processing solution for gathering, processing, storing, and analyzing large amounts of data. What is Apache Kafka Used For?

Kafka 52

Kafka Banking Healthcare Medical 52

Confluent

MAY 9, 2019

Event-first thinking enables us to build a new atomic unit: the event. Four pillars of event streaming. Pillar 4 – Operational plane: Event logging, DLQs and automation. To read the other articles in this series, see: Journey to Event Driven – Part 1: Why Event-First Thinking Changes Everything.

Kafka 94

Kafka Pipeline-centric Architecture Database-centric 94

Confluent

JUNE 26, 2019

I see this pattern coming up more and more in the field in conjunction with Apache Kafka ®. In these projects, microservice architectures use Kafka as an event streaming platform. Apache Kafka – An event streaming platform for microservices. What technology should you use to build microservice architectures?

Kafka 109

Kafka Designing Architecture Coding 109

Cloudera

SEPTEMBER 13, 2023

Soon after, I became a huge fan of Apache Kafka. Coupled with amazing technology was an amazing team that only grew and improved with the merger with Cloudera. It’s the Apache Software Foundation’s biggest event of the year. The post A Talented Team, Innovative Technology, and The Opportunity to Grow.

Technology 83

Technology Hadoop Kafka Project 83

Data Engineering Podcast

MARCH 2, 2020

Summary Building applications on top of unbounded event streams is a complex endeavor, requiring careful integration of multiple disparate systems that were engineered in isolation. The ksqlDB project was created to address this state of affairs by building a unified layer on top of the Kafka ecosystem for stream processing.

Kafka 100

Kafka Process PostgreSQL MySQL 100

Confluent

MAY 23, 2019

Let us cut to the chase: Kafka Summit London session videos are available! This year’s London event was almost double the size of last year, so I’m especially happy to see these numbers on the page: 2 days of Apache Kafka ®. World-class Kafka keynotes. And if you weren’t there? Well, dig in and start learning! 3 keynotes.

Kafka 109

Kafka Cloud Database Engineering 109

Cloudera

OCTOBER 4, 2022

This blog post will provide guidance to administrators currently using or interested in using Kafka nodes to maintain cluster changes as they scale up or down to balance performance and cloud costs in production deployments. Kafka brokers contained within host groups enable the administrators to more easily add and remove nodes.

Kafka 78

Kafka Data Cloud Big Data 78

Confluent

FEBRUARY 13, 2019

Part 1 of this series discussed why you need to embrace event-first thinking, while this article builds a rationale for different styles of event-driven architectures and compares and contrasts scaling, persistence and runtime models. Event-driven architecture. Event-driven, reactive architecture. Being event first.

Architecture 92

Architecture Programming Kafka Database-centric 92

Confluent

AUGUST 4, 2021

Today, I’m very excited to announce an all-new website dedicated to Apache Kafka®, event streaming, and associated cloud technologies. The site is called Confluent Developer, and it represents a significant […].

Kafka 85

Kafka Cloud Technology IT 85

Christophe Blefari

NOVEMBER 8, 2024

On November 19th, I'm organising with dltHub folks the first dlt Paris community meetup , the event will take place at 42 and will start a 16h. We are very proud and honoured, along with the organising committee, that around 300 people took a paid ticket to the event. Why is Kafka always late?

Data 130

Data Kafka Food Data Preparation 130

Data Engineering Podcast

NOVEMBER 26, 2019

Summary The team at Sentry has built a platform for anyone in the world to send software errors and events. Go to dataengineeringpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today. Closing Announcements Thank you for listening!

Data Warehouse 100

Data Warehouse Building PostgreSQL Kafka 100

Rockset

AUGUST 16, 2022

Streams for Everyone If you have come this far it means you have already considered or are considering using event streaming in your data architecture for the wide variety of benefits it can offer. In either case, both Amazon Kinesis and Apache Kafka can help but which one is the right fit for you and your goals.

Kafka 52

Kafka AWS Cloud Java 52

Scott Logic

APRIL 22, 2024

In my previous blog post I looked at various technologies for sending data asynchronously between services including RabbitMQ, Kafka, AWS EventBridge. I’ve coloured the data entities according to their types and we see there’s a few different patterns like events and state which we’ll discuss in a moment.

AWS 52

AWS Metadata Systems Kafka 52

Knowledge Hut

NOVEMBER 19, 2023

With advances in technology, wearable devices do provide some trace of your health. Importance of Data Science in IoT IoT and Data Science are amongst the most defining technologies of our age. Challenges of Data Science for IoT Any great technology brings along its own unique set of challenges. Here is the broad.

Data Science 52

Data Science Technology Machine Learning Big Data 52

Confluent

FEBRUARY 28, 2019

Only a little more than one month after the first release, we are happy to announce another milestone for our Kafka integration. Today, you can grab the Kafka Connect Neo4j Sink from Confluent Hub. . Neo4j extension – Kafka sink refresher. Then it is really up to you what you want to with the event data.

Kafka 88

Kafka Java Programming Language Data Pipeline 88

Data Engineering Podcast

JULY 1, 2019

Scaling the volume of events that can be processed in real-time can be challenging, so Paul Brebner from Instaclustr set out to see how far he could push Kafka and Cassandra for this use case. Coming up this fall is the combined events of Graphorum and the Data Architecture Summit.

Kafka 100

Kafka Finance Media Architecture 100

Confluent

MAY 13, 2019

In the last year, we’ve experienced enormous growth on Confluent Cloud, our fully managed Apache Kafka ® service. As Confluent Cloud has grown, we’ve noticed two gaps that very clearly remain to be filled in managed Apache Kafka services. Five seconds to Kafka (or, never make another cluster again!).

Kafka 100

Kafka Cloud Building Management 100

Rockset

NOVEMBER 6, 2019

Events are messages that are sent by a system to notify operators or other systems about a change in its domain. With event-driven architectures powered by systems like Apache Kafka becoming more prominent, there are now many applications in the modern software stack that make use of events and messages to operate effectively.

Kafka 40

Kafka Data Lake SQL Hadoop 40

Confluent

JUNE 13, 2019

This data pipeline is a great example of a use case for Apache Kafka ®. Astronomers need to be able to collect, process, characterize, and distribute data on these objects in near real time, especially for time-sensitive events. The case for Apache Kafka. Astronomy in real time.

Kafka 102

Kafka Bytes Data Pipeline Python 102

Confluent

JUNE 27, 2019

Confluent’s clients for Apache Kafka ® recently passed a major milestone—the release of version 1.0. Magnus Edenhill first started developing librdkafka about seven years ago, later joining Confluent in the very early days to help foster the community of Kafka users outside the Java ecosystem. In the v0.11.6

Kafka 106

Kafka Designing Java Coding 106

Data Engineering Podcast

MAY 21, 2023

In this episode David Yaffe and Johnny Graettinger share the story behind the business and technology and how you can start using it today to build a real-time data lake without all of the headache. Stream processing technologies have been around for around a decade. Can you describe what Estuary is and the story behind it?

Data Lake 162

Data Lake Machine Learning Kafka Data Warehouse 162

Confluent

JULY 16, 2019

How cool would it be to build your own burglar alarm system that can alert you before the actual event takes place simply by using a few network-connected cameras and analyzing the camera images with Apache Kafka ® , Kafka Streams, and TensorFlow? Uploading your images into Kafka. Setting up your burglar alarm.

Machine Learning 101

Machine Learning Kafka Java Datasets 101

Christophe Blefari

JULY 13, 2024

AI News 🤖 OpenAI — Always the biggest news provider, announcements or dramas OpenAI was hacked, revealing internal secrets and raising national security concerns — The hacker reached OpenAI’s internal messaging systems early last year, stealing details of how OpenAI's technologies work from employees.

Kafka 130

Kafka AWS Data Database 130

Confluent

FEBRUARY 20, 2019

How many Kafka Summits should there be in a year? Some say there should be one giant event where everybody gathers at once. Others say you should live every day like it’s Kafka Summit. Anyway, let me remind you of the dates and places: Kafka Summit New York – April 2, 2019. Kafka Summit London – May 13-14, 2019.

Kafka 66

Kafka Programming Architecture Technology 66

Data Engineering Weekly

JUNE 10, 2024

[link] Sponsored: DoubleCloud - Production-ready managed Apache Kafka in just about 10 minutes Apache Kafka is the #1 open-source event streaming service, but the challenge of setting up and maintaining clusters can be a dealbreaker for many companies.

Data Engineering 103

Data Engineering Data Engineer Engineering Kafka 103

Pinterest Engineering

NOVEMBER 18, 2024

Change Data Capture (CDC) is a crucial technology that enables organizations to efficiently track and capture changes in their databases. This configuration ensures that if the host goes down due to an EC2® event or any other reason, it will be automatically reprovisioned.

Kafka 46

Kafka MySQL Database Software Engineering 46

Precisely

DECEMBER 28, 2023

Enterprise technology is having a watershed moment; no longer do we access information once a week, or even once a day. Streaming data pipelines, by extension, offer an architecture capable of handling large volumes of data, accommodating millions of events in near real time. But insights derived from day-old data don’t cut it.

Data Pipeline 91

Data Pipeline Building Kafka Big Data 91

Lyft Engineering

NOVEMBER 29, 2023

It also allowed us to optimize for handling time-series data and event data at scale. Real-time Ingestion Events from our real-time analytics pipeline were configured to be sent into our internal Flink application, streamed to Kafka, and written into Druid. ioConfig: Kafka server info, topic names, etc. (ex.

Kafka 104

Kafka Data Ingestion Datasets Architecture 104

Rockset

APRIL 26, 2022

As Kafka Summit is in full swing in London this week and the topic of event streaming is all over my Linkedin feed, I saw a post asking " Is streaming dead? That streaming is rocking and with Kafka Summit this week, I thought it a good time to emphasize the importance of streaming data in today’s modern real-time data stack.

Kafka 52

Kafka Data Warehouse Database Data 52

Scribd Technology

MAY 18, 2021

Streaming data from Apache Kafka into Delta Lake is an integral part of Scribd’s data platform, but has been challenging to manage and scale. We use Spark Structured Streaming jobs to read data from Kafka topics and write that data into Delta Lake tables. To serve this need, we created kafka-delta-ingest.

Kafka 52

Kafka Data Warehouse Bytes Metadata 52

Christophe Blefari

AUGUST 24, 2024

Events ✨ As you may know I'm co-organising the Forward Data Conference on November 25th in Paris. Get tickets for Forward Data Conference Transition to another event. BigTable supports SQL (and this is something) and GCP can run managed Kafka. More details soon in a 2-parts articles I'm writing about it.

BI 130

BI Food Data SQL 130

Cloudera

OCTOBER 17, 2023

Recently, Confluent hosted Current 2023 (formerly Kafka summit) in San Jose on Sept 26th and 27th. With few conferences curating content specific to streaming developers, Current has historically been an important event for anyone trying to keep a pulse on what’s happening in the streaming space. Expecting a product to be GA’d.

Kafka 78

Kafka Database-centric Pipeline-centric Database 78