Hevo

AUGUST 27, 2024

Over the past few years, MongoDB has become a popular choice for NoSQL Databases. Catering to real-time processing requirements, MongoDB introduced a powerful feature to track data […] With the rise of modern data tools, real-time data processing is no longer a dream.

MongoDB 52

MongoDB Kafka NoSQL Data 52

ProjectPro

JUNE 6, 2025

Use Kafka for real-time data ingestion, preprocess with Apache Spark, and store data in Snowflake. This architecture shows that simulated sensor data is ingested from MQTT to Kafka. The data in Kafka is analyzed with Spark Streaming API and stored in a column store called HBase.

Data Engineering 59

Data Engineering Data Engineer Project Engineering 59

Confluent

JUNE 8, 2020

We are excited to announce the preview release of the fully managed MongoDB Atlas source and sink connectors in Confluent Cloud, our fully managed event streaming service based on Apache […].

MongoDB 77

MongoDB Cloud Management Kafka 77

AltexSoft

OCTOBER 21, 2022

Kafka can continue the list of brand names that became generic terms for the entire type of technology. In this article, we’ll explain why businesses choose Kafka and what problems they face when using it. In this article, we’ll explain why businesses choose Kafka and what problems they face when using it. What is Kafka?

Kafka 93

Kafka Hadoop ETL Tools Java 93

Rockset

SEPTEMBER 25, 2020

According to over 40,000 developers, MongoDB is the most popular NOSQL database in use right now. From a developer perspective, MongoDB is a great solution for supporting modern data applications. This blog post will look at three of them: tailing MongoDB with an oplog, using MongoDB change streams, and using a Kafka connector.

MongoDB 52

MongoDB Kafka Database NoSQL 52

Confluent

JUNE 11, 2019

Here in part 4 of the Spring for Apache Kafka Deep Dive blog series, we will cover: Common event streaming topology patterns supported in Spring Cloud Data Flow. Create and manage event streaming pipelines, including a Kafka Streams application using Spring Cloud Data Flow.

Kafka 86

Kafka Cloud Java MongoDB 86

Confluent

JUNE 5, 2020

Most organisations maintain fleets, a collection of vehicles put to use for day-to-day operations. Telcos use a variety of vehicles including cars, vans, and trucks for service, delivery, and maintenance. […].

MongoDB 108

MongoDB Cloud Management Kafka 108

Rockset

JUNE 9, 2020

In the course of implementing the Rockset connector to MongoDB , we did a fair amount of research on the MongoDB user experience, both online and through user interviews. Sharding What is MongoDB Sharding and the Best Practices? This was a recurring theme we heard when speaking with MongoDB users.

MongoDB 52

MongoDB NoSQL Kafka Database 52

Rockset

JUNE 16, 2020

MongoDB.live took place last week, and Rockset had the opportunity to participate alongside members of the MongoDB community and share about our work to make MongoDB data accessible via real-time external indexing. We would be responsible for building and maintaining pipelines from these sources to MongoDB.

MongoDB 52

MongoDB Data Lake PostgreSQL Kafka 52

Rockset

OCTOBER 1, 2020

The data sources available include: users (MongoDB): Core customer data such as name, age, gender, address. online_orders (MongoDB): Online purchase data including product details and delivery addresses. instore_orders (MongoDB): In-store purchase data again including product details and store location. SELECT users.id

MongoDB 40

MongoDB Kafka Building Data Warehouse 40

Data Engineering Podcast

SEPTEMBER 11, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Email hosts@dataengineeringpodcast.com ) with your story. Email hosts@dataengineeringpodcast.com ) with your story.

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Data Engineering Podcast

AUGUST 6, 2017

Contact Information Email @rywalker on Twitter Links Astronomer Kiss Metrics Segment Marketing tools chart Clickstream HIPAA FERPA PCI Mesos Mesos DC/OS Airflow SSIS Marathon Prometheus Grafana Terraform Kafka Spark ELK Stack React GraphQL PostGreSQL MongoDB Ceph Druid Aries Vault Adapter Pattern Docker Kinesis API Gateway Kong AWS Lambda Flink Redshift (..)

MongoDB 100

MongoDB PostgreSQL Kafka Data Pipeline 100

Data Engineering Podcast

MAY 27, 2018

It’s easy to get one started but difficult to manage as new requirements are added and greater scalability becomes necessary.

Data Pipeline 100

Data Pipeline MongoDB Scala Kafka 100

Monte Carlo

OCTOBER 31, 2024

Data Engineering Tools Data engineers need to be comfortable using essential tools for data pipeline management and workflow orchestration, including Apache Kafka, Apache Spark, Airflow, Dagster, dbt, and many more. Get familiar with data warehouses, data lakes, and data lakehouses, including MongoDB , Cassandra, BigQuery, Redshift and more.

Data Engineering 52

Data Engineering Data Engineer Engineering Unstructured Data 52

Confluent

MAY 12, 2020

Imagine you’ve got a stream of data; it’s not “big data,” but it’s certainly a lot. Within the data, you’ve got some bits you’re interested in, and of those bits, […].

Kafka 141

Kafka Building Big Data MongoDB 141

Data Engineering Podcast

AUGUST 21, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs.

Lambda Architecture 100

Lambda Architecture MongoDB Scala MySQL 100

Confluent

JULY 13, 2021

In Data Science projects, we distinguish between descriptive analytics and statistical models running in production. Overall, these can be seen as one process. You start with analyzing historical data to […].

Data Analysis 138

Data Analysis Kafka Data Science Data 138

Confluent

DECEMBER 31, 1969

Connect any database to MongoDB using Confluent's cloud-native data streaming platform. Modernize any database, build streaming data pipelines, and empower real-time data in minutes.

MongoDB 52

MongoDB Database Kafka Data Pipeline 52

ProjectPro

JUNE 6, 2025

In addition, to extract data from the eCommerce website, you need experts familiar with databases like MongoDB that store reviews of customers. You can use big-data processing tools like Apache Spark , Kafka , and more to create such pipelines. However, it is not straightforward to create data pipelines.

Data Pipeline 40

Data Pipeline Architecture Kafka Data Lake 40

Confluent

MAY 6, 2021

Today, Confluent is announcing the general availability (GA) of the fully managed MongoDB Atlas Source and MongoDB Atlas Sink Connectors within Confluent Cloud. Now, with just a few simple clicks, […].

MongoDB 52

MongoDB Cloud Management Kafka 52

Data Engineering Podcast

JANUARY 13, 2019

Contact Info Ajay @acoustik on Twitter LinkedIn Mike LinkedIn Website @michaelfreedman on Twitter Timescale Website Documentation Careers timescaledb on GitHub @timescaledb on Twitter Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Database 100

Database PostgreSQL SQL MongoDB 100

Data Engineering Podcast

OCTOBER 5, 2021

How did the constraint of supporting the Kafka API influence your implementation strategy for transaction semantics? How did the constraint of supporting the Kafka API influence your implementation strategy for transaction semantics? What are the elements of streaming systems that make atomic transactions a complex problem?

Engineering 100

Engineering MongoDB Kafka Data Lake 100

ProjectPro

JUNE 6, 2025

Step 2: Master Big Data Tools and Technologies Familiarize yourself with the core Big Data technologies and frameworks, such as Hadoop , Apache Spark, and Apache Kafka. Apache Kafka: Kafka is a distributed event streaming platform. Learning Scala is valuable when you focus on real-time data processing and analytics.

Big Data 40

Big Data Hadoop Scala NoSQL 40

Data Engineering Podcast

FEBRUARY 11, 2018

Links Timescale PostGreSQL Citus Timescale Design Blog Post MIT NYU Stanford SDN Princeton Machine Data Timeseries Data List of Timeseries Databases NoSQL Online Transaction Processing (OLTP) Object Relational Mapper (ORM) Grafana Tableau Kafka When Boring Is Awesome PostGreSQL RDS Google Cloud SQL Azure DB Docker Continuous Aggregates Streaming Replication (..)

PostgreSQL 100

PostgreSQL NoSQL Google Cloud MongoDB 100

Data Engineering Podcast

MARCH 8, 2021

To help other people find the show please leave a review on iTunes and tell your friends and co-workers Join the community in the new Zulip chat workspace at dataengineeringpodcast.com/chat Links Molecula Pilosa Podcast Episode The Social Dilemma Feature Store Cassandra Elasticsearch Podcast Episode Druid MongoDB SwimOS Podcast Episode Kafka Kafka (..)

IT 100

IT MongoDB Data Warehouse Kafka 100

ProjectPro

JUNE 6, 2025

Prepare for Your Next Big Data Job Interview with Kafka Interview Questions and Answers 5. When should you utilize BigQuery in place of more established databases like MongoDB or MySQL? BigQuery SQL Interview Questions Here are some GCP BigQuery SQL interview questions to help you master your BigQuery skills.

Google Cloud 40

Google Cloud Cloud Storage SQL MongoDB 40

ProjectPro

JUNE 6, 2025

Source Code: Build a Similar Image Finder Top 3 Open Source Big Data Tools This section consists of three leading open-source big data tools- Apache Spark , Apache Hadoop, and Apache Kafka. Additionally, you will learn how to integrate Spark with Kafka and MongoDB. This also boosts Kafka's resilience and prevents server failure.

Big Data Tools 40

Big Data Tools Big Data Hadoop BI 40

Data Engineering Podcast

APRIL 19, 2020

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA Support Data Engineering Podcast Summary Modern applications frequently require access to real-time data, but building and maintaining the systems that make that possible is a complex and time consuming endeavor.

Building 100

Building PostgreSQL MongoDB SQL 100

Data Engineering Podcast

NOVEMBER 20, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Hevo]([link] Are you sick of repetitive, time-consuming ELT work?

Systems 130

Systems Metadata MongoDB Data Pipeline 130

Rockset

DECEMBER 14, 2022

Rockset’s native connector for Amazon Managed Streaming for Apache Kafka (MSK) makes it simpler and faster to ingest streaming data for real-time analytics. Amazon MSK is a fully managed AWS service that gives users the ability to build and run applications using Apache Kafka.

Kafka 52

Kafka MongoDB SQL AWS 52

Rockset

AUGUST 25, 2020

In Part One , we discussed how to first identify slow queries on MongoDB using the database profiler, and then investigated what the strategies the database took doing during the execution of those queries to understand why our queries were taking the time and resources that they were taking.

MongoDB 40

MongoDB NoSQL SQL Relational Database 40

Rockset

SEPTEMBER 14, 2021

We’re introducing a new Rockset Integration for Apache Kafka that offers native support for Confluent Cloud and Apache Kafka, making it simpler and faster to ingest streaming data for real-time analytics. With the Kafka Integration, users no longer need to build, deploy or operate any infrastructure component on the Kafka side.

Kafka 52

Kafka SQL MongoDB Computer Science 52

Confluent

AUGUST 19, 2019

It points to best practices for anyone writing Kafka Connect connectors. In a nutshell, the document states that sources and sinks are verified as Gold if they’re functionally equivalent to Kafka Connect connectors. Over the years, we’ve since seen wide adoption of Kafka Connect.

Programming 15

Programming Kafka Database-centric MongoDB 15

ProjectPro

JUNE 6, 2025

Prepare for Your Next Big Data Job Interview with Kafka Interview Questions and Answers 2. Is MongoDB better than PostgreSQL in terms of performance? It's difficult to determine whether MongoDB is significantly faster than PostgreSQL since database performance depends on numerous parameters.

Data Warehouse 53

Data Warehouse NoSQL PostgreSQL Relational Database 53

Data Engineering Podcast

DECEMBER 4, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs.

Data Lake 100

Data Lake MongoDB MySQL Data Warehouse 100

Data Engineering Podcast

JULY 10, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Email hosts@dataengineeringpodcast.com ) with your story. Email hosts@dataengineeringpodcast.com ) with your story.

Metadata 100

Metadata MongoDB MySQL BI 100

Data Engineering Podcast

SEPTEMBER 25, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs.

Food 100

Food MongoDB Scala MySQL 100