Data Engineering Podcast

AUGUST 21, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Lambda Architecture 100

Lambda Architecture MongoDB Scala MySQL 100

Rockset

AUGUST 30, 2021

It eliminates the cost and complexity around data preparation, performance tuning and operations, helping to accelerate the movement from batch to real-time analytics. The latest Rockset release, SQL-based rollups, has made real-time analytics on streaming data a lot more affordable and accessible.

SQL 52

SQL Kafka MongoDB MySQL 52

Rockset

MAY 17, 2022

Limitations of NoSQL SQL supports complex queries because it is a very expressive, mature language. Complex SQL queries have long been commonplace in business intelligence (BI). And when systems such as Hadoop and Hive arrived, it married complex queries with big data for the first time. This is intentionally not their forte.

SQL 52

SQL NoSQL Hadoop MongoDB 52

Data Engineering Podcast

SEPTEMBER 11, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Data Engineering Podcast

JUNE 12, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Unstructured Data 130

Unstructured Data MongoDB Scala MySQL 130

Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

MongoDB 100

MongoDB Google Cloud Scala MySQL 100

Data Engineering Podcast

OCTOBER 30, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

OCTOBER 16, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Lake 100

Data Lake Food MongoDB Scala 100

Data Engineering Podcast

JULY 24, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

MongoDB 100

MongoDB Scala MySQL Data Lake 100

Data Engineering Podcast

JULY 3, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Great Expectations, Soda SQL, etc.)

Data Integration 100

Data Integration MongoDB Scala MySQL 100

Edureka

AUGUST 23, 2024

Current and up-to-date data helps enhance the efficiency of services, improve customer experiences, and drive innovation. Data Ingestion Data from different streams, such as applications, sensors, etc., Enabling Data Access Once the data processing is complete, the real-time data is available in the data stream.

AWS 52

AWS Kafka Amazon Web Services Medical 52

Data Engineering Podcast

JUNE 5, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Metadata 130

Metadata MongoDB Scala MySQL 130

Data Engineering Podcast

JULY 17, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

OCTOBER 2, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

IT 100

IT Food PostgreSQL MongoDB 100

Data Engineering Podcast

AUGUST 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Metadata 100

Metadata MongoDB Scala MySQL 100

DareData

JANUARY 30, 2023

Our goal is to help data scientists better manage their models deployments or work more effectively with their data engineering counterparts, ensuring their models are deployed and maintained in a robust and reliable way. DigDag: An open-source orchestrator for data engineering workflows. Stanford's Relational Databases and SQL.

Data Engineering 52

Data Engineering Data Engineer NoSQL Engineering 52

Data Engineering Podcast

SEPTEMBER 25, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Hospitality 100

Hospitality Food MongoDB Scala 100

Rockset

JANUARY 9, 2023

Technical Challenges Our original data infrastructure was built around an on-premises MongoDB database that ingested and stored all user transaction data. If we didn’t have the right indexes already built, or the SQL query wasn’t optimized, the MySQL query would inevitably drag or time out. It took just one week.

MySQL 52

MySQL MongoDB Recruitment Data Ingestion 52

Data Engineering Podcast

AUGUST 28, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Building 100

Building MongoDB Scala MySQL 100

Data Engineering Podcast

JULY 31, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Analysis 100

Data Analysis MongoDB Algorithm Scala 100

Data Engineering Podcast

JULY 10, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

OCTOBER 9, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

AUGUST 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Machine Learning 100

Machine Learning Pipeline-centric Database-centric MongoDB 100

Data Engineering Podcast

AUGUST 6, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Machine Learning 100

Machine Learning Database MySQL PostgreSQL 100

Data Engineering Podcast

JUNE 26, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Management 100

Data Management Management MongoDB Scala 100

Striim

OCTOBER 11, 2024

Striim supported American Airlines by implementing a comprehensive data pipeline solution to modernize and accelerate operations. To achieve this, the TechOps team implemented a real-time data hub using MongoDB, Striim, Azure, and Databricks to maintain seamless, large-scale operations.

Data Pipeline 52

Data Pipeline MongoDB Unstructured Data Data Lake 52

Data Engineering Podcast

SEPTEMBER 4, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

MongoDB 100

MongoDB Scala MySQL Machine Learning 100

AltexSoft

JUNE 26, 2023

Read our article on Hotel Data Management to have a full picture of what information can be collected to boost revenue and customer satisfaction in hospitality. While all three are about data acquisition, they have distinct differences. Structured data is modeled to be easily searchable and occupy minimal storage space.

Data Collection 52

Data Collection Machine Learning Unstructured Data Non-relational Database 52

Knowledge Hut

APRIL 25, 2023

It allows data engineers to easily build and customize data models and create interactive dashboards and reports. Looker also provides an SQL-based interface for querying and analyzing data, which makes it easy for data engineers to integrate with existing tools and applications.

Data Engineering 52

Data Engineering Data Engineer Engineering Google Cloud 52

Rockset

MARCH 15, 2021

While there’s typically some amount of data engineering required here, there are ways to minimize it. For example, instead of denormalizing the data, you could use a query engine that supports joins. This will avoid unnecessary processing during data ingestion and reduce the storage bloat due to redundant data.

MongoDB 40

MongoDB Data Ingestion Analytics Application Kafka 40

Rockset

NOVEMBER 28, 2022

We’ve already built CDC-based data connectors for many common sources: DynamoDB , MongoDB , and more. For some background, when you ingest data into Rockset you can specify a SQL query, called an ingest transformation , that is evaluated on your source data.

Data Warehouse 52

Data Warehouse PostgreSQL MongoDB SQL 52

Rockset

MAY 12, 2022

However, these databases tend to sacrifice support for complex SQL queries at any scale. Instead, these database makers have offloaded complex analytics onto application code and their developers, who have neither the skills nor the time to constantly update queries as data sets evolve. One layer processes batches of historic data.

Analytics Application 52

Analytics Application Lambda Architecture Hadoop Database 52

AltexSoft

JULY 29, 2022

To facilitate data ingestion, there are Apache Flume aggregating log data from multiple servers and Apache Sqoop designed to transport information between Hadoop and relational (SQL) databases. The most common language for data analysis is SQL but barebone Hadoop doesn’t support it.

Hadoop 59

Hadoop Big Data Google Cloud NoSQL 59

Rockset

OCTOBER 4, 2022

They possess distributed architectures that allow for scalability to handle performance or data volume requirements. Both offer SQL support and are capable of ingesting streaming data from Kafka. This means users need to configure their streams to batch data ahead of loading into ClickHouse.

MySQL 52

MySQL Kafka Aggregated Data Architecture 52

ProjectPro

AUGUST 24, 2021

Data Engineering Project for Beginners If you are a newbie in data engineering and are interested in exploring real-world data engineering projects, check out the list of data engineering project examples below. This big data project discusses IoT architecture with a sample use case.

Data Engineering 52

Data Engineering Data Engineer Coding Project 52