Data Engineering Podcast

MAY 13, 2021

If you want to build a warehouse that gives you both control and flexibility then you might consider building on top of the venerable PostgreSQL project. In this episode Thomas Richter and Joshua Drake share their advice on how to build a production ready data warehouse with Postgres.

PostgreSQL 100

PostgreSQL Data Warehouse Building MySQL 100

Hevo

AUGUST 27, 2024

Building and managing effective data pipelines is becoming more important due to the growing demand for data-based technologies. Connecting Airflow to a robust database like MySQL further enhances its capabilities. Connecting Airflow to a robust database like MySQL further enhances its capabilities. Moreover, […]

MySQL 52

MySQL Data Pipeline Database Data Engineering 52

Data Engineering Podcast

DECEMBER 14, 2020

They share the journey that they went through to build a scalable and maintainable system for web scraping, how to make it reliable and resilient to errors, and the lessons that they learned in the process. This was a great conversation about real world experiences in building a successful data-oriented business.

Data Analytics 100

Data Analytics Building MySQL Data 100

Knowledge Hut

NOVEMBER 22, 2023

MySQL Database Administrators makes Netflix binging, booking an Uber ride, and shopping on Amazon possible. MySQL database administrator salary ranges from USD 45,000 to USD 150,000+ based on the individual's skills and experiences. Who is a MySQL Database Administrator?

MySQL 52

MySQL Database Banking Certification 52

Knowledge Hut

DECEMBER 21, 2023

That is why we have compiled a MySQL tools list to consider in 2024. From powerful query builders to intuitive user interfaces, the top picks are designed to get the most out of your MySQL databases. You can also enroll for the MySQL online course to deepen your knowledge in this fierce market competition. What is MySQL?

MySQL 52

MySQL PostgreSQL SQL Programming Language 52

Snowflake

JUNE 13, 2024

Snowflake is launching native integrations with some of the most popular databases, including PostgreSQL and MySQL. Ingest and transform easily in a single system without having to stitch solutions together or build additional data pipelines to move data around. Additionally, developers have the option to build their own connectors.

Data Ingestion 115

Data Ingestion MySQL PostgreSQL Data Pipeline 115

Data Engineering Podcast

DECEMBER 25, 2022

Summary Five years of hosting the Data Engineering Podcast has provided Tobias Macey with a wealth of insight into the work of building and operating data systems at a variety of scales and for myriad purposes. Learning: there is likely going to be some painful pieces missing, especially around metadata, as you build out your platform.

Building 100

Building Metadata Business Intelligence Data Lake 100

Data Engineering Podcast

SEPTEMBER 11, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Systems 100

Systems Metadata Building MongoDB 100

Data Engineering Podcast

SEPTEMBER 25, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Building 100

Building Metadata MongoDB MySQL 100

Data Engineering Podcast

SEPTEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Building 100

Building Metadata MongoDB MySQL 100

Pinterest Engineering

NOVEMBER 18, 2024

Introducing Generic CDC To address these challenges, we have decided to build a Generic CDC solution based on Red Hat Debezium (™). While the open source Debezium connector, such as the MySQL connector, works seamlessly for a single shard, the challenge lies in making it compatible with our distributed databases. or its affiliates.

Kafka 50

Kafka MySQL Database Software Engineer 50

Hevo

JULY 12, 2024

Building a data lake for reporting, analytics, and machine learning needs has become general practice. Data lakes allow us to ingest data from multiple sources in their raw formats in real time. This will enable us to scale any data size and save time in defining its schema and transformations. This blog describes a simpler […]

Data Lake 52

Data Lake MySQL Building Machine Learning 52

ThoughtSpot

OCTOBER 27, 2023

Every business that analyzes their operational (or transactional) data needs to build a custom data pipeline involving several batch or streaming jobs to extract transactional data from relational databases , transform it, and load it into the data warehouse. For example, “what is the qtysold over period of time?”

Building 97

Building MySQL Data Warehouse SQL 97

Data Engineering Podcast

OCTOBER 23, 2022

In this episode field CTO Manjot Singh shares his experiences as an early user of MySQL and MariaDB and explains how the suite of products being built on top of the open source foundation address the growing needs for advanced storage and analytical capabilities. Mention the podcast to get a free "In Data We Trust World Tour" t-shirt.

Database 100

Database MySQL Cloud MongoDB 100

Knowledge Hut

NOVEMBER 16, 2023

JSP certification JSP is a Web page-building tool that allows for dynamic content. DBA – MySQL – SQL Server In this highly competitive as well as dynamic Software/IT industry, there is one course the one course, which is very popular and can give you a stable career, DBA.

Technology 98

Technology MySQL MongoDB Google Cloud 98

Data Engineering Podcast

DECEMBER 11, 2022

Planetscale is a serverless option for your MySQL workloads that lets you focus on your applications without having to worry about managing the database or fight with differences between development and production. RudderStack helps you build a customer data platform on your warehouse or data lake.

Database 100

Database MySQL Data Lake MongoDB 100

Christophe Blefari

JULY 26, 2024

Building Pinterest Canvas, a text-to-image foundation model — Great article about creating a image generation model for product backgrounds. Multimodal RAG pipeline — A notebook explaining how you can index and build a RAG on deck slides. This articles tries to explain why and what to do.

MySQL 130

MySQL Data PostgreSQL Datasets 130

Knowledge Hut

OCTOBER 27, 2023

Furthermore, project-based learning contributes to building a compelling portfolio that demonstrates your expertise and captivates potential employers. Undertaking real-life projects equips you with a deep understanding of programming languages, tools, and frameworks, preparing you to face intricate challenges and devise efficient solutions.

Software Engineering 98

Software Engineering Software Engineer Coding Project 98

Monte Carlo

MAY 9, 2023

That’s why we’re excited to announce the Monte Carlo data observability platform now integrates with Postgres , MySQL , and Microsoft SQL Server databases. Postgres and MySQL are free and open-source, making them especially easy to adopt, and of course they are still a standard part of many applications. Let’s talk.

MySQL 52

MySQL Database SQL Relational Database 52

Christophe Blefari

JUNE 21, 2024

Native CDC for Postgres and MySQL — Snowflake will be able to connect to Postgres and MySQL to natively move data from your databases to the warehouse. Tabular is building a catalog for Apache Iceberg and Tabular employs a few part of Iceberg open-source contributors. But what is doing Tabular? What's Iceberg?

Metadata 147

Metadata Data Warehouse BI MySQL 147

Cloudera

OCTOBER 6, 2020

The latter makes COD accessible to developers who are used to building applications that use MySQL, Postgres, etc. To build this application, we will be using Phoenix, one of the underlying components of COD, along with Flask. Auto-heal – resolve operational problems automatically (coming soon). .

Database 81

Database Building Bytes NoSQL 81

Data Engineering Podcast

JANUARY 22, 2023

Building your own scripts to replicate data from production is time consuming and error-prone. What are the units of composition that you are building to allow for evolution and expansion of your product? Building your own scripts to replicate data from production is time consuming and error-prone.

PostgreSQL 130

PostgreSQL MongoDB MySQL Machine Learning 130

Netflix Tech

DECEMBER 17, 2019

CDC is becoming increasingly popular for use cases that require keeping multiple heterogeneous datastores in sync (like MySQL and ElasticSearch) and addresses challenges that exist with traditional techniques like dual-writes and distributed transactions [3][4]. These conditions are fulfilled by systems like MySQL, PostgreSQL, MariaDB, etc.

MySQL 88

MySQL PostgreSQL Database Transportation 88

Data Engineering Podcast

JUNE 12, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Unstructured Data 130

Unstructured Data MongoDB MySQL Scala 130

Knowledge Hut

DECEMBER 26, 2023

Most Popular Programming Certifications C & C++ Certifications Oracle Certified Associate Java Programmer OCAJP Certified Associate in Python Programming (PCAP) MongoDB Certified Developer Associate Exam R Programming Certification Oracle MySQL Database Administration Training and Certification (CMDBA) CCA Spark and Hadoop Developer 1.

Certification 98

Certification Programming MongoDB R (Programming) 98

Data Engineering Podcast

NOVEMBER 20, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Lake 130

Data Lake Data Ingestion MongoDB MySQL 130

Data Engineering Weekly

JULY 21, 2024

The author highlights key lessons learning building an analytical org from scratch with key highlights, 1. Build reliable data, maintain usable data models, and ensure the data is interpreted correctly for decision-making. Vimeo discusses its Retrieval-Augmented Generation (RAG) system design for building a knowledge management system.

Data Engineering 118

Data Engineering Data Engineer Engineering Generalist 118

Data Engineering Podcast

JUNE 22, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Database 100

Database MySQL PostgreSQL Data Warehouse 100

Data Engineering Podcast

MARCH 25, 2023

The peril is that building a robust and scalable streaming architecture is always more complicated and error-prone than you think it's going to be. It's supposed to make building smarter, faster, and more flexible data infrastructures a breeze. We feel your pain. It ends up being anything but that.

MySQL 130

MySQL Python Architecture Machine Learning 130

Data Engineering Podcast

NOVEMBER 6, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

MongoDB 130

MongoDB MySQL Scala Machine Learning 130

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 130

Metadata MongoDB MySQL Scala 130

Data Engineering Podcast

JUNE 26, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130

Data Engineering Podcast

OCTOBER 30, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Business Intelligence 130

Business Intelligence Metadata MongoDB MySQL 130

Data Engineering Podcast

DECEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. Or tried to understand what a column name means?

Metadata 130

Metadata Business Intelligence Data Lake BI 130

Data Engineering Podcast

AUGUST 6, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Sifflet also offers a 2-week free trial.

Machine Learning 100

Machine Learning Database MySQL MongoDB 100

LinkedIn Engineering

APRIL 4, 2023

It is critical for our engineering leaders to understand how efficiently and effectively their teams are operating to continuously deliver value-added features for our members and build an industry-leading engineering culture. Over the years we’ve created several metrics and dashboards for measuring developer productivity.

MySQL 122

MySQL Datasets Software Engineer Software Engineering 122

Data Engineering Podcast

DECEMBER 28, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode.

Data Lake 130

Data Lake Data Warehouse Data Pipeline MongoDB 130