Data Engineering Podcast

MAY 18, 2020

Summary The PostgreSQL database is massively popular due to its flexibility and extensive ecosystem of extensions, but it is still not the first choice for high performance analytics. If you are trying to get more performance out of your database then this episode is for you! Can you start by explaining what Swarm64 is?

PostgreSQL 100

PostgreSQL Database Data Warehouse Machine Learning 100

Data Engineering Podcast

AUGUST 5, 2018

Summary One of the longest running and most popular open source database projects is PostgreSQL. It is difficult to capture any single facet of this database in a single conversation, let alone the entire surface area, but in this episode Jonathan Katz does an admirable job of it.

PostgreSQL 100

PostgreSQL MySQL Banking Database 100

Rock the JVM

MAY 22, 2024

We’ll also need to insert messages into our database, for this we’ll need another case class, create a message.scala file under domain , and add the following code: package rockthejvm.websockets.domain import java.util.UUID import java.time.LocalDateTime import rockthejvm.websockets.domain.user. object message {. object message {.

PostgreSQL 136

PostgreSQL Scala Database SQL 136

Towards Data Science

DECEMBER 10, 2024

From governance processes to costly tools to dbt implementationdata quality projects never seem to want to besmall. TL;DR Take advantage of old school database tricks, like ENUM data types, and column constraints. And would you believe all of this was available to us since the release of PostgreSQL 6.5 Precise decimal handling.

PostgreSQL 75

PostgreSQL Data Python SQL 75

Data Engineering Podcast

JANUARY 13, 2019

release, how the use cases for timeseries data have proliferated, and how they are continuing to simplify the task of processing your time oriented events. How has the market for timeseries databases changed since we last spoke? How has the market for timeseries databases changed since we last spoke? release of Timescale.

Database 100

Database PostgreSQL SQL MongoDB 100

Data Engineering Podcast

APRIL 28, 2019

However, managing the database layer is still a separate concern. In this episode Tamal Saha explains how the KubeDB project got started, why you might want to run your database with Kubernetes, and how to get started. Can you talk through how KubeDB simplifies the process of deploying and maintaining databases?

Database 100

Database PostgreSQL MongoDB MySQL 100

Workfall

MARCH 28, 2023

This blog will demonstrate to you how Hasura and PostgreSQL can help you accelerate app development and easily launch backends. In this blog, we will cover: GraphQL Hasura PostgreSQL Hands-on Conclusion GraphQL GraphQL is an API query language and runtime for answering queries with existing data. Why Hasura is Fast?

PostgreSQL 75

PostgreSQL Coding MySQL Database 75

Data Engineering Podcast

JUNE 22, 2021

Summary The database is the core of any system because it holds the data that drives your entire experience. Andy Pavlo researches autonomous database systems, and out of that research he created OtterTune to find the optimal set of parameters to use for your specific workload. How does it relate to your work with NoisePage?

Database 100

Database MySQL PostgreSQL Data Warehouse 100

ThoughtSpot

NOVEMBER 5, 2024

In the realm of modern analytics platforms, where rapid and efficient processing of large datasets is essential, swift metadata access and management are critical for optimal system performance. Optimizing the server initialization process for Atlas is vital for maintaining the high availability and performance of the ThoughtSpot system.

Metadata 52

Metadata PostgreSQL Java Database 52

Data Engineering Podcast

MARCH 2, 2020

The ksqlDB project was created to address this state of affairs by building a unified layer on top of the Kafka ecosystem for stream processing. You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management.

Kafka 100

Kafka Process PostgreSQL MySQL 100

Cloudera

AUGUST 10, 2022

Cloudera has a strong track record of providing a comprehensive solution for stream processing. Cloudera Stream Processing (CSP), powered by Apache Flink and Apache Kafka, provides a complete stream management and stateful processing solution. Cloudera Stream Processing Community Edition.

Process 91

Process Kafka PostgreSQL MySQL 91

Data Engineering Podcast

AUGUST 19, 2018

For a substantial number of use cases, the optimal format for storing and querying that information is as a graph, however databases architected around that use case have historically been difficult to use at scale or for serving fast, distributed queries. Interview Introduction How did you get involved in the area of data management?

Database 100

Database PostgreSQL NoSQL Transportation 100

Data Engineering Podcast

AUGUST 6, 2022

For machine learning applications relational models require additional processing to be directly useful, which is why there has been a growth in the use of vector databases. Who is the target audience for this database? What are the use cases for a vector database and similarity search of vector embeddings?

Machine Learning 100

Machine Learning Database MySQL PostgreSQL 100

Data Engineering Podcast

MAY 29, 2022

Singlestore aims to cut down on the number of database engines that you need to run so that you can reduce the amount of copying that is required. By supporting fast, in-memory row-based queries and columnar on-disk representation, it lets your transactional and analytical workloads run in the same database.

Database 100

Database Architecture Data Architecture PostgreSQL 100

Snowflake

JUNE 13, 2024

Snowflake is launching native integrations with some of the most popular databases, including PostgreSQL and MySQL. With other ingestion improvements and our new database connectors, we are smoothing out the data ingestion process, making it radically simple and efficient to bring data to Snowflake.

Data Ingestion 113

Data Ingestion MySQL PostgreSQL Data Pipeline 113

Data Engineering Podcast

JANUARY 29, 2023

Data analysts create reports that are used by the business to understand and direct the business, but the process is very labor and time intensive. Materialize breaks down those barriers with a true cloud-native streaming database - not simply a database that connects to streaming systems.

Business Intelligence 130

Business Intelligence PostgreSQL Building BI 130

DoorDash Engineering

MARCH 7, 2023

We knew we’d be deploying a Docker container to Fargate as well as using an Amazon Aurora PostgreSQL database and Terraform to model our infrastructure as code. Set up a locally running containerized PostgreSQL database. This isn’t necessary for your application, but it definitely speeds up the development process.

AWS 119

AWS PostgreSQL Database SQL 119

Monte Carlo

APRIL 8, 2025

Let’s walk through how to build this system step by step, using PostgreSQL examples to make it real and actionable. Maybe its just your admin team, or maybe one super-paranoid person in IT who guards the database like a dragon guards gold. Step 2: Hunt Down the Sensitive Stuff Now its time to play detective in your database.

PostgreSQL 52

PostgreSQL Medical Database Data 52

dbt Developer Hub

APRIL 19, 2023

Close alignment with actual business processes : Business processes and metrics are modeled and calculated as part of dimensional modeling. Part 1: Setup dbt project and database ​ Step 1: Install project dependencies ​ Before you can get started: You must have either DuckDB or PostgreSQL installed.

Building 145

Building PostgreSQL BI Database 145

Meltano

OCTOBER 19, 2022

PostgreSQL and MySQL are among the most popular open-source relational database management systems (RDMS) worldwide. For all of their similarities, PostgreSQL and MySQL differ from one another in many ways. That’s because MySQL isn’t fully SQL-compliant, while PostgreSQL is.

PostgreSQL 52

PostgreSQL MySQL Database SQL 52

DoorDash Engineering

FEBRUARY 7, 2023

This increased volume of items created high latency and high failure rate in the fulfillment backend mainly caused by database scalability problems. It’s hosted in the PostgreSQL and used to serve item metadata to the Dasher, our name for delivery drivers, during order fulfillment.

PostgreSQL 119

PostgreSQL SQL Retail Database 119

Cloudera

DECEMBER 2, 2021

Also, for other industries like retail, telecom or public sector that deal with large amounts of customer data and operate multi-tenant environments, sometimes with end users who are outside of their company, securing all the data may be a very time intensive process. CDW uses various Azure services to provide the infrastructure it requires.

PostgreSQL 123

PostgreSQL Data Lake Data Warehouse Retail 123

Hevo

JUNE 19, 2024

Many organizations are drawn to PostgreSQL’s robust features, open-source nature, and cost-effectiveness, and hence they look to migrate their data from their existing database to PostgreSQL. In this guide, we’ll discuss the Oracle to PostgreSQL migration process.

PostgreSQL 52

PostgreSQL Database Process Data 52

Zalando Engineering

DECEMBER 1, 2021

This blog post explains to you which tools to use to serve geospatial data from a database system (PostgreSQL) to your web browser. All you need is a database server for the data, a web map application for the frontend and a small service in between to transfer user requests. pg_tileserv is such a solution.

PostgreSQL 52

PostgreSQL Metadata Database SQL 52

Data Engineering Podcast

FEBRUARY 11, 2018

The landscape of time series databases is extensive and oftentimes difficult to navigate. release of PostGreSQL had on the design of the project? Which came first, Timescale the business or Timescale the database, and what is your strategy for ensuring that the open source project and the company around it both maintain their health?

PostgreSQL 100

PostgreSQL NoSQL Google Cloud MongoDB 100

Knowledge Hut

DECEMBER 21, 2023

In the database ecosystem, Postgres is one of the top open-source databases, and one of the most widely used PSQL tools for managing PostgreSQL is pgAdmin. To run PostgreSQL instances on the Azure cloud, Azure offers Azure Database for PostgreSQL. What are PostgreSQL Tools? What are PostgreSQL Tools?

PostgreSQL 52

PostgreSQL MySQL MongoDB SQL 52

Netflix Tech

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Triggering repairs at any time.

MySQL 89

MySQL PostgreSQL Database Transportation 89

Towards Data Science

FEBRUARY 9, 2024

This involves getting data from an API and storing it in a PostgreSQL database. In the second phase, we’ll develop an application that uses a language model to interact with this database. The second article, which will come later, will delve into creating agents using tools like LangChain to communicate with external databases.

Kafka 98

Kafka Data Engineering Data Engineer PostgreSQL 98

Striim

NOVEMBER 11, 2024

These challenges stemmed from a complex and fragmented data environment, which included: Siloed Data Sources : The retailer’s on-premise databases were spread across various locations, creating silos that made it difficult to consolidate and manage data effectively.

Retail 52

Retail Database-centric Google Cloud PostgreSQL 52

Data Engineering Podcast

JANUARY 8, 2023

Materialize breaks down those barriers with a true cloud-native streaming database - not simply a database that connects to streaming systems. How did that factor into your design process for how to structure the dialect extensions for job scheduling? SQL has traditionally been challenging to compose.

PostgreSQL 130

PostgreSQL Data Lake Data Warehouse BI 130

Data Engineering Podcast

JANUARY 15, 2023

Materialize breaks down those barriers with a true cloud-native streaming database - not simply a database that connects to streaming systems. What was your process for identifying the "must have" vs "nice to have" features for launching the platform?

Coding 130

Coding Building PostgreSQL Data Lake 130

Netflix Tech

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Triggering repairs at any time.

MySQL 83

MySQL PostgreSQL Database Transportation 83

Hevo

SEPTEMBER 20, 2023

Amazon RDS, with its support for the PostgreSQL database, is a popular choice for businesses looking for reliable relational database services. However, the increasing need for advanced analytics and large-scale data processing requires migrating data to more efficient platforms like Databricks.

PostgreSQL 52

PostgreSQL Relational Database Database Data 52

Cloudera

MARCH 15, 2023

Writing to a database and sending messages to a message bus is not atomic, which means that if one of these operations fails, the state of the application can become inconsistent. InventoryService) or processing a payment (eg. After the transaction commits, the record will be available for external consumers. PaymentService).

PostgreSQL 91

PostgreSQL Kafka Database Data 91

Rockset

MARCH 1, 2023

Every database built for real-time analytics has a fundamental limitation. When you deconstruct the core database architecture, deep in the heart of it you will find a single component that is performing two distinct competing functions: real-time data ingestion and query serving. So they are not suitable for real-time analytics.

Data Ingestion 144

Data Ingestion Database Architecture SQL 144

Netflix Tech

FEBRUARY 25, 2021

Graphile During early GraphQL exploration efforts, Netflix engineers became aware of the Graphile library for presenting PostgreSQL database objects (tables, views, and functions) as a GraphQL API. Documentation can even be embedded in the database comments such that it displays in the GraphQL schema generated by Graphile.

PostgreSQL 82

PostgreSQL Database Entertainment SQL 82

Zalando Engineering

JULY 29, 2015

Recently my team and I observed in our PostgreSQL databases a sporadic increase in the execution time of stored procedures (see the graph above). To find answers, we tested how different configurations of PostgreSQL influenced the results of the query planner. PostgreSQL also addresses non-uniform distributions.

PostgreSQL 40

PostgreSQL Database Process Systems 40