Data Engineering Podcast

OCTOBER 29, 2023

Summary Databases are the core of most applications, whether transactional or analytical. In recent years the selection of database products has exploded, making the critical decision of which engine(s) to use even more difficult. What are the aspects of the database market that keep you interested as a VP of product?

Database 189

Database SQL BI Machine Learning 189

Data Engineering Podcast

APRIL 14, 2024

Summary Databases come in a variety of formats for different use cases. The default association with the term "database" is relational engines, but non-relational engines are also used quite widely. Can you describe what constitutes a NoSQL database? If you were to start from scratch today, what database would you build?

Non-relational Database 147

Non-relational Database Relational Database Database Designing 147

KDnuggets

OCTOBER 18, 2023

Cloud databases have made it easier and cheaper to develop enterprise-level applications, offering flexibility, convenience, and standard database functionality. See what KDnuggets recommends.

Database 141

Database Cloud IT Data Engineering 141

Data Engineering Podcast

MARCH 17, 2024

Summary A significant portion of data workflows involve storing and processing information in database engines. Your host is Tobias Macey and today I'm welcoming back Gleb Mezhanskiy to talk about how to reconcile data in database environments Interview Introduction How did you get involved in the area of data management?

Database 147

Database Data Lake High Quality Data Data Workflow 147

Striim

NOVEMBER 19, 2024

SQL2Fabric Mirroring is a new fully managed service offered by Striim to mirror on premise SQL Databases. It’s a collaborative service between Striim and Microsoft based on Fabric Open Mirroring that enables real-time data replication from on-premise SQL Server databases to Azure Fabric OneLake.

SQL 52

SQL Database Data Warehouse Data Pipeline 52

KDnuggets

SEPTEMBER 13, 2024

Learn how to install ClickHouse DBMS, create a database, and run SQL queries using native and Python clients.

Database 123

Database SQL Python Data Engineering 123

KDnuggets

JUNE 28, 2024

Explore the world of modern databases that are fast, secure, and cost-efficient, designed to tackle large-scale and diverse data challenges.

Database 138

Database SQL Designing Data 138

KDnuggets

MARCH 5, 2024

Looking to learn SQL and databases to level up your data science skills? Learn SQL, database internals, and much more with these free university courses.

SQL 144

SQL Database Data Science Data 144

Data Engineering Podcast

FEBRUARY 25, 2024

Summary Building a database engine requires a substantial amount of engineering effort and time investment. In this episode he explains how he used the combination of Apache Arrow, Flight, Datafusion, and Parquet to lay the foundation of the newest version of his time-series database.

Database 162

Database Technology Data Lake High Quality Data 162

KDnuggets

NOVEMBER 2, 2023

We will explore their use cases, key features, performance metrics, supported programming languages, and more to provide a comprehensive and unbiased overview of each database.

Database 134

Database Programming Language Programming 134

KDnuggets

NOVEMBER 17, 2023

The top vector databases are known for their versatility, performance, scalability, consistency, and efficient algorithms in storing, indexing, and querying vector embeddings for AI applications.

Database 143

Database Algorithm 143

KDnuggets

FEBRUARY 28, 2024

Exploring the limitless possibilities of AI and making it context-aware.

Deep Learning 156

Deep Learning Database IT 156

Simon Späti

MARCH 28, 2025

The database landscape has reached 394 ranked systems across multiple categoriesrelational, document, key-value, graph, search engine, time series, and the rapidly emerging vector databases. What fundamental differences exist between AI-focused vector databases and analytical vector engines like DuckDB or DataFusion?

Technology 130

Technology PostgreSQL MySQL Database 130

Analytics Vidhya

FEBRUARY 22, 2023

Introduction Data replication is also known as database replication, which is copying data to ensure that all information remains consistent across all data resources in real-time. data replication is like a safety net that keeps your information safe from disappearing or falling through the cracks. In most cases, data alters.

Database 276

Database Data NoSQL Datasets 276

Jesse Anderson

FEBRUARY 11, 2025

Semih is a researcher and entrepreneur with a background in distributed systems and databases. He then pursued his doctoral studies at Stanford University, delving into the complexities of database systems. Dont forget to subscribe to my YouTube channel to get the latest on Unapologetically Technical!

Computer Science 130

Computer Science Database Design Software Engineer Software Engineering 130

The Pragmatic Engineer

OCTOBER 17, 2024

The current database includes 2,000 server types in 130 regions and 340 zones. Results are stored in git and their database, together with benchmarking metadata. Databases:  SQLite files used to publish data Duck DB to query these files in the public APIs Cockroach DB : used to collect and store historical data.

Cloud 332

Cloud AWS Metadata Cloud Computing 332

Analytics Vidhya

MARCH 7, 2023

Introduction Apache Cassandra is a NoSQL database management system that is open-source and distributed. It is meant to handle massive volumes of data across many commodity servers while maintaining high availability with no single point of failure. Facebook created Cassandra, which ultimately became an Apache Software Foundation project.

NoSQL 259

NoSQL Database Project Systems 259

Seattle Data Guy

APRIL 15, 2025

Everyone is talking about AI, chatbots, LLMs, vector databases, and whether your data stack is “AI-ready.” Planning out your data infrastructure in 2025 can feel wildly different than it did even five years ago. The ecosystem is louder, flashier, and more fragmented.

Database 130

Database Data IT Big Data 130

The Pragmatic Engineer

NOVEMBER 21, 2023

  We didn’t build our applications in neat containers, but in bulky monoliths which commingled business, database, backend, and frontend logic. We dabbled in network engineering, database management, and system administration. Our deployments were initially manual. were in english only.

Engineering 345

Engineering Bytes Cloud Computing AWS 345

Analytics Vidhya

JANUARY 30, 2023

Whether we are analyzing IoT data streams, managing scheduled events, processing document uploads, responding to database changes, etc. Azure functions allow developers […] The post How to Develop Serverless Code Using Azure Functions? appeared first on Analytics Vidhya.

Coding 245

Coding Database Management Process 245

Simon Späti

FEBRUARY 26, 2025

Traditionally, answering this question would require expensive GIS (Geographic Information Systems) software or complex database setups. ” This seemingly simple question requires analyzing competitor locations, population density, traffic patterns, and demographicsall spatial data.

Database 130

Database Data Engineering Data Engineer Accessible 130

Snowflake

APRIL 2, 2025

Data storage has been evolving, from databases to data warehouses and expansive data lakes, with each architecture responding to different business and data needs. Traditional databases excelled at structured data and transactional workloads but struggled with performance at scale as data volumes grew.

Data Lake 97

Data Lake Metadata Cloud Storage Data Warehouse 97

Analytics Vidhya

MARCH 1, 2023

Introduction Cassandra is an Apache-developed free and open-source distributed NoSQL database management system. It manages huge volumes of data across many commodity servers, ensures fault tolerance with the swift transfer of data, and provides high availability with no single point of failure.

NoSQL 230

NoSQL Java Database Big Data 230

Analytics Vidhya

FEBRUARY 6, 2023

Especially while working with databases, it is often considered a good practice to follow a design pattern. Introduction A design pattern is simply a repeatable solution for problems that keep on reoccurring. The pattern is not an actual code but a template that can be used to solve problems in different situations.

Accessible 239

Accessible Accessibility Python Database 239

Analytics Vidhya

MARCH 14, 2023

This generated data is stored in the database and will maintain it. SQL is a structured query language used to read and write these databases. Introduction In today’s world, technology has increased tremendously, and many people are using the internet. This results in the generation of so much data daily.

SQL 204

SQL Database Technology Data 204

Seattle Data Guy

NOVEMBER 8, 2024

Whether it was moving data from a local database instance to S3 or some other data storage layer. Recently, I’ve encountered a few projects that used AWS DMS, which is almost like an ELT solution. It was interesting to see AWS DMS used in this manner. But it’s not what DMS was built for.

AWS 130

AWS IT Data Storage Database 130

databricks

DECEMBER 21, 2023

We are excited to announce that Gartner has recognized Databricks as a Leader for a third consecutive year in the 2023 Gartner® Magic.

Database 145

Database Systems Cloud Management 145

Christophe Blefari

JANUARY 11, 2025

Materialization of data warehouse layers — What are the consideration for every materialisation you should pick in your data warehouse layer: view, tables, schema vs. databases, etc. The best code is the code you never wrote — Every line of code is a form of debt—a liability that must be maintained and understood.

Data 130

Data Data Warehouse Coding Programming Language 130

Analytics Vidhya

JUNE 24, 2023

As a new developer, a robust data modeling foundation is crucial for effectively working with databases. Introduction In the era of data-driven decision-making, having accurate data modeling tools is essential for businesses aiming to stay competitive.

Database 218

Database Utilities Data Data Science 218

Analytics Vidhya

FEBRUARY 21, 2023

Introduction SQL is a database programming language created for managing and retrieving data from Relational databases like MySQL, Oracle, and SQL Server. SQL(Structured Query Language) is the common language for all databases. In other terms, SQL is a language that communicates with databases.

SQL 172

SQL MySQL Programming Language Relational Database 172

Monte Carlo

DECEMBER 18, 2024

In the Winter 2025 awards, we were recognized as #1 across the entire Data Observability, Data Quality, and Database Monitoring categories , as well as #1 in the Enterprise Relationship Index. All told, we’re featured in 71 reports, including the #1 spot in 31 reports, and we received 44 badges, including eight Leader badges!

Database 93

Database High Quality Data Data Software Engineer 93

Analytics Vidhya

FEBRUARY 28, 2023

Introduction Structured Query Language is a powerful language to manage and manipulate data stored in databases. After being introduced in the 70s, it has become the standard querying language for relational databases. […] The post Step-by-Step Roadmap to Learn SQL in 2023 appeared first on Analytics Vidhya.

SQL 230

SQL Relational Database Data Science Database 230

Pinterest Engineering

NOVEMBER 18, 2024

Change Data Capture (CDC) is a crucial technology that enables organizations to efficiently track and capture changes in their databases. In this blog post, we’ll explore what CDC is, why it’s important, and our journey of implementing Generic CDC solutions for all online databases at Pinterest. What is Change Data Capture?

Kafka 50

Kafka MySQL Database Software Engineer 50

Analytics Vidhya

OCTOBER 7, 2024

Microsoft’s open-source tool, Drasi, addresses this need by effortlessly detecting, monitoring, and responding to data changes across platforms, including relational and graph databases.

Systems 175

Systems Management Database Data 175