Simon Späti

MARCH 28, 2025

The database landscape has reached 394 ranked systems across multiple categoriesrelational, document, key-value, graph, search engine, time series, and the rapidly emerging vector databases. As AI applications multiply quickly, vector technologies have become a frontier that data engineers must explore.

Technology 130

Technology PostgreSQL MySQL Database 130

databricks

MAY 27, 2025

Until now, sharing data between enterprise systems often meant complex pipelines, duplication, and lock-in. With Oracles support for Delta Sharing, thats no longer the case.

BI 82

BI Database Systems Data 82

ProjectPro

JUNE 6, 2025

In today's data-driven world, the ability to efficiently manage and manipulate data is a skill that transcends industries. Whether you're a data analyst, a web developer, or a business professional, Structured Query Language, or SQL, is a fundamental tool in your arsenal. This is where our SQLCheat Sheet comes to the rescue.

SQL 49

SQL Database Data Analysis Data Science 49

ProjectPro

JUNE 6, 2025

Explore the world of data analytics with the top AWS databases! Check out this blog to discover your ideal database and uncover the power of scalable and efficient solutions for all your data analytical requirements. Let’s understand more about AWS Databases in the following section.

AWS 40

AWS Database Amazon Web Services MySQL 40

Engineering at Meta

JANUARY 22, 2025

Data lineage is an instrumental part of Metas Privacy Aware Infrastructure (PAI) initiative, a suite of technologies that efficiently protect user privacy. It is a critical and powerful tool for scalable discovery of relevant data and data flows, which supports privacy controls across Metas systems.

Data Warehouse 116

Data Warehouse SQL Programming Language Data 116

ProjectPro

JUNE 6, 2025

With a CAGR of 30%, the NoSQL Database Market is likely to surpass USD 36.50 Businesses worldwide are inclining towards analytical solutions to optimize their decision-making abilities based on data-driven techniques. Two of the most popular NoSQL database services available in the industry are AWS DynamoDB and MongoDB.

NoSQL 40

NoSQL MongoDB Database Amazon Web Services 40

ProjectPro

JUNE 6, 2025

Traditional search methods face computational bottlenecks, especially when dealing with high-dimensional data, leading to slow query times and high resource usage. Want to find similar images in a massive database? This blog explores the FAISS Vector Database, a versatile tool applicable to various applications.

Database 40

Database Datasets Algorithm Hadoop 40

ProjectPro

JUNE 6, 2025

” This blog will align with that vision by exploring what Pinecone Vector Database is, how to use Pinecone Vector Database, and explore a comprehensive Pinecone Vector Database tutorial with a simple example. Table of Contents What is a Pinecone Vector Database? Pinecone is helpful in this situation.

Database 40

Database Project Metadata Unstructured Data 40

ProjectPro

JUNE 6, 2025

Imagine you're a detective trying to identify a suspect from a database of millions of mugshots. Embeddings are numerical representations of data, like images, text, or audio. Each movie in your database has a description or review. Embedding Function: A function that calculates embeddings from raw data.

Database 40

Database Metadata Medical Recruitment 40

KDnuggets

JUNE 10, 2025

By Josep Ferrer , KDnuggets AI Content Specialist on June 10, 2025 in Python Image by Author DuckDB is a fast, in-process analytical database designed for modern data analysis. DuckDB is a free, open-source, in-process OLAP database built for fast, local analytics. Let’s dive in! What Is DuckDB? What Are DuckDB’s Main Features?

Python 114

Python Data Science SQL Machine Learning 114

Precisely

JANUARY 9, 2025

Does the LLM capture all the relevant data and context required for it to deliver useful insights? Not to mention the crazy stories about Gen AI making up answers without the data to back it up!) Are we allowed to use all the data, or are there copyright or privacy concerns? But simply moving the data wasnt enough.

Data Integration 119

Data Integration Hadoop Data Lake Data Warehouse 119

Christophe Blefari

MARCH 15, 2024

Saying mainly that " Sora is a tool to extend creativity " Last point Mira has been mocked and criticised online because as a CTO she wasn't able to say on which public / licensed data Sora has been trained on. Pandera, a data validation library for dataframes, now supports Polars.

Metadata 272

Metadata Software Engineer Software Engineering Data Warehouse 272

Towards Data Science

JANUARY 30, 2025

Building more efficient AI TLDR : Data-centric AI can create more efficient and accurate models. I experimented with data pruning on MNIST to classify handwritten digits. What if I told you that using just 50% of your training data could achieve better results than using the fulldataset? Image byauthor.

Database-centric 106

Database-centric Datasets Data Architecture 106

Towards Data Science

JANUARY 7, 2025

How CDC tools use MySQL Binlog and PostgreSQL WAL with logical decoding for real-time data streaming Photo by Matoo.Studio on Unsplash CDC (Change Data Capture) is a term that has been gaining significant attention over the past few years. Log-based CDC : This method utilizes the databases transaction log to capture every change made.

PostgreSQL 84

PostgreSQL MySQL Bytes Data Lake 84

Monte Carlo

NOVEMBER 26, 2024

Here’s where leading futurist and investor Tomasz Tunguz thinks data and AI stands at the end of 2024—plus a few predictions of my own. 2025 data engineering trends incoming. Small data is the future of AI (Tomasz) 7. The lines are blurring for analysts and data engineers (Barr) 8. Table of Contents 1.

Data Engineering 105

Data Engineering Data Engineer Engineering Unstructured Data 105

Monte Carlo

DECEMBER 18, 2024

As we turn the corner into 2025, were excited to announce that for the 7th quarter in a row, Monte Carlo has been named G2s #1 Data Observability Platform, as well as #1 in the Data Quality category. Knowing our products are helping our customers achieve their data goals means everything to us. Image courtesy of G2.

High Quality Data 93

High Quality Data Database Data Software Engineer 93

Snowflake

APRIL 2, 2025

Data storage has been evolving, from databases to data warehouses and expansive data lakes, with each architecture responding to different business and data needs. Traditional databases excelled at structured data and transactional workloads but struggled with performance at scale as data volumes grew.

Data Lake 97

Data Lake Cloud Storage Metadata Data Warehouse 97

Towards Data Science

DECEMBER 10, 2024

Three Zero-Cost Solutions That Take Hours, NotMonths A data quality certified pipeline. Source: unsplash.com In my career, data quality initiatives have usually meant big changes. Whats more, fixing the data quality issues this way often leads to new problems. Create a custom dashboard for your specific data qualityproblem.

PostgreSQL 75

PostgreSQL Data SQL Python 75

Snowflake

NOVEMBER 26, 2024

Many of our customers — from Marriott to AT&T — start their journey with the Snowflake AI Data Cloud by migrating their data warehousing workloads to the platform. Today we’re focusing on customers who migrated from a cloud data warehouse to Snowflake and some of the benefits they saw. million in cost savings annually.

Data Warehouse 105

Data Warehouse Cloud PostgreSQL Data Lake 105

Snowflake

NOVEMBER 11, 2024

It’s easy these days for an organization’s data infrastructure to begin looking like a maze, with an accumulation of point solutions here and there. Snowflake is committed to doing just that by continually adding features to help our customers simplify how they architect their data infrastructure. Here’s a closer look.

Data Architecture 105

Data Architecture Architecture Data Lake Kafka 105

ProjectPro

JUNE 6, 2025

Today, businesses use traditional data warehouses to centralize massive amounts of raw data from business operations. Since data needs to be accessible easily, organizations use Amazon Redshift as it offers seamless integration with business intelligence tools and helps you train and deploy machine learning models using SQL commands.

Data Pipeline 64

Data Pipeline AWS Project Building 64

databricks

JUNE 11, 2025

Events Data + AI Summit Data + AI World Tour Data Intelligence Days Event Calendar Blog and Podcasts Databricks Blog Explore news, product announcements, and more Databricks Mosaic Research Blog Discover the latest in our Gen AI research Data Brew Podcast Let’s talk data! REGISTER Ready to get started?

Entertainment 95

Entertainment Manufacturing Consulting Retail 95

Snowflake

JUNE 10, 2025

Data transformations are the engine room of modern data operations — powering innovations in AI, analytics and applications. As the core building blocks of any effective data strategy, these transformations are crucial for constructing robust and scalable data pipelines. This puts data engineers in a critical position.

Data Pipeline 89

Data Pipeline SQL Python Building 89

The Pragmatic Engineer

OCTOBER 17, 2024

The current database includes 2,000 server types in 130 regions and 340 zones. Storing data: data collected is stored to allow for historical comparisons. Results are stored in git and their database, together with benchmarking metadata. Visualizing the data: the frontend that allows querying of live and historic data.

Cloud 326

Cloud Metadata AWS Cloud Computing 326

Snowflake

JANUARY 15, 2025

Over the last three geospatial-centric blog posts, weve covered the basics of what geospatial data is, how it works in the broader world of data and how it specifically works in Snowflake based on our native support for GEOGRAPHY , GEOMETRY and H3. But there is so much more you can do with geospatial data in your Snowflake account!

Transportation 97

Transportation BI Database-centric Metadata 97

ProjectPro

JUNE 6, 2025

Data modeling is a crucial skill for every big data professional, but it can be challenging to master. So, if you are preparing for a data modelling interview, you have landed on the right page. We have compiled the top 50 data modelling interview questions and answers from beginner to advanced levels. billion by 2028.

Data Warehouse 53

Data Warehouse NoSQL PostgreSQL Relational Database 53

ProjectPro

JUNE 6, 2025

Discover 50+ Azure Data Factory interview questions and answers for all experience levels. A report by ResearchAndMarkets projects the global data integration market size to grow from USD 12.24 A report by ResearchAndMarkets projects the global data integration market size to grow from USD 12.24 billion in 2020 to USD 24.84

Data Lake 60

Data Lake Metadata SQL Datasets 60

Towards Data Science

DECEMBER 16, 2024

Agentic AI, small data, and the search for value in the age of the unstructured datastack. Heres where leading futurist and investor Tomasz Tunguz thinks data and AI stands at the end of 2024plus a few predictions of myown. 2025 data engineering trends incoming. Search: tools that leverage a corpus of data to answer questions 3.

Unstructured Data 80

Unstructured Data Data Food Data Engineering 80

Data Engineering Weekly

MAY 25, 2025

Dagster Components is now here Components provides a modular architecture that enables data practitioners to self-serve while maintaining engineering quality. Understanding this fact will help data tools break new ground with the advancement of AI agents. and Lite 2.0) to pinpoint drop-offs and high retention sections.

Data Engineering 62

Data Engineering Data Engineer Engineering PostgreSQL 62

Jesse Anderson

FEBRUARY 11, 2025

Semih is a researcher and entrepreneur with a background in distributed systems and databases. He then pursued his doctoral studies at Stanford University, delving into the complexities of database systems. Dont forget to subscribe to my YouTube channel to get the latest on Unapologetically Technical!

Computer Science 130

Computer Science Database Design Software Engineer Software Engineering 130

ProjectPro

JUNE 6, 2025

Data is often referred to as the new oil, and just like oil requires refining to become useful fuel, data also needs a similar transformation to unlock its true value. This transformation is where data warehousing tools come into play, acting as the refining process for your data. Why Choose a Data Warehousing Tool?

Data Storage 58

Data Storage PostgreSQL Data Warehouse AWS 58

Seattle Data Guy

NOVEMBER 8, 2024

Whether it was moving data from a local database instance to S3 or some other data storage layer. As… Read more The post What Is AWS DMS And Why You Shouldn’t Use It As An ELT appeared first on Seattle Data Guy. It was interesting to see AWS DMS used in this manner. But it’s not what DMS was built for.

AWS 130

AWS IT Data Storage Database 130

ProjectPro

JUNE 6, 2025

In the thought process of making a career transition from ETL developer to data engineer job roles? Read this blog to know how various data-specific roles, such as data engineer, data scientist, etc., differ from ETL developer and the additional skills you need to transition from ETL developer to data engineer job roles.

Data Engineer 59

Data Engineer Data Engineering Engineering ETL Tools 59

ProjectPro

JUNE 6, 2025

Are you a data science enthusiast looking to enhance your Python Flask skills? Check out these exciting python flask projects that will help you apply your Flask knowledge to solve real-world data science challenges. Here is the list of the best Python Flask projects ideal for data experts. This is where Python Flask comes in.

Data Science 69

Data Science Python Project Google Cloud 69