Rockset

MARCH 27, 2019

You have complex, semi-structured data—nested JSON or XML, for instance, containing mixed types, sparse fields, and null values. It's messy, you don't understand how it's structured, and new fields appear every so often. Organizations will typically build hard-to-maintain ETL pipelines to feed data into their SQL systems.

Raw Data 40

Raw Data SQL NoSQL Datasets 40

Striim

SEPTEMBER 11, 2024

Data pipelines are the backbone of your business’s data architecture. Implementing a robust and scalable pipeline ensures you can effectively manage, analyze, and organize your growing data. Most importantly, these pipelines enable your team to transform data into actionable insights, demonstrating tangible business value.

Data Pipeline 98

Data Pipeline Designing Data Lake Data Warehouse 98

Rockset

JUNE 13, 2019

We love SQL — our mission is to bring fast, real-time queries to messy, semi-structured real-world data and SQL is a core part of our effort. Datagrip , Jupyter , RStudio ) and data exploration / visualization (e.g. A SQL API allows our product to fit neatly into the stacks of our users without any workflow re-architecting.

SQL 52

SQL Structured Data Building Raw Data 52

ThoughtSpot

MARCH 5, 2024

Managing complex data pipelines is a major challenge for data-driven organizations looking to accelerate analytics initiatives. While AI-powered, self-service BI platforms like ThoughtSpot can fully operationalize insights at scale by delivering visual data exploration and discovery, it still requires robust underlying data management.

BI 111

BI Datasets SQL Raw Data 111

Rockset

NOVEMBER 19, 2020

In this blog post, we show how Rockset’s Smart Schema feature lets developers use real-time SQL queries to extract meaningful insights from raw semi-structured data ingested without a predefined schema. This is particularly true given the nature of real-world data.

Structured Data 40

Structured Data SQL NoSQL Raw Data 40

Edureka

APRIL 14, 2025

In today’s data-driven landscape, organizations need robust solutions for managing, analyzing, and visualizing information. Microsoft offers two standout platforms that fulfill these needs, each addressing different stages of the data lifecycle. Its purpose is to simplify data exploration for users across skill levels.

BI 40

BI Business Intelligence Raw Data Retail 40

Precisely

MARCH 9, 2023

As cloud computing platforms make it possible to perform advanced analytics on ever larger and more diverse data sets, new and innovative approaches have emerged for storing, preprocessing, and analyzing information. In this article, we’ll focus on a data lake vs. data warehouse.

Data Lake 58

Data Lake Data Warehouse Hadoop Raw Data 58

Precisely

OCTOBER 5, 2023

How much data is your business generating each day? While answers will vary by organization, chances are there’s one commonality: it’s more data than ever before. But what do you do with all that data? How do you turn that raw data into actionable insights? That’s where data enrichment comes in.

Raw Data 68

Raw Data Insurance Datasets Telecommunication 68

Knowledge Hut

DECEMBER 26, 2023

Data Engineering is typically a software engineering role that focuses deeply on data – namely, data workflows, data pipelines, and the ETL (Extract, Transform, Load) process. What is Data Science? What are the roles and responsibilities of a Data Engineer? What is the need for Data Science?

Data Engineer 98

Data Engineer Data Engineering Engineering Hadoop 98

Knowledge Hut

JANUARY 29, 2024

In today's data-driven world, where information reigns supreme, businesses rely on data to guide their decisions and strategies. However, the sheer volume and complexity of raw data from various sources can often resemble a chaotic jigsaw puzzle. What Is Data Wrangling? Why Is Data Wrangling Important?

Raw Data 52

Raw Data Data Mining Data Preparation Structured Data 52

Knowledge Hut

JANUARY 18, 2024

I’ve often noticed that people use terms like Data Science and Artificial Intelligence ( AI ) interchangeably. The key connection between Data Science and AI is data. Some may argue that AI and Machine Learning fall within the broader category of Data Science , but it's essential to recognize the subtle differences.

Data Science 98

Data Science Deep Learning Business Analyst Data Mining 98

Monte Carlo

AUGUST 25, 2023

Modern companies are ingesting, storing, transforming, and leveraging more data to drive more decision-making than ever before. Data teams need to balance the need for robust, powerful data platforms with increasing scrutiny on costs. But, the options for data storage are evolving quickly. Let’s dive in.

Data Lake 52

Data Lake Data Warehouse Unstructured Data Raw Data 52

Snowflake

MAY 23, 2024

In this edition, we’ll learn why the founders of data tools company TDAA, Andrew Curran and Jon Farr, chose Snowflake as the platform to deliver their app Pancake , as well as the ways they’re effectively leveraging the Snowflake Native App model. For many data sources, the schema of the data source can change without warning.

Data Pipeline 104

Data Pipeline Raw Data Data Schemas Technology 104

The Modern Data Company

MAY 8, 2023

Data lakes, data warehouses, data hubs, data lakehouses, and data operating systems are data management and storage solutions designed to meet different needs in data analytics, integration, and processing. This feature allows for a more flexible exploration of data.

Data Management 52

Data Management Management Data Lake Data Governance 52

The Modern Data Company

MAY 8, 2023

Data lakes, data warehouses, data hubs, data lakehouses, and data operating systems are data management and storage solutions designed to meet different needs in data analytics, integration, and processing. This feature allows for a more flexible exploration of data.

Data Management 52

Data Management Management Data Lake Data Governance 52

The Modern Data Company

MAY 8, 2023

Data lakes, data warehouses, data hubs, data lakehouses, and data operating systems are data management and storage solutions designed to meet different needs in data analytics, integration, and processing. This feature allows for a more flexible exploration of data.

Data Management 52

Data Management Management Data Lake Data Governance 52

U-Next

SEPTEMBER 7, 2022

The terms “ Data Warehouse ” and “ Data Lake ” may have confused you, and you have some questions. Structuring data refers to converting unstructured data into tables and defining data types and relationships based on a schema. What is Data Warehouse? . What is Data Lake? .

Data Lake 52

Data Lake Data Warehouse Unstructured Data Amazon Web Services 52

Sync Computing

NOVEMBER 7, 2024

Data is central to modern business and society. Depending on what sort of leaky analogy you prefer, data can be the new oil , gold , or even electricity. Of course, even the biggest data sets are worthless, and might even be a liability, if they arent organized properly.

Data Lake 52

Data Lake Data Warehouse Business Intelligence Unstructured Data 52

The Modern Data Company

MAY 10, 2023

In our previous post, The Pros and Cons of Leading Data Management and Storage Solutions , we untangled the differences among data lakes, data warehouses, data lakehouses, data hubs, and data operating systems. What factors are most important when building a data management ecosystem?

Data Management 52

Data Management Management Data Lake Data Warehouse 52

The Modern Data Company

MAY 10, 2023

In our previous post, The Pros and Cons of Leading Data Management and Storage Solutions , we untangled the differences among data lakes, data warehouses, data lakehouses, data hubs, and data operating systems. What factors are most important when building a data management ecosystem?

Data Management 52

Data Management Management Data Lake Data Warehouse 52

The Modern Data Company

MAY 10, 2023

In our previous post, The Pros and Cons of Leading Data Management and Storage Solutions , we untangled the differences among data lakes, data warehouses, data lakehouses, data hubs, and data operating systems. What factors are most important when building a data management ecosystem?

Data Management 52

Data Management Management Data Lake Data Warehouse 52

Knowledge Hut

JANUARY 30, 2024

In today's world, where data rules the roost, data extraction is the key to unlocking its hidden treasures. As someone deeply immersed in the world of data science, I know that raw data is the lifeblood of innovation, decision-making, and business progress. What is data extraction?

ETL Tools 52

ETL Tools Database-centric Data Mining Raw Data 52

AltexSoft

MAY 12, 2023

In today’s data-driven world, organizations amass vast amounts of information that can unlock significant insights and inform decision-making. A staggering 80 percent of this digital treasure trove is unstructured data, which lacks a pre-defined format or organization. How much data was generated in a minute in 2013 and 2022.

Unstructured Data 59

Unstructured Data NoSQL Hadoop Data Lake 59

Knowledge Hut

JUNE 28, 2023

In our data-driven world, our lives are governed by big data. The TV shows we watch, the social media we follow, the news we read, and even the optimized routes we take to work are all influenced by the power of big data analytics. The answer lies in the strategic utilization of business intelligence for data mining (BI).

Data Mining 52

Data Mining Business Intelligence BI Structured Data 52

Edureka

AUGUST 2, 2023

Organisations and businesses are flooded with enormous amounts of data in the digital era. Raw data, however, is frequently disorganised, unstructured, and challenging to work with directly. Data processing analysts can be useful in this situation. What Does a Data Processing Analyst Do?

Data Process 52

Data Process Process Data Cleanse Data Mining 52

Towards Data Science

MAY 14, 2024

Dataform enables the application of software engineering best practices such as testing, environments, version control, dependencies management, orchestration and automated documentation to data pipelines. js for data transformations and logic. It is a serverless, SQL workflow orchestration workhorse within GCP.

Data Pipeline 97

Data Pipeline Coding Raw Data Accessibility 97

Hepta Analytics

FEBRUARY 14, 2022

DE Zoomcamp 2.2.1 – Introduction to Workflow Orchestration Following last weeks blog , we move to data ingestion. We already had a script that downloaded a csv file, processed the data and pushed the data to postgres database. This week, we got to think about our data ingestion design. It is called Idempotency.

Data Ingestion 52

Data Ingestion Data Engineer Data Engineering Engineering 52

Grouparoo

JANUARY 11, 2022

When it comes to storing large volumes of data, a simple database will be impractical due to the processing and throughput inefficiencies that emerge when managing and accessing big data. This article looks at the options available for storing and processing big data, which is too large for conventional databases to handle.

Data Lake 52

Data Lake Data Warehouse Unstructured Data Raw Data 52

Knowledge Hut

APRIL 23, 2024

Data Science and Business intelligence are popular terms in every business domain these days. Though both have data as the fundamental aspect, their uses, and operations vary. Data Science is the field that focuses on gathering data from multiple sources using different tools and techniques.

Business Intelligence 52

Business Intelligence Data Science BI Unstructured Data 52

AltexSoft

AUGUST 29, 2023

In 2010, a transformative concept took root in the realm of data storage and analytics — a data lake. The term was coined by James Dixon , Back-End Java, Data, and Business Intelligence Engineer, and it started a new era in how organizations could store, manage, and analyze their data. What is a data lake?

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

ProjectPro

FEBRUARY 16, 2023

ETL is a critical component of success for most data engineering teams, and with teams harnessing it with the power of AWS, the stakes are higher than ever. Data Engineers and Data Scientists require efficient methods for managing large databases, which is why centralized data warehouses are in high demand.

AWS 52

AWS Data Management ETL Tools Management 52

ProjectPro

DECEMBER 7, 2021

Data pipelines are a significant part of the big data domain, and every professional working or willing to work in this field must have extensive knowledge of them. As data is expanding exponentially, organizations struggle to harness digital information's power for different business use cases. What is a Big Data Pipeline?

Data Pipeline 52

Data Pipeline Architecture Kafka AWS 52

Monte Carlo

MAY 28, 2024

A data ingestion architecture is the technical blueprint that ensures that every pulse of your organization’s data ecosystem brings critical information to where it’s needed most. Ensuring all relevant data inputs are accounted for is crucial for a comprehensive ingestion process.

Data Ingestion 52

Data Ingestion Architecture Designing Hadoop 52

Monte Carlo

FEBRUARY 15, 2023

Since the inception of the cloud, there has been a massive push to store any and all data. Cloud data warehouses solve these problems. Belonging to the category of OLAP (online analytical processing) databases, popular data warehouses like Snowflake, Redshift and Big Query can query one billion rows in less than a minute.

Data Warehouse 52

Data Warehouse Unstructured Data AWS Business Intelligence 52

AltexSoft

JUNE 26, 2023

All successful companies do it: constantly collect data. While today’s world abounds with data, gathering valuable information presents a lot of organizational and technical challenges, which we are going to address in this article. What is data collection?

Data Collection 52

Data Collection Machine Learning Unstructured Data Non-relational Database 52

Workfall

JULY 4, 2023

Reading Time: 8 minutes In the world of data engineering, a mighty tool called DBT (Data Build Tool) comes to the rescue of modern data workflows. Imagine a team of skilled data engineers on an exciting quest to transform raw data into a treasure trove of insights. These guards are tests in DBT.

Data Warehouse 69

Data Warehouse SQL Database PostgreSQL 69

AltexSoft

MARCH 14, 2023

As the volume and complexity of data continue to grow, organizations seek faster, more efficient, and cost-effective ways to manage and analyze data. In recent years, cloud-based data warehouses have revolutionized data processing with their advanced massively parallel processing (MPP) capabilities and SQL support.

IT 59

IT Data Warehouse Data Governance Data Lake 59