Towards Data Science

JANUARY 16, 2024

Result: Companies started to sell pre-configured data warehouses as products. The concept of `Data Marts` was introduced. Image by the author 2004 to 2010 — The elephant enters the room New wave of applications emerged — Social Media, Software observability, etc. New data formats emerged — JSON, Avro, Parquet, XML etc.

Data Warehouse 64

Data Warehouse Data Architecture Relational Database NoSQL 64

Zalando Engineering

FEBRUARY 28, 2018

For the sake of comparison, let’s recap the Hadoop way of working: Hadoop saves intermediate states to disk and communicates over a network. The processes that run the computation and store data of your application are executors: Returns computed data to the driver. Provides in memory storage for cached RDD’s.

Data Analysis 40

Data Analysis Hadoop Datasets SQL 40

ProjectPro

JUNE 6, 2015

Solocal has taken big data to the next stage of BI by designing a novel vision of BI with the open source distributed computing framework Hadoop. It replaced its traditional BI structure by integrating big data and Hadoop."-April The goal of BI is to create intelligence through Data. So what is BI?

Business Intelligence 40

Business Intelligence Big Data BI Hadoop 40

Knowledge Hut

DECEMBER 26, 2023

The job of a data engineer is to develop models using machine learning to scan, label and organize this unstructured data. This process helps convert the unstructured data into structured data, which can easily be collected and interpreted using analytical tools.

Data Engineer 98

Data Engineer Data Engineering Engineering Pipeline-centric 98