Jesse Anderson

DECEMBER 12, 2022

They created MapReduce and GFS in 2004. Doug Cutting took those papers and created Apache Hadoop in 2005. They were the first companies to commercialize open source big data technologies and pushed the marketing and commercialization of Hadoop. Hadoop was hard to program, and Apache Hive came along in 2010 to add SQL.

Data Engineer 130

Data Engineer Data Engineering Engineering Hadoop 130

Towards Data Science

JANUARY 16, 2024

My personal take on justifying the existence of Data Mesh A senior stakeholder at one my projects mentioned that they wanted to decentralise their data platform architecture and democratise data across the organisation. Result: Hadoop & NoSQL frameworks emerged. The concept of `Data Marts` was introduced.

Data Warehouse 63

Data Warehouse Data Architecture Relational Database NoSQL 63

Cloudera

DECEMBER 12, 2018

I was part of this migration project, and then after undergrad, I went on to be a software engineer for a utility company, who was using DB2 on the mainframe and migrating to Oracle on Unix. Greg Rahn: Toward the end of that eight-year stint, I saw this thing coming up called Hadoop and an engine called Hive. Michael Moreno: Nice!

Data Warehouse 41

Data Warehouse Relational Database Hadoop Database 41

Zalando Engineering

FEBRUARY 28, 2018

For the sake of comparison, let’s recap the Hadoop way of working: Hadoop saves intermediate states to disk and communicates over a network. In fact, in a 2004 mapReduce research paper the designer states that key-value pairs is a key choice in designing mapReduce. Provides in memory storage for cached RDD’s.

Data Analysis 40

Data Analysis Hadoop Datasets SQL 40

ProjectPro

JUNE 6, 2015

Solocal has taken big data to the next stage of BI by designing a novel vision of BI with the open source distributed computing framework Hadoop. It replaced its traditional BI structure by integrating big data and Hadoop."-April For example, say we get a project on analyzing Twitter data. So what is BI? So what is BI?

Business Intelligence 40

Business Intelligence Big Data BI Hadoop 40

Knowledge Hut

DECEMBER 26, 2023

Facebook It is a social media platform created originally by Mark Zuckerberg for college students in 2004. Most of the Data engineers working in the field enroll themselves in several other training programs to learn an outside skill, such as Hadoop or Big Data querying, alongside their Master's degree and PhDs.

Data Engineer 98

Data Engineer Data Engineering Engineering Pipeline-centric 98