Uber Engineering

OCTOBER 17, 2018

Uber is committed to delivering safer and more reliable transportation across our global markets. To accomplish this, Uber relies heavily on making data-driven decisions at every level, from forecasting rider demand during high traffic events to identifying and addressing bottlenecks … The post Uber’s Big Data Platform: 100+ Petabytes with Minute Latency appeared first on Uber Engineering Blog.

Big Data 109

Big Data Transportation Data Engineering 109

Data Engineering Podcast

OCTOBER 14, 2018

Summary With the growth of the Hadoop ecosystem came a proliferation of implementations for the Hive table format. Unfortunately, with no formal specification, each project works slightly different which increases the difficulty of integration across systems. The Hive format is also built with the assumptions of a local filesystem which results in painful edge cases when leveraging cloud object storage for a data lake.

Data Lake 100

Data Lake Big Data Cloud Hadoop 100

Rockset

OCTOBER 19, 2018

I love open-source but open-source software for data infrastructure is on the way out. There, I said it. And you might think I've got a screw loose, given the broad adoption of open source today, but hear me out. Yes, open source is ubiquitous in data management today, but the era of open-source innovation is all but over. In the age of public cloud, there is no longer a reason to build or use open source for data infrastructure, and a new category of software I'm labeling open services will ren

MongoDB 40

MongoDB Hadoop Kafka Data Warehouse 40

Zalando Engineering

OCTOBER 17, 2018

The six month journey of the customer inbox multi-disciplinary team The customer inbox multi-disciplinary area operates in the Fashion Store pillar of the Zalando platform organization. The purpose of the Customer Inbox Unit is to serve customers personal and practical fashion messages, through multiple channels, i.e. “Target the customers at the right time, at the right place.

Consulting 40

Consulting Management Engineering IT 40

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture