Sat.Mar 31, 2018 - Fri.Apr 06, 2018

article thumbnail

Scaling Uber’s Apache Hadoop Distributed File System for Growth

Uber Engineering

Three years ago, Uber Engineering adopted Hadoop as the storage ( HDFS ) and compute ( YARN ) infrastructure for our organization’s big data analysis. This analysis powers our services and enables the delivery of more seamless and reliable user … The post Scaling Uber’s Apache Hadoop Distributed File System for Growth appeared first on Uber Engineering Blog.

Hadoop 109
article thumbnail

ThreatStack: Data Driven Cloud Security with Pete Cheslock and Patrick Cable - Episode 25

Data Engineering Podcast

Summary Cloud computing and ubiquitous virtualization have changed the ways that our applications are built and deployed. This new environment requires a new way of tracking and addressing the security of our systems. ThreatStack is a platform that collects all of the data that your servers generate and monitors for unexpected anomalies in behavior that would indicate a breach and notifies you in near-realtime.

article thumbnail

The Perks of Being in a Hackathon

Zalando Engineering

How stepping out of our comfort zone led to a hackathon victory Zalando Tech doesn't just put on hackathons , we love to attend them too! Here, we catch up with software engineers, Lisa Knolle and Izabela Bratovic about their time at #picturepunk. At the end of last year we took part in a hackathon. We came to this decision for the sake of exposing ourselves to new experiences, new people, and new technologies.

Media 52
article thumbnail

Collaborating for systems change

Cloudera

[This blog by Claudia Juech, Executive Director of the Cloudera Foundation, highlights how increased collaboration between different philanthropic organizations can result in better funding for critical social issues. By adopting technologies like machine learning and analytics, these organizations can optimize how they spend funds for social good. She highlights how this technology can impact society.].

Systems 46
article thumbnail

Apache Airflow® Best Practices for ETL and ELT Pipelines

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

article thumbnail

Recap of Hadoop News for March 2018

ProjectPro

News on Hadoop - March 2018 Kyvos Insights to Host Session "BI on Big Data - With Instant Response Times" at the Gartner Data and Analytics Summit 2018.PRNewswire.com, March 5, 2018 The big data analytics company Kyos Insights announced that it will host a session “BI on Big Data - With Instant Response Times” at the Gartner Data and Analytics Summit 2018 conference in Grapevine, Texas from March 5-8, 2018.

Hadoop 40