Uber Engineering

MARCH 8, 2019

First introduced by Uber in November 2018, Peloton , a unified resource scheduler, manages resources across distinct workloads, combining separate compute clusters. Peloton is designed for web-scale companies like Uber with millions of containers and tens of thousands of nodes. … The post Open Sourcing Peloton, Uber’s Unified Resource Scheduler appeared first on Uber Engineering Blog.

Engineering 109

Engineering Designing Management Architecture 109

Data Engineering Podcast

MARCH 4, 2019

Summary Customer analytics is a problem domain that has given rise to its own industry. In order to gain a full understanding of what your users are doing and how best to serve them you may need to send data to multiple services, each with their own tracking code or APIs. To simplify this process and allow your non-engineering employees to gain access to the information they need to do their jobs Segment provides a single interface for capturing data and routing it to all of the places that you

Google Cloud 100

Google Cloud Kafka Media Data Warehouse 100

Confluent

MARCH 4, 2019

Cloud is one of the key drivers for innovation. Innovative companies experiment with data to come up with something useful. It usually starts with the opening of a firehose that continuously broadcasts tons of events before they start mining it to create music out of simply noise. Today, companies from all around the world are witnessing an explosion of event generation coming from everywhere, including their own internal systems.

Cloud 89

Cloud Banking Kafka NoSQL 89

Netflix Tech

MARCH 5, 2019

Netflix’s engineering culture is predicated on Freedom & Responsibility, the idea that everyone (and every team) at Netflix is entrusted with a core responsibility and they are free to operate with freedom to satisfy their mission. This freedom allows teams and individuals to move fast to deliver on innovation and feel responsible for quality and robustness of their delivery.

Cloud 74

Cloud Building Amazon Web Services Metadata 74

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Uber Engineering

MARCH 7, 2019

Reliability engineering teams at Uber build the tools, libraries, and infrastructure that enable engineers to operate our thousands of microservices reliably at scale. At its essence, reliability engineering boils down to actively preventing outages that affect the mean time between … The post Using Machine Learning to Ensure the Capacity Safety of Individual Microservices appeared first on Uber Engineering Blog.

Machine Learning 105

Machine Learning Engineering Building IT 105

Teradata

MARCH 7, 2019

Teradata CEO Oliver Ratzesberger celebrates International Women's Day.

Building 69

Building 69

Advancing Analytics: Data Engineering

MARCH 5, 2019

Today is my first day back in the office after attending SQLBits in Manchester last week. SQLBits is the UK's largest Microsoft Data Platform conference. What makes this event special is that it is organised for the community, by the community and is not for profit - All the proceeds go in to funding the event and in particular, the awesome Friday night party.

Deep Learning 52

Deep Learning Big Data Machine Learning Data Science 52

Domino Data Lab: Data Engineering

MARCH 2, 2019

Paco Nathan covers recent research on data infrastructure as well as adoption of machine learning and AI in the enterprise.

Machine Learning 52

Machine Learning Deep Learning Data Data Science 52

Teradata

MARCH 4, 2019

Find out how data scientists can contribute to solving real-life business problems, as well as how Teradata Vantage can help data scientists enable a better ROI for their company.

Data 49

Data 49

Netflix Tech

MARCH 6, 2019

FlameScope sub-second heatmap visualization. Less than a year ago, FlameScope was released as a proof of concept for a new profile visualization. Since then, it helped us, and many other users, to easily find and fix performance issues, and allowed us to see patterns that we had never noticed before in our profiles. As a tool, FlameScope was limited.

Programming Language 40

Programming Language Coding Programming Cloud 40

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Netflix Tech

MARCH 6, 2019

MezzFS?—?Mounting object storage in Netflix’s media processing platform By Barak Alon (on behalf of Netflix’s Media Cloud Engineering team) MezzFS (short for “Mezzanine File System”) is a tool we’ve developed at Netflix that mounts cloud objects as local files via FUSE. It’s used extensively in our media processing platform, which includes services like Archer and runs features like video encoding and title image generation on tens of thousands of Amazon EC2 instances.

Media 88

Media Bytes Process Accessibility 88

Netflix Tech

MARCH 7, 2019

Design Principles for Mathematical Engineering in Experimentation Platform at Netflix Jeffrey Wong, Senior Modeling Architect, Experimentation Platform Colin McFarland, Director, Experimentation Platform At Netflix, we have data scientists coming from many backgrounds such as neuroscience, statistics and biostatistics, economics, and physics; each of these backgrounds has a meaningful contribution to how experiments should be analyzed.

Designing 70

Designing Engineering Data Science Python 70