Data Engineering Podcast

OCTOBER 28, 2018

Summary Jupyter notebooks have gained popularity among data scientists as an easy way to do exploratory analysis and build interactive reports. However, this can cause difficulties when trying to move the work of the data scientist into a more standard production environment, due to the translation efforts that are necessary. At Netflix they had the crazy idea that perhaps that last step isn’t necessary, and the production workflows can just run the notebooks directly.

Scala 100

Scala Python Data Engineering Data Engineer 100

Uber Engineering

OCTOBER 30, 2018

Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware. At Uber, cluster management … The post Peloton: Uber’s Unified Resource Scheduler for Diverse Cluster Workloads appeared first on Uber Engineering Blog.

Engineering 103

Engineering Management Technology Hadoop 103

Zalando Engineering

OCTOBER 31, 2018

Why I advocate a practice of no estimates as a software engineer Before I get to the topic, I would like to clarify one thing: I don’t want to ban estimations generally from software development, as there are good and solid reasons for it. In a nutshell, business needs to be predictable. I want to show a software developer's view on how to reduce or even get rid of endless estimations meetings with doubtful outcomes.

Software Engineer 52

Software Engineer Software Engineering Coding Building 52

Netflix Tech

OCTOBER 31, 2018

Netflix Media Database?—?the Media Timeline Data Model In the previous post in this series, we described some important Netflix business needs as well as traits of the media data system?—?called “ N etflix M edia D ata B ase” (NMDB) that is used to address them. The curious reader might have noticed that a majority of these characteristics relate to properties of the data managed by NMDB.

Media 54

Media Metadata Data MongoDB 54

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Cloudera

OCTOBER 30, 2018

451 Research Analyst Sheryl Kingstone, and Cloudera’s Steve Totman recently discussed how a growing number of organizations are replacing legacy Customer 360 systems with Customer Insights Platforms ( watch the replay here ). In this blog post, Sheryl outlines how next-gen CIP applications are delivering a better customer experience, and why businesses are relying on CIPs as their preferred path to customer insights.

Unstructured Data 42

Unstructured Data Data Lake Algorithm Machine Learning 42

Uber Engineering

OCTOBER 30, 2018

Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware. At Uber, cluster management … The post Peloton: Uber’s Unified Resource Scheduler for Diverse Cluster Workloads appeared first on Uber Engineering Blog.

Engineering 62

Engineering Management Technology Hadoop 62

Rockset

NOVEMBER 1, 2018

As Peter Bailis put it in his post , querying unstructured data using SQL is a painful process. Moreover, developers frequently prefer dynamic programming languages, so interacting with the strict type system of SQL is a barrier. We at Rockset have built the first schemaless SQL data platform. In this post and a few others that follow, we'd like to introduce you to our approach.

SQL 40

SQL NoSQL Programming Language Bytes 40

Rockset

OCTOBER 30, 2018

Prior to Rockset, I spent eight years at Facebook building out their big data infrastructure and online data infrastructure. All the software we wrote was deployed in Facebook's private data centers, so it was not till I started building on the public cloud that I fully appreciated its true potential. Facebook may be the very definition of a web-scale company, but getting hardware still required huge lead times and extensive capacity planning.

Cloud 40

Cloud IT MongoDB Hadoop 40

Cloudera

OCTOBER 30, 2018

Federated Learning is a technology that allows you to build machine learning systems when your datacenter can’t get direct access to model training data. The data remains in its original location, which helps to ensure privacy and reduces communication costs. Privacy and reduced communication make federated learning a great fit for smartphones and edge hardware, healthcare and other privacy-sensitive use cases, and industrial applications such as predictive maintenance.

Machine Learning 40

Machine Learning Healthcare Manufacturing Accessibility 40