Uber Engineering
MARCH 12, 2017
With the evolution of storage formats like Apache Parquet and Apache ORC and query engines like Presto and Apache Impala , the Hadoop ecosystem has the potential to become a general-purpose, unified serving layer for workloads that can tolerate latencies … The post Hudi: Uber Engineering’s Incremental Processing Framework on Apache Hadoop appeared first on Uber Engineering Blog.
Data Engineering Podcast
MARCH 18, 2017
Summary If you like the features of Cassandra DB but wish it ran faster with fewer resources then ScyllaDB is the answer you have been looking for. In this episode Eyal Gutkind explains how Scylla was created and how it differentiates itself in the crowded database market. Preamble Hello and welcome to the Data Engineering Podcast, the show about modern data infrastructure Go to dataengineeringpodcast.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Zalando Engineering
MARCH 21, 2017
At Zalando adtech lab in Hamburg, machine learning drives many of our production systems to build great user experiences. Our most recent product requires precise estimates of future interests of Zalando consumers based on their history of interacting with the fashion platform. For example, we want to predict a consumer's interest in ordering selected fashion articles.
ProjectPro
MARCH 1, 2017
News on Hadoop-February 2017 Big data brings breast cancer research forwards by 'decades'. ScienceDaily.com, February 1, 2017. Researchers analysed data of more than 28000 different genes and millions of images of 300,000 breast cancer cells and found that any cell shape changes caused by physical pressures on the tumours are converted into gene activity.
Advertisement
In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate
Data Engineering Podcast
MARCH 4, 2017
Summary What exactly is data engineering? How has it evolved in recent years and where is it going? How do you get started in the field? In this episode, Maxime Beauchemin joins me to discuss these questions and more. Transcript provided by CastSource Preamble Hello and welcome to the Data Engineering Podcast, the show about modern data infrastructure Go to dataengineeringpodcast.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
Zalando Engineering
MARCH 15, 2017
Since joining Zalando, I have had the opportunity to dive into some open source projects like ESLint , a pluggable JavaScript linter. Here is my take on what ESLint is, a brief description of linting in general, and why it is so important. What is linting? Generally speaking, linting is a tool for static code analysis and therefore part of white-box testing.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
|
Zalando Engineering
MARCH 28, 2017
Every year, Zalando’s Hack Week gives us the opportunity to join together in cross-disciplinary teams to solve a wide variety of problems (you can check this year’s amazing winners here ). The projects come from any point of the organization and we are encouraged to band together with other employees across locations and business units. For our 2016 edition of Hack Week, we implemented a PySpark version of Hidden Markov Model (HMM).
Zalando Engineering
MARCH 20, 2017
RxJava is a valuable part of the Java developer toolset and the number one language improvement framework for Android developers. Many of us want to learn it better, read some blogs and sources, but often miss practice to consolidate collected knowledge. See below for how you can challenge yourself with coding tasks and improve your practical RxJava skills.
Zalando Engineering
MARCH 13, 2017
The macOS Server 5.2 is a new fruit. It was released (almost) in parallel with Xcode 8 and might come as no surprise that it is the minimum required version by Xcode 8, which also spans new territory. Most importantly, it’s the name change. Say goodbye to OS X Server as now you have macOS Server. But changes go beyond that: While not being mentioned in the changelog, the good old “_xcsbuildd” user is now gone.
Expert insights. Personalized for you.
105 
Let's personalize your content