Data Engineering Podcast

MARCH 18, 2017

Summary If you like the features of Cassandra DB but wish it ran faster with fewer resources then ScyllaDB is the answer you have been looking for. In this episode Eyal Gutkind explains how Scylla was created and how it differentiates itself in the crowded database market. Preamble Hello and welcome to the Data Engineering Podcast, the show about modern data infrastructure Go to dataengineeringpodcast.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch

Database 100

Database Data Engineer Data Engineering Architecture 100

Uber Engineering

MARCH 12, 2017

With the evolution of storage formats like Apache Parquet and Apache ORC and query engines like Presto and Apache Impala , the Hadoop ecosystem has the potential to become a general-purpose, unified serving layer for workloads that can tolerate latencies … The post Hudi: Uber Engineering’s Incremental Processing Framework on Apache Hadoop appeared first on Uber Engineering Blog.

Hadoop 94

Hadoop Process Engineering Data Architecture 94

Zalando Engineering

MARCH 21, 2017

At Zalando adtech lab in Hamburg, machine learning drives many of our production systems to build great user experiences. Our most recent product requires precise estimates of future interests of Zalando consumers based on their history of interacting with the fashion platform. For example, we want to predict a consumer's interest in ordering selected fashion articles.

Deep Learning 52

Deep Learning Raw Data Machine Learning AWS 52

ProjectPro

MARCH 1, 2017

News on Hadoop-February 2017 Big data brings breast cancer research forwards by 'decades'. ScienceDaily.com, February 1, 2017. Researchers analysed data of more than 28000 different genes and millions of images of 300,000 breast cancer cells and found that any cell shape changes caused by physical pressures on the tumours are converted into gene activity.

Hadoop 40

Hadoop Food Data Lake Banking 40

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Data Engineering Podcast

MARCH 4, 2017

Summary What exactly is data engineering? How has it evolved in recent years and where is it going? How do you get started in the field? In this episode, Maxime Beauchemin joins me to discuss these questions and more. Transcript provided by CastSource Preamble Hello and welcome to the Data Engineering Podcast, the show about modern data infrastructure Go to dataengineeringpodcast.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.

Data Engineer 100

Data Engineer Data Engineering Engineering Software Engineer 100

Zalando Engineering

MARCH 15, 2017

Since joining Zalando, I have had the opportunity to dive into some open source projects like ESLint , a pluggable JavaScript linter. Here is my take on what ESLint is, a brief description of linting in general, and why it is so important. What is linting? Generally speaking, linting is a tool for static code analysis and therefore part of white-box testing.

Coding 52

Coding Programming Language Project Software Engineer 52

Zalando Engineering

MARCH 28, 2017

Every year, Zalando’s Hack Week gives us the opportunity to join together in cross-disciplinary teams to solve a wide variety of problems (you can check this year’s amazing winners here ). The projects come from any point of the organization and we are encouraged to band together with other employees across locations and business units. For our 2016 edition of Hack Week, we implemented a PySpark version of Hidden Markov Model (HMM).

Project 40

Project Algorithm Datasets Python 40

Zalando Engineering

MARCH 20, 2017

RxJava is a valuable part of the Java developer toolset and the number one language improvement framework for Android developers. Many of us want to learn it better, read some blogs and sources, but often miss practice to consolidate collected knowledge. See below for how you can challenge yourself with coding tasks and improve your practical RxJava skills.

Java 40

Java Coding Project IT 40

Zalando Engineering

MARCH 13, 2017

The macOS Server 5.2 is a new fruit. It was released (almost) in parallel with Xcode 8 and might come as no surprise that it is the minimum required version by Xcode 8, which also spans new territory. Most importantly, it’s the name change. Say goodbye to OS X Server as now you have macOS Server. But changes go beyond that: While not being mentioned in the changelog, the good old “_xcsbuildd” user is now gone.

Building 40

Building Project Coding Process 40