Maxime Beauchemin

AUGUST 28, 2017

This post follows up on The Rise of the Data Engineer , a recent post that was an attempt at defining data engineering and described how this new role relates to historical and modern roles in the data space. In this post, I want to expose the challenges and risks that cripple data engineers and enumerates the forces that work against this discipline as it goes through its adolescence.

Data Engineering 174

Data Engineering Data Engineer Engineering Software Engineering 174

Data Engineering Podcast

AUGUST 6, 2017

Summary Building a data pipeline that is reliable and flexible is a difficult task, especially when you have a small team. Astronomer is a platform that lets you skip straight to processing your valuable business data. Ry Walker, the CEO of Astronomer, explains how the company got started, how the platform works, and their commitment to open source.

PostgreSQL 100

PostgreSQL MongoDB Data Pipeline Kafka 100

Zalando Engineering

AUGUST 23, 2017

Some months ago I had the opportunity, with two fellow Zalandos, to organize the “Dortmund 5PM”; a gathering across all Dortmund teams, scheduled once a month on Fridays in our local event space. We want to foster further cross-team collaboration between individuals, making these meetings a memorable experience for all. We opted for running The Marshmallow Challenge ; a funny design exercise that encourages teams to experience simple yet profound lessons in collaboration, innovation, and creativ

Deep Learning 52

Deep Learning Education Designing Project 52

Zalando Engineering

AUGUST 29, 2017

At Zalando, we have many teams working on their respective systems and domains where they operate like small startups. They take responsibility for these systems and have a feeling of ownership, constantly engaging with other teams to solve bigger challenges and enable new features. Sometimes during these alignments and feature requests, priorities collide and we can’t support others.

Coding 40

Coding Project Designing Systems 40

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineering

Zalando Engineering

AUGUST 15, 2017

As Zalando continues taking steps towards becoming a fully-fledged platform, we want to move fast, validate the ways that our big strategic moves pay off, and capture the full value of our products by continuous optimization. To this end, we wanted to ensure that we’re bringing data-informed decision making to the forefront of our processes by establishing a true data and experimentation culture that could ultimately become a competitive advantage in today’s fast-changing world.

Education 40

Education Data Governance Government Data 40

Zalando Engineering

AUGUST 13, 2017

Almost two years ago, the Talent Acquisition Team (Technology) at Zalando began their work on Candidate Experience (CX) research. We were determined to learn our tech candidates’ needs and expectations, and identify the factors that influence their experience the most. This research allows us to identify how we can deliver a superb and meaningful candidate experience and thereby continue to strengthen our employer brand, securing a steady flow of strong candidates.

Recruitment 40

Recruitment Coding Skills Software Engineering Software Engineer 40