Remove Big Data Tools Remove Blog Remove Building
article thumbnail

Data Engineering Annotated Monthly – January 2022

Big Data Tools

Apache Hop 1.1 — The number of no-code tools is snowballing. We all know Apache NiFi, a stream processing tool with its own processing engine. It has a web interface, allowing you to build the pipeline you need. That wraps up January’s Data Engineering Annotated. Apache Hop is different in many ways.

article thumbnail

Data Engineering Annotated Monthly – January 2022

Big Data Tools

Apache Hop 1.1 — The number of no-code tools is snowballing. We all know Apache NiFi, a stream processing tool with its own processing engine. It has a web interface, allowing you to build the pipeline you need. That wraps up January’s Data Engineering Annotated. Apache Hop is different in many ways.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Data Engineering Annotated Monthly – August 2021

Big Data Tools

How Uber Achieves Operational Excellence in the Data Quality Experience – Uber is known for having a huge Hadoop installation in Kubernetes. This blog post is more about data quality, though, describing how they built their data quality platform. 100% test coverage sounds amazing, too, so good job!

article thumbnail

Data Engineering Annotated Monthly – July 2021

Big Data Tools

Building Data Pipelines Using Kotlin – Surprisingly, big companies are using Kotlin for data pipelines, too! Salesforce shares its experience of using Kotlin everywhere in data engineering but Spark, and we’re in touch about using the Kotlin API for Apache Spark, too! Read the article to find out how they did it.

article thumbnail

Data Engineering Annotated Monthly – July 2021

Big Data Tools

Building Data Pipelines Using Kotlin – Surprisingly, big companies are using Kotlin for data pipelines, too! Salesforce shares its experience of using Kotlin everywhere in data engineering but Spark, and we’re in touch about using the Kotlin API for Apache Spark, too! Read the article to find out how they did it.

article thumbnail

Data Engineering Annotated Monthly – November 2021

Big Data Tools

Usually, I say that data engineering starts when there is not enough data for Excel to handle. Sometimes we need to explore data and not build some heavy-weight process on top of it. And VisiData is a tool that helps us do so. Next I want to build a table that will show me what genre was most popular in which year.

article thumbnail

Data Engineering Annotated Monthly – November 2021

Big Data Tools

Usually, I say that data engineering starts when there is not enough data for Excel to handle. Sometimes we need to explore data and not build some heavy-weight process on top of it. And VisiData is a tool that helps us do so. Next I want to build a table that will show me what genre was most popular in which year.