Maxime Beauchemin

JANUARY 20, 2017

By the time I left in 2013, I was a data engineer. We were developing new skills, new ways of doing things, new tools, and — more often than not — turning our backs to traditional methods. We were data engineers! Data Engineering? Like data scientists, data engineers write code. We were pioneers.

Data Engineering 130

Data Engineering Data Engineer Engineering ETL Tools 130

Hevo

JULY 1, 2024

As data continues to grow in volume and complexity, the need for an efficient ETL tool becomes increasingly critical for a data professional. ETL tools not only streamline the process of extracting data from various sources but also transform it into a usable format and load it into a system of your choice.

ETL Tools 40

ETL Tools Systems Process IT 40

KDnuggets

JANUARY 3, 2022

Also: 6 Predictive Models Every Beginner Data Scientist Should Master; The Best ETL Tools in 2021; Write Clean Python Code Using Pipes; Three R Libraries Every Data Scientist Should Know (Even if You Use Python).

Python 132

Python Coding ETL Tools Data 132

Ascend.io

OCTOBER 28, 2024

What is Data Transformation? Data transformation is the process of converting raw data into a usable format to generate insights. It involves cleaning, normalizing, validating, and enriching data, ensuring that it is consistent and ready for analysis. This understanding forms the basis for effective data transformation.

Raw Data 52

Raw Data Datasets Aggregated Data Data Pipeline 52

Hevo

APRIL 26, 2024

Amazon Redshift is a serverless, fully managed leading data warehouse in the market, and many organizations are migrating their legacy data to Redshift for better analytics. In this blog, we will discuss the best Redshift ETL tools that you can use to load data into Redshift.

ETL Tools 40

ETL Tools Data Warehouse Management Data 40

KDnuggets

AUGUST 19, 2019

What’s the best way to execute your data integration tasks: writing manual code or using ETL tool? Find out the approach that best fits your organization’s needs and the factors that influence it.

Data Integration 96

Data Integration Coding ETL Tools Data 96

Hevo

JUNE 3, 2023

Tableau is a robust Business Intelligence tool that helps users visualize data simply and elegantly. Tableau has helped numerous organizations understand their customer data better through their Visual Analytics platform.

ETL Tools 52

ETL Tools Business Intelligence Data Warehouse Data 52

Hevo

MAY 31, 2024

As data continues to grow at an unprecedented rate, the need for an efficient and scalable open-source ETL solution becomes increasingly pressing. However, with every organisation’s varying needs and the cluttered market for ETL tools, finding and choosing the right tool can be strenuous.

ETL Tools 40

ETL Tools Data Data Integration 40

Monte Carlo

MARCH 28, 2025

Its how you integrate AI with your first-party data to deliver new business value that sets you apart. And its not sufficient to simply build these data + AI applications – as in any other technological discipline, you have to do it reliably, too. So, what does it mean to achieve trusted data + AI?

Coding 52

Coding Systems Data Pipeline ETL Tools 52

Hevo

SEPTEMBER 27, 2024

In the data engineering industry, managing your data is critical for driving business. Data is gathered from various sources in all shapes and forms, and without the right set of tools, it is impossible to use this data for meaning analysis.

ETL Tools 52

ETL Tools Data Engineering Data Engineer Engineering 52

Knowledge Hut

MAY 3, 2023

Did you know that data is now an essential component of modern business operations? With companies increasingly relying on data-driven insights to make informed decisions, there has never been a greater need for skilled specialists who can manage and evaluate vast amounts of data.

Data Engineering 98

Data Engineering Data Engineer Engineering Data Cleanse 98

Hevo

MAY 1, 2023

Modern businesses are data-driven – they use data in daily operations and decision-making. Data is collected from a variety of data storage systems, formats, and locations, and data engineers have a hefty job structuring, cleaning, and integrating this data.

ETL Tools 52

ETL Tools Data Storage Data Engineering Data Engineer 52

Hevo

SEPTEMBER 29, 2024

If you are a data-driven business, then you must know how crucial it is to extract meaningful insights from your data. That’s where Reverse ETL comes into play. I’m guessing you might know what ETL (Extract, Transform, Load) is. It is the process of bringing data into your warehouses.

ETL Tools 52

ETL Tools Process IT Data 52

Hevo

SEPTEMBER 10, 2024

Two platforms are most commonly associated with automating your data processes: Fivetran vs Supermetrics.

ETL Tools 52

ETL Tools Data Process Process Data 52

Hevo

JANUARY 25, 2023

Business leaders use business intelligence (BI) to turn data into valuable insights and make strategic decisions within the company. And this strategy starts with data aggregation and integration. Realising the importance of ETL many businesses […]

ETL Tools 52

ETL Tools Business Intelligence BI Data 52

Cloudera

AUGUST 13, 2021

Every enterprise is trying to collect and analyze data to get better insights into their business. Whether it is consuming log files, sensor metrics, and other unstructured data, most enterprises manage and deliver data to the data lake and leverage various applications like ETL tools, search engines, and databases for analysis.

Data Pipeline 111

Data Pipeline Data Lake ETL Tools Unstructured Data 111

Data Engineering Podcast

JANUARY 15, 2022

Summary Applications of data have grown well beyond the venerable business intelligence dashboards that organizations have relied on for decades. Given this increased level of importance it has become necessary for everyone in the business to treat data as a product in the same way that software applications have driven the early 2000s.

Engineering 100

Engineering Electronics ETL Tools Data Pipeline 100

Hevo

OCTOBER 17, 2024

ETL stands for Extract, Transform, and Load. ETL is a process of transferring data from various sources to target destinations/data warehouses and performing transformations in between to make data analysis ready. Managing data is a tedious task if done manually and leads to no guarantee of accuracy.

ETL Tools 40

ETL Tools Data Warehouse Data Analysis Management 40

Confluent

MAY 15, 2019

Red Pill Analytics was hired to first design and th en implement all the necessary data integration processes required to connect Oracle WMS Cloud with their on-premises systems. This left us with only two data integration requirements to solve: relational tables and REST APIs. Streaming data into Oracle WMS Cloud.

Kafka 92

Kafka ETL Tools Cloud Data Integration 92

Hevo

APRIL 26, 2024

AWS Glue is a serverless ETL solution that helps organizations move data into enterprise-class data warehouses. It provides close integration with other AWS services, which appeals to businesses already invested significantly in AWS.

ETL Tools 40

ETL Tools AWS Data Warehouse IT 40

Towards Data Science

NOVEMBER 4, 2023

Platform Specific Tools and Advanced Techniques Photo by Christopher Burns on Unsplash The modern data ecosystem keeps evolving and new data tools emerge now and then. In this article, I want to talk about crucial things that affect data engineers. Are your data pipelines efficient? Data warehouse exmaple.

Data Engineering 98

Data Engineering Data Engineer Engineering BI 98

Towards Data Science

JUNE 16, 2023

How to Identify Your Business-Critical Data Practical steps to identifying business-critical data models and dashboards and drive confidence in your data Source: synq.io This article has been co-written with Lindsay Murphy Not all data is created equal. There’s a good reason for this. Source: synq.io

BI 98

BI Data ETL Tools Machine Learning 98

Knowledge Hut

DECEMBER 26, 2023

According to the World Economic Forum, the amount of data generated per day will reach 463 exabytes (1 exabyte = 10 9 gigabytes) globally by the year 2025. Thus, almost every organization has access to large volumes of rich data and needs “experts” who can generate insights from this rich data.

Data Science 97

Data Science BI Machine Learning Business Intelligence 97

Hevo

SEPTEMBER 2, 2024

As far as data pipeline construction and maintenance are concerned, ETL (Extract, Transform, Load) tools play a crucial role, and their selection determines success. When considering the market offerings, AWS Glue vs Matillion frequently stands out. In this blog, we […]

ETL Tools 40

ETL Tools AWS Data Pipeline Data 40

AltexSoft

OCTOBER 30, 2021

Explaining the difference, especially when they both work with something intangible such as data , is difficult. If you’re an executive who has a hard time understanding the underlying processes of data science and get confused with terminology, keep reading. Data science vs data engineering.

Data Engineering 98

Data Engineering Data Engineer Engineering Machine Learning 98

Hevo

JULY 1, 2024

Need a better way to handle all that customer and marketing data in HubSpot. This article will explain how you can transfer your HubSpot data into Google BigQuery through various means, be it HubSpot’s API or an automated ETL tool like Hevo Data, which does it […] Transfer it to BigQuery.

ETL Tools 52

ETL Tools Data IT Data Integration 52

Ascend.io

MAY 31, 2023

In the modern world of data engineering, two concepts often find themselves in a semantic tug-of-war: data pipeline and ETL. Fast forward to the present day, and we now have data pipelines. However, they are not just an upgraded version of ETL. The data sources themselves are not built to perform analytics.

Data Pipeline 52

Data Pipeline ETL Tools Pipeline-centric Data Warehouse 52

Lyft Engineering

OCTOBER 3, 2023

High-quality data is necessary for the success of every data-driven company. It is now the norm for tech companies to have a well-developed data platform. This makes it easy for engineers to generate, transform, store, and analyze data at the petabyte scale. What and Where is Data Quality?

Big Data 87

Big Data Metadata Data Warehouse Data 87

Netflix Tech

JULY 15, 2021

Data Engineers of Netflix?—?Interview Interview with Kevin Wylie This post is part of our “Data Engineers of Netflix” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Kevin Wylie is a Data Engineer on the Content Data Science and Engineering team.

Data Engineering 79

Data Engineering Data Engineer Engineering Entertainment 79

Data Engineering Weekly

DECEMBER 18, 2023

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. I included this post because I often see high-pitched LinkedIn posts stating it is the human fault, especially around data quality issues.

Data Engineering 68

Data Engineering Data Engineer Engineering Food 68

Silectis

FEBRUARY 25, 2021

Data lineage can be a tremendously useful tool for data engineering and analytics, but is often treated as an afterthought both because of the challenges in implementation and the fact that it has not been broadly available within organizations. Lineage is dependencies – What is upstream of the final data that we are accessing?

ETL Tools 52

ETL Tools Metadata Data Data Engineering 52

Ascend.io

DECEMBER 21, 2022

The last three years have seen a remarkable change in data infrastructure. ETL changed towards ELT. Now, data teams are embracing a new approach: reverse ETL. Cloud data warehouses, such as Snowflake and BigQuery, have made it simpler than ever to combine all of your data into one location. Let’s dive in!

ETL Tools 52

ETL Tools ETL System Data Warehouse Data Consolidation 52

DataKitchen

MAY 4, 2023

All the cool kids are talking about Data Products and Data Mesh. The data companies have gotten ahold of terms and started to say their twenty-year-old ETL tools are the perfect tools to do that fashionable product-meshy stuff. What is going on?

ETL Tools 52

ETL Tools Data 52

ProjectPro

FEBRUARY 16, 2023

ETL is a critical component of success for most data engineering teams, and with teams harnessing it with the power of AWS, the stakes are higher than ever. Data Engineers and Data Scientists require efficient methods for managing large databases, which is why centralized data warehouses are in high demand.

AWS 52

AWS Data Management ETL Tools Management 52

Knowledge Hut

OCTOBER 31, 2023

Businesses are increasingly depending on cloud platforms to manage and analyze their data in today's data-driven environment. Two of the most well-known cloud service providers, Amazon Web Services (AWS) and Microsoft Azure, provide reliable data engineering solutions. Azure Data Factory, Databricks, etc.

Data Engineering 52

Data Engineering Data Engineer AWS Engineering 52

Snowflake

MARCH 30, 2023

A 2016 data science report from data enrichment platform CrowdFlower found that data scientists spend around 80% of their time in data preparation (collecting, cleaning, and organizing of data) before they can even begin to build machine learning (ML) models to deliver business value.

Engineering 105

Engineering Raw Data Data Science Machine Learning 105

Hevo

AUGUST 16, 2024

Data integration is central to making informed business decisions for any organization in this data-driven world. ETL tools are central to this since they enable organizations to manage their data from different sources effectively and integrate it efficiently.

ETL Tools 52

ETL Tools Data Integration Management Data 52