Hepta Analytics

FEBRUARY 14, 2022

DE Zoomcamp 2.2.1 – Introduction to Workflow Orchestration Following last weeks blog , we move to data ingestion. We already had a script that downloaded a csv file, processed the data and pushed the data to postgres database. This week, we got to think about our data ingestion design.

Data Ingestion 52

Data Ingestion Data Engineering Data Engineer Engineering 52

Ascend.io

DECEMBER 21, 2022

21, 2022 – Ascend.io , The Data Automation Cloud, today announced they have partnered with Snowflake , the Data Cloud company, to launch Free Ingest , a new feature that will reduce an enterprise’s data ingest cost and deliver data products up to 7x faster by ingesting data from all sources into the Snowflake Data Cloud quickly and easily.

Data Ingestion 52

Data Ingestion Google Cloud Data Lake Cloud 52

Hevo

AUGUST 16, 2024

This blog explores the world of open source data orchestration tools, highlighting their importance in managing and automating complex data workflows. From Apache Airflow to Google Cloud Composer, we’ll walk you through ten powerful tools to streamline your data processes, enhance efficiency, and scale your growing needs.

Google Cloud 52

Google Cloud Data Workflow Data Data Engineering 52

Monte Carlo

JUNE 14, 2023

Why is data pipeline architecture important? Databricks – Databricks, the Apache Spark-as-a-service platform, has pioneered the data lakehouse, giving users the options to leverage both structured and unstructured data and offers the low-cost storage features of a data lake.

Data Pipeline 52

Data Pipeline Architecture Data Lake Data Warehouse 52

Knowledge Hut

MARCH 28, 2024

Role Level: Intermediate Responsibilities Design and develop big data solutions using Azure services like Azure HDInsight, Azure Databricks, and Azure Data Lake Storage. Implement data ingestion, processing, and analysis pipelines for large-scale data sets.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

AltexSoft

SEPTEMBER 21, 2023

Accessible via a unified API, these new features enhance search relevance and are available on Elastic Cloud. The Elastic Stacks Elasticsearch is integral within analytics stacks, collaborating seamlessly with other tools developed by Elastic to manage the entire data workflow — from ingestion to visualization.

Engineering 52

Engineering NoSQL Programming Language Java 52

phData: Data Engineering

JANUARY 3, 2022

This is a config driven tool that is made by HashiCorp and is supported by over 1000+ providers such as: AWS Azure Google Cloud Oracle Alibaba Okta Kubernetes As you can see, there’s support for all the major cloud providers and various other auxiliary tooling that enterprises frequently leverage.

IT 52

IT AWS Software Engineer Software Engineering 52