KDnuggets

FEBRUARY 22, 2024

Top list of open-source tools for building and managing workflows.

Data 153

Data Building Management Data Engineer 153

Start Data Engineering

FEBRUARY 22, 2024

1. Introduction 2. Setup & Logging architecture 3. Data Pipeline Logging Best Practices 3.1. Metadata: Information about pipeline runs, & data flowing through your pipeline 3.2. Obtain visibility into the code’s execution sequence using text logs 3.3. Understand resource usage by tracking Metrics 3.4. Monitoring UI & Traceability 3.5.

Metadata 130

Metadata Data Engineer Data Engineering Engineering 130

KDnuggets

FEBRUARY 22, 2024

Interested in learning data science? Check out these free micro-courses from Kaggle to learn essential data science skills.

Data Science 152

Data Science Data 152

databricks

FEBRUARY 22, 2024

In today's environment, proactive cybersecurity is crucial to any public sector agency. For many organizations, log data that security professionals need for effective.

Data Management 122

Data Management Management Data 122

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

ArcGIS

FEBRUARY 22, 2024

Join us for an exciting Partner Conference and Developer Summit! Discover the latest in ArcGIS Location Referencing and connect with experts.

Transportation 119

Transportation 119

Snowflake

FEBRUARY 22, 2024

A lot of the buzz around AI focuses on its future potential. And we get it — we’re talking about a transformative technology that presents seemingly limitless possibilities. But an important aspect of this world-changing tech story that gets lost in the hype is understanding exactly what AI solutions are available for you and your team to employ right now, today.

Technology 95

Technology Machine Learning Utilities Designing 95

ArcGIS

FEBRUARY 22, 2024

ArcGIS Aviation Airports and ArcGIS Aviation Charting are extensions to ArcGIS Pro that allow users to do their best aviation work with the power of the next generation of desktop software. The tools in these two extensions are enhanced and incorporated in ArcGIS Pro to support your airport, charting, data management, migration, and design needs.

Designing 81

Designing Data Management Management Transportation 81

Rockset

FEBRUARY 22, 2024

Introduction Indexes are a crucial part of proper data modeling for all databases, and DynamoDB is no exception. DynamoDB's secondary indexes are a powerful tool for enabling new access patterns for your data. In this post, we'll look at DynamoDB secondary indexes. First, we'll start with some conceptual points about how to think about DynamoDB and the problems that secondary indexes solve.

Database 80

Database Relational Database Accessibility Accessible 80

DataKitchen

FEBRUARY 22, 2024

The Art of Data Buck-Passing 101: Mastering the Blame Game in Data and Analytic Teams Welcome, dear readers, to the hallowed halls of Data Buck-Passing University, where the motto is “ Per Alios Culpa Transfertur ” (Blame is Transferred to Others). In the world of data and analytics, one skill stands timeless and universal: the art of blaming someone else when things go sideways.

Data 78

Data Data Governance Government Cloud 78

Confluent

FEBRUARY 22, 2024

The 2024 Data Streaming Startup Challenge Finalists will compete for the grand prize on stage at Kafka Summit London.

Kafka 69

Kafka Data 69

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Precisely

FEBRUARY 22, 2024

SAP automation and digital transformation Automation plays a key role in digital transformation initiatives overall, enabling organizations to realize the outcomes they need to remain competitive: agility, speed, and improved data quality and integrity. As automation technologies have evolved over the past decade, organizations have come to recognize that employing automation to streamline business processes is an imperative for success.

Transportation 64

Transportation Process Technology Architecture 64

RandomTrees

FEBRUARY 22, 2024

How to Migrate Your Business to the Cloud Moving to the google cloud workload is one of the most impressive things your business can utilize to build your adaptability, flexibility, and productivity. It will furnish your business with 24/7 admittance to key information and center work processes when you want it, regardless of where you are and when it makes the biggest difference.

Google Cloud 52

Google Cloud Cloud Utilities Machine Learning 52