Data Engineering Podcast

JUNE 30, 2024

Summary This episode features an insightful conversation with Petr Janda, the CEO and founder of Synq. Petr shares his journey from being an engineer to founding Synq, emphasizing the importance of treating data systems with the same rigor as engineering systems. He discusses the challenges and solutions in data reliability, including the need for transparency and ownership in data systems.

Pipeline-centric 147

Pipeline-centric Engineering Data Lake High Quality Data 147

ArcGIS

JUNE 30, 2024

In June 2024, elevation layers have been updated with lidar derived DTM’s of Slovakia, Belgium, San Mateo County (USA) along with USGS 3DEP.

Data 105

Data 105

Data Engineering Weekly

JUNE 30, 2024

Experience Enterprise-Grade Apache Airflow Astro augments Airflow with enterprise-grade features to enhance productivity, meet scalability and availability demands across your data pipelines, and more. Learn More → Ozge Demirci, Jonas Hannane & Xinrong Zhu: Who Is AI Replacing? The Impact of Generative AI on Online Freelancing Platforms The economic impact of Gen AI is widely speculated, and we see few signs of impact.

Data Engineering 102

Data Engineering Data Engineer Engineering Unstructured Data 102

Knowledge Hut

JUNE 30, 2024

When it comes to top cybersecurity certifications, CISSP and Security+ stand out. Both are respected and can lead to high-paying jobs. But which is right for you? The CISSP is the gold standard in cybersecurity certifications, geared towards experienced professionals with at least five years in the field. The exam is notoriously difficult, but it offers a comprehensive credential.

Certification 52

Certification Government Consulting Architecture 52

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline