ArcGIS

MAY 12, 2025

ArcGIS Pro on Windows 365 GPU-enabled Cloud PCs

Cloud 64

Cloud 64

databricks

MAY 12, 2025

Driving business transformation with data and AI takes more than the right tools it needs the right strategy.

Data 64

Data IT 64

Waitingforcode

MAY 7, 2025

One of the biggest changes for PySpark has been the DataFrame API. It greatly reduces the JVM-to-PVM communication overhead and improves the performance. However, it also complexities the code. Probably, some of you have already seen, written, or worked with the code like this.

Accessible 130

Accessible Accessibility Designing Coding 130

databricks

MAY 7, 2025

As organizations consolidate analytics workloads to Databricks, they often need to adapt traditional data warehouse techniques.

Data Warehouse 71

Data Warehouse SQL Data 71

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline

Snowflake

MAY 8, 2025

Data providers want their data available to their customers, no matter where in the world or on which cloud service provider the customer is located. However, egress costs can contribute up to 70% of total data transfer costs. Providers have historically had to balance the desire to increase the availability of their data to any relevant Snowflake regions with the need to manage egress costs.

AWS 59

AWS Cloud Algorithm Data Security 59

Edureka

MAY 8, 2025

When you purchase online, do you ever find yourself pondering how your tastes get changed into suggestions for products that are uniquely suited to you? Or how self-driving cars get through very complicated situations with amazing accuracy? These are the ways that data engineering improves our lives in the real world. The field of data engineering turns unstructured data into ideas that can be used to change businesses and our lives.

Data Engineering 52

Data Engineering Data Engineer Engineering Telecommunication 52

Striim

MAY 9, 2025

Get More Insights In Your Inbox Sol Rashidi has built AI, data, and digital strategies inside some of the worlds biggest companiesand shes seen the same mistakes play out again and again. In this episode, she unpacks why AI initiatives often stall, how executives misread what transformation really requires, and why the future of AI success isnt technicalits cultural.

Data 52

Data 52

Data Engineering Weekly

MAY 11, 2025

Dagster Running Dagster: Our Open Platform We’re pulling back the curtain. Join us on May 13 for a live deep dive into how Dagster Labs runs Dagster in production. One of our lead data engineers will walk through our real-world implementation, architecture decisions, and the lessons we've learned scaling the platform. Register now Editor’s Note: OpenXData Conference - 2025 - A Free Virtual Event A free virtual event on open data architectures - Iceberg, Hudi, lakehouses, query engine

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

ArcGIS

MAY 14, 2025

Learn more about exciting new functionality and improvements made to ArcGIS Utility Network with the 2025 Network Management Release.

Utilities 52

Utilities Management 52

KDnuggets

MAY 14, 2025

Add these 4 data analytic-based projects to your resume to land your next job.

Data Analytics 49

Data Analytics Project Data 49

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Pipeline-centric

databricks

MAY 12, 2025

The oil and gas industry relies heavily on seismic data to explore and extract hydrocarbons safely and efficiently.

Data Pipeline 52

Data Pipeline Data Process 52

Striim

MAY 9, 2025

Introduction This recipe shows how you can build a data pipeline to read data from ServiceNow and write to BigQuery. Striim’s ServiceNow Reader will first read the existing tables from the configured ServiceNow dataset and then write them to the target BigQuery project using the BigQuery Writer, a process called “initial load” in Striim and “historical sync” or “initial snapshot” by others.

Data Pipeline 40

Data Pipeline Data Datasets Data Warehouse 40

Snowflake

MAY 8, 2025

We are thrilled to announce that as part of our strategic partnership with Anthropic, Snowflake customers will now have access to Claude 3.7 Sonnet in Snowflake Cortex AI. Anthropic and Snowflake entered a multi-year partnership in November 2024 to help enterprises develop and scale easy, efficient and trusted AI products, starting with the launch of Claude 3.5.

Unstructured Data 52

Unstructured Data Coding Structured Data Government 52

Seattle Data Guy

MAY 13, 2025

Maybe youre luckier than me. Maybe youve never opened a.sql file or anAirflow DAG only to be greeted by a 5,000+ line query…a true monster of a script that leaves you wondering where to begin. Ive seen plenty of these, and every time, I ask myself:Why in the world do these exist? And, more… Read more The post How To Write Better SQL – Simplifying Complex SQL appeared first on Seattle Data Guy.

SQL 130

SQL Data Data Pipeline Big Data 130

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

KDnuggets

MAY 7, 2025

Learn to build AI agents, fine-tune reasoning models, and master practical AI skills with these courses.

Building 140

Building 140

databricks

MAY 14, 2025

Today, we are excited to announce that we have agreed to acquire Neon, a developer-first, serverless Postgres company.

52

52

Robinhood

MAY 13, 2025

WonderFi will join Robinhood Crypto and continue to deliver crypto products to Canadian customers WonderFi shareholders to receive all-cash consideration of C$0.36 per Common Share Purchase price represents an attractive premium of approximately 41% to the closing price, and 71% to the 30-day VWAP, as of May 12 Robinhood Markets, Inc. has entered into an agreement to acquire WonderFi, a Canadian leader in digital asset products and services.

Finance 111

Finance Accessibility Accessible Technology 111

Snowflake

MAY 9, 2025

In today's complex enterprise environments, managing data security is a daunting challenge. Organizations are grappling with a growing number of data stores, data sharing, increasing use of data for AI and increasingly sophisticated threats. This complexity necessitates a shift toward automated, AI-driven solutions that simplify security governance and accelerate threat detection.

Metadata 52

Metadata Data Governance Government Data Security 52

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

Seattle Data Guy

MAY 9, 2025

After working in data for over a decade, one thing that remains the same is the need to create data pipelines. Whether you call them ETLs/ELTs or something else, companies need to move and process data for analytics. The question becomes how companies are actually building their data pipelines. What ETL tools are they actually… Read more The post 6 Real-World ETL Use Cases with Estuary Flow appeared first on Seattle Data Guy.

ETL Tools 130

ETL Tools Data Pipeline Building Process 130

KDnuggets

MAY 12, 2025

Master LLMs through books, courses, tutorials, exercises, projects, and comprehensive guides that cover everything from foundational concepts to advanced techniques.

Project 134

Project 134

ArcGIS

MAY 13, 2025

In ArcGIS Pro 3.5, we have just launched the first of a new toolset of cartogram generating tools, Generate Contiguous Cartogram.

101

101

Engineering at Meta

MAY 8, 2025

Meta and NVIDIA collaborated to accelerate vector search on GPUs by integrating NVIDIA cuVS into Faiss v1.10 , Metas open source library for similarity search. This new implementation of cuVS will be more performant than classic GPU-accelerated search in some areas. For inverted file (IVF) indexing, NVIDIA cuVS outperforms classical GPU-accelerated IVF build times by up to 4.7x; and search latency is reduced by as much as 8.1x.

Algorithm 81

Algorithm Datasets Machine Learning Database 81

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Snowflake

MAY 12, 2025

We are thrilled to announce that as part of our strategic partnership with Anthropic, Snowflake customers will soon have access to Claude 3.7 Sonnet in Snowflake Cortex AI. Anthropic and Snowflake entered a multi-year partnership in November 2024 to help enterprises develop and scale easy, efficient and trusted AI products with Claude 3.5. We are building on this momentum with Claude 3.7 Sonnet.

Unstructured Data 74

Unstructured Data Coding Structured Data Government 74

Netflix Tech

MAY 9, 2025

Kinesh Satiya Introduction In a digital advertising platform, a robust feedback system is essential for the lifecycle and success of an ad campaign. This system comprises of diverse sub-systems designed to monitor, measure, and optimize ad campaigns. At Netflix, we embarked on a journey to build a robust event processing platform that not only meets the current demands but also scales for future needs.

Process 71

Process Building Metadata Kafka 71

KDnuggets

MAY 12, 2025

Looking to get the most out of NotebookLM? These five expert tips will help you use it better and improve your productivity.

IT 105

IT 105

ArcGIS

MAY 12, 2025

The latest release of ArcGIS Pro brings powerful new tools and enhancements specifically designed for working with imagery. Read on to learn how ArcGIS Pro will improve your geospatial analysis capabilities, streamline workflows, and provide more accurate and actionable insights.

Designing 93

Designing 93

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineering

databricks

MAY 7, 2025

Introduction Stateful processing in Apache Spark Structured Streaming has evolved significantly to meet the growing demands of complex streaming applications.

Process 69

Process 69

Yelp Engineering

MAY 7, 2025

It has been over 3 years since we published our Nrtsearch blog post and over 4 years since we started using Nrtsearch, our Lucene-based search engine, in production. We have since migrated over 90% of Elasticsearch traffic to Nrtsearch. We are excited to announce the release of Nrtsearch 1.0.0 with several new features and improvements from the initial release.

AWS 59

AWS Cloud Engineering IT 59

Netflix Tech

MAY 7, 2025

Enhancing Member Experience Through Strategic Collaboration Ozzie Sutherland , Iroro Orife , Chih-Wei Wu , BhanuSrikanth At Netflix, delivering the best possible experience for our members is at the heart of everything we do, and we know we cant do it alone. Thats why we work closely with a diverse ecosystem of technology partners, combining their deep expertise with our creative and operational insights.

Digital Media 70

Digital Media Entertainment Media Machine Learning 70

KDnuggets

MAY 8, 2025

Breaking into data science has never been easy. In this tutorial, well make your life easier by providing you with a step-by-step roadmap for data science beginners.

Data Science 102

Data Science Data 102

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data