KDnuggets

MARCH 28, 2025

Learn model serving, CI/CD, ML orchestration, model deployment, local AI, and Docker to streamline ML workflows, automate pipelines, and deploy scalable, portable AI solutions effectively.

Machine Learning 126

Machine Learning Project 126

Simon Späti

MARCH 28, 2025

The database landscape has reached 394 ranked systems across multiple categoriesrelational, document, key-value, graph, search engine, time series, and the rapidly emerging vector databases. As AI applications multiply quickly, vector technologies have become a frontier that data engineers must explore. The essential questions to be answered are: When should you choose specialized vector solutions like Pinecone, Weaviate, or Qdrant over adding vector extensions to established databases like Post

Technology 130

Technology PostgreSQL MySQL Database 130

Netflix Tech

MARCH 28, 2025

By Ko-Jen Hsiao , Yesu Feng and Sudarshan Lamkhede Motivation Netflixs personalized recommender system is a complex system, boasting a variety of specialized machine learned models each catering to distinct needs including Continue Watching and Todays Top Picks for You. (Refer to our recent overview for more details). However, as we expanded our set of personalization algorithms to meet increasing business needs, maintenance of the recommender system became quite costly.

Metadata 66

Metadata Bytes Entertainment Data Mining 66

Wayne Yaddow

MARCH 28, 2025

Unleashing GenAIEnsuring Data Quality at Scale (Part2) Transitioning from individual repository source systems to consolidated AI LLM pipelines, the importance of automated checks, end-to-end observability, and compliance with enterprise businessrules. T Introduction There are several opportunities (and needs!) to improve operational effectiveness and analytical capacity when integrating data repository systems for AI Large Language Model (LLM) pipelines.

Data Integration 52

Data Integration Data Governance Datasets Government 52

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineering

Monte Carlo

MARCH 28, 2025

AI can do a lot these days. At this very moment, an army of SaaS companies are hard at work infusing AI assistants and copilots into every horizontal B2B workflow currently known to humankind. ChatGPT can summarize the web to help sales prospects. Gemini can polish Google documents for research teams. GitHub copilot can even code alongside you like your own pocket-sized Steve Wozniak.

Coding 52

Coding Systems Data Pipeline ETL Tools 52

Precisely

MARCH 28, 2025

In today’s fast-paced digital world, maintaining high standards and addressing contemporary requirements is crucial for any company. One of our customers, a leading automotive manufacturer, relies on the IBM Z for its computing power and rock-solid reliability. However, they faced a growing challenge: integrating and accessing data across a complex environment.

Data Integration 52

Data Integration Pipeline-centric Database-centric Kafka 52

Confessions of a Data Guy

MARCH 28, 2025

Running dbt on Databricks has never been easier. The integration between dbtcore and Databricks could not be more simple to set up and run. Wondering how to approach running dbt models on Databricks with SparkSQL? Watch the tutorial below. The post dbt on Databricks appeared first on Confessions of a Data Guy.

Data 100

Data Big Data Data Engineer Data Engineering 100

databricks

MARCH 28, 2025

Qwen models, developed by Alibaba, have shown strong performance in both code completion and instruction tasks. In this blog, well show how you can register.

Coding 81

Coding 81

ArcGIS

MARCH 28, 2025

Poles of inaccessibility are the locations furthest from the coast in land masses or the ocean.

58

58

Confluent

MARCH 28, 2025

Singapore's Tech Acceleration Lab helps other government agencies modernize by onboarding them to a streaming-capable government cloud, which features Confluent as a key vendor.

Government 58

Government Cloud Building 58

Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage

When tasked with building a fundamentally new product line with deeper insights than previously achievable for a high-value client, Ben Epstein and his team faced a significant challenge: how to harness LLMs to produce consistent, high-accuracy outputs at scale. In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation m

Software Engineer

Cloudyard

MARCH 28, 2025

Read Time: 3 Minute, 56 Second Snowflake’s S erverless Tasks are designed to automate workflows without requiring a dedicated warehouse. Instead, Snowflake dynamically allocates compute resources based on workload demands, making it a cost-effective and scalable solution for scheduling data transformations, ELT processes, and real-time analytics.

Utilities 52

Utilities Designing Process Systems 52

Picnic Engineering

MARCH 28, 2025

TL;DR Over the past decade, Picnic transformed its approach to dataevolving from a single, all-purpose data team into multiple specialized teams using a lean, scalable tech stack. We empowered analysts by giving them access and training to the same tools as engineers, dramatically increasing speed and impact. Our investments in a lakeless data warehouse, modern analytics platform, and strong master data practices have made data a core strategic capability.

Data Warehouse 40

Data Warehouse PostgreSQL Python Machine Learning 40

ArcGIS

MARCH 28, 2025

In March 2025, esri elevation layers have been updated with lidar derived DTMs of New Zealand and USA along with bathymetry of Australia.

Data 66

Data 66

WeCloudData

MARCH 28, 2025

The word cloud may evoke images of fluffy white shapes in the sky, right? But in the tech world, it represents the backbone of modern innovation. Cloud computing is an essential technology that powers everything from social media platforms to enterprise applications. Cloud computing is the on-demand delivery of computing resources, such as servers, storage, […] The post Cloud Computing appeared first on WeCloudData.

Cloud Computing 52

Cloud Computing Cloud Media Technology 52

Advertisement

Apache Airflow® is the open-source standard to manage workflows as code. It is a versatile tool used in companies across the world from agile startups to tech giants to flagship enterprises across all industries. Due to its widespread adoption, Airflow knowledge is paramount to success in the field of data engineering.

Datasets