Netflix Tech
APRIL 8, 2025
By Cheng Xie , Bryan Shultz , and Christine Xu In a previous blog post , we described how Netflix uses eBPF to capture TCP flow logs at scale for enhanced network insights. In this post, we delve deeper into how Netflix solved a core problem: accurately attributing flow IP addresses to workload identities. A BriefRecap FlowExporter is a sidecar that runs alongside all Netflix workloads.
InData Labs
APRIL 10, 2025
The retail sector is among the most competitive markets, making it exceptionally difficult for businesses to not only thrive but even survive. Business intelligence in retail industry can be a colossal game changer for organizations struggling to compete. BI for retail allows companies to leverage Big data analytics and machine learning techniques to extract valuable.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
DataKitchen
APRIL 9, 2025
Why Data Quality Isnt Worth The Effort : Data Quality Coffee With Uncle Chip Data quality has become one of the most discussed challenges in modern data teams, yet it remains one of the most thankless and frustrating responsibilities. In the first of the Data Quality Coffee With Uncle Chip series, he highlights the persistent tension between the need for clean, reliable data and its overwhelming complexity.
Pinterest Engineering
APRIL 7, 2025
Jia Zhan, Senior Staff Software Engineer, Pinterest Sachin Holla, Principal Solution Architect, AWS Summary Pinterest is a visual search engine and powers over 550 million monthly active users globally. Pinterests infrastructure runs on AWS and leverages Amazon EC2 instances for its compute fleet. In recent years, while managing Pinterests EC2 infrastructure, particularly for our essential online storage systems, we identified a significant challenge: the lack of clear insights into EC2s network
Advertisement
With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you
KDnuggets
APRIL 7, 2025
Doing data science projects can be demanding, but it doesnt mean it has to be boring. Here are four projects to introduce more fun to your learning and stand out from the masses.
Snowflake
APRIL 10, 2025
Organizations across industries are achieving unprecedented efficiency and scale along with robust compliance by using data and AI. At Snowflakes most recent virtual events for industries, Accelerate Retail & Consumer Goods , in partnership with Microsoft, and Accelerate Advertising, Media & Entertainment , attendees heard how industry leaders are accelerating innovation, business insights, customer experience and more with robust enterprise AI and data strategies.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Waitingforcode
APRIL 8, 2025
Data quality is one of the key factors of a successful data project. Without a good quality, even the most advanced engineering or analytics work will not be trusted, therefore, not used. Unfortunately, data quality controls are very often considered as a work item to implement in the end, which sometimes translates to never.
Data Engineering Weekly
APRIL 6, 2025
Introducing Apache Airflow® 3.0 Be among the first to see Airflow 3.0 in action and get your questions answered directly by the Astronomer team. You won't want to miss this live event on April 23rd! Save Your Spot → Thoughtworks: Macro trends in the tech industry That raises an important question: not whether AI becomes foundational infrastructure, but how we prepare for that without getting caught flat-footed.
Monte Carlo
APRIL 8, 2025
Data classification is about putting things in the right place based on how sensitive or important they are. Think of it like sorting your inbox: there’s spam, random newsletters, personal messages, and those critical project updates that require immediate attention. In practical terms, this means creating a system where everyone in your organization understands what data they’re handling and how to treat it appropriately, with safeguards if someone accidentally tries to mishandle se
Snowflake
APRIL 7, 2025
Welcome to Snowflakes Startup Spotlight, where we learn about amazing companies building businesses on Snowflake. This time, were casting the spotlight on Innova-Q , where the founders are stirring things up in the food and beverage industry. With the power of modern generative AI, theyre improving product safety, streamlining operations and simplifying regulatory compliance.
Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage
When tasked with building a fundamentally new product line with deeper insights than previously achievable for a high-value client, Ben Epstein and his team faced a significant challenge: how to harness LLMs to produce consistent, high-accuracy outputs at scale. In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation m
Picnic Engineering
APRIL 7, 2025
A little over a year ago, we shared a blog post about our journey to enhance customers meal planning experience with personalized recipe recommendations. We discussed the challenge of finding culinary inspiration when personal preferences arent fully consideredlike encountering that one veggie youd rather avoid. We explained how a system that learns from your tastes and habits could solve this issue, ultimately making the daily task of choosing meals both effortless and inspiring.
ThoughtSpot
APRIL 7, 2025
Everyone associated with Business Intelligence (BI) applications is talking about their Artificial Intelligence (AI) journey and the integration of AI in analytics. Artificial intelligence encompasses a broad spectrum of categories, including machine learning, natural language processing, computer vision, and automated insights. ThoughtSpot has been a leader in augmented analytics , leveraging AI to automate insights and empower users to make data-driven decisions.
WeCloudData
APRIL 9, 2025
The ability to extract information from vast amounts of text has made question-answering (QA) systems essential in the modern era of AI-driven apps. RAG-based question-answering systems use large language models to generate human-like responses to user queries. Whether it’s for research, customer support, or general knowledge retrieval, a Retrieval-Augmented Generation system enhances traditional QA models […] The post Building a Question-Answering System Using RAG appeared first on
Monte Carlo
APRIL 8, 2025
Over the past couple of months Ive spoken to dozens of data teams who are actively building and deploying AI applications. While some of these applications can thrive without perfect accuracy, others demand high reliability as scale, visibility and business impact increase. This post explores the patterns that drive when and why trust becomes an imperative.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Snowflake
APRIL 9, 2025
The traditional five-year anniversary gift is wood. Since snowboards often have a wooden core, and because a snowboard is the traditional trophy for the Snowflake Startup Challenge, were going to go ahead and say that the snowboard trophy qualifies as a present for the fifth anniversary of our Startup Challenge. The only difference is that instead of receiving the gift, well be giving it to one of the 10 semifinalists listed below!
ArcGIS
APRIL 8, 2025
Explore the analytical and 3D visualization capabilities of Space Time Kernel Density tool with time and elevation data and Voxel layer.
Striim
APRIL 10, 2025
As data volumes surge and the need for fast, data-driven decisions intensifies, traditional data processing methods no longer suffice. This growing demand for real-time analytics, scalable infrastructures, and optimized algorithms is driven by the need to handle large volumes of high-velocity data without compromising performance or accuracy. To stay competitive, organizations must embrace technologies that enable them to process data in real time, empowering them to make intelligent, on-the-fly
Precisely
APRIL 7, 2025
A former colleague recently asked me to explain my role at Precisely. After my (admittedly lengthy) explanation of what I do as the EVP and GM of our Enrich business, she summarized it in a very succinct, but new way: “Oh, you manage the appending datasets.” That got me thinking. We often use different terms when were talking about the same thing in this case, data appending vs. data enrichment.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
KDnuggets
APRIL 9, 2025
Tired of dragging messy data through bloated tools? This handbook shows how to clean and transform datasets with Bash.
Snowflake
APRIL 5, 2025
At Snowflake, we are committed to providing our customers with industry-leading LLMs. Were pleased to bring Metas latest Llama 4 models to Snowflake Cortex AI! Llama 4 models deliver performant inference so customers can build enterprise-grade generative AI applications and deliver personalized experiences. The Llama 4 Maverick and Llama 4 Scout models can be accessed within the secure Snowflake perimeter on Cortex AI.
databricks
APRIL 6, 2025
At Databricks, we believe the future of business intelligence is powered by AI. Thats why were thrilled to announce the Databricks Smart Business Insights Challenge.
Start Data Engineering
APRIL 5, 2025
1. Introduction 1.1. Code and setup 2. MERGE INTO is used to UPDATE/DELETE/INSERT rows into a target table based on data in the source table 3. SCD2 table pipeline: INSERT new data, UPDATE existing data, and DELETE stale data 3.1. Source includes 2 versions of upstream customer data: one for insert and the other for update 3.2. Updates to the target table 4.
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
Edureka
APRIL 8, 2025
The job market is constantly evolving and shifting rapidly these days, so workers need to know about reskilling and upskilling to stay ahead of the competition. Continuous learning was once considered a luxury, but as businesses change and new technologies come out, it’s become a must. This blog post talks about the differences between upskilling and reskilling, as well as their value, benefits, and how to do them effectively.
KDnuggets
APRIL 10, 2025
In this article, well explain how to use mind maps within NotebookLM to enhance your productivity and comprehension.
ArcGIS
APRIL 8, 2025
A guide on common approaches of migrating data directly from ArcGIS Online to an enterprise geodatabase.
databricks
APRIL 5, 2025
Thousands of enterprises already use Llama models on the Databricks Data Intelligence Platform to power AI applications, agents, and workflows.
Advertisement
Apache Airflow® is the open-source standard to manage workflows as code. It is a versatile tool used in companies across the world from agile startups to tech giants to flagship enterprises across all industries. Due to its widespread adoption, Airflow knowledge is paramount to success in the field of data engineering.
Monte Carlo
APRIL 7, 2025
Machine learning (ML) orchestration tools are like the stage managers of your data science production. They dont write the script or act in the play, but without them, everything falls apart: lights dont come on, cues get missed, and suddenly your model is predicting total nonsense. That’s why you need someone calling the shots backstage, and thats exactly what these tools do.
Edureka
APRIL 10, 2025
When you hear the term System Hacking, it might bring to mind shadowy figures behind computer screens and high-stakes cyber heists. In reality, system hacking encompasses a wide range of techniques aimed at exploiting computer systems, whether for unauthorized access by malicious actors or ethical penetration testing by security professionals. In this blog, we’ll explore the definition, purpose, process, and methods of prevention related to system hacking, offering a detailed overview to h
KDnuggets
APRIL 7, 2025
It's the must-learn data science skill to land a job at big tech.
ArcGIS
APRIL 8, 2025
Learn how to use the Migration Toolset to migrate data to a utility network and deploy it to an ArcGIS Enterprise environment.
Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
Expert insights. Personalized for you.
67 
Let's personalize your content