Inside recommendations: how a recommender system recommends
KDnuggets
NOVEMBER 17, 2021
We describe types of recommender systems, more specifically, algorithms and methods for content-based systems, collaborative filtering, and hybrid systems.
Inside Facebook’s video delivery system
Engineering at Meta
DECEMBER 10, 2024
Were explaining the end-to-end systems the Facebook app leverages to deliver relevant content to people. At Facebooks scale, the systems built to support and overcome these challenges require extensive trade-off analyses, focused optimizations, and architecture built to allow our engineers to push for the same user and business outcomes.
Systems 
68
68 Join 37,000+
Insiders
Sign Up for our Newsletter
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Webinars
Agent Tooling: Connecting AI to Your Tools, Systems & Data
How to Modernize Manufacturing Without Losing Control
Mastering Apache Airflow® 3.0: What’s New (and What’s Next) for Data Orchestration
Trending Sources
Establishing a Large Scale Learned Retrieval System at Pinterest
Pinterest Engineering
JANUARY 31, 2025
Modern large-scale recommendation systems usually include multiple stages where retrieval aims at retrieving candidates from billions of candidate pools, and ranking predicts which item a user tends to engage from the trimmed candidate set retrieved from early stages [2]. General multi-stage recommendation system design in Pinterest.
Systems 67
67 
Bring Your Own Algorithm to Anomaly Detection
Pinterest Engineering
OCTOBER 17, 2023
Warden started off as a Java Thrift service built around the EGADs open-source library, which contains Java implementations of various time-series anomaly detection algorithms. They found the existing selection of anomaly detection algorithms in EGADs to be limiting. Each job is load-balanced to a node in the Warden cluster.
Algorithm 110
110 
Difference between distributed learning versus federated learning algorithms
KDnuggets
NOVEMBER 19, 2021
Want to know the difference between distributed and federated learning? Read this article to find out.
Algorithm 160
160 
Time Complexity: Significance, Types, Algorithms
Knowledge Hut
JANUARY 29, 2024
” In this article, we are going to discuss time complexity of algorithms and how they are significant to us. Nobody would want to use a system which takes a lot of time to process large input size. The Time complexity of an algorithm is the actual time needed to execute the particular codes.
Algorithm 98
98 
Enhancing Distributed System Load Shedding with TCP Congestion Control Algorithm
Zalando Engineering
APRIL 22, 2024
But our system is event driven, all requests we process are delivered as events via Nakadi. We know if our system runs within its normal limits that we meet our SLOs. If we would control the ingestion of message requests into our system we would be able to process the task in a timely manner.
Algorithm 76
76 
Title Launch Observability at Netflix Scale
Netflix Tech
JANUARY 6, 2025
In this case, the main stakeholders are: - Title Launch Operators Role: Responsible for setting up the title and its metadata into our systems. In this context, were focused on developing systems that ensure successful title launches, build trust between content creators and our brand, and reduce engineering operational overhead.
Metadata 87
87 
Movie Recommendation System: Definition, Strategies, Usecase
Knowledge Hut
FEBRUARY 1, 2024
Not only could this recommendation system save time browsing through lists of movies, it can also give more personalized results so users don’t feel overwhelmed by too many options. What are Movie Recommendation Systems? Recommender systems have two main categories: content-based & collaborative filtering.
Systems 98
98 
What are the Commonly Used Machine Learning Algorithms?
Knowledge Hut
APRIL 26, 2024
There is no end to what can be achieved with the right ML algorithm. Machine Learning is comprised of different types of algorithms, each of which performs a unique task. U sers deploy these algorithms based on the problem statement and complexity of the problem they deal with.
Real-Time AI for Crisis Management: Responding Faster with Smarter Systems
Striim
JANUARY 30, 2025
By integrating AI/ML models directly into these data streams, organizations gain deeper insights: advanced algorithms can spot emerging patterns, predict cascading effects, and recommend interventionsall in the moment. Systems must be capable of handling high-velocity data without bottlenecks.
Systems 52
52 
Interesting startup idea: benchmarking cloud platform pricing
The Pragmatic Engineer
OCTOBER 17, 2024
We recently covered how CockroachDB joins the trend of moving from open source to proprietary and why Oxide decided to keep using it with self-support , regardless Web hosting: Netlify : chosen thanks to their super smooth preview system with SSR support.
Cloud 332
332 
The Future of Reliable Data + AI—Observing the Data, System, Code, and Model
Monte Carlo
MARCH 28, 2025
Table of Contents Understanding How Data + AI Can Break Data System Code Model Data + AI observability must cover inputs and outputs it is all or nothing Understanding How Data + AI Can Break Data + AI applications are complex. But code takes on new weight in the data + AI system.
Coding 52
52 
How Precision Time Protocol handles leap seconds
Engineering at Meta
FEBRUARY 3, 2025
This is particularly true in the data center space, where new protocols like Precision Time Protocol (PTP) are allowing systems to be synchronized down to nanosecond precision. The difference between two approaches can mean differences of over 100 microseconds, creating challenges for services that consume time from both systems.
Algorithm 87
87 
Scaling the Instagram Explore recommendations system
Engineering at Meta
AUGUST 9, 2023
Explore is one of the largest recommendation systems on Instagram. Using more advanced machine learning models, like Two Towers neural networks, we’ve been able to make the Explore recommendation system even more scalable and flexible. locally popular media), which further contributes to system scalability.
Systems 90
90 
Arcadia: An end-to-end AI system performance simulator
Engineering at Meta
SEPTEMBER 7, 2023
We’re introducing Arcadia, Meta’s unified system that simulates the compute, memory, and network performance of AI training clusters. We need a systemized source of truth that can simulate various performance factors across compute, storage, and network collectively. For instance, the AI Research SuperCluster for AI research.
Systems 106
106 
Mainframe Data Meets AI: Reducing Bias and Enhancing Predictive Power
Precisely
DECEMBER 12, 2024
Key Takeaways : The significance of using legacy systems like mainframes in modern AI. The challenges and solutions involved in integrating legacy data with modern AI systems. These systems store massive amounts of historical datadata that has been accumulated, processed, and secured over decades of operation.
Healthcare 64
64 
An educational side project
The Pragmatic Engineer
JUNE 1, 2023
Juraj included system monitoring parts which monitor the server’s capacity he runs the app on: The monitoring page on the Rides app And it doesn’t end here. Juraj created a systems design explainer on how he built this project, and the technologies used: The systems design diagram for the Rides application The app uses: Node.js
Education 364
364 
Foundation Model for Personalized Recommendation
Netflix Tech
MARCH 28, 2025
By Ko-Jen Hsiao , Yesu Feng and Sudarshan Lamkhede Motivation Netflixs personalized recommender system is a complex system, boasting a variety of specialized machine learned models each catering to distinct needs including Continue Watching and Todays Top Picks for You. Refer to our recent overview for more details).
Metadata 84
84 
Handling Online-Offline Discrepancy in Pinterest Ads Ranking System
Pinterest Engineering
JANUARY 18, 2024
In particular, our machine learning powered ads ranking systems are trying to understand users’ engagement and conversion intent and promote the right ads to the right user at the right time. Our engineers are constantly discovering new algorithms and new signals to improve the performance of our machine learning models.
Systems 98
98 
Ensuring the Successful Launch of Ads on Netflix
Netflix Tech
JUNE 1, 2023
We used this simulation to help us surface problems of scale and validate our Ads algorithms. Replay traffic enabled us to test our new systems and algorithms at scale before launch, while also making the traffic as realistic as possible. We also constructed and checked our ad monitoring and alerting system during this period.
Algorithm 139
139 
Ensuring Even Ad Spend on the Zalando Homepage: How Our New Bidding Algorithm Maximizes Value for Advertisers and Shoppers
Zalando Engineering
SEPTEMBER 16, 2024
Currently, ad placements on the homepage are determined by a real-time bidding system. This system can lead to situations where your ad budget is exhausted early in the campaign period, limiting your potential reach. Ad spend: old vs. new algorithm However, a hidden hurdle exists: uneven ad spend. The Consequence?
Algorithm 87
87 
Top 10 Data Engineering Trends in 2025
Edureka
APRIL 22, 2025
To meet this need, people who work in data engineering will focus on making systems that can handle ongoing data streams with little delay. Cloud-Native Data Engineering These days, cloud-based systems are the best choice for data engineering infrastructure because they are flexible and can grow as needed.
They Handle 500B Events Daily. Here’s Their Data Engineering Architecture.
Monte Carlo
NOVEMBER 12, 2024
Every time you play, skip, or save a song, Spotify notes the behavior and passes it to their recommendation system through Kafka. Meta’s algorithms consider a range of engagement data, like which posts catch your attention, how long you view them, and who you engage with most often.
Architecture 96
96 
Building a Media Understanding Platform for ML Innovations
Netflix Tech
MARCH 14, 2023
Earlier we shared the details of one of these algorithms , introduced how our platform team is evolving the media-specific machine learning ecosystem , and discussed how data from these algorithms gets stored in our annotation service. Some ML algorithms are computationally intensive. Processing took several hours to complete.
Media 119
119 
How GenAI is Transforming Quality Control and Safety in the F&B Industry.
RandomTrees
DECEMBER 17, 2024
These problems have created a situation where AI systems, especially GenAI, need to be integrated to improve and automate quality control systems. Regulatory Updates: AI algorithms perform and analyze the news and changes related to regulations free of charge, making compliance simple for businesses.
Food 52
52 
Smart Utilities in Action: Generative AI’s Role in Real-Time Fault Detection
RandomTrees
JANUARY 30, 2025
Utility systems are completely revolutionized into intelligent, self-sustained ecosystems through an interconnected AI network. The grid allows AI systems to gather information via IoT sensors, smart utility meters, and other devices for real-time analysis.
Utilities 52
52 
From Event-Driven Chaos to a Blazingly Fast Serving API
Zalando Engineering
MARCH 6, 2025
To serve the presentation view of a Product Offer, a multi-stage event-driven system merged Product, Price, and Stock events into a single structure. To solve this, we leveraged a powerful load balancing algorithm for our products component, Consistent Hash Load Balancing (CHLB).
Algorithm 63
63 
Introducing the Robinhood Crypto Trading API
Robinhood
MAY 30, 2024
With the Robinhood Crypto trading API, customers can write their own programs to engage with cryptocurrency markets in real-time, leveraging algorithms and strategies to execute trades swiftly and efficiently. We also engage third-party security experts to test our systems, helping us build some of the most secure systems in the industry.
Insurance 142
142 
The Emerging Role of AI Data Engineers - The New Strategic Role for AI-Driven Success
Data Engineering Weekly
JANUARY 15, 2025
The answer lies in unstructured data processing—a field that powers modern artificial intelligence (AI) systems. To address these challenges, AI Data Engineers have emerged as key players, designing scalable data workflows that fuel the next generation of AI systems. How does a self-driving car understand a chaotic street scene?
Scaling Media Machine Learning at Netflix
Netflix Tech
FEBRUARY 13, 2023
This feature store is equipped with a data replication system that enables copying data to different storage solutions depending on the required access patterns. Amber is a suite of multiple infrastructure components that offers triggering capabilities to initiate the computation of algorithms with recursive dependency resolution.
Media 119
119 
Building Holiday Finds: How Pinterest Engineers Reimagined Gift Discovery
Pinterest Engineering
MARCH 26, 2025
Unified Logging System: We implemented comprehensive engagement tracking that helps us understand how users interact with gift content differently from standardPins. Unified Logging System: We implemented comprehensive engagement tracking that helps us understand how users interact with gift content differently from standardPins.
Building 59
59 
Engineering Privacy: A Technical Overview of Privacy in Data Systems
Data Engineering Weekly
SEPTEMBER 26, 2024
Most cloud providers offer built-in encryption options and key management systems (KMS) , making it easier to stay compliant without sacrificing security. Two prevalent models are: Role-Based Access Control (RBAC): This system assigns permissions based on predefined organizational roles (e.g., data analyst, marketing manager).
Systems 68
68 
Grid Modernization with AI for More Connected Utilities
RandomTrees
NOVEMBER 29, 2024
With the increasing demand for energy and renewable energy transition progressing forward, the older grid systems fall behind because they cannot meet the efficiency, reliability, and sustainability targets. The Need for Grid Modernization The lack of an energy transfer system worldwide has negatively affected economies and businesses.
Utilities 52
52 
10 Skills to Ace Your Data Engineering Interviews
Start Data Engineering
OCTOBER 11, 2021
Leetcode: data structures and algorithms 4. Distributed system fundamentals 7. System design 9. Introduction Skills 1. Data modeling 4.1 Data warehousing 4.2 Data pipelines 6. Event streaming 8. Business questions 10. Cloud computing 11.
Data Engineering 130
130 
Unlocking Real-Time Decision-Making with High-Velocity Data Analytics
Striim
APRIL 10, 2025
This growing demand for real-time analytics, scalable infrastructures, and optimized algorithms is driven by the need to handle large volumes of high-velocity data without compromising performance or accuracy. Enhanced Accuracy: Real-time processing utilizes sophisticated algorithms.
The Quest to Understand Metric Movements
Pinterest Engineering
FEBRUARY 11, 2025
How we are analyzing the metric segments takes inspiration from the algorithm in Linkedins ThirdEye. a new recommendation algorithm). For analytics tools like anomaly detection or root-cause analysis, the results are often mere suggestions for users who may not have a clear idea of the algorithms involved or how to tune them.
Algorithm 46
46 
Generative AI vs. Predictive AI: Understanding the Differences
Edureka
JUNE 7, 2024
Unlike traditional AI systems that operate on pre-existing data, generative AI models learn the underlying patterns and relationships within their training data and use that knowledge to create novel outputs that did not previously exist. paintings, songs, code) Historical data relevant to the prediction task (e.g., stock market trends).
Deep Learning 114
114 
Boosting Media & Entertainment Production Efficiency with AI and Cloud
RandomTrees
NOVEMBER 13, 2024
However, AI-assisted editing tools are transforming the systems that are capable of eliminating tough jobs from the editing process. Machine learning algorithms are capable of absorbing a specific editor or director’s editing style and utilizing those principles for new projects, leading to quicker and more consistent edits.
Title Launch Observability at Netflix Scale
Netflix Tech
DECEMBER 17, 2024
To achieve this, we are committed to building robust systems that deliver comprehensive observability, enabling us to take full accountability for every title on ourservice. Each title represents countless hours of effort and creativity, and our systems need to honor that uniqueness. Yet, these pages couldnt be more different.
Metadata 68
68 
Introducing Configurable Metaflow
Netflix Tech
DECEMBER 19, 2024
Many of these projects are under constant development by dedicated teams with their own business goals and development best practices, such as the system that supports our content decision makers , or the system that ranks which language subtitles are most valuable for a specific piece ofcontent.
How Meta trains large language models at scale
Engineering at Meta
JUNE 12, 2024
Solving this problem requires a robust and high-speed network infrastructure as well as efficient data transfer protocols and algorithms. This includes developing new algorithms and techniques for efficient large-scale training and integrating new software tools and frameworks into our infrastructure.
Algorithm 129
129 
The Power of Predictive Analytics: Leveraging Data to Forecast Business Trends
RandomTrees
MARCH 10, 2025
Revenue Growth: Marketing teams use predictive algorithms to find high-value leads, optimize campaigns, and boost ROI. Data Integration: Combine data from several sources, including as CRM systems, social media, and IoT devices, to generate a holistic perspective.
Retail 52
52 
New Applied ML Prototypes Now Available in Cloudera Machine Learning
Cloudera
NOVEMBER 17, 2021
In the hands of an experienced practitioner, AutoML holds much promise for automating away some of the tedious parts of building machine learning systems. TPOT is a library for performing sophisticated search over whole ML pipelines, selecting preprocessing steps and algorithm hyperparameters to optimize for your use case.
Machine Learning 124
124 
Let's personalize your content