Pinterest Engineering

SEPTEMBER 19, 2024

Manager, Engineering | At Pinterest, we operate a large-scale online machine learning inference system, where feature caching plays a critical role to achieve optimal efficiency. Background Recommender systems are fundamental to Pinterest’s mission to inspire users to create a life they love.

Systems 56

Systems Utilities Architecture Software Engineering 56

The Modern Data Company

JANUARY 31, 2023

In 2023, The Modern Data Company (Modern) hopes to reach more companies and organizations with our data operating system, build incredible value from existing and upcoming data assets, and share insights into major shifts in what it means to be data-driven. These were our most popular blog posts in 2022 according to reader statistics.

Retail 98

Retail Healthcare Data Architecture Data Pipeline 98

Data Engineering Weekly

SEPTEMBER 26, 2024

Most cloud providers offer built-in encryption options and key management systems (KMS) , making it easier to stay compliant without sacrificing security. Two prevalent models are: Role-Based Access Control (RBAC): This system assigns permissions based on predefined organizational roles (e.g., data analyst, marketing manager).

Systems 67

Systems Engineering Data Warehouse Architecture 67

Pinterest Engineering

AUGUST 8, 2024

In our previous system, which operated on a daily budget allocation model, the system relied on predicting daily budgets for individual users on a daily basis, constraining the flexibility and responsiveness required for dynamic user engagement and content changes. Figure 1 below shows the overview of the system architecture.

Systems 43

Systems Machine Learning Utilities Architecture 43

databricks

NOVEMBER 12, 2024

Monolithic to Modular The proof of concept (POC) of any new technology often starts with large, monolithic units that are difficult to characterize.

Systems 105

Systems Engineering Technology Data 105

Pinterest Engineering

JANUARY 18, 2024

In particular, our machine learning powered ads ranking systems are trying to understand users’ engagement and conversion intent and promote the right ads to the right user at the right time. Specifically, such discrepancies unfold into the following scenarios: Bug-free scenario : Our ads ranking system is working bug-free.

Systems 94

Systems Machine Learning Data Mining Algorithm 94

Zalando Engineering

MAY 13, 2024

As a design system evolves alongside with the brand it represents, there are often multiple occasions when a need to introduce variations arises. The previous article on this blog gives a wider overview of the Zalando Design System. The previous article on this blog gives a wider overview of the Zalando Design System.

Designing 59

Designing Systems Architecture Coding 59

databricks

NOVEMBER 13, 2024

As organizations integrate AI agent systems into. Generative AI has become a powerful reality, transforming industries by enhancing customer experiences and automating decisions.

Systems 79

Systems 79

Data Engineering Podcast

NOVEMBER 20, 2022

Summary The majority of blog posts and presentations about data engineering and analytics assume that the consumers of those efforts are internal business users accessing an environment controlled by the business. The biggest challenge with modern data systems is understanding what data you have, where it is located, and who is using it.

Systems 130

Systems Metadata Data Pipeline MongoDB 130

Cloudera

JUNE 2, 2022

This blog aims to answer two questions: What is a universal data distribution service? In a recent customer workshop with a large retail data science media company, one of the attendees, an engineering leader, made the following observation: “Everytime I go to your competitor website, they only care about their system.

Systems 104

Systems Data Lake Google Cloud Cloud 104

databricks

SEPTEMBER 4, 2024

Recommender systems (RecSys) have become an integral part of modern digital experiences, powering personalized content suggestions across various platforms. These sophisticated systems and.

Systems 72

Systems Data Science Data Engineering Data Engineer 72

Eventbrite Engineering

JUNE 13, 2022

Obviously, the answer is that we need to … Continue reading "Monitoring Your System" The post Monitoring Your System appeared first on Engineering Blog.

Systems 52

Systems Engineering Technology IT 52

Cloudera

SEPTEMBER 15, 2022

In this blog post, we will talk about a single Ozone cluster with the capabilities of both Hadoop Core File System (HCFS) and Object Store (like Amazon S3). A unified storage architecture that can store both files and objects and provide a flexible, scalable, and high-performance system. Bucket types. release version.

Systems 87

Systems Hadoop Metadata Telecommunication 87

Cloudera

SEPTEMBER 20, 2021

In this blog series, we will discuss each of these deployments and the deployment choices made along with how they impact reliability. In Part 1, the discussion is related to: Serial and Parallel Systems Reliability as a concept, Kafka Clusters with and without Co-Located Apache Zookeeper, and Kafka Clusters deployed on VMs. .

Kafka 115

Kafka Systems Utilities Bytes 115

DataKitchen

DECEMBER 9, 2022

This can include the use of tools for data preparation, model training, and deployment, as well as technologies for monitoring and managing data-related systems and processes. One of the key benefits of DataOps observability is the ability to improve collaboration and communication across teams and systems.

Machine Learning 52

Machine Learning Data Preparation Government Data Analytics 52

Uber Engineering

SEPTEMBER 9, 2021

One such technology is called Schemaless , which enables the modeling of related entries in one single row of multiple columns, as well as … The post Jellyfish: Cost-Effective Data Tiering for Uber’s Largest Storage System appeared first on Uber Engineering Blog.

Systems 118

Systems Data Technology Engineering 118

Cloudera

SEPTEMBER 19, 2022

The identity team at Cloudera has been working to add the System for Cross-domain Identity Management (SCIM) support to Cloudera Data Platform (CDP) and we’re happy to announce the general availability of SCIM on Azure Active Directory! SCIM (System for Cross-domain Identity Management): an Introduction. Are revocable.

Systems 100

Systems Management Accessible Accessibility 100

Striim

JULY 30, 2024

Ensuring system resilience is critical for maintaining a competitive edge in today’s data-driven world. As businesses rely on real-time data to fuel decision-making, it’s essential that their systems can withstand disruptions and maintain functionality. Check out this blog.)

Systems 52

Systems Healthcare Utilities Data Integration 52

Knowledge Hut

JANUARY 3, 2024

A major computer system component is its operating system (OS). A computer would be a little more than a useless computer without an operating system. And at least one operating system must be installed on your computer to run simple programs like browsers. What is Operating System (OS)?

Systems 52

Systems Programming Python Accessible 52

Lyft Engineering

APRIL 25, 2023

Building a large scale unsupervised model anomaly detection system — Part 2 Building ML Models with Observability at Scale By Rajeev Prabhakar , Han Wang , Anindya Saha Photo by Octavian Rosca on Unsplash In our previous blog we discussed the different challenges we faced for model monitoring and our strategy for addressing some of these problems.

Systems 76

Systems Building Machine Learning Datasets 76

Knowledge Hut

JANUARY 18, 2024

CISSP stands for Certified Information Systems Security Professional, and it is a certification in cyber security. This professional certification is developed and offered by (ISC)2, also known as International Information Systems Security Certification Consortium. This blog will help you understand the question “what is CISSP?

Systems 52

Systems Certification Consulting Education 52

Cloudera

MAY 25, 2021

Recommendation systems have become a cornerstone of modern life, spanning sectors that include online retail, music and video streaming, and even content publishing. These systems help us navigate the sheer volume of content on the internet, allowing us to discover what’s interesting or important to us.

Systems 62

Systems Algorithm Retail Utilities 62

Knowledge Hut

JULY 26, 2023

But what precisely is operating system virtualization? This blog will provide you all the information about the Operating System virtualization along with AWS Solution Architect syllabus. What is Operating System Virtualization? Uses of Operating System Virtualization?

Systems 52

Systems Cloud Computing Utilities Cloud 52

Edureka

JULY 18, 2024

It understands concepts like ambiguity and nuance – the two biggest blindspots of traditional computer systems This shift from generic AI to context-aware systems paves the way for a more natural and effective human-machine interaction. Think about a smart home system. New to generative AI? How Does Contextual AI Work?

Systems 40

Systems Retail Food Education 40

Uber Engineering

FEBRUARY 23, 2023

In this blog learn how we automated column-level drift detection in batch datasets at Uber scale, reducing the median time to detect issues in critical datasets by 5X. Data quality is of paramount importance at Uber, powering critical decisions and features.

Systems 85

Systems Datasets Data 85

Zalando Engineering

JUNE 30, 2020

Our Engineering Blog was launched in June 2020 after a long break of the previous tech blog. What customizations we applied to design the blog and the publishing process. Static Site Generator Our previous tech blog used a CMS which only a limited number of people had access to. So which static site generator to choose?

Engineering 40

Engineering Bytes AWS Python 40

LinkedIn Engineering

DECEMBER 6, 2022

With a reasonably sizable footprint of servers in data centers, LinkedIn is responsible for ensuring that these hosts are always on an operating system (OS) version deemed the ���latest and greatest��� for all intents and purposes.

Systems 55

Systems Metadata Datasets Designing 55

Rockset

DECEMBER 19, 2023

From his early days at Quora to leading projects at Facebook and his current venture at Fennel (a real-time feature store for ML), Nikhil has traversed the evolving landscape of machine learning engineering and machine learning infrastructure specifically in the context of recommendation systems.

Systems 52

Systems Machine Learning Deep Learning Media 52

Netflix Tech

FEBRUARY 10, 2021

Stranger Things imagery showcasing the inspiration for the Hawkins Design System by Hawkins team member Joshua Godi ; with art contributions by Wiki Chaves Hawkins may be the name of a fictional town in Indiana, most widely known as the backdrop for one of Netflix’s most popular TV series “Stranger Things,” but the name is so much more.

Designing 93

Designing Systems Building Entertainment 93

Data Science Blog: Data Engineering

SEPTEMBER 19, 2023

In the contemporary age of Big Data, Data Warehouse Systems and Data Science Analytics Infrastructures have become an essential component for organizations to store, analyze, and make data-driven decisions. Scalability Tailored for Data Needs Data Warehouse Systems often require to scale quickly to handle larger datasets or more queries.

Data Warehouse 52

Data Warehouse Coding Systems Cloud 52

Robinhood

NOVEMBER 13, 2024

With lower barriers to entry, we believe crypto presents an opportunity for those who have been historically left behind by the traditional financial system.” Robinhood Crypto is licensed to engage in virtual currency business activity by the New York State Department of Financial Services.

Insurance 140

Insurance Accessible Accessibility Project 140

U-Next

DECEMBER 19, 2022

They are not only always trying to find loopholes in any security system but also land up succeeding most of the time. The post Top 10 Emerging Technologies Blogs To Read In 2023 appeared first on Jigsaw Academy. Types Of Hackers Based On Their Intent | Who Are Ethical Hackers? .

Technology 52

Technology Cloud Computing Java Cloud Storage 52

Uber Engineering

APRIL 5, 2018

This analysis powers our services and enables the delivery of more seamless and reliable user … The post Scaling Uber’s Apache Hadoop Distributed File System for Growth appeared first on Uber Engineering Blog.

Hadoop 109

Hadoop Systems Big Data Data Analysis 109

Zalando Engineering

JULY 20, 2022

Yet Another "What is a Design System?" There is a lot of literature and countless blog posts around the very definition of the concept of design systems. You can also find more information about the creation process from a design point of view in this blog post.

Designing 52

Designing Systems Electronics Architecture 52

Data Engineering Podcast

FEBRUARY 18, 2019

Summary Distributed storage systems are the foundational layer of any big data stack. Alluxio is a distributed virtual filesystem which integrates with multiple persistent storage systems to provide a scalable, in-memory storage layer for scaling computational workloads independent of the size of your data.

Systems 100

Systems Java Media Algorithm 100

Pinterest Engineering

JANUARY 10, 2023

Liang Ma | Software Engineer, Core Eng; Wei Zhu | Software Engineer, Observability In early 2020, during a critical iOS out of memory incident (we have a blogpost for that), we realized that we didn’t have much visibility of how the app is running or a good system to look up for monitoring and troubleshooting.

Systems 57

Systems Building Software Engineering Software Engineer 57

The Pragmatic Engineer

JUNE 1, 2023

Juraj included system monitoring parts which monitor the server’s capacity he runs the app on: The monitoring page on the Rides app And it doesn’t end here. Juraj created a systems design explainer on how he built this project, and the technologies used: The systems design diagram for the Rides application The app uses: Node.js

Education 363

Education Project PostgreSQL Software Engineering 363