The Modern Data Company

JANUARY 31, 2023

In 2023, The Modern Data Company (Modern) hopes to reach more companies and organizations with our data operating system, build incredible value from existing and upcoming data assets, and share insights into major shifts in what it means to be data-driven. These were our most popular blog posts in 2022 according to reader statistics.

Retail 98

Retail Healthcare Data Architecture Data Pipeline 98

Pinterest Engineering

JANUARY 10, 2023

Liang Ma | Software Engineer, Core Eng; Wei Zhu | Software Engineer, Observability In early 2020, during a critical iOS out of memory incident (we have a blogpost for that), we realized that we didn’t have much visibility of how the app is running or a good system to look up for monitoring and troubleshooting.

Systems 57

Systems Building Software Engineer Software Engineering 57

phData: Data Engineering

NOVEMBER 8, 2024

The world we live in today presents larger datasets, more complex data, and diverse needs, all of which call for efficient, scalable data systems. In this blog, we will discuss: What is the Open Table format (OTF)? These systems are built on open standards and offer immense analytical and transactional processing flexibility.

Architecture 52

Architecture Systems Data Lake Google Cloud 52

Rockset

SEPTEMBER 2, 2022

I recently had the good fortune to host a small-group discussion on personalization and recommendation systems with two technical experts with years of experience at FAANG and other web-scale companies. Nikhil Garg is CEO and co-founder of Fennel AI , a startup working on building the future of real-time machine learning infrastructure.

Systems 52

Systems Building Machine Learning NoSQL 52

Data Engineering Podcast

NOVEMBER 20, 2022

Summary The majority of blog posts and presentations about data engineering and analytics assume that the consumers of those efforts are internal business users accessing an environment controlled by the business. The biggest challenge with modern data systems is understanding what data you have, where it is located, and who is using it.

Systems 130

Systems Metadata Data Pipeline MongoDB 130

Pinterest Engineering

JULY 15, 2024

David Chang; Staff Software Engineer | Develocity, formerly known as Gradle Enterprise, is a powerful tool that speeds up local and CI build time, helps troubleshoot your builds, and analyzes your data. At Pinterest, we have a dedicated team, Mobile Builds, and we ensure that developers can build fast and often.

Building 64

Building Metadata Software Engineer Software Engineering 64

Zalando Engineering

MAY 13, 2024

As a design system evolves alongside with the brand it represents, there are often multiple occasions when a need to introduce variations arises. The previous article on this blog gives a wider overview of the Zalando Design System. The previous article on this blog gives a wider overview of the Zalando Design System.

Designing 59

Designing Systems Architecture Coding 59

Cloudera

JUNE 2, 2022

Over the last two years, the Cloudera DataFlow team has been hard at work building Cloudera DataFlow for the Public Cloud (CDF-PC). This blog aims to answer two questions: What is a universal data distribution service? How to onboard data into their system? I don’t care about their system. What is the modern data stack?

Systems 103

Systems Data Lake Google Cloud Cloud 103

Pinterest Engineering

JANUARY 4, 2024

In 2020, anticipating the growing needs of the business and to simplify our storage offerings, we decided to consolidate our different key-value systems in the company into a single unified service called KVStore. In order to build a distributed and replicated service using RocksDB, we built a real time replicator library: Rocksplicator.

Database 93

Database Building Datasets Relational Database 93

Zalando Engineering

OCTOBER 16, 2024

Introduction Context and Purpose Our team is part of the Transport teams within the Logistics department, where we build and manage software for internal users, including finance teams, warehouses, and, in the future, our third-party partners. Modularity was key to enabling independent feature development and deployment across teams.

Building 68

Building Transportation Coding Designing 68

Uber Engineering

SEPTEMBER 29, 2021

a driver starting a trip) and system actions … The post Building Uber’s Fulfillment Platform for Planet-Scale using Google Cloud Spanner appeared first on Uber Engineering Blog. The platform handles billions of database transactions each day, ranging from user actions (e.g.,

Google Cloud 137

Google Cloud Cloud Building Database 137

Netflix Tech

FEBRUARY 10, 2021

Stranger Things imagery showcasing the inspiration for the Hawkins Design System by Hawkins team member Joshua Godi ; with art contributions by Wiki Chaves Hawkins may be the name of a fictional town in Indiana, most widely known as the backdrop for one of Netflix’s most popular TV series “Stranger Things,” but the name is so much more.

Designing 93

Designing Systems Building Entertainment 93

Netflix Tech

MARCH 14, 2023

We build creator tooling to enable these colleagues to focus their time and energy on creativity. We implemented a batch processing system for users to submit their requests and wait for the system to generate the output. This limited pilot system greatly reduced the time spent by our users to manually analyze the content.

Media 118

Media Building Algorithm Machine Learning 118

Cloudera

DECEMBER 1, 2023

It focuses on five key pillars: investing in research and development; unleashing government AI resources; setting standards and policy; building the AI workforce; and advancing trust and security. AI systems must operate within a framework that promotes ethical practices, transparency, and accountability. million), among others.

Building 108

Building Government Transportation Data Governance 108

DataKitchen

DECEMBER 9, 2022

This can include the use of tools for data preparation, model training, and deployment, as well as technologies for monitoring and managing data-related systems and processes. This can help organizations to build trust in their data-related workflows, and to drive better outcomes from their data analytics and machine learning initiatives.

Machine Learning 52

Machine Learning Data Preparation Government Data Analytics 52

Tweag

JULY 26, 2023

The vast majority of the Rust projects are using Cargo as a build tool. Cargo is great when you are developing and packaging a single Rust library or application, but when it comes to a fast-growing and complex workspace, one could be attracted to the idea of using a more flexible and scalable build system.

Building 97

Building Metadata Coding Project 97

LinkedIn Engineering

DECEMBER 20, 2023

At LinkedIn, trust is the cornerstone for building meaningful connections and professional relationships. In this blog post, we discuss how we are harnessing AI to help us with abuse prevention and share an overview of our infrastructure and the role it plays in identifying and mitigating abusive behavior on our platform.

Building 91

Building Algorithm Kafka Machine Learning 91

Yelp Engineering

MARCH 7, 2024

This blog post covers how we leverage Yelp’s extensive streaming infrastructure to build robust data abstractions for our offline and streaming data consumers. Key terminology Let’s start by covering certain key terms used throughout the post: Offline systems - data warehousing platforms such as AWS Redshift or.

Building 59

Building AWS Data Pipeline Data 59

Lyft Engineering

JUNE 28, 2023

In early 2022, Lyft already had a comprehensive Machine Learning Platform called LyftLearn composed of model serving , training , CI/CD, feature serving , and model monitoring systems. However, streaming data was not supported as a first-class citizen across many of the platform’s systems — such as training, complex monitoring, and others.

Machine Learning 116

Machine Learning Building Metadata Kafka 116

Striim

JULY 30, 2024

Ensuring system resilience is critical for maintaining a competitive edge in today’s data-driven world. As businesses rely on real-time data to fuel decision-making, it’s essential that their systems can withstand disruptions and maintain functionality. Check out this blog.)

Systems 52

Systems Healthcare Utilities Data Integration 52

Knowledge Hut

JANUARY 18, 2024

CISSP stands for Certified Information Systems Security Professional, and it is a certification in cyber security. This professional certification is developed and offered by (ISC)2, also known as International Information Systems Security Certification Consortium. This blog will help you understand the question “what is CISSP?

Systems 52

Systems Certification Consulting Education 52

Edureka

APRIL 19, 2024

In other words, you don’t need to be a programmer or have any coding background to dive into this creative process of building a chatbot using prompt engineering. In this tutorial blog, we’ll be building a chatbot using prompt engineering inspired by Steve Harvey, offering tailored advice on life motivation and personal growth.

Building 52

Building Engineering Coding Systems 52

LinkedIn Engineering

DECEMBER 6, 2022

With a reasonably sizable footprint of servers in data centers, LinkedIn is responsible for ensuring that these hosts are always on an operating system (OS) version deemed the ���latest and greatest��� for all intents and purposes. How do we build an OS snapshot?

Systems 55

Systems Metadata Datasets Designing 55

The Pragmatic Engineer

JUNE 1, 2023

Juraj included system monitoring parts which monitor the server’s capacity he runs the app on: The monitoring page on the Rides app And it doesn’t end here. Juraj created a systems design explainer on how he built this project, and the technologies used: The systems design diagram for the Rides application The app uses: Node.js

Education 363

Education Project PostgreSQL Software Engineer 363

Knowledge Hut

JANUARY 3, 2024

A major computer system component is its operating system (OS). A computer would be a little more than a useless computer without an operating system. And at least one operating system must be installed on your computer to run simple programs like browsers. What is Operating System (OS)?

Systems 52

Systems Programming Python Accessible 52

Lyft Engineering

APRIL 24, 2024

See our previous blog post for more on the motivations behind TLC. The Technical Learning workstream offers Lyft scientists the opportunity to build or brush up their core data science skills across multiple areas. Some recent examples can be found in our tech blog: reinforcement learning and structural causal modeling.

Data Science 67

Data Science Building Computer Science Software Engineer 67

Zalando Engineering

JUNE 30, 2020

Our Engineering Blog was launched in June 2020 after a long break of the previous tech blog. What customizations we applied to design the blog and the publishing process. Static Site Generator Our previous tech blog used a CMS which only a limited number of people had access to. So which static site generator to choose?

Engineering 40

Engineering Bytes AWS Python 40

Edureka

JULY 18, 2024

It understands concepts like ambiguity and nuance – the two biggest blindspots of traditional computer systems This shift from generic AI to context-aware systems paves the way for a more natural and effective human-machine interaction. Think about a smart home system. New to generative AI? How Does Contextual AI Work?

Systems 40

Systems Retail Food Education 40

Cloudera

JUNE 17, 2022

In the second blog of the Universal Data Distribution blog series , we explored how Cloudera DataFlow for the Public Cloud (CDF-PC) can help you implement use cases like data lakehouse and data warehouse ingest, cybersecurity, and log optimization, as well as IoT and streaming data collection. What are inbound connections?

Data Pipeline 93

Data Pipeline Building Kafka Java 93

Rockset

DECEMBER 19, 2023

From his early days at Quora to leading projects at Facebook and his current venture at Fennel (a real-time feature store for ML), Nikhil has traversed the evolving landscape of machine learning engineering and machine learning infrastructure specifically in the context of recommendation systems.

Systems 52

Systems Machine Learning Deep Learning Media 52

Data Engineering Podcast

FEBRUARY 18, 2019

Summary Distributed storage systems are the foundational layer of any big data stack. Alluxio is a distributed virtual filesystem which integrates with multiple persistent storage systems to provide a scalable, in-memory storage layer for scaling computational workloads independent of the size of your data.

Systems 100

Systems Java Media Algorithm 100

Workfall

FEBRUARY 7, 2023

With the introduction of WebAssembly, it became possible to build frontend web apps in Rust , such as the one we just built, expanding development opportunities for developers. Hands-on We are assuming that Rust is installed in your system. In case you need to install, you can refer to our Rust blog.

Building 87

Building PostgreSQL Coding AWS 87

Data Engineering Podcast

NOVEMBER 26, 2019

In this episode James Cunningham and Ted Kaemming describe the process of rearchitecting a production system, what they learned in the process, and some useful tips for anyone else evaluating Clickhouse. What did the previous system look like? What was your design criteria for building a new platform?

Data Warehouse 100

Data Warehouse Building PostgreSQL Kafka 100

Cloudera

FEBRUARY 8, 2021

This is part 2 in this blog series. This blog series follows the manufacturing, operations and sales data for a connected vehicle manufacturer as the data goes through stages and transformations typically experienced in a large manufacturing company on the leading edge of current technology. Conclusion.

Data Pipeline 117

Data Pipeline Building Manufacturing Data Warehouse 117

Data Engineering Podcast

APRIL 19, 2020

Summary Modern applications frequently require access to real-time data, but building and maintaining the systems that make that possible is a complex and time consuming endeavor. This was an interesting inside look at building a business on top of open source stream processing frameworks and how to reduce the burden on end users.

Building 100

Building PostgreSQL MongoDB SQL 100

Picnic Engineering

SEPTEMBER 20, 2022

In this blog we will dive into our plans to further scale data science at Picnic. Data Science at Picnic We have been building and maintaining our ML systems as a central Data Science team with a lot of emphasis on software engineering skills. So far, this has been working great for Picnic.

Data Science 52

Data Science Building Machine Learning Software Engineer 52

Cloudera

OCTOBER 31, 2023

However, it remains difficult to quickly build these capabilities given the challenges with finding readily available talent and resources to get started rapidly on the AI journey. Customers can quickly and easily build generative AI applications using these new features available in Cloudera.

Building 102

Building Amazon Web Services AWS Machine Learning 102