article thumbnail

Data Migration Strategies For Large Scale Systems

Data Engineering Podcast

Summary Any software system that survives long enough will require some form of migration or evolution. When that system is responsible for the data layer the process becomes more challenging. This episode is supported by Code Comments, an original podcast from Red Hat. My thanks to the team at Code Comments for their support.

Systems 130
article thumbnail

Automating dead code cleanup

Engineering at Meta

Meta’s Systematic Code and Asset Removal Framework (SCARF) has a subsystem for identifying and removing dead code. SCARF combines static and dynamic analysis of programs to detect dead code from both a business and programming language perspective. These are combined and form an augmented dependency graph.

Coding 133
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Why using Infrastructure as Code for developing Cloud-based Data Warehouse Systems?

Data Science Blog: Data Engineering

In the contemporary age of Big Data, Data Warehouse Systems and Data Science Analytics Infrastructures have become an essential component for organizations to store, analyze, and make data-driven decisions. Infrastructure as Code (IaC) can be a game-changer in this scenario. So why using IaC for Cloud Data Infrastructures?

article thumbnail

A Tour Around Buck2, Meta's New Build System

Tweag

Buck2 is a from-scratch rewrite of Buck , a polyglot, monorepo build system that was developed and used at Meta (Facebook), and shares a few similarities with Bazel. As you may know, the Scalable Builds Group at Tweag has a strong interest in such scalable build systems. fix the code # fix code 7.

Systems 141
article thumbnail

Monetizing Analytics Features: Why Data Visualizations Will Never Be Enough

Think your customers will pay more for data visualizations in your application? Five years ago they may have. But today, dashboards and visualizations have become table stakes. Discover which features will differentiate your application and maximize the ROI of your embedded analytics. Brought to you by Logi Analytics.

article thumbnail

Movie Recommendation System: Definition, Strategies, Usecase

Knowledge Hut

Not only could this recommendation system save time browsing through lists of movies, it can also give more personalized results so users don’t feel overwhelmed by too many options. What are Movie Recommendation Systems? Recommender systems have two main categories: content-based & collaborative filtering.

Systems 98
article thumbnail

Build faster with Buck2: Our open source build system

Engineering at Meta

Buck2, our new open source, large-scale build system , is now available on GitHub. Buck2 is an extensible and performant build system written in Rust and designed to make your build experience faster and more efficient. In our internal tests at Meta, we observed that Buck2 completed builds 2x as fast as Buck1. Why rebuild Buck?

Building 144