The Pragmatic Engineer

MARCH 12, 2024

” Brooks agrees with this observation, and suggests a radical solution: have as few senior programmers as possible, and build a team around each one – a bit like how a hospital surgeon leads a whole team. CI/CD : running automated tests on all changes, and deploying code to production automatically. The toolsmith.

Engineering 274

Engineering Programming Language Hospitality Programming 274

The Pragmatic Engineer

JUNE 1, 2023

Juraj included system monitoring parts which monitor the server’s capacity he runs the app on: The monitoring page on the Rides app And it doesn’t end here. Juraj created a systems design explainer on how he built this project, and the technologies used: The systems design diagram for the Rides application The app uses: Node.js

Education 363

Education Project PostgreSQL Software Engineer 363

Engineering at Meta

MARCH 12, 2024

By the end of 2024, we’re aiming to continue to grow our infrastructure build-out that will include 350,000 NVIDIA H100 GPUs as part of a portfolio that will feature compute power equivalent to nearly 600,000 H100s. RSC has accelerated our open and responsible AI research by helping us build our first generation of advanced AI models.

Building 145

Building Portfolio Utilities Data Storage 145

Data Engineering Podcast

OCTOBER 15, 2023

Summary Building streaming applications has gotten substantially easier over the past several years. Decodable was built with a mission of eliminating all of the painful aspects of developing and deploying stream processing systems for engineering teams. How can you get the best results for your use case?

Process 182

Process Building SQL BI 182

The Pragmatic Engineer

NOVEMBER 21, 2023

If you had a continuous deployment system up and running around 2010, you were ahead of the pack: but today it’s considered strange if your team would not have this for things like web applications.    He then worked at the casual games company Zynga, building their in-game advertising platform.

Engineering 302

Engineering Bytes Cloud Computing AWS 302

Data Engineering Podcast

FEBRUARY 18, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. As a listener to the Data Engineering Podcast you can get a special discount of 20% off your ticket by using the promo code dataengpod20.

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

The Pragmatic Engineer

MARCH 19, 2024

A first, smaller wave of these stories included Magic.dev raising $100M in funding from Nat Friedman (CEO of GitHub from 2018-2021,) and Daniel Gross (cofounder of search engine Cue which Apple acquired in 2013,) to build a “superhuman software engineer.” years ago, and it became the leading AI coding assistant almost overnight.

Software Engineer 276

Software Engineer Software Engineering Programming Language Media 276

The Pragmatic Engineer

OCTOBER 17, 2024

A €150K ($165K) grant, three people, and 10 months to build it. We recently covered how CockroachDB joins the trend of moving from open source to proprietary and why Oxide decided to keep using it with self-support , regardless Web hosting:  Netlify : chosen thanks to their super smooth preview system with SSR support.

Cloud 278

Cloud AWS Metadata Cloud Computing 278

Knowledge Hut

FEBRUARY 1, 2024

Not only could this recommendation system save time browsing through lists of movies, it can also give more personalized results so users don’t feel overwhelmed by too many options. What are Movie Recommendation Systems? Recommender systems have two main categories: content-based & collaborative filtering.

Systems 98

Systems Entertainment Algorithm Datasets 98

Engineering at Meta

DECEMBER 6, 2023

For example, we recently published an updated white paper, “ Meta’s Approach to Safer Private Messaging on Messenger and Instagram Direct Messaging ,” t hat sets out the industry-leading safety systems and tools available on Messenger. The post Building end-to-end security for Messenger appeared first on Engineering at Meta.

Building 145

Building Designing Consulting Accessible 145

Data Engineering Podcast

DECEMBER 25, 2022

Recognizing this shortcoming and the capabilities that could be unlocked by a robust solution Rishabh Poddar helped to create Opaque Systems as an outgrowth of his PhD studies. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Machine Learning 130

Machine Learning Systems Data Lake Data Warehouse 130

Knowledge Hut

OCTOBER 29, 2023

Hospital Management System Type : Application development, Database management, Programming There is no shortage of computer science project topics out there. But if you are looking for something that's both technically challenging and socially relevant, consider a hospital management system. Source Code: Weather Forecast App 3.

Computer Science 111

Computer Science Coding Project Hospitality 111

Data Engineering Podcast

AUGUST 27, 2023

Summary Data persistence is one of the most challenging aspects of computer systems. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. Why Postgres?

Database 130

Database Building PostgreSQL BI 130

The Pragmatic Engineer

OCTOBER 10, 2023

He’s solved interesting engineering challenges along the way, too – like building observability for Amazon’s EC2 offering, and being one of the first engineers on Uber’s observability platform. From learning to code in Australia, to working in Silicon Valley How did I learn to code?

Software Engineer 268

Software Engineer Software Engineering Architecture Media 268

Data Engineering Podcast

APRIL 16, 2023

As the capabilities of these systems has improved and become more accessible, the target of what self-serve means changes. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Who are the target users?

Business Intelligence 147

Business Intelligence Building Data Lake BI 147

Data Engineering Podcast

NOVEMBER 20, 2022

In this episode Ian Schweer shares his experiences at Riot Games supporting player-focused features such as machine learning models and recommeder systems that are deployed as part of the game binary. The biggest challenge with modern data systems is understanding what data you have, where it is located, and who is using it.

Systems 130

Systems Metadata Data Pipeline MongoDB 130

The Pragmatic Engineer

SEPTEMBER 26, 2023

Every day, there’s more code at a tech company, not less. This means more repositories are needed, which are fast enough to build and work with, but which increase fragmentation. However, monorepos result in codebases growing large, so that even checking out the code or updating to the head can be time consuming.

Cloud 258

Cloud Software Engineer Software Engineering Cloud Computing 258

Lyft Engineering

JUNE 28, 2023

In early 2022, Lyft already had a comprehensive Machine Learning Platform called LyftLearn composed of model serving , training , CI/CD, feature serving , and model monitoring systems. However, streaming data was not supported as a first-class citizen across many of the platform’s systems — such as training, complex monitoring, and others.

Machine Learning 117

Machine Learning Building Metadata Kafka 117

Snowflake

JUNE 5, 2024

Enterprises can utilize gen AI to extract more value from their data and build conversational interfaces for customer and employee applications. It provides access to industry-leading large language models (LLMs), enabling users to easily build and deploy AI-powered applications. Adopting AI solutions can enhance overall efficiency.

Coding 93

Coding Building Management Government 93

Data Engineering Podcast

NOVEMBER 6, 2022

In this episode Nick King discusses how you can be intentional about data creation in your applications and services to reduce the friction and errors involved in building data products and ML applications. What technical systems are required to generate and collect those interactions?

Building 130

Building IT Metadata MongoDB 130

The Pragmatic Engineer

SEPTEMBER 22, 2023

Bun was mostly built by Jared Sumner , a former Stripe engineer, and recipient of the Thiel Fellowship (a grant of $100,000 for young people to drop out of school and build things, founded by venture capitalist, Peter Thiel). Bun has other contributors, but Jared writes the lion’s share of code.

Programming Language 255

Programming Language Project Coding Engineering 255

Engineering at Meta

DECEMBER 10, 2024

Were explaining the end-to-end systems the Facebook app leverages to deliver relevant content to people. At Facebooks scale, the systems built to support and overcome these challenges require extensive trade-off analyses, focused optimizations, and architecture built to allow our engineers to push for the same user and business outcomes.

Systems 74

Systems Architecture Engineering Data Pipeline 74

Data Engineering Podcast

SEPTEMBER 10, 2023

Summary Data systems are inherently complex and often require integration of multiple technologies. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. Rudderstack]([link] Introducing RudderStack Profiles.

BI 208

BI SQL Data Machine Learning 208

Engineering at Meta

JANUARY 29, 2024

Slow build times and inefficiencies in packaging and distributing execution files were costing our ML/AI engineers a significant amount of time while working on our training stack. Our goal in improving build speed was to minimize the need for extensive rebuilding.

Machine Learning 97

Machine Learning Building Python Utilities 97

Engineering at Meta

DECEMBER 19, 2024

Were sharing details about Glean , Metas open source system for collecting, deriving and working with facts about source code. In this blog post well talk about why a system like Glean is important, explain the rationale for Gleans design, and run through some of the ways were using Glean to supercharge our developer tooling at Meta.

Coding 55

Coding Programming Language SQL Programming 55

Data Engineering Podcast

OCTOBER 29, 2023

RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration.

Database 189

Database BI SQL Machine Learning 189

The Pragmatic Engineer

SEPTEMBER 19, 2024

Use tech debt payments to get into the flow and stay in it A good reason to add new comments to old code before you change it is to speed up a code review. When it takes me time to learn what code does, writing something down helps me remember what I figured out. Clarifying the code is even better.

Recruitment 176

Recruitment Java Coding Project 176

The Pragmatic Engineer

MAY 1, 2024

A first, smaller wave of these stories included Magic.dev raising $100M in funding from Nat Friedman (CEO of GitHub from 2018-2021,) and Daniel Gross (cofounder of search engine Cue which Apple acquired in 2013,) to build a “superhuman software engineer.” years ago, and it became the leading AI coding assistant almost overnight.

Software Engineer 200

Software Engineer Software Engineering Programming Language Media 200

Engineering at Meta

NOVEMBER 15, 2023

Meta is building for the future of AI at every level – from hardware like MTIA v1 , Meta’s first-generation AI inference accelerator to publicly released models like Llama 2 , Meta’s next-generation large language model, as well as new generative AI (GenAI) tools like Code Llama.

Building 103

Building Engineering Transportation Software Engineer 103

Data Engineering Podcast

MARCH 3, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Visit dataengineeringpodcast.com/data-council and use code dataengpod20 to register today! Your first 30 days are free!

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Data Engineering Podcast

FEBRUARY 25, 2024

Summary Building a database engine requires a substantial amount of engineering effort and time investment. Over the decades of research and development into building these software systems there are a number of common components that are shared across implementations. Your first 30 days are free!

Database 162

Database Technology Data Lake High Quality Data 162

Data Engineering Podcast

JUNE 9, 2024

To address this shortcoming Datorios created an observability platform for Flink that brings visibility to the internals of this popular stream processing system. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat.

Process 147

Process Data Lake High Quality Data Machine Learning 147

Knowledge Hut

DECEMBER 21, 2023

Meanwhile, if you’ve always been fascinated by the world of coding and design and looking for the ideal place to get started, then Web Development Certificate online is one of the best certifications course you can consider. This can include features such as smart grids, renewable energy sources, and energy-efficient buildings.

Coding 98

Coding Transportation Medical Retail 98

Cloudera

OCTOBER 31, 2023

However, it remains difficult to quickly build these capabilities given the challenges with finding readily available talent and resources to get started rapidly on the AI journey. Customers can quickly and easily build generative AI applications using these new features available in Cloudera.

Building 109

Building Amazon Web Services AWS Machine Learning 109

Engineering at Meta

FEBRUARY 6, 2023

The app makes heavy use of code generation, spurred by Buck , our custom build system. Without heavy caching from our build system, engineers would have to spend an entire workday waiting for the app to build. If News Feed wanted to have a declarative UI, the team would have to build a new UI framework.

Architecture 145

Architecture Coding Engineering Systems 145

Data Engineering Podcast

JUNE 2, 2024

Summary Modern businesses aspire to be data driven, and technologists enjoy working through the challenge of building data systems to support that goal. This episode is supported by Code Comments, an original podcast from Red Hat. My thanks to the team at Code Comments for their support.

Data Governance 147

Data Governance Government Data Lake High Quality Data 147

Data Engineering Podcast

FEBRUARY 4, 2024

Summary Stream processing systems have long been built with a code-first design, adding SQL as a layer on top of the existing framework. Dagster offers a new approach to building and running data platforms and data pipelines. Can you describe a typical workflow for teams that are building on top of RisingWave?

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173