The Pragmatic Engineer

JUNE 1, 2023

Juraj included system monitoring parts which monitor the server’s capacity he runs the app on: The monitoring page on the Rides app And it doesn’t end here. Juraj created a systems design explainer on how he built this project, and the technologies used: The systems design diagram for the Rides application The app uses: Node.js

Education 363

Education Project PostgreSQL Software Engineer 363

Data Engineering Podcast

JANUARY 15, 2023

At the Modern Data Company they created the DataOS platform as a means of driving your full analytics lifecycle through code, while providing automatic knowledge graphs and data discovery. In this episode Srujan Akula explains how the system is implemented and how you can start using it today with your existing data systems.

Coding 130

Coding Building PostgreSQL Data Lake 130

The Pragmatic Engineer

MARCH 12, 2024

” They write the specification, code, tests it, and write the documentation. Code reviews reduce the need to pair while working on a task, allowing engineers to keep up with changes and learn from each other. CI/CD : running automated tests on all changes, and deploying code to production automatically. The copilot.

Engineering 274

Engineering Programming Language Hospitality Programming 274

The Pragmatic Engineer

NOVEMBER 21, 2023

If you had a continuous deployment system up and running around 2010, you were ahead of the pack: but today it’s considered strange if your team would not have this for things like web applications.  Backend code I wrote and pushed to prod took down Amazon.com for several hours. and hand-rolled C -code.

Engineering 302

Engineering Bytes Cloud Computing AWS 302

Knowledge Hut

FEBRUARY 1, 2024

Not only could this recommendation system save time browsing through lists of movies, it can also give more personalized results so users don’t feel overwhelmed by too many options. What are Movie Recommendation Systems? Recommender systems have two main categories: content-based & collaborative filtering.

Systems 98

Systems Entertainment Algorithm Datasets 98

Knowledge Hut

OCTOBER 29, 2023

Hospital Management System Type : Application development, Database management, Programming There is no shortage of computer science project topics out there. But if you are looking for something that's both technically challenging and socially relevant, consider a hospital management system. Source Code: Weather Forecast App 3.

Computer Science 111

Computer Science Coding Project Hospitality 111

Data Engineering Podcast

MAY 18, 2024

Summary The purpose of business intelligence systems is to allow anyone in the business to access and decode data to help them make informed decisions. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat.

Building 278

Building Data Lake High Quality Data Business Intelligence 278

Engineering at Meta

DECEMBER 10, 2024

Were explaining the end-to-end systems the Facebook app leverages to deliver relevant content to people. At Facebooks scale, the systems built to support and overcome these challenges require extensive trade-off analyses, focused optimizations, and architecture built to allow our engineers to push for the same user and business outcomes.

Systems 76

Systems Architecture Engineering Data Pipeline 76

Engineering at Meta

DECEMBER 19, 2024

Were sharing details about Glean , Metas open source system for collecting, deriving and working with facts about source code. In this blog post well talk about why a system like Glean is important, explain the rationale for Gleans design, and run through some of the ways were using Glean to supercharge our developer tooling at Meta.

Coding 55

Coding Programming Language SQL Programming 55

Data Engineering Podcast

FEBRUARY 18, 2024

As a listener to the Data Engineering Podcast you can get a special discount of 20% off your ticket by using the promo code dataengpod20. What are the pain points that are still prevalent in lakehouse architectures as compared to warehouse or vertically integrated systems? Promo Code: depod20 Starburst : ![Starburst

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

The Pragmatic Engineer

MARCH 19, 2024

The company says: “Devin is the new state-of-the-art on the SWE-Bench coding benchmark, has successfully passed practical engineering interviews from leading AI companies, and has even completed real jobs on Upwork. years ago, and it became the leading AI coding assistant almost overnight. It’s more a copilot.

Software Engineer 276

Software Engineer Software Engineering Programming Language Media 276

Analytics Vidhya

FEBRUARY 3, 2023

Introduction “Let’s containerize your code to ship worldwide!” Say Harish and Lisa are two people working on the same project but on two different systems(say windows and […] The post Getting Started with The Basics of Docker appeared first on Analytics Vidhya. Well, my friend, this is what Docker is.

Coding 257

Coding Project Systems IT 257

The Pragmatic Engineer

OCTOBER 17, 2024

We recently covered how CockroachDB joins the trend of moving from open source to proprietary and why Oxide decided to keep using it with self-support , regardless Web hosting:  Netlify : chosen thanks to their super smooth preview system with SSR support. Internal comms: Chat: Slack Coordination / project management: Linear 3.

Cloud 278

Cloud AWS Metadata Cloud Computing 278

The Pragmatic Engineer

SEPTEMBER 19, 2024

Use tech debt payments to get into the flow and stay in it A good reason to add new comments to old code before you change it is to speed up a code review. When it takes me time to learn what code does, writing something down helps me remember what I figured out. Clarifying the code is even better.

Recruitment 176

Recruitment Java Coding Project 176

The Pragmatic Engineer

OCTOBER 10, 2023

However, Martin had not written a line of production code for the last four years, as he’s taken on the role of CEO, and heads up observability scaleup Chronosphere – at more than 250 people and growing. From learning to code in Australia, to working in Silicon Valley How did I learn to code?

Software Engineer 268

Software Engineer Software Engineering Architecture Media 268

The Pragmatic Engineer

SEPTEMBER 26, 2023

Every day, there’s more code at a tech company, not less. However, monorepos result in codebases growing large, so that even checking out the code or updating to the head can be time consuming. Concern about code leaks. Open source VS Code Server. In 2021, Microsoft open sourced VS Code Server.

Cloud 258

Cloud Software Engineer Software Engineering Cloud Computing 258

Scott Logic

DECEMBER 14, 2024

Advent of Code (AoC) is an annual, christmas-themed, coding competition that has been running for the past years and is something that I participate in at times. It was writing code to complete puzzles that took me half an hour or more, in just seconds! Do not describe your approach, only return the JavaScript code.

Coding 64

Coding Datasets Software Engineer Software Engineering 64

Data Engineering Podcast

SEPTEMBER 10, 2023

Summary Data systems are inherently complex and often require integration of multiple technologies. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy.

BI 208

BI SQL Data Machine Learning 208

Data Engineering Podcast

OCTOBER 29, 2023

This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration.

Database 189

Database BI SQL Machine Learning 189

The Pragmatic Engineer

SEPTEMBER 22, 2023

Bun has other contributors, but Jared writes the lion’s share of code. Its top focus is performance, and according to benchmarks shared in the launch video , around 10x performance increases can be observed when building packages, running code, or handling inbound requests on a server.

Programming Language 255

Programming Language Project Coding Engineering 255

Simon Späti

APRIL 6, 2022

With burnout and mental stress at every level of our lives, I find my Personal Knowledge Management (PKM) system even more valuable. As a developer and knowledge worker, I re-use code snippets or create new things. That’s why a PKM system such as a Second Brain to store all of it in a sustainable way is crucial to me.

Management 189

Management Coding Systems IT 189

Snowflake

JUNE 5, 2024

Snowflake AI & ML Studio for LLMs (private preview): Enable users of all technical levels to utilize AI with no-code development. Tabular data, figure synthesis: Get answers from non-free-text structures in your data — such as data specifically formatted and often found in common data storage systems, like databases and spreadsheets.

Coding 93

Coding Building Management Government 93

The Pragmatic Engineer

MAY 1, 2024

The company says: “Devin is the new state-of-the-art on the SWE-Bench coding benchmark, has successfully passed practical engineering interviews from leading AI companies, and has even completed real jobs on Upwork. years ago, and it became the leading AI coding assistant almost overnight. It’s more a copilot.

Software Engineer 200

Software Engineer Software Engineering Programming Language Media 200

Start Data Engineering

MAY 30, 2024

Introduction Data is stored on disk and processed in memory Running the code Run on Codespaces Run on your laptop Using python REPL Python basics Python is used for extracting data from sources, transforming it, & loading it into a destination [Extract & Load] Read and write data to any system [Transform] Process data in Python or instruct (..)

Python 147

Python Data Engineer Data Engineering Engineering 147

Data Engineering Podcast

MARCH 3, 2024

As a listener to the Data Engineering Podcast you can get a special discount off regular priced and late bird tickets by using the promo code dataengpod20. Visit dataengineeringpodcast.com/data-council and use code dataengpod20 to register today! Promo Code: dataengpod20 Starburst : ![Starburst When is AI the wrong choice?

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

The Pragmatic Engineer

FEBRUARY 29, 2024

But first, a few current cases of systems whose developers didn’t: In Sweden, card payments are down at a leading supermarket chain. Airline Avianca printed tickets dated as 3/1 instead of 2/29, thanks to their system not accounting for the leap day. The system was almost fully restored before noon.”

Software Engineer 220

Software Engineer Software Engineering Banking Engineering 220

Data Engineering Podcast

JUNE 9, 2024

To address this shortcoming Datorios created an observability platform for Flink that brings visibility to the internals of this popular stream processing system. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat.

Process 147

Process Data Lake High Quality Data Machine Learning 147

The Pragmatic Engineer

NOVEMBER 9, 2023

What would you do if you learned your company is up to something illegal like stealing customer funds, or you’re asked to make code changes that will enable something illegal to happen, like misleading investors, or defrauding customers? Sign up to The Pragmatic Engineer to get articles like this earlier in your inbox.

Software Engineer 227

Software Engineer Software Engineering Engineering Coding 227

Engineering at Meta

FEBRUARY 6, 2023

The app makes heavy use of code generation, spurred by Buck , our custom build system. Without heavy caching from our build system, engineers would have to spend an entire workday waiting for the app to build. The app’s ‘module’ system gave each product ungoverned access to all the app’s resourcing.

Architecture 145

Architecture Coding Engineering Systems 145

Netflix Tech

MAY 4, 2023

Behind the scenes, a myriad of systems and services are involved in orchestrating the product experience. These backend systems are consistently being evolved and optimized to meet and exceed customer and product expectations. This technique facilitates validation on multiple fronts.

Utilities 139

Utilities Systems Architecture Coding 139

Jesse Anderson

OCTOBER 5, 2023

In another post , I introduced a concept from Justin Coffey about how much systems can be changed from their original design. Teams must know this, or you’ll have vendor lock-in on an ostensibly open-source system. Queues Queues are often used with pub/sub systems like Kafka. As of now, Kafka makes for a terrible queuing system.

Kafka 195

Kafka Finance Cloud Designing 195

Snowflake

APRIL 24, 2024

Members of the Snowflake AI Research team pioneered systems such as ZeRO and DeepSpeed , PagedAttention / vLLM , and LLM360 which significantly reduced the cost of LLM training and inference, and open sourced them to make LLMs more accessible and cost-effective for the community. license provides ungated access to weights and code.

Amazon Web Services 145

Amazon Web Services SQL AWS Architecture 145

Data Engineering Podcast

SEPTEMBER 17, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Hex brings everything together.

Building 189

Building SQL BI Python 189

Data Engineering Podcast

JUNE 2, 2024

Summary Modern businesses aspire to be data driven, and technologists enjoy working through the challenge of building data systems to support that goal. This episode is supported by Code Comments, an original podcast from Red Hat. My thanks to the team at Code Comments for their support. Starburst : ![Starburst

Data Governance 147

Data Governance Government Data Lake High Quality Data 147

Data Engineering Podcast

OCTOBER 15, 2023

Decodable was built with a mission of eliminating all of the painful aspects of developing and deploying stream processing systems for engineering teams. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration.

Process 182

Process Building SQL BI 182

Data Engineering Podcast

FEBRUARY 25, 2024

Over the decades of research and development into building these software systems there are a number of common components that are shared across implementations. As a listener to the Data Engineering Podcast you can get a special discount off regular priced and late bird tickets by using the promo code dataengpod20.

Database 162

Database Technology Data Lake High Quality Data 162

Data Engineering Podcast

MAY 12, 2024

The services and systems need to be kept up to date, but so does the code that controls their behavior. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat. Data lakes are notoriously complex.

Management 147

Management Data Lake High Quality Data Machine Learning 147