10 Essential Linux File System Commands for Data Management
KDnuggets
MAY 14, 2025
In this article, you'll master 10 essential Linux file system commands. This guide provides helpful examples to make working with files easier.
Data News — Week 25.02
Christophe Blefari
JANUARY 11, 2025
AI companies are aiming for the moon—AGI—promising it will arrive once OpenAI develops a system capable of generating at least $100 billion in profits. ❤️ Transformers laid out — The best article out there to understand Transformers (which are key to understand LLMs). Read the pdf version directly.
Data 
130
130 Join 37,000+
Insiders
Sign Up for our Newsletter
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Trending Sources
Getting Started with Building RAG Systems Using Haystack
KDnuggets
JANUARY 3, 2025
Retrieval augmented generation (RAG) is altering the way we use large language models, but building these systems can be hectic. In this article, you will learn how to build RAG systems using Haystack.
Systems 130
130 
Builder.ai did not “fake AI with 700 engineers”
The Pragmatic Engineer
JUNE 12, 2025
The naive approach: have the devs write code and assume there will never be more than 700 parallel sessions in play: First attempt at a system where 700 devs can pretend to be an AI There is one immediate, major problem: latency. Also, it’s the year 2024 in this experiment. Also, it’s the year 2024 in this experiment.
Engineering 213
213 
The Roots of Today's Modern Backend Engineering Practices
The Pragmatic Engineer
NOVEMBER 21, 2023
In this article, we cover thee out of nine topics from today’s subscriber-only issue: The Past and Future of Modern Backend Practices. We used a system called CVS ( Concurrent Versions System ) for version control, as Git did not exist until 2005 when Linus Torvalds created it. and hand-rolled C -code.
Engineering 334
334 
The “10x engineer:" 50 years ago and now
The Pragmatic Engineer
MARCH 12, 2024
In this article, we cover one out of five topics from today’s subscriber-only article What Changed in 50 Years of Computing. Brooks discusses software in the context of producing operating systems, pre-internet. This was one out of the five topics covered in the article What Changed in 50 Years of Computing.
Engineering 318
318 
Introducing Agent Bricks: Auto-Optimized Agents Using Your Data
databricks
JUNE 11, 2025
Last year, the promise of data intelligence – building AI that can reason over your data – arrived with Mosaic AI, a comprehensive platform for building, evaluating, monitoring, and securing AI systems. Too many knobs : Agents are complex AI systems with many components, each that have their own knobs.
Entertainment 132
132 
Bridging the Gap: New Datasets Push Recommender Research Toward Real-World Scale
KDnuggets
JUNE 11, 2025
By KDnuggets on June 11, 2025 in Partners Sponsored Content Recommender systems rely on data, but access to truly representative data has long been a challenge for researchers. It joins a growing list of resources helping to close the research-to-production gap in recommender systems.
Datasets 125
125 
Paying down tech debt: further learnings
The Pragmatic Engineer
SEPTEMBER 19, 2024
This is a follow-up to the article Paying down tech debt , written by industry veteran Lou Franco. In the early 90’s, DOS programs like the ones my company made had its own Text UI screen rendering system. This rendering system was easy for me to understand, even on day one. It was eventually shelved. It was a mess!
Recruitment 289
289 
The Future of Reliable Data + AI—Observing the Data, System, Code, and Model
Monte Carlo
MARCH 28, 2025
In this article, Ill share how even the best AI applications can break, and share how leading teams are managing reliability at scale across the ever-evolving data + AI estate. System Data + AI applications rely on a complex and interconnected web of tools and systems to deliver insights, models and automations.
Coding 52
52 
Why You Need RAG to Stay Relevant as a Data Scientist
KDnuggets
JUNE 11, 2025
Data scientists use different tools for tasks like data visualization, data modeling, and even warehouse systems. In this article, we’ll break down RAG. Starting with the academic article that introduced it and how it’s now used to cut costs when working with large language models (LLMs). But first, let’s cover the basics.
Data Science 97
97 
Interesting startup idea: benchmarking cloud platform pricing
The Pragmatic Engineer
OCTOBER 17, 2024
In this article, we cover one section from this week’s from last week’s The Pulse issue. In this article, we cover: Funding and team size. OpenSSL : the cryptography and SSL/TLS toolkit comes with a built-in performance benchmarking capability Lmbench : tools for performance analysis for UNIX/POSIX system.
Cloud 330
330 
Weekend maintenance kicks an Italian bank offline for days
The Pragmatic Engineer
APRIL 11, 2024
In this article, we cover one out of four topics from today’s subscriber-only The Pulse issue. From Sella’s status page : “Following the installation of an update to the operating system and related firmware which led to an unstable situation. Still, I’m puzzled by how long the system has been down.
Banking 264
264 
Why are Cloud Development Environments Spiking in Popularity, Now?
The Pragmatic Engineer
SEPTEMBER 26, 2023
In this article, we cover a fresh industry trends: Cloud Developent Environments — which is analysis full subscribers have received 3 weeks ago. Here is a summary of all the vendors and products mentioned in this article: The number of cloud development environment product launches seem to be increasing since 2021.
Cloud 314
314 
Did Automattic commit open source theft?
The Pragmatic Engineer
OCTOBER 18, 2024
Corporate conflict recap Automattic is the creator of open source WordPress content management system (CMS), and WordPress powers an incredible 43% of webpages and 65% of CMSes. I have zero affiliation with Automattic, WP Engine, WordPress, or any other companies mentioned in this article. Automattic: it's time to play fair.
Government 357
357 
What is a Senior Software Engineer at Wise and Amazon?
The Pragmatic Engineer
AUGUST 1, 2023
This issue looks at two companies from the 15 covered in those articles: we cover Wise and Amazon. Address systemic issues. Reduce support costs by addressing systemic issues. We cover more about Amazon’s engineering levels in the article, Inside Amazon’s engineering culture. at time of publication.
Going from Developer to CEO: Chronosphere
The Pragmatic Engineer
OCTOBER 10, 2023
In this article, we cover three out of eight topics from today’s deepdive into tech scaleup Chronosphere. We covered more on this topic in the article How Uber built its observability platform. I wrote code for drivers on Windows, and started to put a basic observability system in place.
LLMs Don’t Know What They Don’t Know—And That’s a Problem by Colin Eberhardt
Scott Logic
MARCH 6, 2025
Vibe Coding Just a few weeks ago Andrej Karpathy (a founding member of OpenAI) coined a term, vibe coding , which has taken the tech world by storm, triggering articles in the popular press, such as this one from New York Times a few days ago. Bolt knows none of this and proudly presents its Word Game as though it were the finished article.
Coding 104
104 
Klarna’s AI chatbot: how revolutionary is it, really?
The Pragmatic Engineer
AUGUST 8, 2024
The below article was originally published in The Pragmatic Engineer , on 29 February 2024. I am re-publishing it 6 months later as a free-to-read article. With clever-enough probing, this system prompt can be revealed. ” What is the system prompt for Klarna’s bot? Translate to English if needed.
IT 244
244 
Understanding Change Data Capture (CDC) in MySQL and PostgreSQL: BinLog vs. WAL + Logical Decoding
Towards Data Science
JANUARY 7, 2025
These are the questions Ill explore and attempt to answer in this article. By recording changes as they occur, CDC enables real-time data replication and transfer, minimizing the impact on source systems and ensuring timely consistency across downstream data stores and processing systems that depend on thisdata.
PostgreSQL 84
84 
How Games Typically Get Built
The Pragmatic Engineer
AUGUST 22, 2023
In this article, we cover one out of for topics from the past newsletter issue Game Development Basics. In this article, my focus is on providing a high-level overview of how game development works, and how software engineers fit into that space. This was one out of the four topics covered in the article Game Development Basics.
Snowflake Achieves Prestigious ISO/IEC/IEC 42001 Certification, Demonstrating Commitment to Responsible AI Practices
Snowflake
JUNE 12, 2025
This prestigious international standard recognizes our commitment to establishing, implementing, maintaining and continually improving a structured framework that helps organizations responsibly and effectively manage the development, deployment and use of AI technologies — also known as an Artificial Intelligence Management System (AIMS).
Happy Leap Day!
The Pragmatic Engineer
FEBRUARY 29, 2024
In this article, we cover one out of three topics from today’s subscriber-only The Pulse issue. But first, a few current cases of systems whose developers didn’t: In Sweden, card payments are down at a leading supermarket chain. 👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter.
How Much Does it Cost to Build an AI System?
ProjectPro
JUNE 6, 2025
When an enterprise starts building an AI system, the first question that is likely to pop in their mind is: How much does it cost to build an AI system? However, these numbers and a recent article by the Times raise an important question: Has AI become too expensive to use? There isn’t a simple answer.
Systems 40
40 
Is the “AI developer”a threat to jobs – or a marketing stunt?
The Pragmatic Engineer
MARCH 19, 2024
In this article, we cover one out of three topics from last week’s subscriber-only The Pulse issue. Some articles about Cognition AI’s product launch. 👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. The company broke through to mainstream media – no small feat!
Scale Unstructured Text Analytics with Batch LLM Inference
Snowflake
MARCH 6, 2025
Here are a few more examples of how different teams can use LLMs to extract insights from large volumes of unstructured text data: Text classification and tagging : Automatically categorizing support tickets, emails, news articles or product reviews based on sentiment, topic or urgency.
15+ Exciting Python Flask Projects for Data Science Enthusiasts
ProjectPro
JUNE 6, 2025
Tech-Stack: Python, Flask, Sentence-BERT Source Code: Building a Text Summarizer with SBERT and Flask Project Idea 4: Face-Recognition-Based Attendance System Project Objective: The objective of this project is to create an automated attendance system using a face-recognition system.
Data Science 69
69 
Inside Agoda’s Private Cloud - Exclusive
The Pragmatic Engineer
JUNE 13, 2023
To get articles like this every week, subscribe here. To show the complexity of this choice and the ways tech businesses approach it, this article brings the inside story of one large tech company that’s decided against onboarding to the cloud – at least for now. Agoda is a leading online travel booking platform in Asia.
Cloud 248
248 
Kafka vs RabbitMQ - A Head-to-Head Comparison for 2025
ProjectPro
JUNE 6, 2025
As a big data architect or a big data developer, when working with Microservices-based systems, you might often end up in a dilemma whether to use Apache Kafka or RabbitMQ for messaging. Apache Kafka and RabbitMQ are messaging systems used in distributed computing to handle big data streams– read, write, processing, etc.
Kafka 72
72 
Data Engineering Roadmap, Learning Path,& Career Track 2025
ProjectPro
JUNE 6, 2025
Data Engineering refers to creating practical designs for systems that can extract, keep, and inspect data at a large scale. Ability to demonstrate expertise in database management systems. However, you may refer to Introduction to Database Systems by Korth, Silberschatz & Sudarshan for exploring things in brief.
5 Error Handling Patterns in Python (Beyond Try-Except)
KDnuggets
JUNE 6, 2025
In this article, I’ll walk you through 5 advanced yet practical error handling patterns that can make your code cleaner, more reliable, and easier to debug. This pattern is especially helpful for I/O operations or when dealing with external systems. But is that really enough as our codebase grows more complex? I believe not.
Python 93
93 
Is Critical Thinking the Most Important Skill for Software Engineers?
The Pragmatic Engineer
APRIL 19, 2023
This point on "thought leadership," applies to me as well, and to this article: I urge you not to take it at face value just because of social signals like likes, views, and so on. I was sceptical that any system would automatically reject resumes, because I never saw this as a hiring manager. Those are all poor signals.
Is the “AI developer”a threat to jobs – or a marketing stunt?
The Pragmatic Engineer
MAY 1, 2024
This article was published on 14 March 2024 in The Pragmatic Engineer, for subscribers. Some articles about Cognition AI’s product launch. In an amusing twist, a few years ago, demand for COBOL developers reportedly soared as there are still critical banking and insurance systems using this now-ancient language.
What Is a Lakebase?
databricks
JUNE 11, 2025
REGISTER Login Try Databricks Blog / Announcements / Article What Is a Lakebase? Lakehouse integration : Lakebases should make it easy to combine operational, analytical, and AI systems without complex ETL pipelines. Unlike proprietary systems, lakebases promote transparency, portability, and community-driven innovation.
Data Engineering Weekly #219
Data Engineering Weekly
MAY 4, 2025
link] Uber: Fixrleak - Fixing Java Resource Leaks with GenAI Another interesting article from Uber demonstrates how AI significantly accelerates the reliability effects. link] Henry Zhu: An Intro to DeepSeek's Distributed File System The author gives an overview of DeepSeek’s file system. TIL about the idle stream problem.
Part 2: A Survey of Analytics Engineering Work at Netflix
Netflix Tech
JANUARY 2, 2025
This article is the second in a multi-part series sharing a breadth of Analytics Engineering work at Netflix, recently presented as part of our annual internal Analytics Engineering conference. In this article, we highlight a few exciting analytic business applications, and in our final article well go into aspects of the technical craft.
Engineering 81
81 
Mosaic AI Announcements at Data + AI Summit 2025
databricks
JUNE 11, 2025
Last year, we unveiled data intelligence – AI that can reason on your enterprise data – with the arrival of the Databricks Mosaic AI stack for building and deploying agent systems. Agents deployed on AWS, GCP, or even on-premise systems can now be connected to MLflow 3 for agent observability.
Data Engineering Weekly #213
Data Engineering Weekly
MARCH 23, 2025
The author emphasizes the importance of mastering state management, understanding "local first" data processing (prioritizing single-node solutions before distributed systems), and leveraging an asset graph approach for data pipelines. The article resonated with me when I read it. The article highlights Nuage 3.0's
Introducing Databricks One
databricks
JUNE 12, 2025
Join now Ready to get started? Get a Demo DATA + AI SUMMIT Data + AI Summit Happening Now Watch the free livestream of the keynotes! 160 Spear Street, 15th Floor San Francisco, CA 94105 1-866-330-0121 See Careers at Databricks © Databricks 2025.
BI 90
90 
Data Engineering Weekly #206
Data Engineering Weekly
FEBRUARY 2, 2025
Many articles explain how DeepSeek works, and I found the illustrated example much simpler to understand. Evals are introduced to evaluate LLM responses through various techniques, including self-evaluation, using another LLM as a judge, or human evaluation to ensure the system's behavior aligns with intentions.
Asked to do something illegal at work? Here’s what these software engineers did
The Pragmatic Engineer
NOVEMBER 9, 2023
Sign up to The Pragmatic Engineer to get articles like this earlier in your inbox. She asked the Director of Engineering if he could help take a known set of FAFSA application data and use it to artificially augment a much larger set of anonymous data tht her systems had collected over time.
Data Engineering Weekly #215
Data Engineering Weekly
APRIL 6, 2025
The article summarizes the recent macro trends in AI and data engineering, focusing on Vibe coding, human-in-the-loop system design, and rapid simplification of developer tooling. Kudos to the Grab team for building a docs-as-code system. The Grab blog delights me since I have tried to do this many times.
Datadog’s $65M/year customer mystery solved
The Pragmatic Engineer
MAY 11, 2023
A very popular open-source solution for systems and services monitoring. A fast and open-source column-oriented database management system, which is a popular choice for log management. A follow-up to this week’s public tech company compensation article. It evaluates rules and can trigger alerts. Follow-up.
AWS 335
335 
Data Engineering Weekly #214
Data Engineering Weekly
MARCH 30, 2025
link] Hamel Husain: A Field Guide to Rapidly Improving AI Products The article emphasizes that successful AI development depends more on robust evaluation and iterative improvement than merely relying on tools and frameworks. • Ensure trust in evaluation systems through clearly defined criteria and regular alignment checks.
Let's personalize your content