WeCloudData

APRIL 9, 2025

The ability to extract information from vast amounts of text has made question-answering (QA) systems essential in the modern era of AI-driven apps. RAG-based question-answering systems use large language models to generate human-like responses to user queries.

Systems 52

Systems Building IT Data Science 52

Striim

MARCH 21, 2025

The changes are captured without making application level changes and without having to scan operational tables, both of which add additional workload and reduce source systems’ performance. Business transactions captured in relational databases are critical to understanding the state of business operations.

IT 52

IT Data Lake Relational Database Data Warehouse 52

Snowflake

FEBRUARY 27, 2025

Innovation is vital to power financial systems in order to unlock value for and meet the needs of our customers." Developers can build an app and market, monetize and distribute it to customers across the AI Data Cloud ecosystem through Snowflake Marketplace , all within Snowflakes secure and governed platform.

Cloud 78

Cloud IT Amazon Web Services Healthcare 78

Edureka

FEBRUARY 12, 2025

Flexibility and Modularity : The modular design of LangChain lets coders change how parts work, connect them to other systems, and try out different setups. LangChain is a dynamic framework designed to supercharge the potential of Large Language Models (LLMs) by seamlessly integrating them with tools, APIs, and memory. What is LangChain?

IT 52

IT Database Google Cloud Coding 52

RandomTrees

NOVEMBER 25, 2024

There are obligations on telecommunications providers to ensure that their systems of AI are accountable and understandable to clients and regulatory authorities. The scope of telecom services is growing in size and complexity, owing to technologies such as 5G, the Internet of Things (IoT), and cloud technology.

Telecommunication 52

Telecommunication IT Unstructured Data Data Mining 52

Engineering at Meta

JANUARY 22, 2025

It is a critical and powerful tool for scalable discovery of relevant data and data flows, which supports privacy controls across Metas systems. It enhances the traceability of data flows within systems, ultimately empowering developers to swiftly implement privacy controls and create innovative products.

Data Warehouse 120

Data Warehouse SQL Programming Language Data 120

Monte Carlo

MARCH 28, 2025

Table of Contents Understanding How Data + AI Can Break Data System Code Model Data + AI observability must cover inputs and outputs it is all or nothing Understanding How Data + AI Can Break Data + AI applications are complex. The commodities just keep coming. So, what does it mean to achieve trusted data + AI?

Coding 52

Coding Systems Data Pipeline ETL Tools 52

The Pragmatic Engineer

NOVEMBER 21, 2023

If you had a continuous deployment system up and running around 2010, you were ahead of the pack: but today it’s considered strange if your team would not have this for things like web applications. To get full issues twice a week, subscribe here. With that, it’s over to Joshua: 1. and hand-rolled C -code.

Engineering 337

Engineering Bytes Cloud Computing AWS 337

The Pragmatic Engineer

SEPTEMBER 19, 2024

In the early 90’s, DOS programs like the ones my company made had its own Text UI screen rendering system. This rendering system was easy for me to understand, even on day one. Our rendering system was very memory inefficient, but that could be fixed. By doing so, I got to see every screen of the system.

Recruitment 285

Recruitment Java Coding Project 285

Christophe Blefari

JANUARY 11, 2025

AI companies are aiming for the moon—AGI—promising it will arrive once OpenAI develops a system capable of generating at least $100 billion in profits. We announced the AI Product Day , a 1-day conference that will take place in Paris on March 31. We are looking for sponsors and the ticketing is open.

Data 130

Data Data Warehouse Coding Programming Language 130

Snowflake

APRIL 22, 2025

We're talking about potential headaches around data quality, the amount of data your systems can handle, the risk of bias getting amplified and those privacy concerns the ones about proprietary business information or customer personal data possibly leaking out in the outputs. These essential steps introduce some real challenges.

IT 53

IT Unstructured Data Government Cloud 53

Netflix Tech

NOVEMBER 12, 2024

Failures in a distributed system are a given, and having the ability to safely retry requests enhances the reliability of the service. Implementing idempotency would likely require using an external system for such keys, which can further degrade performance or cause race conditions.

Datasets 103

Datasets Computer Science Systems Kafka 103

Data Engineering Weekly

DECEMBER 28, 2024

Investment in an Agent Management System (AMS) is crucial, as it offers a framework for scaling, monitoring, and refining AI agents. AI engineers, in particular, will find their skills in high demand as they navigate managing and optimizing agents to ensure reliability within enterprise systems.

Unstructured Data 112

Unstructured Data Metadata Data Government 112

Precisely

JANUARY 9, 2025

The simple idea was, hey how can we get more value from the transactional data in our operational systems spanning finance, sales, customer relationship management, and other siloed functions. Yet along with the AI hype and excitement comes very appropriate sanity-checks asking whether AI is ready for prime-time. Can it do it without bias?

Data Integration 118

Data Integration Hadoop Data Warehouse Data Lake 118

The Pragmatic Engineer

JUNE 1, 2023

Juraj included system monitoring parts which monitor the server’s capacity he runs the app on: The monitoring page on the Rides app And it doesn’t end here. Juraj included system monitoring parts which monitor the server’s capacity he runs the app on: The monitoring page on the Rides app And it doesn’t end here.

Education 364

Education Project PostgreSQL Software Engineering 364

The Pragmatic Engineer

MARCH 12, 2024

Tools and approaches at our disposal, which didn’t exist in 1975, or were not widespread in 1995, include: Git – the now-dominant version control system used by much of the industry, with exceptions for projects with very large assets, like video games Code reviews : these became common in parallel with version control.

Engineering 319

Engineering Programming Language Hospitality Programming 319

Jesse Anderson

MAY 5, 2025

Finally, Shane outlines how observability is crucial for emerging AI/ML workflows like RAG pipelines, discussing the monitoring of vector databases (like Pinecone), unstructured data, and the entire AI system lifecycle, concluding with a look at Monte Carlo’s exciting roadmap, including AI-powered troubleshooting agents.

Unstructured Data 130

Unstructured Data Finance Metadata Architecture 130

Jesse Anderson

FEBRUARY 11, 2025

Semih is a researcher and entrepreneur with a background in distributed systems and databases. He then pursued his doctoral studies at Stanford University, delving into the complexities of database systems.

Computer Science 130

Computer Science Database Design Software Engineer Software Engineering 130

Snowflake

JANUARY 8, 2025

Understanding AI as an attack vector Last year, we published an AI security framework that identifies 20 attack vectors against large language models and generative AI systems. Though AI is (still) the hottest technology topic, its not the overriding issue for enterprise security in 2025.

Data Lake 103

Data Lake Data Security Machine Learning Technology 103

The Pragmatic Engineer

FEBRUARY 29, 2024

But first, a few current cases of systems whose developers didn’t: In Sweden, card payments are down at a leading supermarket chain. Airline Avianca printed tickets dated as 3/1 instead of 2/29, thanks to their system not accounting for the leap day. Subscribe to get issues like this in your inbox, every week.

Software Engineering 269

Software Engineering Software Engineer Banking Engineering 269

Engineering at Meta

APRIL 28, 2025

Meta’s vast and diverse systems make it particularly challenging to comprehend its structure, meaning, and context at scale. We discovered that a flexible and incremental approach was necessary to onboard the wide variety of systems and languages used in building Metas products.

Metadata 86

Metadata Data Utilities Data Warehouse 86

Simon Späti

MARCH 11, 2025

Therefore, you’ve probably come across terms like OLAP (Online Analytical Processing) systems, data warehouses, and, more recently, real-time analytical databases. Postgres is powerful, reliable, and flexible enough to handle both transactional and basic analytical workloads.

Database 130

Database Data Warehouse Data Engineering Data Engineer 130

The Pragmatic Engineer

OCTOBER 17, 2024

We recently covered how CockroachDB joins the trend of moving from open source to proprietary and why Oxide decided to keep using it with self-support , regardless Web hosting:  Netlify : chosen thanks to their super smooth preview system with SSR support. Spare Cores attempts to make it easier to compare prices across cloud providers.

Cloud 326

Cloud AWS Metadata Cloud Computing 326

The Pragmatic Engineer

APRIL 11, 2024

From Sella’s status page : “Following the installation of an update to the operating system and related firmware which led to an unstable situation. Still, I’m puzzled by how long the system has been down. If it was an update to Oracle, or to the operating system, then why not roll back the update?

Banking 261

Banking Utilities Database Engineering 261

The Pragmatic Engineer

OCTOBER 18, 2024

Corporate conflict recap Automattic is the creator of open source WordPress content management system (CMS), and WordPress powers an incredible 43% of webpages and 65% of CMSes. Imagine Apple decided Spotify was a big enough business threat that it had to take unfair measures to limit Spotify’s growth on the App Store.

Engineering 356

Engineering Government Project AWS 356

Engineering at Meta

FEBRUARY 3, 2025

This is particularly true in the data center space, where new protocols like Precision Time Protocol (PTP) are allowing systems to be synchronized down to nanosecond precision. The difference between two approaches can mean differences of over 100 microseconds, creating challenges for services that consume time from both systems.

Algorithm 101

Algorithm Utilities Systems Engineering 101

Netflix Tech

MARCH 28, 2025

By Ko-Jen Hsiao , Yesu Feng and Sudarshan Lamkhede Motivation Netflixs personalized recommender system is a complex system, boasting a variety of specialized machine learned models each catering to distinct needs including Continue Watching and Todays Top Picks for You. Refer to our recent overview for more details).

Metadata 88

Metadata Bytes Entertainment Data Mining 88

The Pragmatic Engineer

MAY 12, 2023

For a realtime alerting system! I have since talked with engineers on the OpsGenie team who said that it felt that Atlassian rushed the OpsGenie integration - after buying the company - onto their unified internal stack, ignoring warnings that an outage in their identity system would take OpsGenie down. Incident management.

Systems 276

Systems Management Engineering IT 276

Netflix Tech

JANUARY 6, 2025

In this case, the main stakeholders are: - Title Launch Operators Role: Responsible for setting up the title and its metadata into our systems. In this context, were focused on developing systems that ensure successful title launches, build trust between content creators and our brand, and reduce engineering operational overhead.

Algorithm 91

Algorithm Metadata Systems Building 91

Analytics Vidhya

FEBRUARY 27, 2023

Introduction The Hadoop Distributed File System (HDFS) is a Java-based file system that is Distributed, Scalable, and Portable. Still, it does include shell commands and Java Application Programming Interface (API) functions that are similar to other file systems.

Hadoop 240

Hadoop Java Data Storage Programming 240

Jesse Anderson

OCTOBER 29, 2024

He sees logs as a treasure trove of insights and believes effective log analysis is critical in today’s complex systems. We discussed his early experiences with distributed systems, including his work on creating graphs and entity resolution. Lastly, we go in-depth into Scanner.dev, covering what it is and how it works.

Data Engineering 100

Data Engineering Data Engineer Systems Engineering 100

Engineering at Meta

FEBRUARY 4, 2025

Here we explore initial system designs we considered, an overview of the current architecture, and some important principles Meta takes into account in making data accessible and easy to understand. Users have a variety of tools they can use to manage and access their information on Meta platforms. What are data logs?

Accessible 92

Accessible Accessibility Raw Data Data Warehouse 92

The Pragmatic Engineer

OCTOBER 10, 2023

We don’t frequently have CEOs on The Pragmatic Engineer: in fact, today is the first issue when we’re talking with a CEO – partially – about their CEO job. I’ve always wondered what it would be like to go from a developer, to eventually become the CEO of a large and growing company.

Software Engineer 318

Software Engineer Software Engineering Architecture Media 318

dbt Developer Hub

APRIL 20, 2025

Both AI agents and business stakeholders will then operate on top of LLM-driven systems hydrated by the dbt MCP context. Todays system is not a full realization of the vision in the posts shared above, but it is a meaningful step towards safely integrating your structured enterprise data into AI workflows. What is MCP?

Structured Data 98

Structured Data SQL BI Project 98