DataKitchen

NOVEMBER 29, 2021

Keep an eye on the eight top trends below that we believe will be significant in the year 2022. The data industry realizes that AI bias is simply a quality problem, and AI systems should be subject to this same level of process control as an automobile rolling off an assembly line. AI Accountability. Data Gets Meshier.

Data Lake 98

Data Lake Manufacturing Architecture Data Architecture 98

Lyft Engineering

APRIL 3, 2023

This blog post focuses on the scope and the goals of the recommendation system, and explores some of the most recent changes the Rider team has made to better serve Lyft’s riders. Introduction: Scope of the Recommendation System The recommendation system covers user experiences throughout the ride journey.

Systems 87

Systems Pipeline-centric Machine Learning Transportation 87

The Pragmatic Engineer

JUNE 1, 2023

Juraj included system monitoring parts which monitor the server’s capacity he runs the app on: The monitoring page on the Rides app And it doesn’t end here. Juraj created a systems design explainer on how he built this project, and the technologies used: The systems design diagram for the Rides application The app uses: Node.js

Education 363

Education Project PostgreSQL Software Engineer 363

The Modern Data Company

JANUARY 31, 2023

In 2023, The Modern Data Company (Modern) hopes to reach more companies and organizations with our data operating system, build incredible value from existing and upcoming data assets, and share insights into major shifts in what it means to be data-driven. These were our most popular blog posts in 2022 according to reader statistics.

Retail 98

Retail Healthcare Data Architecture Data Pipeline 98

The Pragmatic Engineer

MAY 12, 2023

I recommend exercising caution regarding OpsGenie, given how in 2022, OpsGenie was down for hundreds of customers for 2 weeks. For a realtime alerting system! Be wary of OpsGenie's past reliability incident. Yes: 2 for weeks! Unlucky customers had no option but to move to alternative alerting/incident management providers.

Systems 231

Systems Management Engineering IT 231

Scott Logic

JANUARY 16, 2023

Noting that transformation is hard, he uses the term “immune system” to describe how a company/organisation naturally and unintentionally tends to fight/resist change. Everything You Should Know About Web Development in 2022 Stefan Judis took us through the highlights of what’s new in the web space this year.

Generalist 83

Generalist Coding Java Education 83

The Pragmatic Engineer

NOVEMBER 9, 2023

Here is software engineer and writer Molly White summarizes of his evidence : “To hear Singh tell it, he didn’t even really realize what was going on at FTX and Alameda Research until September 2022 — only a month or two before everything came crashing down. (.) FTX collapsed in November 2022.

Software Engineer 227

Software Engineer Software Engineering Engineering Coding 227

Cloudera

JUNE 13, 2022

We are excited to announce that Cloudera is named as a 2022 Gartner Peer Insights Customers’ Choice for Cloud Database Management Systems (DBMS). As of Mar 2022, Cloudera customers provided 81 reviews, and we earned an average Overall Rating of 4.5. What do our customers say about Cloudera? . We take your feedback to heart.

Hadoop 78

Hadoop Manufacturing Finance Media 78

The Pragmatic Engineer

APRIL 13, 2023

Source: Dealroom In 2022, Angel rounds were at the lowest by cumulative value since 2015, which is even worse than we’ve seen in the US. Source: Dealroom Seed investments have dropped since 2021, but in 2022 were around the same level as in 2020 – and well above 2019. Seed investments in Europe.

Finance 193

Finance Media Software Engineer Software Engineering 193

The Pragmatic Engineer

NOVEMBER 21, 2023

If you had a continuous deployment system up and running around 2010, you were ahead of the pack: but today it’s considered strange if your team would not have this for things like web applications.  We dabbled in network engineering, database management, and system administration. and hand-rolled C -code.

Engineering 302

Engineering Bytes Cloud Computing AWS 302

Precisely

SEPTEMBER 23, 2024

More specifically, these companies use IBM Z systems. The most recent version, IBM z16, was released in April of 2022. This growth is driven by the ongoing need for reliable, high-capacity computing systems, particularly in sectors like banking, finance, and government​. Mainframes are used by 71% of Fortune 500 companies.

Banking 116

Banking Insurance Finance Systems 116

Jesse Anderson

JANUARY 9, 2023

2022 was a banner year for activity.” In 2022, many of the Ververica employees left and formed Immerok. You can see a significant drop starting in March 2022. While the technology sector has had a tough time, that started in May 2022. Confluent’s problems began months earlier, in February 2022.

Kafka 147

Kafka Coding Technology SQL 147

The Pragmatic Engineer

JUNE 8, 2023

Shopify’s stock value fell 29% between September 2021 and September 2022: Shopify’s stock price change, versus the S&P 500 index, between September 2021 and 2022 The most common way companies respond to such drastic stock declines are these: Issue higher equity refreshers , intended to counter a falling stock price.

Software Engineer 153

Software Engineer Software Engineering Media Engineering 153

The Pragmatic Engineer

JANUARY 5, 2023

We also recommend customers review internal logs for their systems for any unauthorized access starting from December 21, 2022, through today, January 4, 2023, or upon completion of your secrets rotation. (.)We We take the security of our systems and our customers’ systems extremely seriously.

Pipeline-centric 130

Pipeline-centric Database-centric Coding Accessible 130

Netflix Tech

SEPTEMBER 29, 2022

Timestone: Netflix’s High-Throughput, Low-Latency Priority Queueing System with Built-in Support for Non-Parallelizable Workloads by Kostas Christidis Introduction Timestone is a high-throughput, low-latency priority queueing system we built in-house to support the needs of Cosmos , our media encoding platform. Over the past 2.5

Systems 89

Systems Metadata Media Kafka 89

Data Engineering Podcast

SEPTEMBER 10, 2023

Summary Data systems are inherently complex and often require integration of multiple technologies. Can you start by defining what data orchestration is and how it differs from other types of orchestration systems? Can you start by defining what data orchestration is and how it differs from other types of orchestration systems?

BI 208

BI SQL Data Machine Learning 208

The Pragmatic Engineer

MARCH 19, 2024

The Cognition AI team smashed it on media attention; it feels like their launch garnered almost as much developer attention as ChatGPT’s launch in November 2022. In an amusing twist, a few years ago, demand for COBOL developers reportedly soared as there are still critical banking and insurance systems using this now-ancient language.

Software Engineer 276

Software Engineer Software Engineering Programming Language Media 276

DoorDash Engineering

APRIL 4, 2023

This is the third blog post in a series of articles showcasing our 2022 summer intern projects. Today these changes are entered as single entry systems that are prone to errors and difficult to reconcile. The post 2022 Summer Intern Projects Article #3 appeared first on DoorDash Engineering Blog.

Project 75

Project Banking Kafka Database 75

The Pragmatic Engineer

OCTOBER 10, 2022

Starting from 2022, they run once a year. ” In 2022, the first such quarterly review or touchpoint was April. So yes, the performance review process is new for 2022, but it’s only new because it’s different from the previous one, which was biannual (every 6 months.) So, how will they reach this goal?

Engineering 130

Engineering Software Engineer Software Engineering Recruitment 130

Jesse Anderson

OCTOBER 5, 2023

In another post , I introduced a concept from Justin Coffey about how much systems can be changed from their original design. In 2022, Confluent spent 264 million on research and development. Teams must know this, or you’ll have vendor lock-in on an ostensibly open-source system. What has changed?” I don’t have an answer yet.

Kafka 195

Kafka Finance Cloud Designing 195

Netflix Tech

JUNE 1, 2023

By Jose Fernandez , Ed Barker , Hank Jacobs Introduction In November 2022, we introduced a brand new tier —  Basic with ads. Replay traffic enabled us to test our new systems and algorithms at scale before launch, while also making the traffic as realistic as possible. Basic with ads was launched worldwide on November 3rd.

Algorithm 140

Algorithm Metadata Kafka Systems 140

The Pragmatic Engineer

SEPTEMBER 26, 2023

Codeanywhere : one of the earliest cloud IDEs, founded in 2013 Daytona was founded in 2023, and its founders are the cofounders of Codeanywhere.

Cloud 258

Cloud Software Engineer Software Engineering Cloud Computing 258

The Pragmatic Engineer

OCTOBER 10, 2023

I wrote code for drivers on Windows, and started to put a basic observability system in place. EC2 had no observability system back then: people would spin up EC2 instances but have no idea whether or not they worked. With my team, we built the basics of what is now called AWS Systems Manager.

Software Engineer 268

Software Engineer Software Engineering Architecture Media 268

The Pragmatic Engineer

OCTOBER 10, 2024

Wordpress is the most popular content management system (CMS),  estimated  to power around 43% of all websites; a staggering number! To maintain 2021-2022 valuations, companies need to increase revenue and profitability. This article was originally published a week ago, on 3 October 2024, in The Pragmatic Engineer.

Consulting 306

Consulting AWS Engineering Software Engineer 306

Tweag

MAY 1, 2024

Table of contents Overview Motivation Statistics Retrospective Thoughts on future work Acknowledgements Overview This is a retrospective of my and many other people’s work on documentation in the Nix ecosystem between October 2022 and March 2024. A lot has happened during that time, so this text is quite lengthy.

Architecture 107

Architecture Project Coding Designing 107

Engineering at Meta

JUNE 27, 2023

Sapling: Scaling version control Sapling is a version control system that can scale to huge sizes, but also emphasizes usability. There are three main components to Sapling – a server, a client, and a virtual file system. The final component is the virtual file system. We also have language-specific testing frameworks.

Java 132

Java Programming Language Algorithm Coding 132

Knowledge Hut

APRIL 29, 2024

The stakes are high in the banking and financial industry since substantial financial sums are at risk and the potential for significant economic upheaval if banks and other financial systems are compromised. The Indian government has reported 11,60,000 cyber-attacks in 2022. The attackers entered the system using fake RBI IDs.

Banking 98

Banking Government Media Electronics 98

DoorDash Engineering

FEBRUARY 27, 2024

We reviewed the architecture of our global search at DoorDash in early 2022 and concluded that our rapid growth meant within three years we wouldn’t be able to scale the system efficiently, particularly as global search shifted from store-only to a hybrid item-and-store search experience.

Engineering 130

Engineering Systems Designing Architecture 130

Engineering at Meta

OCTOBER 15, 2024

AI has been at the core of the experiences Meta has been delivering to people and businesses for years, including AI modeling innovations to optimize and improve on features like Feed and our ads system. For example, Llama 3.1 405B , Meta’s largest model, is a dense transformer with 405B parameters and a context window of up to 128k tokens.

Deep Learning 131

Deep Learning Designing Architecture Systems 131

Precisely

AUGUST 12, 2024

Key Takeaways Legacy CCM systems are damaging to customer satisfaction and employee productivity and are no longer sustainable. But many organizations continue using outdated legacy CCM systems, which can significantly hinder both customer satisfaction and employee productivity and retention.

Systems 52

Systems Technology Machine Learning Data Analytics 52

Data Engineering Podcast

JUNE 11, 2023

Summary Data engineering is all about building workflows, pipelines, systems, and interfaces to provide stable and reliable data. July 2021 and July 2022 about data-diff) What are the roadblocks to data testing/validation that you see teams run into most often? Your data can be stable and wrong, but then it isn't reliable.

Project 130

Project Building Data Lake Machine Learning 130

DoorDash Engineering

MARCH 7, 2023

While most engineering tooling at DoorDash is focused on making safe incremental improvements to existing systems, in part by testing in production (learn more about our end-to-end testing strategy ), this is not always the best approach when launching an entirely new business line. No infrastructure required in the cloud.

AWS 119

AWS PostgreSQL Database SQL 119

The Pragmatic Engineer

AUGUST 22, 2023

But even in 2022, the biggest challenge for anyone keen to learn more is that gaming is a sector which is often shrouded in mystery. Programmers are building core systems, artists are establishing the visual aesthetic and style of the game, and designers are establishing gameplay, long-term progression and retention.

Software Engineer 207

Software Engineer Software Engineering Consulting Entertainment 207

Jesse Anderson

DECEMBER 12, 2022

Google looked over the expanse of the growing internet and realized they’d need scalable systems. With an immutable file system like HDFS, we needed scalable databases to read and write data randomly. We lacked a scalable pub/sub system. She wrote Data Mesh in 2022 to provide more information about the subject.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Ripple Engineering

OCTOBER 5, 2023

Since September 2022, the RippleX performance team has collaborated closely with the engineering team responsible for the AMM feature implementation. Overview In the rippled 1.12.0 release, the AMM amendment stands out as a significant feature in both size and scope. The key features of this environment are the following.

AWS 144

AWS BI Designing Database 144

Cloudera

JULY 11, 2022

Their most recent evaluation, Forrester Wave : Enterprise Data Fabric, Q2 2022, came out on June 23, 2022 and ranked Cloudera as a strong performer. Forrester Research identified the paradigm and coined the term in 2006. Since then, they have conducted regular market analyses and rankings of vendors.

Data Architecture 112

Data Architecture Architecture Data Government 112

U-Next

MARCH 12, 2023

Making use of Date class Making use of Calendar class Using Date Class: In this method, we will build a date class in Java object and then use it to use the function toString() method to acquire the system’s current Java DateTime. getTime()); System.out.println(“Prior change ” neda2); neda2.setTime(204587433443L);

Java 96

Java Programming Language Coding Utilities 96