Uber Engineering

SEPTEMBER 29, 2021

a driver starting a trip) and system actions … The post Building Uber’s Fulfillment Platform for Planet-Scale using Google Cloud Spanner appeared first on Uber Engineering Blog. The platform handles billions of database transactions each day, ranging from user actions (e.g.,

Google Cloud 145

Google Cloud Cloud Building Database 145

The Pragmatic Engineer

JUNE 1, 2023

If you’re not yet a full subscriber, you missed this week’s deep-dive on Agoda’s private cloud setup. Change deploys to be a one-command process, instead of multiple steps See blog posts #1-11 for details on all the steps. We cover one out of four topics in today’s subscriber-only The Scoop issue.

Education 364

Education Project PostgreSQL Software Engineer 364

Cloudera

MAY 14, 2024

Hybrid cloud plays a central role in many of today’s emerging innovations—most notably artificial intelligence (AI) and other emerging technologies that create new business value and improve operational efficiencies. The challenge is not solved, though, by simply adopting a hybrid cloud infrastructure. Data comes in many forms.

Cloud 99

Cloud Data Governance Unstructured Data Data Architecture 99

Cloudera

MARCH 22, 2024

As enterprise AI technologies rapidly reshape our digital environment, the foundation of your cloud infrastructure is more critical than ever. Previously, Cloudera customers on public cloud were provided with Virtual Machine images built on CentOS Linux. Let’s dive into what this means and why it’s a game-changer for our customers.

Cloud 115

Cloud Certification Systems Architecture 115

databricks

AUGUST 13, 2024

Databricks SQL Serverless is now Generally Available on Google Cloud Platform (GCP)! SQL Serverless is available in 7 GCP regions and 40+ regions across AWS, Azure and GCP.

Google Cloud 131

Google Cloud SQL Cloud AWS 131

Striim

MARCH 21, 2025

Today were going to talk about five streaming cloud integration use cases. Streaming cloud integration moves data continuously in real time between heterogeneous databases, with in-flight data processing. Use Case #1 Online Migration/Cloud Adoption Lets start with the first one. This unlimited testing minimizes your risks.

Cloud 52

Cloud Database Architecture BI 52

Striim

MARCH 21, 2025

Today were going to talk about five streaming cloud integration use cases. Streaming cloud integration moves data continuously in real time between heterogeneous databases, with in-flight data processing. Use Case #1 Online Migration/Cloud Adoption Lets start with the first one. This unlimited testing minimizes your risks.

Cloud 52

Cloud Database Architecture BI 52

Snowflake

SEPTEMBER 24, 2024

Today, Snowflake is officially launching the AI Data Cloud for Travel and Hospitality. Snowflake’s newest AI Data Cloud offers a unified and secure platform that streamlines AI and ML development to support the growth of travel and hospitality businesses, empowering organizations to harness their data’s full potential.

Hospitality 105

Hospitality Cloud Database-centric Data 105

Picnic Engineering

FEBRUARY 6, 2024

From Cloud-native to Hybrid and back again: Picnic’s on-premises computing journey Many companies are working on their digital transformation, transitioning their traditional on-premises deployment to a cloud setup. Picnic’s infrastructure design focuses on a rapidly scalable cloud solution.

Cloud 97

Cloud Transportation Cloud Computing Systems 97

Striim

OCTOBER 31, 2024

With Striim’s real-time data integration solution, the institution successfully transitioned to a cloud infrastructure, maintaining seamless operations and paving the way for future advancements. However, while the SIS migration was a significant step forward, the institution’s on-premise SQL Server systems remained vital.

Education 52

Education Healthcare Data Integration Cloud 52

Knowledge Hut

NOVEMBER 16, 2023

From nebulous beginnings, the cloud has grown into a platform that has gained universal acceptance and is transforming businesses across industries. Companies that have adopted cloud technology have seen significant payoffs, with cloud-based tools redefining their data storage, data sharing, marketing and project management capabilities.

Cloud Computing 98

Cloud Computing Cloud Cloud Storage AWS 98

Cloudera

DECEMBER 6, 2024

The AI Forecast: Data and AI in the Cloud Era , sponsored by Cloudera, aims to take an objective look at the impact of AI on business, industry, and the world at large. And specifically, I was reading one of your blog posts recently that talked about the dark ages of data. But what does that future look like?

Manufacturing 82

Manufacturing Retail Finance Metadata 82

Knowledge Hut

JANUARY 12, 2024

What comes to your mind when you hear the term 'Cloud'? Well, in a technologically advanced world, Cloud refers to a place where you can store and manage data on a device. Personally, I find it fascinating how saying, "I can handle the Cloud," has become a ticket to professional opportunities. What is Cloud Computing?

Cloud Computing 98

Cloud Computing Cloud Computer Science Education 98

The Pragmatic Engineer

OCTOBER 10, 2024

Redis Labs changed the formerly permissive open source Redis license to a restrictive one, with the goal to have cloud providers pay when they host Redis. As a response, cloud providers started the Valkey project, which could become the “new and still permissive Redis.” Commercial open source companies in trouble?

Consulting 347

Consulting AWS Engineering Software Engineer 347

Cloudera

APRIL 1, 2024

We’re excited to share that Gartner has recognized Cloudera as a Visionary among all vendors evaluated in the 2023 Gartner® Magic Quadrant for Cloud Database Management Systems. Download the complimentary 2023 Gartner Magic Quadrant for Cloud Database Management Systems report.

Cloud 105

Cloud Unstructured Data Metadata Government 105

Towards Data Science

APRIL 30, 2024

This blog post is my note after reading the paper: The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing. In the rest of this blog, we will see how Google enables this contribution. See you next blog!

Google Cloud 92

Google Cloud Process Cloud Lambda Architecture 92

WeCloudData

APRIL 22, 2025

PaaS is a fundamental cloud computing model that offers developers and organizations a robust environment for building, deploying, and managing applications efficiently. This blog provides detailed information on data Platform as a Service (PaaS),, how it differs from other cloud computing models, its working principles, and its benefits.

Cloud Computing 52

Cloud Computing Cloud Building Management 52

Snowflake

OCTOBER 17, 2023

See how ctrl+s provides in-depth insights into supply chain sustainability, while protecting sensitive customer information—all through Snowflake’s powerful, scalable Data Cloud. To this end, the company uses Snowflake’s Data Cloud to support its data activities. Sustainability is an issue at the forefront of most companies’ agendas.

Cloud 105

Cloud Consulting Transportation Manufacturing 105

Knowledge Hut

JANUARY 11, 2024

Are you planning to appear for the AWS Cloud Practitioner Certification in 2023? Cloud Computing is one of the biggest industries in Information Technology.  Cloud Computing is allowing businesses and clients to transact better and incorporate innovative ideas in an effective way and on a massive scale.

AWS 96

AWS Amazon Web Services Cloud Cloud Computing 96

The Pragmatic Engineer

OCTOBER 18, 2024

WP Engine is the challenger for the most popular managed WordPress hosting service – generating likely around $400M/year in revenue ( as per Automattic ), versus Automattic’s circa $500M/year, as per Automattic’s CEO, in a now-edited blog post. Automattic raised $980M in venture funding and was valued at $7.5B

Engineering 357

Engineering Government Project AWS 357

Snowflake

NOVEMBER 2, 2023

From the start, the Snowflake platform has been delivered as a service, consisting of optimized storage, elastic multi-cluster compute, and cloud services. To learn more about how Snowflake supports the architecture patterns described in this blog post, visit our pages for data warehouse , data lake , data lakehouse , and data mesh.

Data Lake 114

Data Lake Data Warehouse Cloud Unstructured Data 114

Cloudera

DECEMBER 17, 2024

Because data management is a key variable for overcoming these challenges, carriers are turning to hybrid cloud solutions, which provide the flexibility and scalability needed to adapt to the evolving landscape 5G enables. The hybrid cloud architecture also positions Vi for seamless future deployments and AI/ML workloads.

Building 62

Building Telecommunication Data Architecture Architecture 62

Netflix Tech

APRIL 8, 2025

By Cheng Xie , Bryan Shultz , and Christine Xu In a previous blog post , we described how Netflix uses eBPF to capture TCP flow logs at scale for enhanced network insights. Netflixs cloud microservices operate across multiple AWS regions. A BriefRecap FlowExporter is a sidecar that runs alongside all Netflix workloads. With 30 c7i.2xlarge

AWS 75

AWS Kafka Cloud Programming 75

Cloudera

AUGUST 27, 2024

It’s nearing the end of the summer in North America, and one report has been a staple on my reading list for more than a decade: the Flexera State of the Cloud Report. Cloud spend remained on top for the second year in a row, with public cloud spend exceeding budgets by an average of 15%.

Cloud 83

Cloud Banking Business Intelligence Healthcare 83

Edureka

APRIL 22, 2025

This blog will explore the significant advancements, challenges, and opportunities impacting data engineering in 2025, highlighting the increasing importance for companies to stay updated. In 2025, this blog will discuss the most important data engineering trends, problems, and opportunities that companies should be aware of.

Data Engineering 52

Data Engineering Data Engineer Engineering Consulting 52

Uber Engineering

OCTOBER 27, 2024

Our latest blog dives into enabling security for Uber’s modernized batch data lake on Google Cloud Storage! Ready to boost your Hadoop Data Lake security on GCP?

Cloud Storage 75

Cloud Storage Google Cloud Data Lake Hadoop 75

Cloudera

NOVEMBER 13, 2024

The combined platform will integrate data – from wherever it originates and wherever it is stored (cloud or on prem) – to deliver real-time insights required for faster decision making and predictive generative AI applications for personalized customer experiences.

Metadata 90

Metadata Management Data Governance Government 90

KDnuggets

SEPTEMBER 23, 2024

In this blog, I will dive into free courses with Google, from programming. If you’ve been keeping up, I have been creating a series of free courses that are actually free, for example, the AI & ML Edition. Type in ‘Free courses that are actually free’ in the search bar to look at the rest.

Google Cloud 96

Google Cloud Cloud Programming 96

Netflix Tech

NOVEMBER 12, 2024

By: Rajiv Shringi , Oleksii Tkachuk , Kartik Sathyanarayanan Introduction In our previous blog post, we introduced Netflix’s TimeSeries Abstraction , a distributed service designed to store and query large volumes of temporal event data with low millisecond latencies. For more information regarding this, refer to our previous blog.

Datasets 101

Datasets Computer Science Systems Kafka 101

databricks

AUGUST 13, 2023

Today we are announcing the general availability of Delta Live Tables (DLT) on Google Cloud. DLT pipelines empower data engineers to build reliable.

Google Cloud 98

Google Cloud Cloud Data Engineering Data Engineer 98

Snowflake

APRIL 30, 2025

Customers want to know the level of protection their cloud providers offer to help mitigate the risk of ransomware. Recovery: Snowflake resiliency A Snowflake region is built across at least three availability zones within a cloud provider region.

Accessible 75

Accessible Accessibility Metadata Cloud 75

databricks

JULY 8, 2024

We are proud to announce two new analyst reports recognizing Databricks in the data engineering and data streaming space: IDC MarketScape: Worldwide Analytic.

Data Pipeline 137

Data Pipeline Process Cloud Data Engineering 137

Cloudera

DECEMBER 9, 2024

The models in the Garden are already optimized for running efficiently on Google’s Cloud infrastructure, offering cost effective inference and enterprise-grade scaling, even on the highest-throughput apps. Stay tuned for future AMPs well build using Cloudera AI and Vertex AI.

Machine Learning 75

Machine Learning Project Banking Accessibility 75

Data Engineering Weekly

MARCH 2, 2025

I found the blog to be a fresh take on the skill in demand by layoff datasets. The blog provides an excellent analysis of smallpond compared to Spark and Daft. link] Netflix: Cloud Efficiency at Netflix Data is the Key Optimization starts with collecting data and asking the right questions. link] Mehdio: DuckDB goes distributed?

Data Engineering 62

Data Engineering Data Engineer Engineering Datasets 62

Snowflake

SEPTEMBER 11, 2023

At the core of Snowflake is data, and the Snowflake Data Cloud is increasingly the central platform for many organizations’ data strategies. This blog post specifically addresses the highlighted sections in P-39.1 – Act respecting the protection of personal information in the private sector.

Cloud 104

Cloud Electronics Government Data Governance 104

Cloudyard

MAY 4, 2025

Traditionally, DBT is known for transformations and Snowflake for its cloud-native warehousing. In this blog, I walk you through a use case where DBT orchestrates an automated S3-to-Snowflake ingestion flow using Snowflake capabilities like file handling, schema inference, and data loading.

ETL Tools 59

ETL Tools Raw Data SQL Python 59

Data Engineering Podcast

APRIL 21, 2024

It is an open-source, cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability, a declarative programming model, and best-in-class testability. Contact Info LinkedIn Blog Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Data Lake 182

Data Lake High Quality Data Machine Learning Data Pipeline 182