Snowflake

NOVEMBER 11, 2024

With Provisional Throughput (public preview soon on AWS), customers can reserve dedicated throughput, ensuring consistent and predictable performance for their workloads. To learn more about these new features and related updates check out our Cortex Analyst blog post.

Unstructured Data 95

Unstructured Data SQL AWS Healthcare 95

Knowledge Hut

JANUARY 11, 2024

Are you planning to appear for the AWS Cloud Practitioner Certification in 2023? Here is a complete and comprehensive guide of all the things you can expect from the AWS exams and how to best prepare for them! Amazon Web Services (AWS) is the global leader in cloud computing. Looking for some guidance in the right direction?

AWS 96

AWS Amazon Web Services Cloud Cloud Computing 96

ProjectPro

FEBRUARY 8, 2023

AWS Glue is here to put an end to all your worries! Read this blog to understand everything about AWS Glue that makes it one of the most popular data integration solutions in the industry. Well, AWS Glue is the answer to your problems! Table of Contents What is AWS Glue? How Does AWS Glue Work?

AWS 98

AWS Scala Metadata Data Lake 98

databricks

NOVEMBER 28, 2023

We’re thrilled to share that Databricks has won the AWS ISV Partner of the Year award for North America. This award recognizes top I.

AWS 111

AWS 111

databricks

MAY 10, 2023

We're excited to announce the general availability of Databricks Fleet clusters on AWS. What are Fleet clusters? Databricks Fleet clusters unlock the potential.

AWS 96

AWS 96

Cloudera

JANUARY 7, 2025

A prominent public health organization integrated data from multiple regional health entities within a hybrid multi-cloud environment (AWS, Azure, and on-premise). A leading meal kit provider migrated its data architecture to Cloudera on AWS, utilizing Cloudera’s Open Data Lakehouse capabilities.

Cloud 82

Cloud Government AWS Unstructured Data 82

Netflix Tech

APRIL 8, 2025

By Cheng Xie , Bryan Shultz , and Christine Xu In a previous blog post , we described how Netflix uses eBPF to capture TCP flow logs at scale for enhanced network insights. Netflixs cloud microservices operate across multiple AWS regions. For instance, a significant portion of flows goes through AWS ELBs. With 30 c7i.2xlarge

AWS 76

AWS Kafka Cloud Programming 76

Workfall

MAY 16, 2023

In this blog, we shall look at how we can create a Dockerfile to create an image with this executable. In this blog, we will cover: Docker AWS EC2 Hands-on Conclusion Docker We are going to containerize our application using Docker to make it easier to deploy on different target platforms. Enter the application name.

AWS 81

AWS Cloud Computing Cloud Coding 81

Data Engineering Weekly

OCTOBER 27, 2024

The blog is an excellent summary of the existing unstructured data landscape. It is exciting to read probably the first blog on building a vector search infrastructure at scale. The blog from Meta discusses how it designed a privacy-preserving storage. Luckily, the Flink community is actively innovating on this.

Data Engineer 57

Data Engineer Data Engineering Engineering Unstructured Data 57

Cloudera

NOVEMBER 3, 2023

The post EMEA Summit Partner Awards for AWS, Accenture, IBM, Talend a Qlik Company, Tech Data, and Dell Technologies appeared first on Cloudera Blog.

AWS 100

AWS Technology Amazon Web Services Insurance 100

Precisely

DECEMBER 14, 2023

As 2023 comes to an end we’re counting down the Top 5 Data Integrity blog posts of the year. #5. Trusted Generative AI using Precisely and AWS Welcome to an era where machines are breaking free from their traditional analytical roles to unleash boundless creativity and innovation. Read more > #2.

Data Integration 68

Data Integration AWS Machine Learning Data 68

KDnuggets

JANUARY 23, 2023

This blog post will make you less likely to run into issues in this 15+ step process. JupyterHub is a multi-user, container-friendly version of the Jupyter Notebook. However, it can be difficult to setup.

AWS 111

AWS Process IT 111

Data Engineering Weekly

NOVEMBER 24, 2024

The blog highlights how moving from 6-character base-64 to 20-digit base-2 file distribution brings more distribution in S3 and reduces request failures. The blog is a good summary of how to use Snowflake QUERY_TAG to measure and monitor query performance. The blog post made me curious to understand DataFusion's internals.

Data Engineer 57

Data Engineer Data Engineering Engineering Insurance 57

databricks

MARCH 22, 2023

We are excited to announce that PrivateLink and using customer-managed keys (CMK) for encryption are now Generally Available (GA) for Databricks on AWS.

AWS 98

AWS Management 98

Cloudera

SEPTEMBER 15, 2023

Next — working backwards — collaboration tooling such as dashboards and developer environments like those on Amazon Web Services (AWS), are also key components to consider. Building your data monetization strategy on Cloudera and AWS AWS provides several products and solutions to support this approach.

Telecommunication 80

Telecommunication AWS Retail Amazon Web Services 80

databricks

JULY 28, 2023

For the final part of our Best Practices and Guidance for Cloud Engineers to Deploy Databricks on AWS series, we'll cover an important.

AWS 98

AWS Cloud Engineering 98

databricks

JANUARY 26, 2023

This is part two of a three-part series in Best Practices and Guidance for Cloud Engineers to deploy Databricks on AWS. You can.

AWS 98

AWS Cloud Engineering 98

Team Data Science

MAY 10, 2020

The remaining tech (stages 3, 4, 7 and 8) are all AWS technologies. What's Next I'll be documenting how I build this setup in the AWS console (with screenshots). I can now begin drafting my data ingestion/ streaming pipeline without being overwhelmed.

Data Pipeline 130

Data Pipeline Data Ingestion AWS Kafka 130

WeCloudData

MAY 2, 2025

This blog explores Infrastructure as Code (IaC), its use cases, benefits, tools, and how AWS and Azure are enhancing IaC practices. Infrastructure as Code (IaC) offers an efficient, reproducible, and error-resistant approach to managing infrastructure.

Coding 52

Coding AWS Management IT 52

Data Engineering Weekly

APRIL 20, 2025

The blog took out the last edition’s recommendation on AI and summarized the current state of AI adoption in enterprises. The simplistic model expressed in the blog made it easy for me to reason about the transactional system design. The popularity also exposes its Achilles heel, the replication and network bottlenecks.

Data Engineer 67

Data Engineer Data Engineering Engineering Kafka 67

Sync Computing

JANUARY 27, 2025

Databricks clusters and AWS EC2 In todays landscape, big data, which is data too large to fit into a single node machine, is transformed and managed by clusters. M6GD instances are general-purpose EC2 instances equipped with AWS Graviton2 processors and local NVMe-based SSD storage, offering a balanced mix of compute, memory, and storage.

AWS 52

AWS Data Lake Machine Learning Big Data 52

The Pragmatic Engineer

OCTOBER 18, 2024

WP Engine is the challenger for the most popular managed WordPress hosting service – generating likely around $400M/year in revenue ( as per Automattic ), versus Automattic’s circa $500M/year, as per Automattic’s CEO, in a now-edited blog post. Automattic raised $980M in venture funding and was valued at $7.5B

Engineering 356

Engineering Government Project AWS 356

Confluent

DECEMBER 7, 2021

In our February 2020 blog post Celebrating Over 100 Supported Apache Kafka® Connectors, we announced support for more than 100 connectors on Confluent Platform. Since then, we have been focused […].

AWS 104

AWS Kafka Management 104

Yelp Engineering

APRIL 30, 2024

This is the next installment of our blog series on improving our autoscaling infrastructure. In the previous blog posts (Open-sourcing Clusterman, Recycling kubernetes nodes) we explained the architecture and inner-working of Clusterman. Spoiler alert: Karpenter blog post is coming soon!)

AWS 64

AWS Architecture 64

Hevo

AUGUST 29, 2024

AWS Glue is a fully managed serverless ETL service that simplifies preparing and loading data for analytics. In this blog, we will discuss the AWS Glue architecture so you can fully understand how it works and optimize your data better. […] But how does it work?

AWS 52

AWS Architecture Management IT 52

databricks

JUNE 23, 2023

This is a collaborative post from Databricks and Amazon Web Services (AWS). We thank Venkat Viswanathan, Data and Analytics Strategy Leader, Partner Solutions.

Amazon Web Services 98

Amazon Web Services AWS Data 98

Pinterest Engineering

NOVEMBER 18, 2024

In this blog post, we’ll explore what CDC is, why it’s important, and our journey of implementing Generic CDC solutions for all online databases at Pinterest. The control plane manages various aspects of the system: It runs on a single host inside an AWS® Auto Scaling Group with a minimum and maximum host count of 1. or its affiliates.

Kafka 53

Kafka MySQL Database Software Engineer 53

Confluent

MAY 14, 2024

In the blog we will focus specifically on real-time analysis of AWS audit logs. Explore the practical applications of using the Destinations EventBridge API to send data in real time to Confluent, enabling a myriad of use cases.

AWS 69

AWS Cloud Data 69

Cloudyard

NOVEMBER 5, 2024

In this blog, we’ll: Show how to split a large file using Snowpark’s DataFrame transformations. AWS S3 CLI (cp utility) : For S3-stored files, the AWS CLI cp command supports parallel uploads and efficient file handling. Though it’s a valuable exercise for users looking to deepen their understanding of Snowpark.

AWS 52

AWS Utilities Database Data Engineer 52

Cloudyard

DECEMBER 18, 2024

Snowflake, with its robust Snowpark API and features like COPY FILES and REMOVE, allows us to develop fully managed solutions without relying on external automation like AWS Lambda or lifecycle policies.

Retail 52

Retail Data Ingestion AWS Data Pipeline 52

Hevo

AUGUST 27, 2024

In this blog, we will explore how to build a data pipeline using AWS Glue S3. AWS Glue is a tool that makes building and managing your data pipelines easier. We will go through every step of the process, and by the end, you will see how straightforward it can be. It’s fully managed, so […]

AWS 52

AWS Building Data Pipeline Management 52

Edureka

APRIL 22, 2025

Enter Amazon EventBridge, a fully managed serverless event bus service that makes it easier to build event-driven applications using data from your AWS services, custom applications, or SaaS providers. Overall, Amazon EventBridge is a foundational service for anyone looking to embrace modern, event-driven architecture on AWS.

AWS 40

AWS Architecture Media Cloud 40

Data Engineering Weekly

NOVEMBER 3, 2024

The blog emphasizes the importance of starting with a clear client focus to avoid over-engineering and ensure user-centric development. link] CapitalOne: Serverless ML - Lessons from Capital One CapitalOne writes about its experience building Serverless ML on top of AWS Lambda.

Data Engineer 62

Data Engineer Data Engineering Engineering Pipeline-centric 62

databricks

APRIL 30, 2024

We are excited to announce that Databricks on AWS GovCloud is now in public preview and that we recently earned our first FedRAMP® High agency ATO! We are ready today to support your International Traffic in Arms Regulations (ITAR) and HIPAA use cases; the Provisional Authorization for DoD Impact Level 5 (IL5) is expected soon.

AWS 59

AWS 59

Cloudera

OCTOBER 23, 2024

Try Cloudera’s open data lakehouse on AWS for 5 days for free here , or try Snowflake for free for 30 days here. The post Cloudera and Snowflake Partner to Deliver the Most Comprehensive Open Data Lakehouse appeared first on Cloudera Blog. Both platforms are free to try today.

Metadata 84

Metadata BI Data Lake Business Intelligence 84

Data Engineering Weekly

JUNE 10, 2024

I will write a separate blog on these announcements after the Databricks conference; in the meantime, I found the blog from Cube Research, a balanced article about Snowflake Summit. Databricks and Snowflake offer a data warehouse on top of cloud providers like AWS, Google Cloud, and Azure.

Data Engineer 103

Data Engineer Data Engineering Engineering Kafka 103

Christophe Blefari

JULY 13, 2024

Fast News ⚡️ End-to-end data lineage in AWS — AWS announced DataZone to bring lineage to your data assets , from the picture it can mixes datasets (?), UI is AWS-tier. You can also read Timo detailed post on Mixpanel blog and why SCD are the best thing for product analytics. His chart are great.

Kafka 130

Kafka AWS Data Database 130