article thumbnail

Datadog’s $65M/year customer mystery solved

The Pragmatic Engineer

The company racked up huge bills for the likes of AWS, Snowflake, and also Datadog. It’s mostly written in Go, with some Java, Python and Ruby parts. However, look closely at the responses I gathered, and “AWS” and “Datadog” are the two most mentioned vendors as targets for cost savings.

AWS 347
article thumbnail

Adopting Spark Connect

Towards Data Science

The appropriate Spark dependencies (spark-core/spark-sql or spark-connect-client-jvm) will be provided later in the Java classpath, depending on the run mode. hadoop-aws since we almost always have interaction with S3 storage on the client side). AWS Spot interruptions). classOf[SparkSession.Builder].getDeclaredMethod("remote",

Scala 75
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Using Trino And Iceberg As The Foundation Of Your Data Lakehouse

Data Engineering Podcast

If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com ) with your story. If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com ) with your story.

Data Lake 262
article thumbnail

My Python/Java/Spring/Go/Whatever Client Won’t Connect to My Apache Kafka Cluster in Docker/AWS/My Brother’s Laptop. Please Help!

Confluent

tl;dr When a client wants to send or receive a message from Apache Kafka®, there are two types of connection that must succeed: The initial connection to a broker (the […].

Kafka 121
article thumbnail

Databricks, Snowflake and the future

Christophe Blefari

In the data world Snowflake and Databricks are our dedicated platforms, we consider them big, but when we take the whole tech ecosystem they are (so) small: AWS revenue is $80b, Azure is $62b and GCP is $37b. you could write the same pipeline in Java, in Scala, in Python, in SQL, etc.—with Here we go again.

Metadata 147
article thumbnail

Addressing The Challenges Of Component Integration In Data Platform Architectures

Data Engineering Podcast

Go to dataengineeringpodcast.com/memphis today to get started! Data lakes are notoriously complex. Go to [dataengineeringpodcast.com/memphis]([link] today to get started! Rudderstack : ! Go to dataengineeringpodcast.com/memphis today to get started! Data lakes are notoriously complex. Rudderstack : !

article thumbnail

What Is Amazon EventBridge?

Edureka

Enter Amazon EventBridge, a fully managed serverless event bus service that makes it easier to build event-driven applications using data from your AWS services, custom applications, or SaaS providers. Overall, Amazon EventBridge is a foundational service for anyone looking to embrace modern, event-driven architecture on AWS.

AWS 40