article thumbnail

DuckDB … reading from s3 … with AWS Credentials and more.

Confessions of a Data Guy

What was not clear, or easy, was trying to figure out how DuckDB would LIKE to read default AWS […] The post DuckDB … reading from s3 … with AWS Credentials and more. appeared first on Confessions of a Data Guy.

AWS 113
article thumbnail

What Is AWS DMS And Why You Shouldn’t Use It As An ELT

Seattle Data Guy

Recently, I’ve encountered a few projects that used AWS DMS, which is almost like an ELT solution. It was interesting to see AWS DMS used in this manner. As… Read more The post What Is AWS DMS And Why You Shouldn’t Use It As An ELT appeared first on Seattle Data Guy. But it’s not what DMS was built for.

AWS 130
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

AWS Lambdas – Python vs Rust. Performance and Cost Savings.

Confessions of a Data Guy

Someone on Linkedin recently brought up the point that companies could save gobs of money by swapping out AWS Python lambdas for Rust ones. At least it’s an excuse to […] The post AWS Lambdas – Python vs Rust. While it raised the ire of many a Python Data Engineer, I thought it sounded like a great idea.

AWS 356
article thumbnail

Polars vs Pandas. Inside an AWS Lambda.

Confessions of a Data Guy

Inside an AWS Lambda. I also enjoy doing simple things, things that have a “real-life” feel to them. I suppose I could be like the others […] The post Polars vs Pandas. appeared first on Confessions of a Data Guy.

AWS 240
article thumbnail

Handling a Regional Outage: Comparing the Response From AWS, Azure and GCP

The Pragmatic Engineer

13 June 2023: AWS. The largest AWS region (us-east-1) degraded heavily for 3 hours, impacting 104 AWS services. We did a deepdive into this incident earlier in AWS’s us-east-1 outage. We’ll also learn how this article contributed to AWS publishing its first public postmortem in two years!

AWS 227
article thumbnail

Best Practices for Your AWS Cloud Migration

Precisely

In reviewing best practices for your AWS cloud migration, it’s crucial to define your business case first, and work from there. Migrating to AWS can unlock incredible value for your business, but it requires careful planning, risk management, and the right technical and organizational strategies.

AWS 64
article thumbnail

AWS Lambdas. Useful for Data Engineering?

Confessions of a Data Guy

For a lot of my Data Engineering career I didn’t really think about or use AWS lambdas, I just saw them as little annoying flies […] The post AWS Lambdas. I guess I’ve taken them for granted over the years, even though they are incredibly useful. Useful for Data Engineering?

AWS 130