The Pragmatic Engineer

NOVEMBER 21, 2023

It’s fascinating how what is considered “modern” for backend practices keep evolving over time; back in the 2000s, virtualizing your servers was the cutting-edge thing to do; while around 2010 if you onboarded to the cloud, you were well ahead of the pack. Joshua has remained technical while working as an executive.

Engineering 325

Engineering Bytes Cloud Computing AWS 325

Analytics Vidhya

FEBRUARY 8, 2023

Introduction Amazon Redshift is a fully managed, petabyte-scale data warehousing Amazon Web Services (AWS). It allows users to easily set up, operate, and scale a data warehouse in the cloud.

Amazon Web Services 220

Amazon Web Services AWS Data Warehouse Cloud 220

Snowflake

DECEMBER 5, 2023

We had a jam-packed week alongside more than 60,000 attendees at Amazon Web Services (AWS) re:Invent, one of the largest hands-on conferences in the cloud computing industry. But without a governed data foundation, you can’t trust results or unlock all that’s possible with these breakaway technologies.

AWS 117

AWS Amazon Web Services Government Cloud Computing 117

Analytics Vidhya

FEBRUARY 25, 2023

Introduction A data lake is a centralized and scalable repository storing structured and unstructured data. The need for a data lake arises from the growing volume, variety, and velocity of data companies need to manage and analyze.

Cloud Storage 157

Cloud Storage Data Lake Cloud Unstructured Data 157

Waitingforcode

MARCH 9, 2023

There is always a gap between a disruption in the data engineering industry and its integration on the cloud. It was not different for table file formats which have started gaining interest on AWS, Azure, GCP recently.

Cloud 130

Cloud AWS Data Engineering Data Engineer 130

Knowledge Hut

NOVEMBER 16, 2023

One of the biggest challenges faced by corporations today when it comes to cloud adoption is the lack of cloud expertise. There is a clear shortage of professionals certified with Amazon Web Services (AWS). As far as AWS certifications are concerned, there is always a certain debate surrounding them. What is AWS?

AWS 98

AWS Certification IT Amazon Web Services 98

Precisely

APRIL 11, 2024

In an era where cloud technology is not just an option but a necessity for competitive business operations, the collaboration between Precisely and Amazon Web Services (AWS) has set a new benchmark for mainframe and IBM i modernization.

AWS 72

AWS Amazon Web Services Cloud Cloud Computing 72

Precisely

OCTOBER 3, 2024

In reviewing best practices for your AWS cloud migration, it’s crucial to define your business case first, and work from there. Migrating to AWS can unlock incredible value for your business, but it requires careful planning, risk management, and the right technical and organizational strategies. Or is a rewrite the best route?

AWS 64

AWS Cloud Amazon Web Services Data Integration 64

DoorDash Engineering

MARCH 7, 2023

As backend developers, we needed to stay unblocked while the infrastructure — in this case AWS resources — was being created. It was fair to assume that we would use other AWS services, particularly SQS and AWS Secrets Manager. Use LocalStack to enable locally running AWS resources. No long waits during startup.

AWS 119

AWS PostgreSQL Database SQL 119

DoorDash Engineering

NOVEMBER 29, 2022

As DoorDash’s business grows, engineers strive for a better network infrastructure to ensure more third-party services could be integrated into our system while keeping data securely transmitted. The data transfer between DoorDash and payment processors needs to be encrypted to protect customers’ privacy and sensitive data.

Cloud 97

Cloud AWS Amazon Web Services Data 97

Towards Data Science

APRIL 6, 2023

Today’s post follows the same philosophy: fitting local and cloud pieces together to build a data pipeline. But, instead of GCP, we’ll be using AWS. AWS is, by far, the most popular cloud computing platform, it has an absurd number of products to solve every type of specific problem you imagine. not sponsored.

Data Pipeline 98

Data Pipeline AWS Amazon Web Services Python 98

Data Engineering Podcast

MAY 22, 2022

Summary Machine learning has become a meaningful target for data applications, bringing with it an increase in the complexity of orchestrating the entire data flow. With simple pricing, fast networking, object storage, and worldwide data centers, you’ve got everything you need to run a bulletproof data platform.

Machine Learning 130

Machine Learning Data Engineering Data Engineer Cloud 130

Cloudera

JANUARY 7, 2025

In todays dynamic digital landscape, multi-cloud strategies have become vital for organizations aiming to leverage the best of both cloud and on-premises environments. As enterprises navigate complex data-driven transformations, hybrid and multi-cloud models offer unmatched flexibility and resilience.

Cloud 82

Cloud Government AWS Unstructured Data 82

Towards Data Science

JANUARY 26, 2023

In this post we consider the case in which our data application requires access to one or more large files that reside in cloud object storage. This continues a series of posts on the topic of efficient ingestion of data from the cloud (e.g., here , here , and here ).

Cloud Storage 98

Cloud Storage Big Data Cloud AWS 98

Snowflake

MARCH 26, 2024

In 2023, we held our first Accelerate event to explore industry trends, track data and technology innovations in financial services, and lay out data strategy case studies for the industry. To learn more about the latest data and AI trends, read our new report, Data Trends 2024. Why Attend Accelerate Manufacturing?

Entertainment 93

Entertainment Manufacturing Media Amazon Web Services 93

Snowflake

JANUARY 31, 2024

Today, we’re excited to announce the general availability of the Snowflake Native App Framework on AWS and Azure! Because Snowflake Native Apps run within the customer’s Snowflake account, the customer doesn’t need to move or copy their data, which increases security and helps maintain governance controls.

AWS 115

AWS Government Programming Data Science 115

Knowledge Hut

MARCH 7, 2024

In the digital era, the demand for cloud computing has increased like never before. Increased security, scalability, reduced costs, and better collaboration are a few benefits of cloud computing. That is why the need for cloud computing companies has increased a lot. It is one of the safest platforms for cloud service.

Cloud Computing 98

Cloud Computing Amazon Web Services Cloud Google Cloud 98

The Pragmatic Engineer

JUNE 29, 2023

Data source: Domain Name Stat Domain providers which software engineers and other techies recommended in responses to my tweet: “What domain registrars would you recommend / do you use? AWS uses Gandi under the hood for many of its domains. Gandi : a French provider, especially popular in Europe. Around since 2003.

AWS 59

AWS Software Engineer Software Engineering Engineering 59

ProjectPro

FEBRUARY 8, 2023

Do ETL and data integration activities seem complex to you? AWS Glue is here to put an end to all your worries! Read this blog to understand everything about AWS Glue that makes it one of the most popular data integration solutions in the industry. Did you know the global big data market will likely reach $268.4

AWS 98

AWS Scala Metadata Data Lake 98

Cloudera

DECEMBER 19, 2024

AWS re:Invent is one of my favorite trade shows. Finally, we hosted a hands-on workshop to walk attendees through a Retrieval-Augmented Generation (RAG) workflow within Cloudera AI to show how easy it is to deploy contextualized models based on organizational data. You can read more about the partnership and its implications here.

AWS 75

AWS Metadata Government Machine Learning 75

Precisely

AUGUST 26, 2024

Key Takeaways: Enhance capabilities through partnerships: AWS, Confluent, and Precisely accelerate mainframe modernization efforts, providing you with essential tools for success. Replicating mainframe data to cloud environments is a challenging but transformative undertaking that boosts your organization’s operational agility and innovation.

AWS 64

AWS Cloud Data Integration Accessible 64

Start Data Engineering

NOVEMBER 8, 2020

Introduction If you are looking for a simple, cheap data pipeline to pull small amounts of data from a stable API and store it in a cloud storage, then serverless functions are a good choice.

AWS 130

AWS Cloud Storage Data Pipeline Data 130

Knowledge Hut

JULY 2, 2024

Cloud computing is changing faster than we ever imagined. Every day, new features and capabilities have been released that change how we think about, use, and administer cloud services. Thus, the cloud computing future looks pretty bright and stable. Here are 12 trends and predictions for the future of cloud computing.

Cloud Computing 98

Cloud Computing Cloud Healthcare Education 98

Cloudera

SEPTEMBER 28, 2020

Cloudera delivers an enterprise data cloud that enables companies to build end-to-end data pipelines for hybrid cloud, spanning edge devices to public or private cloud, with integrated security and governance underpinning it to protect customers data. Data Science and machine learning workloads using CDSW.

Cloud 132

Cloud Kafka Professional Services Metadata 132

Towards Data Science

JANUARY 2, 2025

Introduction Data Proposed Workflow AWS Cloud Components Collecting the Data (Lambda Function 1) Writing the Data to the Table (Lambda Function 2) Converting the data in CSV

AWS 83

AWS Data Pipeline Data Cloud 83

Cloudera

NOVEMBER 3, 2023

These awards recognize Cloudera’s EMEA partners who are dedicated to transforming the future of data by leveraging the most complete hybrid, multi-cloud data platform. Cloudera was named as IBM’s preferred partner of choice for Real-Time Data Movement using Kafka.

AWS 100

AWS Technology Amazon Web Services Insurance 100

Cloudera

DECEMBER 4, 2024

Today enterprises can leverage the combination of Cloudera and Snowflake—two best-of-breed tools for ingestion, processing and consumption of data—for a single source of truth across all data, analytics, and AI workloads.

AWS 89

AWS Raw Data Relational Database Government 89

Knowledge Hut

DECEMBER 26, 2023

Cloud computing enables an organization to use on-demand IT resources and scale up or down as per their requirements. The company does not need to invest in any additional hardware or equipment or purchase physical data centers for storage and management. What Are the Types of Cloud Computing Tools Available? and more 2.

Cloud Computing 98

Cloud Computing Cloud Amazon Web Services AWS 98

Data Engineering Podcast

FEBRUARY 4, 2024

In this episode Yingjun Wu explains how it is architected to power analytical workflows on continuous data flows, and the challenges of making it responsive and scalable. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex.

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Christophe Blefari

MARCH 1, 2023

dbt Core is an open-source framework that helps you organise data warehouse SQL transformation. dbt Labs also develop dbt Cloud which is a cloud product that hosts and runs dbt Core projects. This switch has been lead by modern data stack vision. With the public clouds—e.g. With the public clouds—e.g.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Knowledge Hut

JANUARY 29, 2024

Some techniques add to the development of technology in the business sectors, including Data Science and Cloud Computing, essential aspects of the technology industry. With the help of data science, one can gather all the critical analyses from vast chunks of data stored in clouds.

Cloud Computing 98

Cloud Computing Data Science Cloud Amazon Web Services 98

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. Data lakes are notoriously complex.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Snowflake

NOVEMBER 2, 2023

Over the years, the technology landscape for data management has given rise to various architecture patterns, each thoughtfully designed to cater to specific use cases and requirements. These patterns include both centralized storage patterns like data warehouse , data lake and data lakehouse , and distributed patterns such as data mesh.

Data Lake 107

Data Lake Data Warehouse Cloud Unstructured Data 107

Towards Data Science

MARCH 6, 2023

On-premise and cloud working together to deliver a data product Photo by Toro Tseleng on Unsplash Developing a data pipeline is somewhat similar to playing with lego, you mentalize what needs to be achieved (the data requirements), choose the pieces (software, tools, platforms), and fit them together.

Google Cloud 98

Google Cloud Cloud Storage Data Pipeline Cloud 98

Cloudera

FEBRUARY 10, 2022

After the launch of Cloudera DataFlow for the Public Cloud (CDF-PC) on AWS a few months ago, we are thrilled to announce that CDF-PC is now generally available on Microsoft Azure, allowing NiFi users on Azure to run their data flows in a cloud-native runtime. . Processing Streaming Data. SIEM Optimization.

Cloud 116

Cloud Kafka AWS Data Ingestion 116

Cloudera

SEPTEMBER 15, 2023

The world is awash with data, no more so than in the telecommunications (telco) industry. With some Cloudera customers ingesting multiple petabytes of data every single day — that’s multiple thousands of terabytes! Access and the exchange of data is critical for managing the operations in many industries.

Telecommunication 80

Telecommunication AWS Retail Amazon Web Services 80

Confluent

NOVEMBER 20, 2024

Combine AWS IoT Core with Confluent Cloud to contextualize your IoT data using your other data sources. Learn more and get a full setup tutorial.

AWS 59

AWS Cloud Data 59