This site uses cookies to improve your experience. To help us insure we adhere to various privacy regulations, please select your country/region of residence. If you do not select a country, we will assume you are from the United States. Select your Cookie Settings or view our Privacy Policy and Terms of Use.
Cookies and similar technologies are used on this website for proper function of the website, for tracking performance analytics and for marketing purposes. We and some of our third-party providers may use cookie data for various purposes. Please review the cookie settings below and choose your preference.
Used for the proper function of the website
Used for monitoring website traffic and interactions
Cookies and similar technologies are used on this website for proper function of the website, for tracking performance analytics and for marketing purposes. We and some of our third-party providers may use cookie data for various purposes. Please review the cookie settings below and choose your preference.
Blog Cloud Cloud Storage 
WeCloudData
APRIL 28, 2025
Our digital lives would be much different without cloud storage, which makes it easy to share, access, and protect data across platforms and devices. The cloud market has huge potential and is continuously evolving with the advancement in technology and time.
Towards Data Science
JANUARY 26, 2023
In this post we consider the case in which our data application requires access to one or more large files that reside in cloud object storage. This continues a series of posts on the topic of efficient ingestion of data from the cloud (e.g., here , here , and here ). CPU cores and TCP connections).
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Agent Tooling: Connecting AI to Your Tools, Systems & Data
How to Modernize Manufacturing Without Losing Control
Mastering Apache Airflow® 3.0: What’s New (and What’s Next) for Data Orchestration
Cloudera
SEPTEMBER 10, 2021
Shared Data Experience ( SDX ) on Cloudera Data Platform ( CDP ) enables centralized data access control and audit for workloads in the Enterprise Data Cloud. The public cloud (CDP-PC) editions default to using cloud storage (S3 for AWS, ADLS-gen2 for Azure). Customer 1 – Centralized data authorization management.
Uber Engineering
OCTOBER 27, 2024
Our latest blog dives into enabling security for Uber’s modernized batch data lake on Google Cloud Storage! Ready to boost your Hadoop Data Lake security on GCP?
Cloudera
NOVEMBER 9, 2023
Powered by Apache HBase and Apache Phoenix, COD ships out of the box with Cloudera Data Platform (CDP) in the public cloud. It’s also multi-cloud ready to meet your business where it is today, whether AWS, Microsoft Azure, or GCP. We tested for two cloud storages, AWS S3 and Azure ABFS. runtime version.
ArcGIS
JUNE 5, 2024
This is a blog that will take users through workflows to create cloud storage connection files that can be used in their stac connections.
Cloudera
MARCH 31, 2021
CDP Public Cloud is now available on Google Cloud. The addition of support for Google Cloud enables Cloudera to deliver on its promise to offer its enterprise data platform at a global scale. CDP Public Cloud is already available on Amazon Web Services and Microsoft Azure.
94 
Knowledge Hut
NOVEMBER 16, 2023
From nebulous beginnings, the cloud has grown into a platform that has gained universal acceptance and is transforming businesses across industries. Companies that have adopted cloud technology have seen significant payoffs, with cloud-based tools redefining their data storage, data sharing, marketing and project management capabilities.
Netflix Tech
SEPTEMBER 24, 2021
Our previous tech blog Packaging award-winning shows with award-winning technology detailed our packaging technology deployed on the streaming side. As an example, cloud-based post-production editing and collaboration pipelines demand a complex set of functionalities, including the generation and hosting of high quality proxy content.
96 
Snowflake
MARCH 2, 2023
Rather than streaming data from source into cloud object stores then copying it to Snowflake, data is ingested directly into a Snowflake table to reduce architectural complexity and reduce end-to-end latency. And if you are using Amazon Managed Streaming for Apache Kafka (MSK), you can get started using this guided demo.
137 
Snowflake
DECEMBER 4, 2023
With this public preview, those external catalog options are either “GLUE”, where Snowflake can retrieve table metadata snapshots from AWS Glue Data Catalog, or “OBJECT_STORE”, where Snowflake retrieves metadata snapshots directly from the specified cloud storage location. With these three options, which one should you use?
122 
DataKitchen
DECEMBER 9, 2024
This blog dives into the remarkable journey of a data team that achieved unparalleled efficiency using DataOps principles and software that transformed their analytics and data teams into a hyper-efficient powerhouse. They opted for Snowflake, a cloud-native data platform ideal for SQL-based analysis.
phData: Data Engineering
NOVEMBER 8, 2024
Open Table Format (OTF) architecture now provides a solution for efficient data storage, management, and processing while ensuring compatibility across different platforms. In this blog, we will discuss: What is the Open Table format (OTF)? Amazon S3, Azure Data Lake, or Google Cloud Storage). Why should we use it?
52 
Edureka
APRIL 22, 2025
The alternative, however, provides more multi-cloud flexibility and strong performance on structured data. Snowflake is a cloud-native platform for data warehouses that prioritizes collaboration, scalability, and performance. It provides real multi-cloud flexibility in its operations on AWS , Azure, and Google Cloud.
52 
Cloudera
SEPTEMBER 15, 2020
introduces fine-grained authorization for access to Azure Data Lake Storage using Apache Ranger policies. Cloudera and Microsoft have been working together closely on this integration, which greatly simplifies the security administration of access to ADLS-Gen2 cloud storage. Cloudera Data Platform 7.2.1 What’s next?
Cloudera
JUNE 25, 2021
Many Cloudera customers are making the transition from being completely on-prem to cloud by either backing up their data in the cloud, or running multi-functional analytics on CDP Public cloud in AWS or Azure. Configure the required ports to enable connectivity from CDH to CDP Public Cloud (see docs for details).
73 
Scott Logic
APRIL 10, 2024
This blog post serves as a dev diary of the process, covering our challenges, contributions made and attempts to validate them. The focus of our submission was on calculating the energy cost of object or “blob” storage in the cloud (eg. This was something that the Cloud Carbon Footprint methodology already takes into account.
Cloudera
OCTOBER 26, 2020
Cloudera and Dell/EMC are continuing our long and successful partnership of developing shared storage solutions for analytic workloads running in hybrid cloud. . PowerScale and ECS as the storage layer for CDP Private Cloud Base. For clarity, the scope of the current certification covers CDP-Private Cloud Base.
Cloudera
SEPTEMBER 29, 2020
Performance is one of the key, if not the most important deciding criterion, in choosing a Cloud Data Warehouse service. In this blog post, we compare Cloudera Data Warehouse (CDW) on Cloudera Data Platform (CDP) using Apache Hive-LLAP to Microsoft HDInsight (also powered by Apache Hive-LLAP) on Azure using the TPC-DS 2.9
120 
Cloudera
AUGUST 17, 2022
CDP One is a new service from Cloudera that is the first data lakehouse SaaS offering with cloud compute, cloud storage, machine learning (ML), streaming analytics, and enterprise grade security built-in. It also requires zero cloud, security, or monitoring operations staff for a dramatically lower TCO and reduced risk. .
116 
Cloudera
FEBRUARY 9, 2021
Today, more and more customers are moving workloads to the public cloud for business agility where cost-saving and management are key considerations. Cloud object storage is used as the main persistent storage layer, which is significantly cheaper than block volumes. The Cost-Effective Data Warehouse Architecture.
Cloudera
SEPTEMBER 28, 2021
Cloudera Data platform ( CDP ) provides a Shared Data Experience ( SDX ) for centralized data access control and audit in the Enterprise Data Cloud. The Ranger Authorization Service (RAZ) is a new service added to help provide fine-grained access control (FGAC) for cloud storage. Changes with file access control .
U-Next
DECEMBER 19, 2022
Top 14 Challenges of Cloud Computing . Cloud Computing is the rage today. There isn’t an industry today that is not aware of or reaping the benefits of Cloud Computing. With it evolving rapidly and finding application in newer vertices, the day is not far when every aspect of our lives are somehow connected to the Cloud.
52 
DataKitchen
NOVEMBER 5, 2024
By storing data in its native state in cloud storage solutions such as AWS S3, Google Cloud Storage, or Azure ADLS, the Bronze layer preserves the full fidelity of the data. This foundational layer is a repository for various data types, from transaction logs and sensor data to social media feeds and system logs.
78 
Cloudera
AUGUST 30, 2022
This blog lays out some steps to help you incrementally advance efforts to be a more data-driven, customer-centric organization. Cloudera refers to this as universal data distribution, as explored further in this blog post. Leverage cloud where it makes sense, not because it’s fashionable. Embrace incremental progress.
107 
Cloudera
MAY 18, 2021
Prior the introduction of CDP Public Cloud, many organizations that wanted to leverage CDH, HDP or any other on-prem Hadoop runtime in the public cloud had to deploy the platform in a lift-and-shift fashion, commonly known as “Hadoop-on-IaaS” or simply the IaaS model. Cloudera subscription and compute costs. 1 Year Reserved .
86 
Rockset
MARCH 1, 2023
Cloud data warehouses with compute-storage separation do offer batch data loads running concurrently with query processing, but they provide this capability by giving up on real time. The possibilities are endless for compute-compute separation in the cloud.
144 
ProjectPro
FEBRUARY 6, 2023
With over 10 million active subscriptions, 50 million active topics, and a trillion messages processed per day, Google Cloud Pub/Sub makes it easy to build and manage complex event-driven systems. Google Cloud Pub/Sub is a global, cloud-based messaging framework that has become increasingly popular among data engineers over recent years.
52 
Cloudera
DECEMBER 21, 2021
Early in the year we expanded our Public Cloud offering to Azure providing customers the flexibility to deploy on both AWS and Azure alleviating vendor lock-in. A new capability called Ranger Authorization Service (RAZ) provides fine grained authorization on cloud storage. Test Drive CDP Pubic Cloud.
118 
Cloudera
SEPTEMBER 23, 2022
In a recent blog, Cloudera Chief Technology Officer Ram Venkatesh described the evolution of a data lakehouse, as well as the benefits of using an open data lakehouse, especially the open Cloudera Data Platform (CDP). Modern data lakehouses are typically deployed in the cloud. The first is near unlimited storage.
109 
Cloudera
FEBRUARY 7, 2019
Who knew that in that search, the company would become the first organization to globally run SAS Viya, a cloud-optimized software, with HDP on GCP to enable modern analytics use cases powered by SAS analytics tools. Reducing Analytic Time to Value by More Than 90 Percent.
67 
Cloudera
JUNE 5, 2023
This blog is to congratulate our winner and review the top submissions. DataFlow is a cloud-native data service powered by Apache NiFi with a streamlined user experience for development and deployment enabling true universal data distribution. RK built some simple flows to pull streaming data into Google Cloud Storage and Snowflake.
77 
Cloudera
SEPTEMBER 1, 2020
DDE is a new template flavor within CDP Data Hub in Cloudera’s public cloud deployment option (CDP PC). YARN allows you to use various data processing engines for batch, interactive, and real-time stream processing of data stored in HDFS or cloud storage like S3 and ADLS. data best served through Apache Solr). Prerequisites.
Striim
MARCH 21, 2025
Between continuous real-time collection of data, and its delivery to enterprise and cloud destinations, data has to move in a reliable and scalable way. Taking a streaming-first approach to data integration, Striim supports incremental, real-time views in your streaming layer and your cloud data warehouse.
52 
Data Engineering Podcast
SEPTEMBER 22, 2019
Summary Object storage is quickly becoming the unifying layer for data intensive applications and analytics. Modern, cloud oriented data warehouses and data lakes both rely on the durability and ease of use that it provides. How do you approach project governance and sustainability?
100 
Tweag
APRIL 19, 2023
In this blog post, I will explain the underlying technical challenges and share the solution that we helped implement at kaiko.ai , a MedTech startup in Amsterdam that is building a Data Platform to support AI research in hospitals. A solution is to read the bytes that we need when we need them directly from Blob Storage.
62 
Monte Carlo
JULY 19, 2023
Unlike ogres, however, the cloud data platform isn’t a fairy tale. For small data teams building their first cloud-native platforms and teams making the jump from on-prem for the first time, it’s essential to bias those layers that will have the most immediate impact on business outcomes. Let’s dive into it. Makes sense.
96 
Cloudera
OCTOBER 26, 2017
Clouds (source: Pexels ). Check out Greg Rahn’s session, “ Rethinking data marts in the cloud: Common architectural patterns for analytics ” at the Strata Data Conference in Singapore, December 4-7, 2017, to learn how to architect analytic workloads in the cloud and the core elements of data governance.
43 
Cloudera
JANUARY 21, 2021
While cloud-native, point-solution data warehouse services may serve your immediate business needs, there are dangers to the corporation as a whole when you do your own IT this way. You also do not want to risk your company-wide cloud consumption costs snowballing out of control. Separate storage. Yes there is a better choice!
94 
Knowledge Hut
DECEMBER 4, 2023
This is where cloud computing comes to the rescue. Cloud computing makes the services of a physical machine available to you as per your convenience, demand and budget, that too at the click of a button. A meticulous cloud computing certification will help you learn and use this technology effectively. What a tedious task!
Edureka
APRIL 14, 2025
This blog breaks down how these tools complement and differ from one another to help you identify the best fit for your business. Seamless Azure connectivity: Deep integration with the cloud allows for highly scalable processing and storage, leveraging the full power of Microsoft’s infrastructure.
40 
Hevo
FEBRUARY 2, 2024
Cloud Migration is one of the most crucial areas of the data science ecosystem. Local storage cannot store huge volumes of data, so a transition from on-premises to cloud storage is needed. In this blog, I […]
40 
Knowledge Hut
SEPTEMBER 21, 2023
Azure or Google Cloud—Which is better? This question is often asked as businesses continue to understand the cloud’s usefulness and services. Sometimes, considering the three leading players in the cloud market, businesses search for the right cloud among the three to adopt. So, let’s dive in! What Is Azure?
52 
Knowledge Hut
SEPTEMBER 28, 2023
The relevance of the AWS Cloud Practitioner Certification was something I couldn't ignore as I started on my path to gaining expertise in cloud computing. Anyone entering the cloud technology domain has to start with this fundamental credential. What is AWS Cloud Practitioner Certification?
52
Expert insights. Personalized for you.
We have resent the email to
Are you sure you want to cancel your subscriptions?
Let's personalize your content