Data Engineering Podcast

FEBRUARY 18, 2024

Summary A data lakehouse is intended to combine the benefits of data lakes (cost effective, scalable storage and compute) and data warehouses (user friendly SQL interface). Data lakes are notoriously complex. Join in with the event for the global data community, Data Council Austin.

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Snowflake

APRIL 1, 2024

Snowflake and Salesforce are happy to share that bidirectional data sharing between Snowflake, the Data Cloud company and Salesforce Data Cloud is now generally available. Enterprises generate valuable customer data inside Salesforce applications. It has opened up new frontiers of data collaboration.

Cloud 120

Cloud Retail Entertainment Media 120

Precisely

OCTOBER 24, 2024

Key Takeaways: Cloud migration enhances agility, cuts operational costs, and helps you stay compliant with evolving regulations. Maintaining data integrity during cloud migration is essential to ensure reliable and high-quality data for better decision-making and future use in advanced applications.

Cloud 111

Cloud Data Integration Insurance Data 111

KDnuggets

MAY 18, 2023

This article discusses the key components that contribute to the successful scaling of data science projects. It covers how to collect data using APIs, how to store data in the cloud, how to clean and process data, how to visualize data, and how to harness the power of data visualization through interactive dashboards.

Cloud Computing 137

Cloud Computing Data Science Cloud Project 137

Snowflake

MARCH 27, 2024

As organizations seek to drive more value from their data, observability plays a vital role in ensuring the performance, security and reliability of applications and pipelines while helping to reduce costs. One of our partners in this area is Observe , which offers a SaaS observability product that is built and operated on the Data Cloud.

Cloud 123

Cloud Data Governance Government Data 123

Snowflake

OCTOBER 23, 2023

Every day, we witness approximately 20 million Snowpark queries² driving a spectrum of data engineering and data science tasks, with Python leading the way. We are looking forward to working with the incoming Ponder team and the Modin community to bring additional Python capabilities to the Snowflake Data Cloud.

Python 141

Python Cloud Data Science Machine Learning 141

Christophe Blefari

MARCH 15, 2024

Saying mainly that " Sora is a tool to extend creativity " Last point Mira has been mocked and criticised online because as a CTO she wasn't able to say on which public / licensed data Sora has been trained on. Pandera, a data validation library for dataframes, now supports Polars.

Metadata 272

Metadata Datasets Data Data Warehouse 272

Waitingforcode

DECEMBER 5, 2023

The "vertical scaling" has caught my attention a few times already when I have been reading about cloud updates. I've always considered horizontal scaling as the single true scaling policy for elastic data processing pipelines. Have I been wrong?

Data Process 130

Data Process Process Cloud Data 130

KDnuggets

SEPTEMBER 11, 2024

These Cloud IDEs include AI code assistants and numerous plugins for a fast and efficient development experience. Access a pre-built Python environment with free GPUs, persistent storage, and large RAM.

Cloud 139

Cloud Data Science Python Coding 139

KDnuggets

OCTOBER 18, 2023

Cloud databases have made it easier and cheaper to develop enterprise-level applications, offering flexibility, convenience, and standard database functionality. See what KDnuggets recommends.

Database 148

Database Cloud IT Data Engineering 148

Waitingforcode

FEBRUARY 13, 2024

It's time for another part of "What's new on the cloud for data engineers" Let's see what happened in the last 5 months.

Data Engineering 130

Data Engineering Data Engineer Cloud Engineering 130

Snowflake

MARCH 5, 2024

An estimated 90% of the world’s data is unstructured, much of it in the form of visual content such as images and videos. Insights from analyzing this visual data can open up powerful new use cases that significantly boost productivity and efficiency, but enterprises need sophisticated computer vision technologies to achieve this.

Cloud 129

Cloud Medical Manufacturing Retail 129

Analytics Vidhya

MARCH 21, 2023

Introduction In today’s world, data is growing exponentially with time with digitalization. Organizations are using various cloud platforms like Azure, GCP, etc., to store and analyze this data to get valuable business insights from it.

Data 240

Data Cloud Data Lake Database 240

KDnuggets

OCTOBER 1, 2023

Explore the essentials of Google Cloud Platform for data science and ML, from account setup to model deployment, with hands-on project examples.

Google Cloud 149

Google Cloud Cloud Data Science Project 149

Knowledge Hut

JULY 2, 2024

Cloud computing is changing faster than we ever imagined. Every day, new features and capabilities have been released that change how we think about, use, and administer cloud services. Thus, the cloud computing future looks pretty bright and stable. Here are 12 trends and predictions for the future of cloud computing.

Cloud Computing 98

Cloud Computing Cloud Healthcare Education 98

ArcGIS

JANUARY 2, 2024

Process Landsat and GEDI datasets in the cloud and predict aboveground biomass for the state of Oregon using machine learning in ArcGIS Pro.

Cloud 115

Cloud Datasets Machine Learning Data 115

Snowflake

JULY 8, 2024

Regulated and sovereign markets across the world have stringent requirements stipulating certain important data be kept within geographical borders or even for certain workloads to have dedicated environments, separate from those of other customers. Select usage data will be sent from the zonal to the global repository.

Cloud 102

Cloud Google Cloud Education AWS 102

Cloudera

MARCH 22, 2024

As enterprise AI technologies rapidly reshape our digital environment, the foundation of your cloud infrastructure is more critical than ever. Previously, Cloudera customers on public cloud were provided with Virtual Machine images built on CentOS Linux. Let’s dive into what this means and why it’s a game-changer for our customers.

Cloud 120

Cloud Certification Systems Architecture 120

Snowflake

NOVEMBER 26, 2024

Many of our customers — from Marriott to AT&T — start their journey with the Snowflake AI Data Cloud by migrating their data warehousing workloads to the platform. Today we’re focusing on customers who migrated from a cloud data warehouse to Snowflake and some of the benefits they saw.

Data Warehouse 86

Data Warehouse Cloud PostgreSQL Hadoop 86

databricks

APRIL 8, 2024

We're excited to announce that Databricks has been honored with the 2024 Google Cloud Technology Partner of the Year award for Data -.

Google Cloud 131

Google Cloud Cloud Technology Data 131

Snowflake

APRIL 4, 2024

Data transformation is the process of converting data from one format to another, the “T” in ELT, or extract, load, transform, which enables organizations to get their data analytics-ready and derive insights and value from it. The Coalesce platform is uniquely built for Snowflake.

Cloud 111

Cloud Management Data Pipeline SQL 111

KDnuggets

DECEMBER 11, 2023

This is the BONUS week and we will dive into learning about deploying to the cloud. Welcome back to the KDnuggets’ "Back to Basics" series.

Cloud 145

Cloud Data Engineering Data Engineer Engineering 145

Snowflake

MAY 22, 2024

At Snowflake, we are making massive investments in generative AI and end-to-end machine learning capabilities to help customers build and deploy high-impact AI use cases that maximize the value of their data. TruEra’s capabilities complement the AI and ML data governance functionalities we already provide in the AI Data Cloud.

Cloud 127

Cloud Data Governance Technology Machine Learning 127

Data Engineering Podcast

SEPTEMBER 10, 2023

Summary Data systems are inherently complex and often require integration of multiple technologies. This offers a single location for managing visibility and error handling so that data platform engineers can manage complexity. With Materialize, you can!

BI 208

BI SQL Data Machine Learning 208

Snowflake

MAY 15, 2024

According to Infosys, 35% of AI projects will either fail or experience delays because of poor data quality. There’s a huge gap between the data quality most companies have by default and the data quality needed for successful AI. Metaplane ensures that every company can trust the data that powers their business.

Cloud 114

Cloud Metadata Data Pipeline Government 114

Snowflake

DECEMBER 18, 2023

When businesses share sensitive first-party data with outside partners or customers, they must do so in a way that meets strict governance requirements around security and privacy. Samooha’s intuitive user interface makes it easy to build data clean rooms that run as Snowflake Native Apps in the Data Cloud.

Cloud 120

Cloud Building Entertainment Government 120

Analytics Vidhya

FEBRUARY 21, 2023

Organizations are converting them to cloud-based technologies for the convenience of data collecting, reporting, and analysis. This is where data warehousing is a critical component of any business, allowing companies to store and manage vast amounts of data.

Data Warehouse 206

Data Warehouse IT Data Collection Data 206

Analytics Vidhya

FEBRUARY 20, 2023

Introduction Azure data factory (ADF) is a cloud-based data ingestion and ETL (Extract, Transform, Load) tool. The data-driven workflow in ADF orchestrates and automates data movement and data transformation.

Data Ingestion 183

Data Ingestion Data Cloud Cloud Computing 183

Analytics Vidhya

APRIL 3, 2023

Introduction Companies can access a large pool of data in the modern business environment, and using this data in real-time may produce insightful results that can spur corporate success. Real-time dashboards such as GCP provide strong data visualization and actionable information for decision-makers.

Data Engineering 251

Data Engineering Data Engineer Engineering Data 251

databricks

JULY 8, 2024

We are proud to announce two new analyst reports recognizing Databricks in the data engineering and data streaming space: IDC MarketScape: Worldwide Analytic.

Data Pipeline 137

Data Pipeline Process Cloud Data Engineering 137

KDnuggets

NOVEMBER 8, 2023

This week on KDnuggets: Dive into KDnuggets Back to Basics: Getting Started in 5 Steps series to help you master Python, SQL, Scikit-learn, PyTorch, and Google Cloud Platform. Unlock the Power of SQL in Data Visualization: Master the Art of Preparing Data for Impactful Charts and Graphs. • And much, much more!

Google Cloud 137

Google Cloud SQL Python Cloud 137

Analytics Vidhya

MARCH 5, 2023

Introduction Microsoft Azure HDInsight(or Microsoft HDFS) is a cloud-based Hadoop Distributed File System version. A distributed file system runs on commodity hardware and manages massive data collections. It is a fully managed cloud-based environment for analyzing and processing enormous volumes of data.

Hadoop 246

Hadoop Data Collection Cloud Technology 246

Analytics Vidhya

MAY 24, 2023

As modern companies rely on data, establishing dependable, effective solutions for maintaining that data is a top task for each organization. The complexity of information storage technologies increases exponentially with the growth of data. appeared first on Analytics Vidhya.

Data Storage 179

Data Storage IT Cloud Computing Cloud 179

Analytics Vidhya

FEBRUARY 28, 2023

Introduction Azure Databricks is a fast, easy, and collaborative Apache Spark-based analytics platform that is built on top of the Microsoft Azure cloud. A collaborative and interactive workspace allows users to perform big data processing and machine learning tasks easily.

Big Data 310

Big Data Machine Learning Cloud Data Process 310

KDnuggets

NOVEMBER 2, 2023

Dive into KDnuggets Back to Basics: Getting Started in 5 Steps series to help you master Python, SQL, Scikit-learn, PyTorch, and Google Cloud Platform.

Google Cloud 143

Google Cloud SQL Python Cloud 143

Data Engineering Podcast

FEBRUARY 4, 2024

In this episode Yingjun Wu explains how it is architected to power analytical workflows on continuous data flows, and the challenges of making it responsive and scalable. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex.

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

KDnuggets

DECEMBER 4, 2024

Learn to build, run, and manage data engineering pipelines both locally and in the cloud using popular tools.

Data Engineering 152

Data Engineering Data Engineer Engineering Project 152