KDnuggets

JULY 12, 2024

Discover the essential tools every data scientist should know to elevate their data science game, from Python and R to SQL and advanced visualization tools.

Data Science 153

Data Science SQL Python Data 153

databricks

JULY 8, 2024

We are proud to announce two new analyst reports recognizing Databricks in the data engineering and data streaming space: IDC MarketScape: Worldwide Analytic.

Data Pipeline 137

Data Pipeline Process Cloud Data Engineer 137

Waitingforcode

JULY 10, 2024

Welcome to the first Data+AI Summit 2024 retrospective blog post. I'm opening the series with the topic close to my heart at the moment, stream processing!

Data 130

Data Process 130

ArcGIS

JULY 12, 2024

Creation of a Digital Twin in Seven Days with ArcGIS in Zurich

Government 124

Government 124

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

KDnuggets

JULY 9, 2024

This article is for anyone looking to maximize their use of Amazon Web Services (AWS) generative AI (GenAI) services. Here are eight courses that range from beginner to expert level.

AWS 150

AWS Amazon Web Services 150

databricks

JULY 8, 2024

The Prodvana team joins Databricks to support new innovations in the Data Intelligence Platform infrastructure. Learn more about the vision and what's ahead.

Data 134

Data 134

ArcGIS

JULY 12, 2024

Esri is working with partners (Maxar, TomTom) to enhance our 3D basemaps with high-quality commercial data for elevation and buildings layers.

Building 119

Building Data 119

KDnuggets

JULY 10, 2024

Let's learn how to efficiently merge large Pandas dataframes.

Python 143

Python 143

databricks

JULY 12, 2024

Hallucinations in large language models (LLMs) occur when models produce responses that do not align with factual reality or the provided context. This.

126

126

Confessions of a Data Guy

JULY 9, 2024

When I was young and full of myself, writing Perl and PHP, while your ma was still reading you a bedtime story and giving you a stuffy to fall asleep with, I had to program uphill, both ways, in the rain and snow. Not like you milk toast Data Engineers clickty clicking around Databricks and […] The post The Abstractions Are Making You Dumb (rise of the Shallow Expert) appeared first on Confessions of a Data Guy.

Data Engineer 113

Data Engineer Data Engineering Programming Engineering 113

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Engineering at Meta

JULY 10, 2024

Meta’s advertising business leverages large-scale machine learning (ML) recommendation models that power millions of ads recommendations per second across Meta’s family of apps. Maintaining reliability of these ML systems helps ensure the highest level of service and uninterrupted benefit delivery to our users and advertisers. To minimize disruptions and ensure our ML systems are intrinsically resilient, we have built a comprehensive set of prediction robustness solutions that ensure stability w

Machine Learning 112

Machine Learning Algorithm Systems Engineering 112

KDnuggets

JULY 8, 2024

Text preprocessing is an important step in NLP. Let's learn how to use the Hugging Face Tokenizers Library to preprocess text data.

Data 143

Data 143

Precisely

JULY 12, 2024

Data can be your organization’s most valuable asset, but only if it’s data you can trust. When companies work with data that is untrustworthy for any reason, it can result in incorrect insights, skewed analysis, and reckless recommendations to become data integrity vs data quality. Two terms can be used to describe the condition of data: data integrity and data quality.

Data Integration 105

Data Integration Datasets Data Data Governance 105

ArcGIS

JULY 12, 2024

You can host scene layers and 3D tiles layers in ArcGIS Online or reference datasets in cloud storage in ArcGIS Enterprise.

Cloud Storage 105

Cloud Storage Datasets Cloud Data Management 105

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Cloudera

JULY 10, 2024

There’s nothing worse than wasting money on unnecessary costs. In on-premises data estates, these costs appear as wasted person-hours waiting for inefficient analytics to complete, or troubleshooting jobs that have failed to execute as expected, or at all. They manifest as idle hardware waiting for urgent workloads to come in, ensuring sufficient spare capacity to run them amidst noisy neighbors and resource-hungry, lower-priority workloads.

Metadata 105

Metadata Cloud Management IT 105

KDnuggets

JULY 11, 2024

Here’s how to ace your data analyst interview and land your first job.

Data 135

Data 135

Snowflake

JULY 8, 2024

Regulated and sovereign markets across the world have stringent requirements stipulating certain important data be kept within geographical borders or even for certain workloads to have dedicated environments, separate from those of other customers. In these markets, organizations need a secure and well-governed data foundation with effective controls to help comply with regulatory requirements.

Cloud 102

Cloud Google Cloud Education AWS 102

Engineering at Meta

JULY 10, 2024

Tail utilization is a significant system issue and a major factor in overload-related failures and low compute utilization. The tail utilization optimizations at Meta have had a profound impact on model serving capacity footprint and reliability. Failure rates, which are mostly timeout errors, were reduced by two-thirds; the compute footprint delivered 35% more work for the same amount of resources; and p99 latency was cut in half.

Utilities 100

Utilities Machine Learning Architecture Systems 100

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Data Engineering Weekly

JULY 7, 2024

Experience Enterprise-Grade Apache Airflow Astro augments Airflow with enterprise-grade features to enhance productivity, meet scalability and availability demands across your data pipelines, and more. Learn More → Notion: Building and scaling Notion’s data lake Notion writes about scaling the data lake by bringing critical data ingestion operations in-house.

Data Engineer 98

Data Engineer Data Engineering Engineering Data Lake 98

KDnuggets

JULY 8, 2024

Learn all about introductory statistics with this collection of tutorials from our sister site Statology.

Data Science 135

Data Science Data 135

Scott Logic

JULY 9, 2024

It seems barely a month goes by without a new supply chain attack making the headlines, and malicious code in dependency packages from package registries such as NPM is a common method. My usual sentiments include “oh another one, what a surprise” , before thoughts eventually turn to - someone really ought to be doing something about this. Fortunately, it turns out that quite a few things are indeed being done - there’s progress, activity, and promising ideas for the future.

Manufacturing 96

Manufacturing Food Coding Project 96

Ripple Engineering

JULY 9, 2024

Introduction: Embracing the Future with Ripple's Data Platform Migration Welcome to a pivotal moment in Ripple's data journey. As leaders at the intersection of blockchain technology and financial services, we're excited to share a transformative step in our data management evolution. We recently embarked on a significant data platform migration, transitioning from Hadoop to Databricks, a move motivated by our relentless pursuit of excellence and our contributions to the XRP Ledge

Hadoop 96

Hadoop Data Lake Machine Learning Raw Data 96

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Snowflake

JULY 9, 2024

There is a scene in Mission: Impossible – Rogue Nation where Tom Cruise is hanging onto the outside of a jet as it has taken off. And while, yes, he’s going with it, he’s not really on board or in control. Some data executives feel like that. It’s not enough to establish goals — or, the destination in this metaphor. The data strategy must provide a flight plan for making sure you get there — on time, on budget and, of course, safely on board.

Government 87

Government Education Data Governance Unstructured Data 87

KDnuggets

JULY 9, 2024

Learn how to streamline data and model orchestration for Generative AI success. Explore practical use cases and a comprehensive guide in this blog.

Data 135

Data 135

Confluent

JULY 11, 2024

confluent-kafka-javascript offers two APIs for calling Kafka from JS, one KafkaJS-like & one node-rdkafka-like. Learn it by making an app w/producer, consumer & UI.

Kafka 78

Kafka Building IT 78

Uber Engineering

JULY 11, 2024

Modernizing the fundamentals of log management at Uber: How we used CLP to build a new logging infra that lets users view and analyze their logs seamlessly, at scale!

Building 73

Building Management Engineering 73

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Snowflake

JULY 11, 2024

Leaders in the advertising, media and entertainment industries know all too well the importance of the media supply chain. It’s the backbone that keeps things running smoothly, including everything from content creation and management to content distribution and analytics. But media supply chains are becoming more complex to manage for several reasons.

Media 76

Media Entertainment Cloud Metadata 76

KDnuggets

JULY 8, 2024

Docker tags are important for managing and versioning Docker images. This tutorial will teach you how to use Docker tags effectively.

Management 133

Management Data Engineer Data Engineering Engineering 133

Confluent

JULY 10, 2024

Data streaming with Confluent enables the integration and processing of post-trade data in real time, allowing for compliance with MiFID II. Learn how.

Data 75

Data Process 75

Monte Carlo

JULY 9, 2024

The DuckDB hype is real — this in-process analytical database has skyrocketed in popularity over the last few years. Known for its columnar storage, vectorized query execution, and scale-up approach to SQL analytics, DuckDB fans proclaim it’s faster, more efficient, and more affordable than other databases. DuckDB is also becoming a must-have layer in many AI stacks.

Data Warehouse 69

Data Warehouse Data Pipeline SQL Database 69

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud