KDnuggets

JULY 8, 2024

Text preprocessing is an important step in NLP. Let's learn how to use the Hugging Face Tokenizers Library to preprocess text data.

Data 143

Data 143

databricks

JULY 8, 2024

We are proud to announce two new analyst reports recognizing Databricks in the data engineering and data streaming space: IDC MarketScape: Worldwide Analytic.

Data Pipeline 137

Data Pipeline Process Cloud Data Engineer 137

KDnuggets

JULY 8, 2024

Learn all about introductory statistics with this collection of tutorials from our sister site Statology.

Data Science 135

Data Science Data 135

databricks

JULY 8, 2024

The Prodvana team joins Databricks to support new innovations in the Data Intelligence Platform infrastructure. Learn more about the vision and what's ahead.

Data 134

Data 134

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

KDnuggets

JULY 8, 2024

Docker tags are important for managing and versioning Docker images. This tutorial will teach you how to use Docker tags effectively.

Management 133

Management Data Engineer Data Engineering Engineering 133

Snowflake

JULY 8, 2024

Regulated and sovereign markets across the world have stringent requirements stipulating certain important data be kept within geographical borders or even for certain workloads to have dedicated environments, separate from those of other customers. In these markets, organizations need a secure and well-governed data foundation with effective controls to help comply with regulatory requirements.

Cloud 102

Cloud Google Cloud Education AWS 102

Precisely

JULY 8, 2024

Key Takeaways: Harness the power of location intelligence to drive smarter, data-driven decisions that turn spatial data into a strategic asset. Top location intelligence use cases include efficient territory planning and network optimization, which help maximize productivity and customer satisfaction. Successful location intelligence initiatives require a foundation of high-quality address data, enrichment data, and spatial analytics.

Telecommunication 64

Telecommunication Insurance Retail Banking 64

Scott Logic

JULY 8, 2024

In this episode, I’m joined by Doro Hinrichs and Kira Clark from Scott Logic and Peter Gostev, Head of AI at Moonpig. Together, we explore whether we can ever really trust and secure Generative AI (GenAI), while sharing stories from the front line about getting to grips with this rapidly evolving technology. With its human-like, non-deterministic nature, GenAI frustrates traditional pass/fail approaches to software testing.

Designing 59

Designing Technology Project IT 59

Monte Carlo

JULY 8, 2024

Note: This was originally published on the Mission Lane Tech Blog and has been republished below with permission. Table of Contents Introduction Why we started: “Customer obsessed” coupled with cost savings A quick introduction to a traditional compliance testing approach How manual testing works in practice and its drawbacks High Level of Effort Lack of Scalability Testing is not “always on” Less Auditable The future is now!

Data Warehouse 52

Data Warehouse SQL Coding Architecture 52

DataKitchen

JULY 8, 2024

Christopher Bergh, CEO of DataKitchen, is transforming data analytics with his DataOps approach. By applying principles from agile and lean manufacturing, Bergh aims to eliminate the 70-80% waste in data processes. DataKitchen's suite of open-source tools offers solutions for observability, testing, and automation, addresses challenges in rapid change management, error detection team productivity.

Manufacturing 52

Manufacturing Data Data Analytics Data Process 52

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Hevo

JULY 8, 2024

Managing infrastructure manually across multiple cloud providers leads to consistency, deployment delays, and difficulty in scaling. You need a solution that automates infrastructure provisioning, ensures consistency, and supports rapid deployment across diverse environments, from development to production, while maintaining security and compliance standards.

Cloud 52

Cloud Management Data 52

Towards Data Science

JULY 8, 2024

Leveraging TensorFlow Transform for scaling data pipelines for production environments Photo by Suzanne D. Williams on Unsplash Data pre-processing is one of the major steps in any Machine Learning pipeline. Tensorflow Transform helps us achieve it in a distributed environment over a huge dataset. Before going further into Data Transformation, Data Validation is the first step of the production pipeline process, which has been covered in my article Validating Data in a Production Pipeline: The

Data Preparation 45

Data Preparation Datasets Metadata Data Ingestion 45

Monte Carlo

JULY 8, 2024

Unlike traditional data quality solutions, Monte Carlo was originally designed to reduce data downtime across modern data platforms such as Snowflake, Databricks, Redshift, BigQuery, Azure Synapse and more. As we worked with data teams, we ran into a diverse set of data platforms teams used to power their data products including: Postgres Teradata MySQL Oracle SAP HANA SQL Server Last year we launched custom monitors , or data tests, for these environments to help identify bad data as early in t

SQL 40

SQL MySQL Relational Database Database 40

Hevo

JULY 8, 2024

In the fast-paced world of data management, choosing the right tool to address your requirements has never been more critical than now. Therefore, understanding each option is central to influencing your decisions. This guide compares Hevo Data vs Fivetran, two leading ELT (Extract, Load, Transform) tools in the market.

Data Management 40

Data Management Management Data Data Pipeline 40

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud