Snowflake

DECEMBER 4, 2024

Snowflake leaders offer insight on AI, open source and cybersecurity development — and the fundamental leadership skills required — in the years ahead. As we come to the end of a calendar year, it’s natural to contemplate what the new year will hold for us. It’s an understatement to say that the future is very hard to predict, but it’s possible to both prepare for the likeliest outcomes and stay ready to adapt to the unexpected.

Unstructured Data 97

Unstructured Data Data Lake Deep Learning Structured Data 97

Cloudera

DECEMBER 4, 2024

For more than a decade, Cloudera has been an ardent supporter and committee member of Apache NiFi, long recognizing its power and versatility for data ingestion, transformation, and delivery. Our customers rely on NiFi as well as the associated sub-projects (Apache MiNiFi and Registry) to connect to structured, unstructured, and multi-modal data from a variety of data sources – from edge devices to SaaS tools to server logs and change data capture streams.

Data Pipeline 93

Data Pipeline Data Ingestion Data Preparation Architecture 93

Ascend.io

DECEMBER 4, 2024

As we approach 2025, data teams find themselves at a pivotal juncture. The rapid evolution of technology and the increasing demand for data-driven insights have placed immense pressure on these teams. According to recent research, 95% of data teams are operating at or over capacity, highlighting the urgent need for strategic preparation. This isn’t just about keeping up; it’s about staying ahead so that data teams can deliver the data needed to fuel their organizations.

Data Pipeline 52

Data Pipeline Metadata Data Workflow Data 52

Cloudera

DECEMBER 4, 2024

Welcome to the first installment of a series of posts discussing the recently announced Cloudera AI Inference service. Today, Artificial Intelligence (AI) and Machine Learning (ML) are more crucial than ever for organizations to turn data into a competitive advantage. To unlock the full potential of AI, however, businesses need to deploy models and AI applications at scale, in real-time, and with low latency and high throughput.

Architecture 73

Architecture Machine Learning BI Deep Learning 73

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline

KDnuggets

DECEMBER 4, 2024

Learn to build, run, and manage data engineering pipelines both locally and in the cloud using popular tools.

Data Engineer 151

Data Engineer Data Engineering Engineering Project 151

Ascend.io

DECEMBER 4, 2024

As we approach 2025, data teams find themselves at a pivotal juncture. The rapid evolution of technology and the increasing demand for data-driven insights have placed immense pressure on these teams. According to recent research, 95% of data teams are operating at or over capacity, highlighting the urgent need for strategic preparation. This isn’t just about keeping up; it’s about staying ahead so that data teams can deliver the data needed to fuel their organizations.

Data Pipeline 52

Data Pipeline Metadata Data Workflow Data 52

KDnuggets

DECEMBER 4, 2024

MongoDB is a database that’s great for handling large amounts of diverse data. This article walks you through installing MongoDB and using the MongoDB Shell to manage your data easily.

MongoDB 133

MongoDB Database Management Data 133

Simon Späti

DECEMBER 4, 2024

Modern data teams face growing complexity: multiple data sources, various BI tools, and ever-increasing self-service analytics. Organizations must maintain consistency across their metrics while making data accessible to business users. This is where a semantic layer shines: It provides a single source of truth for business metrics while abstracting away the underlying complexity.

BI 130

BI Architecture Accessibility Accessible 130

KDnuggets

DECEMBER 4, 2024

Here are five free resources in diverse formats and difficulty levels to acquaint with deep learning models at no cost.

Deep Learning 126

Deep Learning 126

RandomTrees

DECEMBER 4, 2024

As one of the most important sectors of the global economy, the food and beverage (F&B) industry works in highly volatile conditions and ensures its success by reducing waste and managing inventories. Managing production and consumption, meeting deadlines, cutting waste, and being environmentally friendly are always a challenge. Old and traditional approaches often fail or become inefficient and unresponsive in real time.

Food 52

Food Management Manufacturing Machine Learning 52

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

databricks

DECEMBER 4, 2024

In recent years, artificial intelligence has transformed from an aspirational technology to a driver of manufacturing innovation and efficiency. Understanding both the current.

Manufacturing 105

Manufacturing Technology Data 105

ArcGIS

DECEMBER 4, 2024

Sample tool to add attachment date taken to an output table. Date taken data can be used in pop-up windows of an active map.

Data 98

Data Data Management Management 98

databricks

DECEMBER 4, 2024

Established in 2020, EVPassport aims to transform the electric vehicle charging experience. Specializing in multi-family residences, hospitality, retail, workplaces, and commercial parking environments.

Hospitality 95

Hospitality Retail 95

Towards Data Science

DECEMBER 4, 2024

Options, options…In this article, you’ll learn what the possibilities are for bringing your on-prem SQL Server data to Microsoft Fabric Continue reading on Towards Data Science »

SQL 75

SQL Data Science Data Data Engineering 75

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

Confluent

DECEMBER 4, 2024

Tableflow easily integrates with Amazon SageMaker Lakehouse, enabling you to quickly materialize your Apache Kafka topics into Iceberg tables stored in S3.

Kafka 64

Kafka Data Process 64

Elder Research

DECEMBER 4, 2024

Take a break from the usual routine and join thousands of data professionals for Advent of Code. It's a great way to sharpen your skills!

Coding 59

Coding Data IT 59

Confluent

DECEMBER 4, 2024

The recent release of mutual TLS (mTLS) on Confluent Cloud and Amazon Redshift has enabled the streaming of Confluent topics to Amazon Redshift materialized views.

Cloud 59

Cloud 59

Uber Engineering

DECEMBER 4, 2024

LLM iteration can happen with speed and safety! Explore how Uber launched a prompt toolkit for LLMs that helps engineers create, manage, and evaluate prompts with dynamic contextualization, batch generation, and robust safety checks.

Engineering 45

Engineering Management Data 45

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

Cloudera

DECEMBER 4, 2024

Today enterprises can leverage the combination of Cloudera and Snowflake—two best-of-breed tools for ingestion, processing and consumption of data—for a single source of truth across all data, analytics, and AI workloads. But now AWS customers will gain more flexibility, data utility, and complexity, supporting the modern data architecture. All this by making it easier for customers to connect their workloads with Snowflake, Cloudera, and unique AWS services such as Amazon Simple Storage Service

AWS 89

AWS Raw Data Relational Database Government 89

Precisely

DECEMBER 4, 2024

At Precisely, we celebrate the women in our organization because we know that while more women are joining the technology industry, there’s still a gender gap. Supporting and advocating for women in technology is a top priority, which is why the Precisely Women in Technology (PWIT) program was established. Every month, a different woman from the program is featured in this Q&A to share her experience working in tech.

Professional Services 59

Professional Services Technology Education Programming 59

Edureka

DECEMBER 4, 2024

It was a regular workday. IT worker Divya was halfway through her coffee when a colleague sent her an important email. The subject line read, WARNING: Critical Virus Detected! According to the email, if she didn’t remove a particular file right away, a new virus would cause her operating system to fail. Divya got scared and immediately tried to safeguard her system.

Media 52

Media Certification Systems Designing 52