Sat.Jul 27, 2024 - Fri.Aug 02, 2024

article thumbnail

My Obsidian Note-Taking Workflow

Simon Späti

A Vim-Inspired Approach to Efficient Note Management with Obsidian and Markdown

article thumbnail

Introducing Apache Kafka® 3.8

Confluent

Apache Kafka 3.8 adds 17 new KIPs (13 for Core, 3 for Streams & 1 for Connect). Highlights include 2 new Docker images, the ability to set task assignors, and more!

Kafka 131
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Ingest data from SQL Server, Salesforce, and Workday with LakeFlow Connect

databricks

We’re excited to announce the Public Preview of LakeFlow Connect for SQL Server, Salesforce, and Workday. These ingestion connectors enable simple and efficient.

SQL 130
article thumbnail

How To Run A Data Team As A New Head Of Data

Seattle Data Guy

What would you do if you became the head or director of data for a 1,000-person company? Yesterday, you were plugging along as an analyst, and now, suddenly, you have all these new responsibilities. Figuring out where to start is part of the job. You’d probably feel a strong temptation to freak out. Who wouldn’t?… Read more The post How To Run A Data Team As A New Head Of Data appeared first on Seattle Data Guy.

Data 130
article thumbnail

Apache Airflow® Best Practices for ETL and ELT Pipelines

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

article thumbnail

Data+AI Summit 2024 - Retrospective - Apache Spark

Waitingforcode

Welcome to the second blog post dedicated to the previous Data+AI Summit. This time I'm going to share with you a summary of Apache Spark talks.

Data 130
article thumbnail

5 Tips for Improving SQL Query Performance

KDnuggets

If you work in data, you’ll write SQL queries all the time. So how do you write efficient SQL queries that are optimized for performance? This tutorial will help you with just that.

SQL 129

More Trending

article thumbnail

Announcing General Availability of Lakehouse Federation

databricks

Today, we are excited to announce that Lakehouse Federation in Unity Catalog is now Generally Available (GA) across AWS, Azure, and GCP! Lakehouse.

AWS 120
article thumbnail

How to make a “peeled edge” area of interest effect in ArcGIS Pro

ArcGIS

Catch eyes and imaginations with this fun technique that draws attention to your area of interest with a bit of style!

113
113
article thumbnail

Building Data Science Pipelines Using Pandas

KDnuggets

Learn to build the end-to-end data science pipelines from data ingestion to data visualization using Pandas pipe method.

article thumbnail

Data Engineering Weekly #182

Data Engineering Weekly

Meta: Introducing Llama 3.1: Our most capable models to date Probability one of the hottest announcements this week is Llama 3.1 release - the first-ever open-sourced frontier AI model competitive with leading foundation models across a range of tasks, including GPT-4, GPT-4o, and Claude 3.5 Sonnet. The Llama3 herd of models is an insightful paper that helps one deeply understand the foundational model.

article thumbnail

Apache Airflow®: The Ultimate Guide to DAG Writing

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

article thumbnail

Accelerate Feature Engineering With Photon

databricks

Training a high-quality machine learning model requires careful data and feature preparation. To fully utilize raw data stored as tables in Databricks, running.

article thumbnail

ArcGIS Solutions introduces Essential Data Models to Utility Network Foundation solutions

ArcGIS

Essential Data Models in the Utility Network Foundations

Utilities 108
article thumbnail

7 Steps to Master the Art of Data Storytelling

KDnuggets

Follow this 7 step recipe to mastering effective insight and information dissemination through compelling data story crafting.

Data 122
article thumbnail

Securely Deploy Custom Apps and Models with Snowpark Container Services, Now Generally Available

Snowflake

Since introducing Snowpark Container Services, we’ve seen overwhelming adoption across industries from customers and partners, including Landing.AI , Relational.AI , H20.AI , SailPoint , AIR MILES , Spark NZ , and Eutelsat OneWeb. These organizations and many more are using Snowpark Container Services capabilities to easily and securely deploy everything from custom front-ends and large-scale ML training and inference to open source and homegrown models, all securely within Snowflake.

article thumbnail

Optimizing The Modern Developer Experience with Coder

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

article thumbnail

Responsible AI with the Databricks Data Intelligence Platform

databricks

The transformative potential of artificial intelligence (AI) is undeniable. From productivity efficiency, to cost savings, and improved decision-making across all industries, AI is.

Data 109
article thumbnail

Beyond Web Mercator: Projected Basemaps Revisited

ArcGIS

More small-scale projected basemaps to add to the set I built in 2023

Project 104
article thumbnail

How to Perform Memory-Efficient Operations on Large Datasets with Pandas

KDnuggets

Let's learn how to perform memory-efficient operations in pandas with large dataset.

Datasets 116
article thumbnail

Snowflake Invests in Contextual AI to Make It Easier for Enterprises to Deploy RAG Applications in the AI Data Cloud

Snowflake

Retrieval Augmented Generation (RAG) allows enterprises to ground responses from Large Language Models in their specific organization’s data. This helps ensure that AI-powered applications provide responses that are not only accurate, relevant, and consistent, but also aligned with business needs. At Snowflake, we make it simple for our customers to implement RAG, while also enabling the strict governance and privacy controls that businesses require.

Cloud 103
article thumbnail

15 Modern Use Cases for Enterprise Business Intelligence

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

article thumbnail

Lakehouse Monitoring GA: Profiling, Diagnosing, and Enforcing Data Quality with Intelligence

databricks

At Data and AI Summit, we announced the general availability of Databricks Lakehouse Monitoring. Our unified approach to monitoring data and AI.

Data 101
article thumbnail

Snowflake is Dying??!! Data Breach!!

Confessions of a Data Guy

The post Snowflake is Dying??!! Data Breach!! appeared first on Confessions of a Data Guy.

Data 100
article thumbnail

Organize, Search, and Back Up Files with Python’s Pathlib

KDnuggets

This tutorial will teach you how to simplifying your file management tasks, from organization to backup, using Python’s pathlib module.

article thumbnail

Accelerating Academic Medical Research with an AI-Driven Data Strategy

Snowflake

Academic medical centers (AMCs) are a critical keystone of healthcare systems worldwide. They serve as major hubs of medical research, pioneering new treatments that advance and set the standard of care throughout medicine. They also educate and train the next generation of healthcare professionals, ensuring that the medical field continues to advance.

Medical 87
article thumbnail

Prepare Now: 2025s Must-Know Trends For Product And Data Leaders

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

article thumbnail

OKR-Centric Delivery Models for Engineering-Focused Enterprises

databricks

Introduction An organization adopting new technologies or on a modernization journey typically focuses on upcoming tools, their features and potential performance/cost improvements under.

article thumbnail

Daft: Distributed Dataframes with Python.

Confessions of a Data Guy

The post Daft: Distributed Dataframes with Python. appeared first on Confessions of a Data Guy.

Python 100
article thumbnail

5 Free Online Courses to Learn Data Engineering Fundamentals

KDnuggets

Kickstart a new career in one of the most popular tech careers where you can earn a 6 figure salary.

article thumbnail

How BT Group Built a Smart Event Mesh with Confluent

Confluent

BT Group's Smart Event Mesh - centralized event streaming with decentralized customer experience, automation, and a foundation—all built on Confluent.

85
article thumbnail

How to Drive Cost Savings, Efficiency Gains, and Sustainability Wins with MES

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

article thumbnail

An Overview of Cloudera’s AI Survey: The State of Enterprise AI and Modern Data Architecture

Cloudera

Enterprise IT leaders across industries are tasked with preparing their organizations for the technologies of the future – which is no simple task. With the use of AI exploding, Cloudera, in partnership with Researchscape, surveyed 600 IT leaders who work at companies with over 1,000 employees in the U.S., EMEA and APAC regions. The survey, ‘ The State of Enterprise AI and Modern Data Architecture ’ uncovered the challenges and barriers that exist with AI adoption, current enterprise AI deployme

article thumbnail

CI/CD for Data Engineers.

Confessions of a Data Guy

The post CI/CD for Data Engineers. appeared first on Confessions of a Data Guy.

article thumbnail

How to Use MultiIndex for Hierarchical Data Organization in Pandas

KDnuggets

Let's learn how to use multiindex pandas for hierarchical data operations.

Data 95
article thumbnail

Flink AI: Real-Time ML and GenAI Enrichment of Streaming Data with Flink SQL on Confluent Cloud

Confluent

Learn how to use Flink SQL on Confluent Cloud to invoke ML and GenAI endpoints to enrich streaming data

SQL 83
article thumbnail

The Cloud Development Environment Adoption Report

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.