Confessions of a Data Guy

JANUARY 7, 2025

Building fun things is a real part of Data Engineering. Using your creative side when building a Lake House is possible, and using tools that are outside the normal box can sometimes be preferable. Checkout this video where I dive into how I build just such a Lake House using Modern Data Stack tools like […] The post Building a Fast, Light, and CHEAP Lake House with DuckDB, Delta Lake, and AWS Lambda appeared first on Confessions of a Data Guy.

AWS 130

AWS Building Data Engineering Data Engineer 130

ArcGIS

JANUARY 7, 2025

Must read article that introduces the SAR analysis toolset in ArcGIS Pro, which helps users extract valuable insights from processed SAR data.

Process 92

Process Education Data 92

KDnuggets

JANUARY 7, 2025

Do you want to learn data wrangling with Python on a budget? No worries, there are (at least) five free courses thatll provide you with solid knowledge.

Python 80

Python Data 80

databricks

JANUARY 7, 2025

At Databricks, our automation vision is to automate all aspects of the business, making it better, faster, and cheaper. For the sales teams.

IT 78

IT Machine Learning 78

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Snowflake

JANUARY 7, 2025

Marketers at financial services companies have their work cut out for them. Their companies have a wealth of data, but that data is often fragmented among different systems and divisions, and protected-class data has a wide range of restrictions on how it can be used for different product lines. Some of the most effective companies in the financial sector are preparing their strategy for long-term success by centralizing first-party data in the Snowflake AI Data Cloud for Financial Services.

Banking 76

Banking Media Government Cloud 76

Cloudera

JANUARY 7, 2025

In todays dynamic digital landscape, multi-cloud strategies have become vital for organizations aiming to leverage the best of both cloud and on-premises environments. As enterprises navigate complex data-driven transformations, hybrid and multi-cloud models offer unmatched flexibility and resilience. Heres a deep dive into why and how enterprises master multi-cloud deployments to enhance their data and AI initiatives.

Cloud 64

Cloud Government AWS Unstructured Data 64

Towards Data Science

JANUARY 7, 2025

How CDC tools use MySQL Binlog and PostgreSQL WAL with logical decoding for real-time data streaming Photo by Matoo.Studio on Unsplash CDC (Change Data Capture) is a term that has been gaining significant attention over the past few years. You might already be familiar with it (if not, dont worrytheres a quick introduction below ). One question that puzzled me, though, was how tools like the Debezium CDC connectors can read changes from MySQL and PostgreSQL databases.

PostgreSQL 57

PostgreSQL MySQL Data Lake Bytes 57

databricks

JANUARY 7, 2025

Aon plc is a leading global firm providing risk, reinsurance, retirement, and health solutions. Focusing on data-driven insights, Aon operates in over 120.

Data 58

Data 58

Cloudyard

JANUARY 7, 2025

Read Time: 2 Minute, 0 Second Streamlit, coupled with Snowpark, provides an excellent platform for creating interactive and user-friendly data applications. This blog highlights the significance of “Multi-Select Filtering with Streamlit and Snowflake” , demonstrating how multi-select filters can simplify data exploration by allowing users to apply multiple filters dynamically.

Electronics 52

Electronics Retail Coding Project 52

databricks

JANUARY 7, 2025

We are excited to announce that egress control for Databricks serverless and Mosaic AI Model Serving workloads is available in Public Preview on.

57

57

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Monte Carlo

JANUARY 7, 2025

Hex is one of the fastest growing collaborative data exploration solutions for a reason, it allows both technical and non-technical users to explore data, derive insights, and build powerful data applications for further visualization and exploration. A sample Project from the demo on the Hex homepage. Hex is a powerful way to bring analytics, data science, and business users together to answer tough business questionsbut only if the data populating the project can be trusted.

Project 52

Project High Quality Data BI Data Warehouse 52

Towards Data Science

JANUARY 7, 2025

Building idempotent and re-playable data pipelines Continue reading on Towards Data Science

Data Pipeline 54

Data Pipeline Data Science Data Building 54

Cloudera

JANUARY 7, 2025

Airports are an interconnected system where one unforeseen event can tip the scale into chaos. For a smaller airport in Canada, data has grown to be its North Star in an industry full of surprises. In order for data to bring true value to operationsand ultimately customer experiencesthose data insights must be grounded in trust. Ryan Garnett, Senior Manager Business Solutions of Halifax International Airport Authority, joined The AI Forecast to share how the airport revamped its approach to data

Finance 61

Finance Building Cloud Engineering 61