The Pragmatic Engineer

SEPTEMBER 19, 2024

This is a follow-up to the article Paying down tech debt , written by industry veteran Lou Franco. Lou has been in the software business for over 30 years as an engineer, EM, and executive. He’s also worked at four startups and the companies that later acquired them; most recently Atlassian as a Principal Engineer on the Trello iOS app. Later this year, he’s publishing a book on tech debt.

Recruitment 297

Recruitment Java Coding Project 297

KDnuggets

SEPTEMBER 19, 2024

In Python, functions often require multiple arguments, and you may find yourself repeatedly passing the same values for certain parameters. This is where partial functions can help. Python’s built-in functools module allows you to create partial functions.

Python 144

Python 144

databricks

SEPTEMBER 19, 2024

Mosaic AI Model Training now supports fine-tuning up to 131K context length for Llama 3.1 models. More efficient training at long sequence lengths is made possible by several optimizations highlighted in this post.

138

138

KDnuggets

SEPTEMBER 19, 2024

Image generated with FLUX.1 [dev] and edited with Canva Pro The 10 GitHub Repository Education Series has been a hit among readers, so here is another list to help you master the basics of deep learning. This collection will guide you through understanding popular deep learning frameworks and various model architectures. In short, you.

Deep Learning 134

Deep Learning Education Architecture 134

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline

databricks

SEPTEMBER 19, 2024

Today, we're excited to introduce Databricks Assistant Quick Fix , a powerful new feature designed to automatically correct common, single-line errors such as.

Designing 122

Designing 122

KDnuggets

SEPTEMBER 19, 2024

ggplot2 is a tool in R for making charts. You can create charts with dots, bars, or lines. You can also add layers to show more details. This article will help you learn how to use ggplot2 to create visualizations. Getting started with ggplot2 Before using ggplot2, you need to install it and.

Data 123

Data IT 123

Confessions of a Data Guy

SEPTEMBER 19, 2024

It is a Brave New World out there these days. The new tools and features come out faster than your mom on Sunday morning getting you ready for church. The same goes for the context and advice being produced on a myriad of platforms, the ole’ Like and Subscribe, and all that bit. It does […] The post Data Modeling in the Brave New Lakehouse World appeared first on Confessions of a Data Guy.

Data 113

Data IT Data Engineering Data Engineer 113

Snowflake

SEPTEMBER 19, 2024

Data is increasingly becoming critical for the public sector — from guiding decisions in higher education to enhancing citizen services and streamlining government operations. Government agencies are overwhelmed with data, whether it be structured, like incident logs, or unstructured, like satellite images. Harnessing the vast amount of data can become a burden for any organization, yet the insights have the potential to significantly improve quality of life and strengthen national security.

Government 101

Government Education Data Analytics Engineering 101

Precisely

SEPTEMBER 19, 2024

Key Takeaways: Data integrity is essential for AI success and reliability – helping you prevent harmful biases and inaccuracies in AI models. Robust data governance for AI ensures data privacy, compliance, and ethical AI use. Proactive data quality measures are critical, especially in AI applications. Using AI systems to analyze and improve data quality both benefits and contributes to the generation of high-quality data.

Data Governance 78

Data Governance Government High Quality Data Datasets 78

Snowflake

SEPTEMBER 19, 2024

Building on the momentum from Snowflake Summit , where Snowflake announced the rollout of dozens of new features, this year’s Industry Day showcased the numerous ways these capabilities can be put to use, particularly in an AI- and ML-driven world. In his keynote address , Snowflake CEO Sridhar Ramaswamy explained how the AI Data Cloud aligns with customers’ AI and data strategies, highlighting the platform’s unique position to achieve enterprise AI goals.

Telecommunication 98

Telecommunication Hospitality Entertainment Retail 98

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

Pinterest Engineering

SEPTEMBER 19, 2024

Li Tang; Sr. Software Engineer | Saurabh Vishwas Joshi; Sr. Staff Software Engineer | Zhiyuan Zhang; Sr. Manager, Engineering | At Pinterest, we operate a large-scale online machine learning inference system, where feature caching plays a critical role to achieve optimal efficiency. In this blog post, we will discuss our decision to adopt Cachelib project by Meta Open Source (“Cachelib”) and how we have built a high-throughput, flexible feature cache by leveraging and expanding upon the capabili

Systems 70

Systems Utilities Architecture Software Engineering 70

ArcGIS

SEPTEMBER 19, 2024

Guide to TEMPO NO2 multidimensional analysis in ArcGIS Pro 3.3. Track air quality changes and gain insights for public health and environmental monitoring.

Data 62

Data 62

Striim

SEPTEMBER 19, 2024

What makes MotherDuck and DuckDB a game-changer for data analytics? Join us as we sit down with Jacob Matson, a renowned expert in SQL Server, dbt, and Excel, who recently became a developer advocate at MotherDuck. During this episode, Jacob shares his compelling journey to MotherDuck, driven by his frequent use of DuckDB for solving data challenges.

BI 52

BI Big Data SQL Utilities 52

Knowledge Hut

SEPTEMBER 19, 2024

Are you working on many big projects with lots of smaller tasks that all depend on each other? If you become a PgMP-certified pro, you can make things run smoothly. You can learn more about this by taking a PgMP certification course. Getting PgMP certified doesn't just help your career—it also teaches you important things about how organizations work.

Certification 52

Certification Education Programming Project 52

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems