KDnuggets

MAY 2, 2024

Get up and running with Docker with this tutorial on containerizing Python applications.

Python 141

Python 141

Christophe Blefari

MAY 2, 2024

My personal collection of the best resources to bootstrap a data team and get inspired from what others are doing.

Building 130

Building Data 130

Snowflake

MAY 2, 2024

To accurately answer business questions using LLMs, companies must augment models with their data. Retrieval Augmented Generation (RAG) is a popular solution to this problem, as it integrates the organization’s factual, real-time data into the prompt for the LLM. While the adoption of RAG has increased, an open question remains: How do enterprises know how effective their system is?

Cloud 128

Cloud Systems Building IT 128

KDnuggets

MAY 2, 2024

Exploring the Test-Driven Development Paradigm in Python

Python 121

Python 121

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline

Snowflake

MAY 2, 2024

Snowflake Marketplace is designed to give customers and organizations a place to easily find, try and buy data, apps and AI products that help solve their most pressing business problems. We have more than 540 providers, offering over 2,400 live, ready-to-use data products (as of Jan 31, 2024), so there are many options to help you enrich your own data resources, build new data apps and leverage the power of AI on Snowflake.

Electronics 111

Electronics Media Accessible Accessibility 111

databricks

MAY 2, 2024

Unlock the power of advanced sports analytics with Databricks Marketplace and Delta Sharing. Discover how these platforms are transforming the sports industry by enabling seamless data access, collaboration, and real-time insights. Leverage a diverse array of data assets to optimize performance, enhance fan engagement, and gain a competitive edge. Explore the future of sports analytics, powered by Databricks.

Data 111

Data Accessible Accessibility Entertainment 111

Knowledge Hut

MAY 2, 2024

Finding the right automation testing tool for your project can be daunting. With so many choices available, knowing which one will best suit your needs and help you achieve desired results can be difficult. This blog post looks at two of the most common tools used in software development––UFT/QTP and Selenium––and discusses some of the key differences between them that you should consider when choosing an automation tool for your projects.

Programming Language 98

Programming Language Java Coding Skills Programming 98

Towards Data Science

MAY 2, 2024

Four advanced tricks to give your data science and machine learning classes the edge you never knew they needed Continue reading on Towards Data Science »

Python 98

Python Data Science Machine Learning Data 98

Knowledge Hut

MAY 2, 2024

If you are a machine learning enthusiast and stay in touch with the latest developments, you would have definitely come across the news “Machine learning identifies links between the world's oceans” Wait, we all know how complex it would be to analyse a concept such as oceans and their behaviour which would undoubtedly involve billions of data points associated with many critical parameters such as wind velocities, temperatures, earth’s rotation and many such.

Machine Learning 98

Machine Learning IT Deep Learning Algorithm 98

Uber Engineering

MAY 2, 2024

Learn how Uber improved mobile testing reliability, and increased productivity for thousands of engineers, using machine learning to create DragonCrawl, a highly stable and low-maintenance testing system.

Machine Learning 85

Machine Learning Engineering Systems 85

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

Knowledge Hut

MAY 2, 2024

Apache Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python, and R and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools, including Spark SQL for SQL and structured data processing, MLlib for machine learning, GraphX for graph processing, and Spark Streaming.

Java 98

Java Hadoop Scala SQL 98

Uber Engineering

MAY 2, 2024

Learn how Uber serves over 40 million reads per second from its in-house, distributed database built on top of MySQL using an integrated caching solution: CacheFront.

MySQL 81

MySQL Database IT 81

Knowledge Hut

MAY 2, 2024

Automation and machine learning have changed our lives. From the most technologically savvy person working in leading digital platform companies like Google or Facebook to someone who is just a smartphone user, there are very few who have not been impacted by artificial intelligence or machine learning in some form or the other; through social media, smart banking, healthcare or even Uber.

Machine Learning 98

Machine Learning Algorithm Datasets Python 98

Uber Engineering

MAY 2, 2024

Accelerating Tomorrow: How Uber Turbocharges AI/ML Frontiers.

81

81

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

Knowledge Hut

MAY 2, 2024

The work scenario today is stretching workplace flexibilities to accommodate the needs of professionals. Globally stationed offices have also made extending flexible workplaces a norm. Working remotely is the new trend that is transcending industries. While working remotely comes with its own set of benefits, it isn’t well-suited for some industries or professions.

IT 98

IT Management Project Building 98

Uber Engineering

MAY 2, 2024

Migrating money data with peace of mind. Learn how Uber moved its Money related data spanning trillion of rows & petabytes in size flawlessly.

Data 80

Data IT Engineering 80

Knowledge Hut

MAY 2, 2024

The main reason most projects move to Agile is they would like to see results fast. These results cannot be achieved quickly if there is a lack of clarity on the outcome, this is where the user story comes in. You might also find it interesting to go through User Stories examples. User stories are like mini single-line business requirements which tell you the Who for, Why, and What to develop.

Business Analyst 98

Business Analyst Project Certification Management 98

Uber Engineering

MAY 2, 2024

Innovatively scaling its chat channel, Uber’s Customer Obsession Team enhanced global support by transitioning 36% of contact volume to chat, leveraging a new architecture that slashed error rates from 46% to 0.45%, showcasing a significant leap in efficiency and customer satisfaction.

Building 75

Building Architecture IT 75

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

Knowledge Hut

MAY 2, 2024

A study published recently in the Journal of Applied Psychology found that, “the pandemic has resulted in people getting more stressed and less engaged at work” Covid-times have brought to the fore the shortcomings of the traditional workplace. Organizations are relying on HR to deal with new age disruptions like lack of engagement, employee retention and motivation.

Recruitment 98

Recruitment Technology Project Government 98

Uber Engineering

MAY 2, 2024

Learn about how Uber presents a consistent view of distributed financial data across earners, spenders, and merchants powered by indexes in Uber’s homegrown ledger-style database, LedgerStore.

Database 75

Database Data 75

Knowledge Hut

MAY 2, 2024

Why We Need Big Data Frameworks Big data is primarily defined by the volume of a data set. Big data sets are generally huge – measuring tens of terabytes – and sometimes crossing the threshold of petabytes. It is surprising to know how much data is generated every minute. As estimated by DOMO : Over 2.5 quintillion bytes of data are created every single day, and it’s only going to grow from there.

Hadoop 96

Hadoop Scala Datasets Java 96

Uber Engineering

MAY 2, 2024

Want to improve the reliability of your Presto cluster with just a few lines of code? Come read how we reduced errors by 90% through improving garbage collection.

Coding 74

Coding Data 74

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineering

Knowledge Hut

MAY 2, 2024

Test automation is one of the most cost-effective and time-saving methods to test software products with long maintenance cycles. TestComplete and Selenium are the two most important automation testing tools which provide an open platform for you to easily build continuous testing frameworks to test non-stop with a lightweight execution engine and distributed testing.

Programming Language 92

Programming Language Java Programming Python 92

Uber Engineering

MAY 2, 2024

With the introduction of Model Excellence Scores at Uber, we’re setting a new standard for measuring, monitoring, and maintaining ML model quality–read how this innovative approach aims to enhance ML governance and provide clearer insights.

Machine Learning 74

Machine Learning Systems Government Data 74

Precisely

MAY 2, 2024

In the digital era, your data is a crucial key to operational success – and the strategic importance of SAP customer master data can’t be overstated. When it comes to customer-related transactions and analytics, your data’s integrity, accuracy, and accessibility directly impact your business’s ability to operate efficiently and deliver value to customers.

Coding Skills 69

Coding Skills Food Data Portfolio 69

Uber Engineering

MAY 2, 2024

In today’s fast-paced digital world, preventing downtime and disruptions is crucial. Learn how uVitals is helping to reduce the time to detect anomalies from days to hours, improving the signal-to-noise ratio by up to 95% at Uber scale.

Systems 64

Systems Data 64

Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage

When tasked with building a fundamentally new product line with deeper insights than previously achievable for a high-value client, Ben Epstein and his team faced a significant challenge: how to harness LLMs to produce consistent, high-accuracy outputs at scale. In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation m

Software Engineer

Knowledge Hut

MAY 2, 2024

Apache Spark was developed by a team at UC Berkeley in 2009. Since then, Apache Spark has seen a very high adoption rate from top-notch technology companies like Google, Facebook, Apple, Netflix etc. The demand has been ever increasing day by day. According to marketanalysis.com survey, the Apache Spark market worldwide will grow at a CAGR of 67% between 2019 and 2022.

Scala 52

Scala Hospitality Machine Learning Healthcare 52

Uber Engineering

MAY 2, 2024

Get behind-the-scenes access to Uber’s financial finesse. Explore Uber’s commitment to flawless financials with data-driven excellence.

Accessible 45

Accessible Accessibility Data Engineering 45

Knowledge Hut

MAY 2, 2024

In this day it’s very common for companies to shuffle teams and move around people depending on where they are needed or where the company is shorthanded. And one of the major challenges faced is that of effective team building. While the companies face the challenge of team building, the individuals have their own issues to deal with - fitting in.

IT 52

IT Project Building 52

Uber Engineering

MAY 2, 2024

Want to effectively mitigate fraud without compromising your user experience? Learn how we secured the Uber app with risk challenges, like penny drop verification—just one of many employed to verify Uber users making credit or debit card payments.

40

40

Advertisement

Apache Airflow® is the open-source standard to manage workflows as code. It is a versatile tool used in companies across the world from agile startups to tech giants to flagship enterprises across all industries. Due to its widespread adoption, Airflow knowledge is paramount to success in the field of data engineering.

Datasets