KDnuggets

JULY 10, 2024

Let's learn how to efficiently merge large Pandas dataframes.

Python 137

Python 137

Waitingforcode

JULY 10, 2024

Welcome to the first Data+AI Summit 2024 retrospective blog post. I'm opening the series with the topic close to my heart at the moment, stream processing!

Data 130

Data Process 130

KDnuggets

JULY 10, 2024

Learn all about probability with this collection of tutorials from our sister site Statology.

Data Science 116

Data Science Data 116

Engineering at Meta

JULY 10, 2024

Meta’s advertising business leverages large-scale machine learning (ML) recommendation models that power millions of ads recommendations per second across Meta’s family of apps. Maintaining reliability of these ML systems helps ensure the highest level of service and uninterrupted benefit delivery to our users and advertisers. To minimize disruptions and ensure our ML systems are intrinsically resilient, we have built a comprehensive set of prediction robustness solutions that ensure stability w

Machine Learning 107

Machine Learning Algorithm Systems Engineering 107

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

Cloudera

JULY 10, 2024

There’s nothing worse than wasting money on unnecessary costs. In on-premises data estates, these costs appear as wasted person-hours waiting for inefficient analytics to complete, or troubleshooting jobs that have failed to execute as expected, or at all. They manifest as idle hardware waiting for urgent workloads to come in, ensuring sufficient spare capacity to run them amidst noisy neighbors and resource-hungry, lower-priority workloads.

Metadata 99

Metadata Cloud Management IT 99

Engineering at Meta

JULY 10, 2024

Tail utilization is a significant system issue and a major factor in overload-related failures and low compute utilization. The tail utilization optimizations at Meta have had a profound impact on model serving capacity footprint and reliability. Failure rates, which are mostly timeout errors, were reduced by two-thirds; the compute footprint delivered 35% more work for the same amount of resources; and p99 latency was cut in half.

Utilities 87

Utilities Machine Learning Architecture Systems 87

Booking.com Engineering

JULY 10, 2024

Written by Deepak Patankar and Mathijs de Jong Introduction At Booking.com we are dedicated to maintaining a secure and trustworthy platform for both our customers and partners. Our work involves addressing a multitude of threats, ranging from payment fraud to the proliferation of fake hotels and reviews, as well as the abuse of marketing campaigns.

Technology 64

Technology Machine Learning Database Banking 64

ArcGIS

JULY 10, 2024

Use this resource to find all the Open Platform focused sessions and activities at the 2024 Esri User Conference you won't want to miss.

Metadata 62

Metadata 62

KDnuggets

JULY 10, 2024

The rise of soft skills in 2024 and why you should not neglect them!

Coding 62

Coding 62

ArcGIS

JULY 10, 2024

Esri users can leverage the Planetary Computer data catalog for geospatial analysis with ArcGIS for Microsoft Planetary Computer.

Data 52

Data 52

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

Cloudyard

JULY 10, 2024

Read Time: 1 Minute, 37 Second Efficient management of Snowflake resources is crucial for optimizing performance and cost. One of the key aspects is monitoring warehouse usage. In this blog post, we’ll explore a use case where we automate the reporting of warehouse usage and send detailed weekly summaries via email. Our Organization is a data-driven organization relying heavily on Snowflake for its data warehousing needs.

Management 52

Management Process IT Data 52

Hevo

JULY 10, 2024

While you can use Snowpipe for straightforward and low-complexity data ingestion into Snowflake, Snowpipe alternatives, like Kafka, Spark, and COPY, provide enhanced capabilities for real-time data processing, scalability, flexibility in data handling, and broader ecosystem integration.

Kafka 52

Kafka Data Ingestion Data Data Process 52

RandomTrees

JULY 10, 2024

Power Automate is an automation tool developed by Microsoft with the intention to allow citizen developers to access the capability of automation in day-to-day tasks. Whether you are an IT, Marketing, Finance, or HR professional, you will be able to use Power Automate. With the help of power automation one can create a flow to send an alert email whenever a new row is created in a SharePoint list.

BI 52

BI Finance Utilities Accessible 52

Striim

JULY 10, 2024

The utilization of predictive analytics has revolutionized nearly every industry, but perhaps none have experienced its transformative impact quite as profoundly as logistics. In an era marked by rapid technological advancements and ever-increasing customer expectations, the ability to accurately predict demand and efficiently mitigate risks can make or break logistics operations.

Management 52

Management Transportation Machine Learning High Quality Data 52

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

Knowledge Hut

JULY 10, 2024

Since the birth of cloud computing, virtualization has become a common practice in IT. It creates virtual versions of physical items like storage devices, desktops, and servers. Cloud service providers offer these resources, allowing companies to rent them without managing physical services. This reduces IT costs and enhances performance by not limiting physical servers to a few applications.

Cloud Computing 52

Cloud Computing Cloud Systems AWS 52

Edureka

JULY 10, 2024

Even though cybersecurity and ethical hacking are related, they are two distinct fields. Ethical hackers/ white hat hackers, actively probe systems for vulnerabilities and fix them. Cybersecurity is a much broader field. Cybersecurity professionals adopt a defensive stance. Their job is to implement and maintain various security controls, such as firewalls, encryption, and access management.

Certification 40

Certification Consulting Algorithm Accessible 40

Hevo

JULY 10, 2024

The choice of data management system determines how quickly and in real-time you can store and access information. Some cloud database architectures, like Snowflake, offer a scalable and flexible environment for processing large datasets.

Datasets 40

Datasets Architecture Database Cloud 40

Edureka

JULY 10, 2024

Reconnaissance is the initial stage of hacking, and it is ethical. It involves acquiring intelligence on a target with the intention of exploiting their weaknesses. This information proves helpful to hackers when planning how to launch an attack. Now, let us understand what reconnaissance in ethical hacking means, why it is necessary, and how to perform it.

Media 40

Media Utilities Systems Accessibility 40

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineer

Data Engineering Weekly

JULY 10, 2024

The origin - a legend The origin of the modern data stack is a topic of intense debate, shrouded in uncertainty and mystery. Some attribute its incubation to Snowflake, Redshift, or Airflow, while others propose different theories. Rather than being the result of a single event, the term "modern data stack" emerged from a series of innovations and industry shifts, adding to the intrigue of its history.

Hadoop 122

Hadoop Data Warehouse Data Big Data 122