KDnuggets

NOVEMBER 22, 2019

If you are a new Data Scientists early in your professional journey, and you’re a bit confused and lost, then follow this advice to figure out how to best contribute to your company.

Data 126

Data 126

Confluent

NOVEMBER 20, 2019

Today marks a new release of KSQL, one so significant that we’re giving it a new name: ksqlDB. Like KSQL, ksqlDB remains freely available and community licensed, and you can […].

IT 111

IT Process Management 111

Uber Engineering

NOVEMBER 19, 2019

A restaurant’s menu is arguably its most important feature. When ordering online or via the app with Uber Eats, potential customers can’t peer in through a restaurant’s windows or smell the scents wafting from their kitchens, so digital menus become … The post Introducing Menu Maker: Uber Eats’ New Menu Management Tool appeared first on Uber Engineering Blog.

Management 103

Management Engineering IT Architecture 103

Data Engineering Podcast

NOVEMBER 18, 2019

Summary With the constant evolution of technology for data management it can seem impossible to make an informed decision about whether to build a data warehouse, or a data lake, or just leave your data wherever it currently rests. What’s worse is that any time you have to migrate to a new architecture, all of your analytical code has to change too.

Data Lake 100

Data Lake Scala Data Warehouse Hadoop 100

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline

KDnuggets

NOVEMBER 22, 2019

To demonstrate the implementation complexity differences along the AutoML highway, let's have a look at how 3 specific software projects approach the implementation of just such an AutoML "solution," namely Keras Tuner, AutoKeras, and automl-gs.

Machine Learning 123

Machine Learning Project Python 123

Confluent

NOVEMBER 21, 2019

ksqlDB is a new kind of database purpose-built for stream processing apps, allowing users to build stream processing applications against data in Apache Kafka® and enhancing developer productivity. ksqlDB simplifies […].

Kafka 108

Kafka Database Process Building 108

Netflix Tech

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Please stop by our “Living Room” for an opportunity to connect or reconnect with Netflixers. We’ve compiled our speaking events below so you know what we’ve been working on. We look forward to seeing you there!

AWS 15

AWS Entertainment Cloud Software Engineering 15

KDnuggets

NOVEMBER 22, 2019

We will focus here exactly on that part of the analysis that transforms words into numbers and texts into number vectors: text encoding.

Data 123

Data 123

Confluent

NOVEMBER 19, 2019

In the past 12 months, games and other forms of content made with the Unity platform were installed 33 billion times reaching 3 billion devices worldwide. Apart from our real-time […].

Cloud 86

Cloud AWS 86

Teradata

NOVEMBER 17, 2019

Moving part of your analytic ecosystem to the cloud requires the inspection of all the ecosystem elements to make sure they perform well over a WAN. Read more.

Cloud 58

Cloud Architecture 58

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

Netflix Tech

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Please stop by our “Living Room” for an opportunity to connect or reconnect with Netflixers. We’ve compiled our speaking events below so you know what we’ve been working on. We look forward to seeing you there!

AWS 40

AWS Entertainment Software Engineering Software Engineer 40

KDnuggets

NOVEMBER 19, 2019

This post will be dedicated to explaining the maths behind Bayes Theorem, when its application makes sense, and its differences with Maximum Likelihood.

IT 122

IT 122

KDnuggets

NOVEMBER 21, 2019

This tutorial will take you through two options that have automated the geocoding process for the user using Python, Selenium and Google Geocoding API.

Python 120

Python Process 120

KDnuggets

NOVEMBER 21, 2019

This blog shows how text data representations can be used to build a classifier to predict a developer’s deep learning framework of choice based on the code that they wrote, via examples of TensorFlow and PyTorch projects.

Process 115

Process Deep Learning Data Coding 115

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

KDnuggets

NOVEMBER 19, 2019

In this article, we are going to talk about popular languages for Data Science and briefly describe each of them.

Programming Language 115

Programming Language Data Science Programming Management 115

KDnuggets

NOVEMBER 21, 2019

Brush up on your Python basics with this post on creating, using, and manipulating tuples.

Python 113

Python Programming 113

KDnuggets

NOVEMBER 21, 2019

This article aims to explain why this drive towards the use of notebooks in production is an anti pattern, giving some suggestions along the way.

Python 110

Python 110

KDnuggets

NOVEMBER 18, 2019

When training a neural network in deep learning, its performance on processing new data is key. Improving the model's ability to generalize relies on preventing overfitting using these important methods.

Deep Learning 101

Deep Learning Process IT Data 101

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

KDnuggets

NOVEMBER 21, 2019

Also: Bring the scientific rigor of reproducibility to your Data Science projects; Neutrinos Lead to Unexpected Discovery in Basic Math ; The media gets really excited about AI. Maybe a bit too excited.

Data Science 99

Data Science Media Data Project 99

KDnuggets

NOVEMBER 19, 2019

As cornerstones of scientific processes, reproducibility and replicability ensure results can be verified and trusted. These two concepts are also crucial in data science, and as a data scientist, you must follow the same rigor and standards in your projects.

Data Science 96

Data Science Data Project Process 96

KDnuggets

NOVEMBER 21, 2019

Autoencoders can be a very powerful tool for leveraging unlabeled data to solve a variety of problem, such as learning a "feature extractor" that helps build powerful classifiers, finding anomalies, or doing a Missing Value Imputation.

Building 93

Building Machine Learning Data 93

KDnuggets

NOVEMBER 20, 2019

In this blog I will be demonstrating how deep learning can be applied even if we don’t have enough data.

Deep Learning 93

Deep Learning Data 93

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineering

KDnuggets

NOVEMBER 20, 2019

Your spectacularly-performing machine learning model could be subject to the common culprits of class imbalance and missing labels. Learn how to handle these challenges with techniques that remain open areas of new research for addressing real-world machine learning problems.

Machine Learning 89

Machine Learning Data Preparation Data 89

KDnuggets

NOVEMBER 20, 2019

The two main takeaways from this paper: firstly, a sharpening of my understanding of the difference between explainability and interpretability, and why the former may be problematic; and secondly some great pointers to techniques for creating truly interpretable models.

Machine Learning 81

Machine Learning 81

KDnuggets

NOVEMBER 20, 2019

The fundamental fact is that more information than ever will need to be analyzed on millions of devices. And that’s where 5G will make accessing data dramatically faster and more efficient. At Samsung, we’re excited about what 5G can truly enable and to be a central player in the new 5G world.

Accessibility 77

Accessibility Accessible Data 77

KDnuggets

NOVEMBER 18, 2019

Since X never, ever marks the spot, this article raids the GitHub repos in search of quality automated machine learning resources. Read on for projects and papers to help understand and implement AutoML.

Machine Learning 70

Machine Learning Project Python 70

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

KDnuggets

NOVEMBER 19, 2019

KDnuggets is calling for original blogs and contributions from new authors on AI, Data Science, Machine Learning, and related topics. The authors of most popular such blogs in December will be profiled in KDnuggets.

Machine Learning 64

Machine Learning Data Science Data 64

KDnuggets

NOVEMBER 20, 2019

Read tips and tricks that helped one Data Scientist to get better at Machine Learning; Learn how to make ML project cost-effective; Consider submitting a blog to KDnuggets - you can be profiled here; and study how to manipulate Python lists.

Machine Learning 64

Machine Learning Project Python Data 64

KDnuggets

NOVEMBER 18, 2019

The new AlphaStar achieved Grandmaster level at StarCraft II overcoming some of the limitations of the previous version. How did it do it?

IT 61

IT 61

KDnuggets

NOVEMBER 19, 2019

Find out how data scientists and AI practitioners can use a machine learning experimentation platform like Comet.ml to apply machine learning and deep learning to methods in the domain of audio analysis.

Deep Learning 57

Deep Learning Machine Learning Data 57

Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage

When tasked with building a fundamentally new product line with deeper insights than previously achievable for a high-value client, Ben Epstein and his team faced a significant challenge: how to harness LLMs to produce consistent, high-accuracy outputs at scale. In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation m

Software Engineer