KDnuggets

NOVEMBER 7, 2019

This post will walkthrough a Python implementation of a vocabulary class for storing processed text data and related metadata in a manner useful for subsequently performing NLP tasks.

Python 123

Python Metadata Process Data Preparation 123

Data Engineering Podcast

NOVEMBER 4, 2019

Summary As data engineers the health of our pipelines is our highest priority. Unfortunately, there are countless ways that our dataflows can break or degrade that have nothing to do with the business logic or data transformations that we write and maintain. Sean Knapp founded Ascend to address the operational challenges of running a production grade and scalable Spark infrastructure, allowing data engineers to focus on the problems that power their business.

Programming Language 100

Programming Language Data Engineering Data Engineer Kafka 100

Netflix Tech

NOVEMBER 4, 2019

by Artem Shtatnov and Ravi Srinivas Ranganathan Almost a year ago we described our learnings from adopting GraphQL on the Netflix Marketing Tech team. We have a lot more to share since then! There are plenty of existing resources describing how to express a search query in GraphQL and paginate the results. This post looks at the other side of search: how to index data and make it searchable.

Kafka 99

Kafka Algorithm Database Relational Database 99

Confluent

NOVEMBER 6, 2019

Today, we are proud to make Confluent Cloud available to companies leveraging the Microsoft Azure ecosystem of services, in addition to the previous rollouts on Google Cloud Platform (GCP) and […].

Cloud 81

Cloud Google Cloud 81

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline

KDnuggets

NOVEMBER 5, 2019

Artificial Intelligence continues to fill the media headlines while scientists and engineers rapidly expand its capabilities and applications. With such explosive growth in the field, there is a great deal to learn. Dive into these 10 free books that are must-reads to support your AI study and work.

Media 123

Media Engineering IT 123

Teradata

NOVEMBER 5, 2019

The people who drive real business innovation in your org may not all be coders. With Vantage Analyst, they can explore data to uncover insights that may lead to that next big thing.

Data 11

Data 11

Confluent

NOVEMBER 7, 2019

Kafka Connect is the part of Apache Kafka® that provides reliable, scalable, distributed streaming integration between Apache Kafka and other systems. Kafka Connect has connectors for many, many systems, and […].

Kafka 74

Kafka Systems 74

KDnuggets

NOVEMBER 4, 2019

The new release of PyTorch includes some impressive open source projects for deep learning researchers and developers.

Deep Learning 123

Deep Learning Project 123

Netflix Tech

NOVEMBER 4, 2019

by Artem Shtatnov and Ravi Srinivas Ranganathan Almost a year ago we described our learnings from adopting GraphQL on the Netflix Marketing Tech team. We have a lot more to share since then! There are plenty of existing resources describing how to express a search query in GraphQL and paginate the results. This post looks at the other side of search: how to index data and make it searchable.

Kafka 46

Kafka Algorithm Database Relational Database 46

Rockset

NOVEMBER 6, 2019

Events are messages that are sent by a system to notify operators or other systems about a change in its domain. With event-driven architectures powered by systems like Apache Kafka becoming more prominent, there are now many applications in the modern software stack that make use of events and messages to operate effectively. In this blog, we will examine the use of three different data backends for event data - Apache Druid , Elasticsearch and Rockset.

Kafka 40

Kafka Data Lake SQL Hadoop 40

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

Teradata

NOVEMBER 3, 2019

Improving the customer experience is the top priority for CMOs. Find out what the top 3 distinct CX strategies are to drive customer loyalty.

40

40

KDnuggets

NOVEMBER 4, 2019

Customer Segmentation can be a powerful means to identify unsatisfied customer needs. This technique can be used by companies to outperform the competition by developing uniquely appealing products and services.

Python 122

Python 122

Netflix Tech

NOVEMBER 4, 2019

by Artem Shtatnov and Ravi Srinivas Ranganathan Almost a year ago we described our learnings from adopting GraphQL on the Netflix Marketing Tech team. We have a lot more to share since then! There are plenty of existing resources describing how to express a search query in GraphQL and paginate the results. This post looks at the other side of search: how to index data and make it searchable.

Kafka 40

Kafka Algorithm Database Relational Database 40

KDnuggets

NOVEMBER 4, 2019

Check out this step-by-step walk through of some of the more confusing aspects of neural nets to guide you to making smart decisions about your neural network architecture.

Designing 122

Designing Architecture 122

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

KDnuggets

NOVEMBER 7, 2019

In this tutorial, we show how to apply mathematical set operations (union, intersection, and difference) to Pandas DataFrames with the goal of easing the task of comparing the rows of two datasets.

Datasets 118

Datasets Data Preparation Data Science Python 118

KDnuggets

NOVEMBER 8, 2019

A boxplot. It can tell you about your outliers and what their values are. It can also tell you if your data is symmetrical, how tightly your data is grouped, and if and how your data is skewed.

IT 111

IT Data Python 111

KDnuggets

NOVEMBER 7, 2019

Careful preprocessing of data for your machine learning project is crucial. This overview describes the process of data cleaning and dealing with noise and missing data.

Machine Learning 109

Machine Learning Data Project Process 109

KDnuggets

NOVEMBER 8, 2019

Do you want to extract csv files with Python and visualize them in R? How does preparing everything in R and make conclusions with Python sound? Both are possible if you know the right libraries and techniques. Here, we’ll walk through a use-case using both languages in one analysis.

Python 109

Python 109

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

KDnuggets

NOVEMBER 6, 2019

This guide explores research centered on a variety of advanced loss functions for machine learning models.

Machine Learning 100

Machine Learning 100

KDnuggets

NOVEMBER 5, 2019

The maths behind Bayes will be better understood if we first cover the theory and maths underlying another fundamental method of probabilistic machine learning: Maximum Likelihood. This post will be dedicated to explaining it.

Machine Learning 91

Machine Learning IT 91

KDnuggets

NOVEMBER 6, 2019

My short answer is this: Yes, another AI Winter will be here if you don’t deploy more ML solutions. You and your Data Science teams are the last line of defense against the AI Winter. You need to solve five key challenges to keep the momentum up.

Data Science 86

Data Science Machine Learning Data 86

KDnuggets

NOVEMBER 8, 2019

Data Natives is an outstanding conference that lets you meet many talented Data Scientists and Data Professionals. Find your dream company or your dream employee and level up for 2020. Use code DN19_KDNuggets_50 to save.

Data 85

Data Coding 85

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineering

KDnuggets

NOVEMBER 5, 2019

Are you interested in starting your career in the data analysis domain? Read this informative blog on how to get your career off the ground.

Healthcare 77

Healthcare Data Analysis Data 77

KDnuggets

NOVEMBER 8, 2019

Also A European Approach to Master's Degrees in Data Science; How YouTube is Recommending Your Next Video.

Data Analysis 69

Data Analysis Data Management Data Science SQL 69

KDnuggets

NOVEMBER 8, 2019

Data Science is pitched as a modern and exciting job offering high satisfaction. Does its reality really live up to the hype? Here, we show what it's really like to work as a Data Scientist.

Data Science 68

Data Science Data IT 68

KDnuggets

NOVEMBER 6, 2019

Learn about statistical fallacies Data Scientists should avoid; New and quite amazing Deep Learning capabilities FB has been quietly open-sourcing; Top Machine Learning tools for Developers; How to build a Neural Network from scratch and more.

Deep Learning 64

Deep Learning Machine Learning Data Building 64

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

KDnuggets

NOVEMBER 6, 2019

Follow these eight headings of an audit sheet that business analysts should address before submitting the results of their analytics project. One recommended approach is to rewrite each step as a question, answer it, and then attach it to your project.

Project 59

Project Business Analyst IT 59

KDnuggets

NOVEMBER 6, 2019

Neebo is a SaaS solution that enables analytics teams to connect to, find, combine and collaborate on trusted data assets in hybrid cloud landscapes, and provides a unified access point where they can more effectively leverage all their analytics assets and knowledge. In this blog, we will highlight some of the features of Neebo and how they can completely transform the way analytics teams operate.

Cloud 57

Cloud Accessibility Accessible Data 57

KDnuggets

NOVEMBER 6, 2019

Which Data Science Skills are core and which are hot/emerging ones?; The 4 Quadrants of Data Science Skills and 7 Principles for Creating a Viral DataViz; Microsoft open sources #SandDance, a visual data exploration tool.

Data Management 52

Data Management Management Data Science Data 52

KDnuggets

NOVEMBER 7, 2019

Catch this Domino webinar on monitoring models at scale, Dec 11 @ 10am PT, covering detecting changes in pattern of real-world data your models are seeing in production, tracking how model accuracy and other quality metrics are changing over time, and getting alerted when health checks fail so that resolution workflows can be triggered.

Data 51

Data 51

Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage

When tasked with building a fundamentally new product line with deeper insights than previously achievable for a high-value client, Ben Epstein and his team faced a significant challenge: how to harness LLMs to produce consistent, high-accuracy outputs at scale. In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation m

Software Engineer