KDnuggets

DECEMBER 19, 2019

The third part in a series on leveraging techniques to take a look inside the black box of AI, this guide considers methods that try to explain each prediction instead of establishing a global explanation.

157

157

Confluent

DECEMBER 18, 2019

The amount of time it takes for a message to move through a system plays a big role in the performance of distributed systems like Apache Kafka®. In Kafka, the […].

Kafka 26

Kafka Systems IT 26

Uber Engineering

DECEMBER 19, 2019

Every day around the world, millions of trips take place across the Uber network, giving users more reliable transportation through ridesharing, bikes, and scooters, drivers and truckers additional opportunities to earn, employees and employers more convenient business travel, and hungry … The post Uber Infrastructure in 2019: Improving Reliability, Driving Customer Satisfaction appeared first on Uber Engineering Blog.

Transportation 127

Transportation Engineering Architecture 127

Data Engineering Podcast

DECEMBER 16, 2019

Summary Building clean datasets with reliable and reproducible ingestion pipelines is completely useless if it’s not possible to find them and understand their provenance. The solution to discoverability and tracking of data lineage is to incorporate a metadata repository into your data platform. The metadata repository serves as a data catalog and a means of reporting on the health and status of your datasets when it is properly integrated into the rest of your tools.

Metadata 100

Metadata PostgreSQL Datasets Data Warehouse 100

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

KDnuggets

DECEMBER 18, 2019

Ready to try to get hired as a data scientist for the first time? Avoiding these common mistakes won’t guarantee an offer, but not avoiding them is a sure fire way for your application to be tossed into the trash bin.

Data 156

Data 156

Confluent

DECEMBER 19, 2019

With Confluent Platform 5.3, we are actively embracing the rising DevOps movement by introducing CP-Ansible, our very own open source Ansible playbooks for deployment of Apache Kafka® and the Confluent […].

Kafka 22

Kafka 22

Netflix Tech

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. CDC is becoming increasingly popular for use cases that require keeping multiple heterogeneous datastores in sync (like MySQL and ElasticSearch) and addresses challenges that exist with traditional techniques like dual-writes and distributed transactions [3][4].

MySQL 89

MySQL PostgreSQL Database Transportation 89

KDnuggets

DECEMBER 18, 2019

Marketing scientist Kevin Gray asks Dr. Anna Farzindar of the University of Southern California about Automatic Text Summarization and the various ways it is used.

IT 152

IT 152

Confluent

DECEMBER 16, 2019

On behalf of the Apache Kafka® community, it is my pleasure to announce the release of Apache Kafka 2.4.0. This release includes a number of key new features and improvements […].

Kafka 21

Kafka Scala IT 21

Dataquest

DECEMBER 18, 2019

Today, Stacey Ustian is a data engineer. But the path that led her here wasn’t always easy, and there were a few bumps and twists along the way. Her journey to data science started in a rather unusual place: the law library. After earning her Master’s degree in Library and Information Science, Stacey had taken a job working in the library of a law firm.

SQL 52

SQL Python Data Engineer Data Engineering 52

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Netflix Tech

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. CDC is becoming increasingly popular for use cases that require keeping multiple heterogeneous datastores in sync (like MySQL and ElasticSearch) and addresses challenges that exist with traditional techniques like dual-writes and distributed transactions [3][4].

MySQL 83

MySQL PostgreSQL Database Transportation 83

KDnuggets

DECEMBER 19, 2019

Over the years new alternative providers have risen to provided a solitary data science environment hosted on the cloud for data scientist to analyze, host and share their work.

Data Science 143

Data Science Cloud Data Cloud Computing 143

Confluent

DECEMBER 17, 2019

As a test class that allows you to test Kafka Streams logic, TopologyTestDriver is a lot faster than utilizing EmbeddedSingleNodeKafkaCluster and makes it possible to simulate different timing scenarios. Not […].

Kafka 19

Kafka Utilities IT Process 19

Teradata

DECEMBER 18, 2019

Carrie Ballinger discusses the techniques for managing concurrency inside the Advanced SQL Engine and the benefits provided. Read more.

SQL 49

SQL Engineering Management 49

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Netflix Tech

DECEMBER 17, 2019

Andreas Andreakis, Ioannis Papapanagiotou Continue reading on Netflix TechBlog ».

Data 52

Data MySQL Database 52

KDnuggets

DECEMBER 16, 2019

Predictions for 2020 from a dozen innovative companies in AI, Analytics, Machine Learning, Data Science, and Data industry.

Machine Learning 136

Machine Learning Data Science Data 136

Preset

DECEMBER 15, 2019

Announcing Elasticsearch in Superset, powered by a new open-source Python library from Preset

Python 40

Python 40

Teradata

DECEMBER 16, 2019

Enterprises either have no data strategy at all or an over-complicated one that under delivers. Find out how to create an effective data strategy by striking balance.

Data 40

Data 40

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

KDnuggets

DECEMBER 20, 2019

Google has started offering a new service for “explainable AI” or XAI, as it is fashionably called. Presently offered tools are modest, but the intent is in the right direction.

IT 134

IT 134

KDnuggets

DECEMBER 17, 2019

Check out this step by step approach to building an intelligent chatbot in Python.

Building 123

Building Python 123

KDnuggets

DECEMBER 20, 2019

By the end of this article you’ll know which technologies are becoming more popular with employers and which are becoming less popular.

Technology 122

Technology Data Data Science 122

KDnuggets

DECEMBER 16, 2019

Once you have deployed your machine learning model into production, differences in real-world data will result in model drift. So, retraining and redeploying will likely be required. In other words, deployment should be treated as a continuous process. This guide defines model drift and how to identify it, and includes approaches to enable model training.

Machine Learning 120

Machine Learning Process IT Data 120

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

KDnuggets

DECEMBER 18, 2019

This post is about working with a mixture of color and grayscale images and needing to transform them into a uniform format - all grayscale. We'll be working in Python using the Pillow, Numpy, and Matplotlib packages.

Python 114

Python Process 114

KDnuggets

DECEMBER 19, 2019

Here are six more lessons based on real life examples that I think we should all remember as people working in machine learning, whether you’re a researcher, engineer, or a decision-maker.

Machine Learning 107

Machine Learning Engineering Algorithm 107

KDnuggets

DECEMBER 17, 2019

Read this overview of a complete pipeline for detecting pedestrians on the road.

Algorithm 84

Algorithm Python 84

KDnuggets

DECEMBER 20, 2019

This third part in a series about how to "ultralearn" data science will guide you through how to optimize your learning through five valuable techniques.

Data Science 83

Data Science Data 83

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

KDnuggets

DECEMBER 16, 2019

The new technique allows the deployment of machine learning models that operate with minimum training data.

Machine Learning 81

Machine Learning Data Preparation Data 81

KDnuggets

DECEMBER 17, 2019

Also: Your AI skills are worth less than you think; Another 10 Free Must-See Courses for Machine Learning and Data Science.

Technology 75

Technology Machine Learning Data Science Data 75

KDnuggets

DECEMBER 17, 2019

This second part in a series about how to "ultralearn" data science will guide you through several techniques to remove those distractions -- because your focus needs more focus.

Data Science 73

Data Science Data Education 73

KDnuggets

DECEMBER 18, 2019

Ontotext Platform 3.0 features significant technology improvements to enable simpler and faster graph navigation, including GraphQL interfaces to make it easier for application developers to access knowledge graphs without tedious development of back-end APIs or complex SPARQL.

Accessibility 65

Accessibility Accessible Technology IT 65

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud