Data Engineering Podcast

OCTOBER 14, 2019

Summary Managing a data warehouse can be challenging, especially when trying to maintain a common set of patterns. Dataform is a platform that helps you apply engineering principles to your data transformations and table definitions, including unit testing SQL scripts, defining repeatable pipelines, and adding metadata to your warehouse to improve your team’s communication.

Data Warehouse 100

Data Warehouse PostgreSQL AWS Programming Language 100

Uber Engineering

OCTOBER 16, 2019

Michelangelo , Uber’s machine learning (ML) platform, supports the training and serving of thousands of models in production across the company. Designed to cover the end-to-end ML workflow, the system currently supports classical machine learning, time series forecasting, and deep … The post Evolving Michelangelo Model Representation for Flexibility at Scale appeared first on Uber Engineering Blog.

Machine Learning 111

Machine Learning Engineering Designing Systems 111

KDnuggets

OCTOBER 16, 2019

A guide covering the things you should learn to become a data scientist, including the basics of business intelligence, statistics, programming, and machine learning.

Business Intelligence 114

Business Intelligence Machine Learning Programming Data 114

Netflix Tech

OCTOBER 16, 2019

By Ammar Khaku Introduction In a microservice architecture such as Netflix’s, propagating datasets from a single source to multiple downstream destinations can be challenging. These datasets can represent anything from service configuration to the results of a batch job, are often needed in-memory to optimize access and must be updated as they change over time.

Datasets 76

Datasets Metadata Bytes Machine Learning 76

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Dataquest

OCTOBER 16, 2019

Exciting news: we just launched a totally revamped Data Engineering path that offers from-scratch training for anyone who wants to become a data engineer or learn some data engineering skills. Looks cool, right? But it begs the question: why learn data engineering in the first place? Typically, data science teams are comprised of data analysts, data scientists, and data engineers.

Data Engineer 52

Data Engineer Data Engineering Engineering Data Science 52

Uber Engineering

OCTOBER 16, 2019

Michelangelo , Uber’s machine learning (ML) platform, supports the training and serving of thousands of models in production across the company. Designed to cover the end-to-end ML workflow, the system currently supports classical machine learning, time series forecasting, and deep … The post Evolving Michelangelo Model Representation for Flexibility at Scale appeared first on Uber Engineering Blog.

Machine Learning 90

Machine Learning Engineering Designing Systems 90

Teradata

OCTOBER 15, 2019

Find out how our UX team is going to radically simplify the Teradata user experience. To be unveiled at Teradata Universe!

65

65

Dataquest

OCTOBER 16, 2019

We’ve got some really exciting news: we’ve just launched a total revamp of our Data Engineering learning path ! This revamped path is designed to be more like our other course paths. You can start it even if you have no prior experience with coding , and it’ll take you from total beginner to experienced practitioner with all of the core skills needed to become a data engineer.

Data Engineer 52

Data Engineer Data Engineering Engineering SQL 52

Netflix Tech

OCTOBER 18, 2019

Faisal Siddiqi Infrastructure for Contextual Bandits and Reinforcement Learning?—? theme of the ML Platform meetup hosted at Netflix, Los Gatos on Sep 12, 2019. Contextual and Multi-armed Bandits enable faster and adaptive alternatives to traditional A/B Testing. They enable rapid learning and better decision-making for product rollouts. Broadly speaking, these approaches can be seen as a stepping stone to full-on Reinforcement Learning (RL) with closed-loop, on-policy evaluation and model objec

Algorithm 51

Algorithm Architecture Machine Learning Deep Learning 51

KDnuggets

OCTOBER 17, 2019

While the average salary for a Software Engineer is around $100,000 to $150,000, to make the big bucks you want to be an AI or Machine Learning (Specialist/Scientist/Engineer.).

Machine Learning 101

Machine Learning Software Engineer Software Engineering Engineering 101

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Confluent

OCTOBER 16, 2019

Trains are an excellent source of streaming data—their movements around the network are an unbounded series of events. Using this data, Apache Kafka ® and Confluent Platform can provide the foundations for both event-driven applications as well as an analytical platform. With tools like KSQL and Kafka Connect, the concept of streaming ETL is made accessible to a much wider audience of developers and data engineers.

Kafka 19

Kafka Building Data Coding 19

KDnuggets

OCTOBER 17, 2019

This post aims to make you get started with putting your trained machine learning models into production using Flask API.

Machine Learning 110

Machine Learning Python 110

KDnuggets

OCTOBER 14, 2019

Selecting the perfect machine learning model is part art and part science. Learn how to review multiple models and pick the best in both competitive and real-world applications.

Machine Learning 83

Machine Learning 83

KDnuggets

OCTOBER 18, 2019

If you want to launch your data science skills into freelance work, then check out these important tips to help you kick start your next adventure in data.

Data Science 87

Data Science Data Consulting 87

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

KDnuggets

OCTOBER 14, 2019

Density estimation is estimating the probability density function of the population from the sample. This post examines and compares a number of approaches to density estimation.

79

79

KDnuggets

OCTOBER 18, 2019

Read this quick overview of neural networks and learn how to implement your first in very few lines using Keras.

Coding 96

Coding Python 96

KDnuggets

OCTOBER 16, 2019

This post is about various evaluation metrics and how and when to use them.

Data 114

Data Machine Learning Python 114

KDnuggets

OCTOBER 15, 2019

The way we control our data isn’t working. Data is as vulnerable as ever. Download this white paper, which outlines lessons about how data science and governance programs can, if implemented properly, reinforce each other’s objective.

Government 65

Government Data Governance Data Science Data 65

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

KDnuggets

OCTOBER 15, 2019

In this research guide, we’ll look at deep learning papers aimed at synthesizing video frames within an existing video.

Deep Learning 80

Deep Learning 80

KDnuggets

OCTOBER 17, 2019

While effective anonymization technology remains elusive, understanding the history of this challenge can guide data science practitioners to address these important concerns through ethical and responsible use of sensitive information.

Data Science 59

Data Science Data Technology 59

KDnuggets

OCTOBER 18, 2019

In this second part we want to outline our own experience building an AI application and reflect on why we chose not to utilise deep learning as the core technology used.

Deep Learning 60

Deep Learning Building Technology Machine Learning 60

KDnuggets

OCTOBER 16, 2019

This week on KDnuggets: Beyond Word Embedding: Key Ideas in Document Embedding; The problem with metrics is a big problem for AI; Activation maps for deep learning models in a few lines of code; There is No Such Thing as a Free Lunch; 8 Paths to Getting a Machine Learning Job Interview; and much, much more.

Deep Learning 57

Deep Learning Machine Learning Coding Data Science 57

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

KDnuggets

OCTOBER 16, 2019

Also: Kannada-MNIST: A new handwritten digits dataset in ML town; Math for Programmers; The 4 Quadrants of Data Science Skills and 7 Principles for Creating a Viral Data Visualization; The Last SQL Guide for Data Analysis You’ll Ever Need.

Process 56

Process Datasets Data Science Data Analysis 56

KDnuggets

OCTOBER 15, 2019

Follow this step-by-step tutorial using Tensorflow to setup a DC/OS Data Science Engine as a PaaS for enabling distributed multi-node, multi-GPU model training.

Data Science 58

Data Science Data Engineering Cloud Computing 58

KDnuggets

OCTOBER 16, 2019

Learn about one of the fundamental theorems of probability with an easy everyday example.

76

76

KDnuggets

OCTOBER 14, 2019

A recent survey outlined the main neural architecture search methods used to automate the design of deep learning systems.

Architecture 62

Architecture Designing Deep Learning Systems 62

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

KDnuggets

OCTOBER 15, 2019

Even though I’m still in my studies, here’s a list of the most important things I’ve learned (as of yet).

Data Science 67

Data Science Data Education 67

KDnuggets

OCTOBER 18, 2019

ODSC West comes to San Francisco on Oct 29 - Nov 1. With over 300 hours of content, 200+ speakers, and thousands of attendees, there is certainly a lot to see, learn, and do at the conference. Register by Friday for 10% off your pass.

Data Science 48

Data Science Data 48

KDnuggets

OCTOBER 14, 2019

Also: Activation maps for deep learning models in a few lines of code; The 4 Quadrants of Data Science Skills and 7 Principles for Creating a Viral Data Visualization; OpenAI Tried to Train AI Agents to Play Hide-And-Seek but Instead They Were Shocked by What They Learned; 10 Great Python Resources for Aspiring Data Scientists.

Data Analysis 48

Data Analysis Deep Learning Process SQL 48

KDnuggets

OCTOBER 17, 2019

Watch UChicago Master of Science in Analytics capstone projects unfold in Real Data, Big Impact and see how students collaborate with their clients to deliver successful analytics projects.

Project 45

Project Data Education 45

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud