Data Engineering Podcast

OCTOBER 28, 2019

Summary Despite the fact that businesses have relied on useful and accurate data to succeed for decades now, the state of the art for obtaining and maintaining that information still leaves much to be desired. In an effort to create a better abstraction for building data applications Nick Schrock created Dagster. In this episode he explains his motivation for creating a product for data management, how the programming model simplifies the work of building testable and maintainable pipelines, and

Building 100

Building Data Pipeline Programming Language Metadata 100

KDnuggets

OCTOBER 7, 2019

Are you looking to learn natural language processing? This collection of 10 free top notch courses will allow you to do just that, with something for every approach to learning NLP and its varied topics.

Process 123

Process IT 123

Uber Engineering

OCTOBER 16, 2019

Michelangelo , Uber’s machine learning (ML) platform, supports the training and serving of thousands of models in production across the company. Designed to cover the end-to-end ML workflow, the system currently supports classical machine learning, time series forecasting, and deep … The post Evolving Michelangelo Model Representation for Flexibility at Scale appeared first on Uber Engineering Blog.

Machine Learning 111

Machine Learning Engineering Designing Systems 111

Netflix Tech

OCTOBER 23, 2019

Jeremy Smith , Jonathan Indig , Faisal Siddiqi We are pleased to announce the open-source launch of Polynote : a new, polyglot notebook with first-class Scala support, Apache Spark integration, multi-language interoperability including Scala, Python, and SQL, as-you-type autocomplete, and more. Polynote provides data scientists and machine learning researchers with a notebook environment that allows them the freedom to seamlessly integrate our JVM-based ML platform ?

Scala 93

Scala Machine Learning Python Coding 93

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Dataquest

OCTOBER 16, 2019

Exciting news: we just launched a totally revamped Data Engineering path that offers from-scratch training for anyone who wants to become a data engineer or learn some data engineering skills. Looks cool, right? But it begs the question: why learn data engineering in the first place? Typically, data science teams are comprised of data analysts, data scientists, and data engineers.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Science 52

Teradata

OCTOBER 21, 2019

With four new offerings, Teradata is helping companies move from analytics to answers wherever they are on their cloud journey. Read more.

Cloud 66

Cloud 66

KDnuggets

OCTOBER 22, 2019

For full-stack data science mastery, you must understand data management along with all the bells and whistles of machine learning. This high-level overview is a road map for the history and current state of the expansive options for data storage and infrastructure solutions.

Data Management 121

Data Management Management Data Storage Machine Learning 121

Uber Engineering

OCTOBER 16, 2019

Michelangelo , Uber’s machine learning (ML) platform, supports the training and serving of thousands of models in production across the company. Designed to cover the end-to-end ML workflow, the system currently supports classical machine learning, time series forecasting, and deep … The post Evolving Michelangelo Model Representation for Flexibility at Scale appeared first on Uber Engineering Blog.

Machine Learning 90

Machine Learning Engineering Designing Systems 90

Netflix Tech

OCTOBER 16, 2019

By Ammar Khaku Introduction In a microservice architecture such as Netflix’s, propagating datasets from a single source to multiple downstream destinations can be challenging. These datasets can represent anything from service configuration to the results of a batch job, are often needed in-memory to optimize access and must be updated as they change over time.

Datasets 76

Datasets Metadata Bytes Machine Learning 76

Dataquest

OCTOBER 16, 2019

We’ve got some really exciting news: we’ve just launched a total revamp of our Data Engineering learning path ! This revamped path is designed to be more like our other course paths. You can start it even if you have no prior experience with coding , and it’ll take you from total beginner to experienced practitioner with all of the core skills needed to become a data engineer.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Teradata

OCTOBER 3, 2019

Predict the future faster with predictive modeling. Learn more about use cases and how to get more value out of your data.

Data 69

Data 69

Data Engineering Podcast

OCTOBER 14, 2019

Summary Managing a data warehouse can be challenging, especially when trying to maintain a common set of patterns. Dataform is a platform that helps you apply engineering principles to your data transformations and table definitions, including unit testing SQL scripts, defining repeatable pipelines, and adding metadata to your warehouse to improve your team’s communication.

Data Warehouse 100

Data Warehouse PostgreSQL AWS Programming Language 100

KDnuggets

OCTOBER 21, 2019

If you are interested in learning more about the latest Youtube recommendation algorithm paper, read this post for details on its approach and improvements.

Algorithm 123

Algorithm IT Systems Engineering 123

Confluent

OCTOBER 31, 2019

The relationship between Apache Kafka® and machine learning (ML) is an interesting one that I’ve written about quite a bit in How to Build and Deploy Scalable Machine Learning in […].

Machine Learning 49

Machine Learning Kafka Building Process 49

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Netflix Tech

OCTOBER 18, 2019

Faisal Siddiqi Infrastructure for Contextual Bandits and Reinforcement Learning?—? theme of the ML Platform meetup hosted at Netflix, Los Gatos on Sep 12, 2019. Contextual and Multi-armed Bandits enable faster and adaptive alternatives to traditional A/B Testing. They enable rapid learning and better decision-making for product rollouts. Broadly speaking, these approaches can be seen as a stepping stone to full-on Reinforcement Learning (RL) with closed-loop, on-policy evaluation and model objec

Algorithm 51

Algorithm Architecture Machine Learning Deep Learning 51

Rockset

OCTOBER 21, 2019

At Rockset, our singular focus is to be the shortest (and most efficient) path from data to applications for our users. We recognize and truly believe that our success lies in the success of our users. We constantly think about improving our workflows, coming up with new ones and iterating on them in ways that takes the user experience to a whole new level.

Unstructured Data 40

Unstructured Data Designing Data Accessibility 40

Teradata

OCTOBER 22, 2019

With our renewed focus on user experience, we’re applying user-centered design principles & conducting ethnographic research on key personas, starting with developers.

Designing 56

Designing 56

Data Engineering Podcast

OCTOBER 7, 2019

Summary The process of exposing your data through a SQL interface has many possible pathways, each with their own complications and tradeoffs. One of the recent options is Rockset, a serverless platform for fast SQL analytics on semi-structured and structured data. In this episode CEO Venkat Venkataramani and SVP of Product Shruti Bhat explain the origins of Rockset, how it is architected to allow for fast and flexible SQL analytics on your data, and how their serverless platform can save you th

Structured Data 100

Structured Data Cloud SQL Programming Language 100

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems

KDnuggets

OCTOBER 2, 2019

Applying a clustering algorithm is much easier than selecting the best one. Each type offers pros and cons that must be considered if you’re striving for a tidy cluster structure.

Algorithm 111

Algorithm Datasets 111

Preset

OCTOBER 8, 2019

Preset Announcement

Systems 40

Systems 40

Netflix Tech

OCTOBER 23, 2019

Jeremy Smith , Jonathan Indig , Faisal Siddiqi We are pleased to announce the open-source launch of Polynote : a new, polyglot notebook with first-class Scala support, Apache Spark integration, multi-language interoperability including Scala, Python, and SQL, as-you-type autocomplete, and more. Polynote provides data scientists and machine learning researchers with a notebook environment that allows them the freedom to seamlessly integrate our JVM-based ML platform ?

Scala 43

Scala Machine Learning Python Coding 43

Rockset

OCTOBER 1, 2019

Kubernetes at Rockset At Rockset, we use Kubernetes (k8s) for cluster orchestration. It runs all our production microservices — from our ingest workers to our query-serving tier. In addition to hosting all the production infrastructure, each engineer has their own Kubernetes namespace and dedicated resources that we use to locally deploy and test new versions of code and configuration.

SQL 40

SQL Systems Metadata Accessibility 40

Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network

In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.

Project

Teradata

OCTOBER 15, 2019

Find out how our UX team is going to radically simplify the Teradata user experience. To be unveiled at Teradata Universe!

65

65

Confluent

OCTOBER 10, 2019

The Internet of Things (IoT) is getting more and more traction as valuable use cases come to light. A key challenge, however, is integrating devices and machines to process the data in real time and at scale. Apache Kafka ® and its surrounding ecosystem, which includes Kafka Connect, Kafka Streams, and KSQL, have become the technology of choice for integrating and processing these kinds of datasets.

Kafka 20

Kafka Google Cloud Architecture Machine Learning 20

KDnuggets

OCTOBER 2, 2019

As data scientists who are the brains behind the AI-based innovations, you need to understand the significance of data preparation to achieve the desired level of cognitive capability for your models. Let’s begin.

Data Preparation 111

Data Preparation Machine Learning IT Data 111

KDnuggets

OCTOBER 16, 2019

A guide covering the things you should learn to become a data scientist, including the basics of business intelligence, statistics, programming, and machine learning.

Business Intelligence 112

Business Intelligence Machine Learning Programming Data 112

Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL

Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.

Government

KDnuggets

OCTOBER 25, 2019

Have you ever wondered how your personal assistant (e.g: Siri) is built? Do you want to build your own? Perfect! Let’s talk about Natural Language Processing.

Process 114

Process Building 114

KDnuggets

OCTOBER 7, 2019

As a data scientist, your most important skill is creating meaningful visualizations to disseminate knowledge and impact your organization or client. These seven principals will guide you toward developing charts with clarity, as exemplified with data from a recent KDnuggets poll.

Data Science 108

Data Science Data Java Python 108

KDnuggets

OCTOBER 24, 2019

In this post, you will see 3 different techniques of how to do Feature Selection to your datasets and how to build an effective predictive model.

Datasets 118

Datasets Building Machine Learning 118

KDnuggets

OCTOBER 1, 2019

Data science education in Europe has been reevaluated and new recommendations are leading the way to the next generation of data science Master's courses to better support and train students.

Data Science 106

Data Science Education Data 106

Advertisement

Entity Resolution Sometimes referred to as data matching or fuzzy matching, entity resolution, is critical for data quality, analytics, graph visualization and AI. Learn what entity resolution is, why it matters, how it works and its benefits. Advanced entity resolution using AI is crucial because it efficiently and easily solves many of today’s data quality and analytics problems.

IT