Data Engineering Podcast

SEPTEMBER 22, 2019

Summary Object storage is quickly becoming the unifying layer for data intensive applications and analytics. Modern, cloud oriented data warehouses and data lakes both rely on the durability and ease of use that it provides. S3 from Amazon has quickly become the de-facto API for interacting with this service, so the team at MinIO have built a production grade, easy to manage storage engine that replicates that interface.

AWS 100

AWS Google Cloud Cloud Storage Data Lake 100

KDnuggets

SEPTEMBER 23, 2019

Our list of deep learning researchers and industry leaders are the people you should follow to stay current with this wildly expanding field in AI. From early practitioners and established academics to entrepreneurs and today’s top corporate influencers, this diverse group of individuals is leading the way into tomorrow’s deep learning landscape.

Deep Learning 118

Deep Learning 118

Netflix Tech

SEPTEMBER 23, 2019

Niosha Behnam | Demand Engineering @ Netflix At Netflix we prioritize innovation and velocity in pursuit of the best experience for our 150+ million global customers. This means that our microservices constantly evolve and change, but what doesn’t change is our responsibility to provide a highly available service that delivers 100+ million hours of daily streaming to our subscribers.

Amazon Web Services 76

Amazon Web Services Electronics Java AWS 76

Airbnb Tech

SEPTEMBER 24, 2019

Scaling a Mature Data Pipeline — Managing Overhead There is often a hidden performance cost tied to the complexity of data pipelines — the overhead. In this post, we will introduce its concept, and examine the techniques we use to avoid it in our data pipelines. Author : Zachary Ennenga The view from the third floor at Airbnb HQ! Background There is often a natural evolution in the tooling, organization, and technical underpinning of data pipelines.

Data Pipeline 52

Data Pipeline Management Data Scala 52

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Confluent

SEPTEMBER 25, 2019

In 2011, Marc Andressen wrote an article called Why Software is Eating the World. The central idea is that any process that can be moved into software, will be. This has become a kind of shorthand for the investment thesis behind Silicon Valley’s current wave of unicorn startups. It’s also a unifying idea behind the larger set of technology trends we see today, such as machine learning, IoT, ubiquitous mobile connectivity, SaaS, and cloud computing.

Database-centric 56

Database-centric Kafka Pipeline-centric Retail 56

KDnuggets

SEPTEMBER 24, 2019

Deep Learning is/has become the hottest skill in Data Science at the moment. There is a plethora of articles, courses, technologies, influencers and resources that we can leverage to gain the Deep Learning skills.

Deep Learning 117

Deep Learning Data Science Technology Data 117

KDnuggets

SEPTEMBER 23, 2019

We show, step-by-step, how to construct a single, generalized, utility function to pull images automatically from a directory and train a convolutional neural net model.

Utilities 109

Utilities Python 109

KDnuggets

SEPTEMBER 24, 2019

How can you keep your machine learning models and data organized so you can collaborate effectively? Discover this new tool set available for better version control designed for the data scientist workflow.

Machine Learning 101

Machine Learning Data Designing 101

KDnuggets

SEPTEMBER 26, 2019

Learn about the the current and future issues of data science and possible solutions from this interview with IADSS Co-founder, Dr. Usama Fayyad following his keynote speech at ODSC Boston 2019.

Data Science 98

Data Science Data 98

KDnuggets

SEPTEMBER 25, 2019

As a data scientist, you can get lost in your daily dives into the data. Consider this advice to be certain to follow in your work for being diligent and more impactful for your organization.

Data 96

Data 96

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

KDnuggets

SEPTEMBER 26, 2019

This article provides a brief introduction to working with natural language (sometimes called “text analytics”) in Python using spaCy and related libraries.

Python 100

Python 100

KDnuggets

SEPTEMBER 26, 2019

This article shows you how to separate your customers into distinct groups based on their purchase behavior. For the R enthusiasts out there, I demonstrated what you can do with r/stats, ggradar, ggplot2, animation, and factoextra.

91

91

KDnuggets

SEPTEMBER 27, 2019

Take me out to the ballgame! Take me out to the crowd! For the 2,829 seasons that have been played for 101 baseball teams since 1880, which seasons were unlike any others? Using SAX Encoding to recognize patterns in time series data, the most special years in baseball can be found.

Data 90

Data 90

KDnuggets

SEPTEMBER 26, 2019

Join the Crunch Data Conference in Budapest, Oct 16-18, with stellar speakers from companies like Facebook, Netflix and LinkedIn. Use the discount code ‘KDNuggets’ to save $100 off your conference ticket.

Data 92

Data Coding Data Analytics Data Science 92

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

KDnuggets

SEPTEMBER 24, 2019

In this article, we’ll look at a couple of papers aimed at solving the problem of automated speech recognition with machine and deep learning.

Deep Learning 99

Deep Learning 99

KDnuggets

SEPTEMBER 25, 2019

Today, as companies have finally come to understand the value that data science can bring, more and more emphasis is being placed on the implementation of data science in production systems. And as these implementations have required models that can perform on larger and larger datasets in real-time, an awful lot of data science problems have become engineering problems.

Data Science 85

Data Science Data Engineering Data Engineer Engineering 85

KDnuggets

SEPTEMBER 27, 2019

This live webinar, Oct 2 2019, will instruct data scientists and machine learning engineers how to build manage and deploy auto-adaptive machine learning models in production. Save your spot now.

Machine Learning 84

Machine Learning Building Engineering Management 84

KDnuggets

SEPTEMBER 23, 2019

Register now for this webinar, Sep 25 @ 12 PM ET, for a clear approach on how to apply machine learning language technology to massive, unstructured data sets in order to create predictive models of what may be the next “it” ingredient, color, flavor or pack size.

Unstructured Data 83

Unstructured Data Machine Learning Data Technology 83

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems

KDnuggets

SEPTEMBER 23, 2019

Also: Explore the world of Bioinformatics with Machine Learning; My journey path from a Software Engineer to BI Specialist to a Data Scientist; 5 Beginner Friendly Steps to Learn Machine Learning and Data Science with Python; 10 Great Python Resources for Aspiring Data Scientists.

Data Science 82

Data Science BI Software Engineer Software Engineering 82

KDnuggets

SEPTEMBER 25, 2019

Penn State’s fully online data analytics program uniquely prepares students to advance their career in data science. Penn State offers 3 intakes every year and reviews applications on a rolling basis. GMAT or GRE waivers are available to highly qualified candidates. Learn more now.

Data Science 82

Data Science Data Analytics Programming Education 82

KDnuggets

SEPTEMBER 27, 2019

The article contains a brief introduction to various concepts related to Hierarchical clustering algorithm.

Algorithm 106

Algorithm Machine Learning Python 106

KDnuggets

SEPTEMBER 24, 2019

Of all data quality characteristics, we consider consistency and accuracy to be the most difficult ones to measure. Here, we describe the challenges that you may encounter and the ways to overcome them.

Data 76

Data 76

Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network

In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.

Project

KDnuggets

SEPTEMBER 25, 2019

This article covers the beta distribution, and explains it using baseball batting averages.

IT 103

IT 103

KDnuggets

SEPTEMBER 27, 2019

Data mapping is a way to organize various bits of data into a manageable and easy-to-understand system.

Machine Learning 89

Machine Learning Data Systems Management 89

KDnuggets

SEPTEMBER 23, 2019

The new open source framework that brings multi-task learning to conversational agents.

89

89

KDnuggets

SEPTEMBER 24, 2019

AI World Conference & Expo has become the industry’s largest independent business event focused on the state of the practice of AI in the enterprise. Join us in Boston, Oct 23-25. Use the discount code 1968-KDN and SAVE $200.

Coding 54

Coding 54

Speaker: Evelyn Chou

Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.

Business Intelligence

KDnuggets

SEPTEMBER 25, 2019

Python Libraries for Interpretable Machine Learning; Scikit-Learn: A silver bullet for basic machine learning; I wasn't getting hired as a Data Scientist. So I sought data on who is; Which Data Science Skills are core and which are hot/emerging ones?

Machine Learning 46

Machine Learning Python Data Science Data 46

KDnuggets

SEPTEMBER 25, 2019

Learn about unexpected risk of AI applied to Big Data; Study 5 Sampling Algorithms every Data Scientist needs to know; Read how one data scientist copes with his boring days of deploying machine learning; 5 beginner-friendly steps to learn ML with Python; and more.

Algorithm 46

Algorithm Big Data Python Machine Learning 46

Confluent

SEPTEMBER 26, 2019

In the early days, many companies simply used Apache Kafka ® for data ingestion into Hadoop or another data lake. However, Apache Kafka is more than just messaging. The significant difference today is that companies use Apache Kafka as an event streaming platform for building mission-critical infrastructures and core operations platforms. Examples include microservice architectures, mainframe integration, instant payment, fraud detection, sensor analytics, real-time monitoring, and many more—dri

Kafka 21

Kafka SQL BI Hadoop 21

Confluent

SEPTEMBER 23, 2019

Kafka Summit San Francisco is just one week away. Conferences can be busy affairs, so here are some tips on getting the most out of your time there. Plan. Go and check out the schedule. Spend a bit of time familiarising yourself with what sessions you want to get to, and mark them on your calendar. How do you pick which sessions to attend? My advice: diversify!

Kafka 18

Kafka Hadoop Architecture Database 18

Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL

Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.

Government