KDnuggets

NOVEMBER 29, 2022

First steps to learning data science & machine learning are the foundations.

Machine Learning 160

Machine Learning Data Science Data 160

The Pragmatic Engineer

NOVEMBER 17, 2022

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one out of five topics in today’s subscriber-only The Scoop issue. To get this newsletter every week, subscribe here. This article was updated in December 2022. In the midst of gloomy news about hiring freezes and layoffs, let's highlight companies which are growing  and hiring.

Software Engineer 146

Software Engineer Software Engineering Engineering Media 146

Confluent

NOVEMBER 29, 2022

Apache Kafka’s Streams API embeds Machine Learning into any app or microservice (Java, Docker, Kubernetes, etc.) to add business value.

Machine Learning 145

Machine Learning Kafka Java Building 145

Christophe Blefari

NOVEMBER 25, 2022

Capturing the news ( credits ) Hello you, I hope this data news finds you well. Time flies to be honest. I've launched in a rush an Advent of Data. The goal is simple, in December: 24 data people will produce 24 data gems. Every day a new piece of content will be release on a dedicated website. If you wanna join the initiative please reply, we are still looking for a few slots to be filled in.

Data 130

Data Java Data Engineering Data Engineer 130

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

Marc Lamberti

NOVEMBER 23, 2022

DuckDB is an in-process OLAP DBMS written in C++ blah blah blah, too complicated. Let’s start simple, shall we? DuckDB is the SQLite for Analytics. It has no dependencies, is extremely easy to set up, and is optimized to perform queries on data. In this hands-on tutorial, you will learn what DuckDB is, how to use it, and why it is essential for you.

Datasets 130

Datasets Python SQL Database 130

Data Engineering Podcast

NOVEMBER 20, 2022

Summary The problems that are easiest to fix are the ones that you prevent from happening in the first place. Sifflet is a platform that brings your entire data stack into focus to improve the reliability of your data assets and empower collaboration across your teams. In this episode CEO and founder Salma Bakouk shares her views on the causes and impacts of "data entropy" and how you can tame it before it leads to failures.

Data Lake 130

Data Lake Data Ingestion MongoDB MySQL 130

The Pragmatic Engineer

NOVEMBER 24, 2022

Originally published on 24 November 2022. 👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one out of five topics in today’s subscriber-only The Scoop issue. To get this newsletter every week, subscribe here. I was really hoping to not report anything more about Twitter, and that software engineers at the company would get space to heal after the traumatic events, and to focus on building the product.

Software Engineer 130

Software Engineer Software Engineering Engineering Coding 130

Confessions of a Data Guy

NOVEMBER 16, 2022

Ok, so I don’t really mean all that. Or do I? I have no idea what the future holds. Sometimes it’s easy to pick out the winners, like Databricks and Snowflake, you can see, feel, and taste the results of those data products, a delicious and delectable bounty to feast upon. Other things are harder […] The post A Diatribe against Data Contracts and their Abuses. appeared first on Confessions of a Data Guy.

Data 130

Data IT Data Engineering Data Engineer 130

Jesse Anderson

NOVEMBER 15, 2022

My guest this week is Kulani Likotsi , the Head of Data Management and Data Governance at one of the four biggest banks in Africa. She’s had a rising career journey going from an analyst, to a Business Intelligence developer, to the data warehouse team, to the data governance team. I was impressed with Kulani’s volunteer spirit. Whenever there was a need, she volunteered.

Data Governance 130

Data Governance Government Banking Business Intelligence 130

Christophe Blefari

NOVEMBER 11, 2022

Mastodon and Hadoop are on a boat. ( credits ) Hey you, 11th of November was usually off for me. Since I've started my freelancing activities I don't really follow the usual calendar, working whenever I need/want. I mainly work 3 to 4 days a week. Which is awesome but it has a major drawback I never took a break longer than 1 week. Which, yeah, kinda sucks.

BI 130

BI Data Warehouse Data Database 130

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

Data Engineering Podcast

NOVEMBER 20, 2022

Summary The majority of blog posts and presentations about data engineering and analytics assume that the consumers of those efforts are internal business users accessing an environment controlled by the business. In this episode Ian Schweer shares his experiences at Riot Games supporting player-focused features such as machine learning models and recommeder systems that are deployed as part of the game binary.

Systems 130

Systems Metadata Data Pipeline MongoDB 130

KDnuggets

NOVEMBER 23, 2022

Yellowbrick for creating machine learning plots with less code.

Machine Learning 160

Machine Learning Coding Data Science Data 160

The Pragmatic Engineer

NOVEMBER 16, 2022

I get a lot of scoop sent by readers (thank you!). Sadly, in 2022, a good part of the scoop is about companies laying off people. Some of this scoop has not been reported before. I don't want to broadcast layoffs on Twitter or LinkedIn continuously, but also don't want this information to be lost. This page collects scoops I receive, some of which might not have been reported elsewhere.

Healthcare 130

Healthcare Software Engineer Software Engineering Data Workflow 130

Confessions of a Data Guy

NOVEMBER 5, 2022

There are probably few things in life that will strike more fear and tumult in the heart of the Data Engineer than historical loads. You know, on the surface it seems like such an innocent thing. How could it possibly be, just take a bunch of data stored somewhere and shove it into a table. […] The post Introduction to Historical Loads – for Data Engineers. appeared first on Confessions of a Data Guy.

Data Engineering 130

Data Engineering Data Engineer Engineering Data 130

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

Jesse Anderson

NOVEMBER 2, 2022

My guest this week is Erick Webbe , Head of Data Science at bol.com. Bol.com is the biggest online retailer in northwestern Europe, serving about 12 million customers, as a general retailer similar to Amazon.com. Erick has a Master’s degree in Applied Physics. His background in physics forms a basis for his philosophy on life and work. That’s a “philosophy that I still apply to my work every single day […] we think about how we can best help them overcome that problem or solve it, and then

Generalist 130

Generalist Data Science Building Retail 130

Netflix Tech

NOVEMBER 11, 2022

By Soheil Esmaeilzadeh , Negin Salajegheh , Amir Ziai , Jeff Boote Introduction Streaming services serve content to millions of users all over the world. These services allow users to stream or download content across a broad category of devices including mobile phones, laptops, and televisions. However, some restrictions are in place, such as the number of active devices, the number of streams, and the number of downloaded titles.

Machine Learning 128

Machine Learning Digital Media Datasets Media 128

Data Engineering Podcast

NOVEMBER 13, 2022

Summary Building data products is an undertaking that has historically required substantial investments of time and talent. With the rise in cloud platforms and self-serve data technologies the barrier of entry is dropping. Shane Gibson co-founded AgileData to make analytics accessible to companies of all sizes. In this episode he explains the design of the platform and how it builds on agile development principles to help you focus on delivering value.

Building 130

Building Metadata MongoDB MySQL 130

KDnuggets

NOVEMBER 18, 2022

The Pandas library is core to any Data Science work in Python. This introduction will walk you through the basics of data manipulating, and features many of Pandas important features.

Data Science 160

Data Science Python Data 160

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineer

The Pragmatic Engineer

NOVEMBER 10, 2022

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one out of five topics in today’s subscriber-only The Scoop issue. To get this newsletter every week, subscribe here. Last Thursday, I covered the turmoil at Twitter , of how people worked long hours through the weekend and how most expected layoffs of about 50%.

Software Engineer 130

Software Engineer Software Engineering MongoDB Engineering 130

Confluent

NOVEMBER 30, 2022

What is stream processing, or complex event processing (CEP), and how does it work? Learn about real-time data and event stream analytics in this tutorial.

Process 126

Process Data IT 126

Teradata

NOVEMBER 17, 2022

"Doing more with less” is a familiar refrain echoing through the halls of many organizations. To answer this call, businesses are searching for efficiency gains & turning to data to unlock savings.

Data 98

Data 98

Netflix Tech

NOVEMBER 17, 2022

by Christos G. Bampis , Li-Heng Chen and Zhi Li When you are binge-watching the latest season of Stranger Things or Ozark, we strive to deliver the best possible video quality to your eyes. To do so, we continuously push the boundaries of streaming video quality and leverage the best video technologies. For example, we invest in next-generation, royalty-free codecs and sophisticated video encoding optimizations.

Media 120

Media Architecture Algorithm Designing 120

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Data Engineering Podcast

NOVEMBER 6, 2022

Summary A lot of the work that goes into data engineering is trying to make sense of the "data exhaust" from other applications and services. There is an undeniable amount of value and utility in that information, but it also introduces significant cost and time requirements. In this episode Nick King discusses how you can be intentional about data creation in your applications and services to reduce the friction and errors involved in building data products and ML applications.

Building 130

Building IT Metadata MongoDB 130

KDnuggets

NOVEMBER 16, 2022

While different ways to learn Data Science for the first time exist, the approach that works for you should be based on how you learn best. One powerful method is to evolve your learning from simple practice into complex foundations, as outlined in this learning path recommended by a physicist who turned into a Data Scientist.

Data Science 160

Data Science IT Data 160

The Pragmatic Engineer

NOVEMBER 3, 2022

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one out of six topics in today’s subscriber-only The Scoop issue. To get this newsletter every week, subscribe here. On Wednesday, 26 October, Elon Musk entered Twitter’s headquarters in San Francisco with a sink, marking his arrival at the company he’d just bought.

Software Engineer 130

Software Engineer Software Engineering Coding Media 130

Confluent

NOVEMBER 29, 2022

An in-depth analysis of the various ways to join streams and tables in Apache Kafka's Streams API.

Kafka 117

Kafka Process 117

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Teradata

NOVEMBER 30, 2022

Teradata has joined the Amazon SageMaker Ready Program which differentiates Teradata as an AWS Partner Network member with a product that works with Amazon SageMaker & fully supports AWS customers.

Programming 97

Programming Designing AWS 97

Netflix Tech

NOVEMBER 15, 2022

By: Peter Cioni (Netflix), Alex Schworer (Netflix), Mac Moore (Conductor Tech.), Rachel Kelley (AWS), Ranjit Raju (AWS) Rendering is core to the the VFX process VFX studios around the world create amazing imagery for Netflix productions. Nearly every show that is produced today includes digital visual effects, from the creatures in Stranger Things , to recreating historic London in Bridgerton.

Cloud 116

Cloud Entertainment AWS Portfolio 116

Data Engineering Podcast

NOVEMBER 6, 2022

Summary Despite the best efforts of data engineers, data is as messy as the real world. Entity resolution and fuzzy matching are powerful utilities for cleaning up data from disconnected sources, but it has typically required custom development and training machine learning models. Sonal Goyal created and open-sourced Zingg as a generalized tool for data mastering and entity resolution to reduce the effort involved in adopting those practices.

MongoDB 130

MongoDB MySQL Scala Machine Learning 130

KDnuggets

NOVEMBER 16, 2022

Knowing git is no longer an option for data professionals. Grab this handy reference sheet now and make sure you know how to git the job done.

Data Science 160

Data Science Data Programming 160

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data