KDnuggets

JANUARY 13, 2022

Learn how to query your Pandas DataFrames using the standard SQL SELECT statement, seamlessly from within your Python code.

SQL 160

SQL Python Coding 160

Confluent

JANUARY 11, 2022

Whether you’re a seasoned Apache Kafka® developer or just getting started you’re likely to hit a snag at some point or another—either in configuring and understanding your clients or setting […].

Kafka 138

Kafka 138

Marc Lamberti

JANUARY 9, 2022

Airflow TaskGroups have been introduced to make your DAG visually cleaner and easier to read. They are meant to replace SubDAGs which was the historic way of grouping your tasks. The problem with SubDAGs is that they are much more than that. They bring a lot of complexity as you need to create a DAG in a DAG, import the SubDagOperator which is in fact a sensor, define the parameters properly, and so on.

Coding 130

Coding Python Process Management 130

Netflix Tech

JANUARY 11, 2022

Martin Tingley with Wenjing Zheng , Simon Ejdemyr , Stephanie Lane , Colin McFarland , Andy Rhines , Sophia Liu , Mihir Tendulkar , Kevin Mercurio , Veronica Hannan , Ting-Po Lee Earlier posts in this series covered the basics of A/B tests ( Part 1 and Part 2 ), core statistical concepts ( Part 3 and Part 4 ), and how to build confidence in decisions based on A/B test results ( Part 5 ).

Data Science 98

Data Science Data Engineering Algorithm 98

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

KDnuggets

JANUARY 13, 2022

Focusing on the important concepts for data scientists.

SQL 160

SQL Data Science Data 160

Confluent

JANUARY 12, 2022

We’re pleased to announce ksqlDB 0.23.1! This release allows you to now perform pull queries on streams, which makes it much easier to find a given record in a topic. […].

IT 98

IT Process 98

Netflix Tech

JANUARY 13, 2022

By Vikram Srivastava and Marcelo Mayworm Netflix has one of the most complex data platforms in the cloud on which our data scientists and engineers run batch and streaming workloads. As our subscribers grow worldwide and Netflix enters the world of gaming , the number of batch workflows and real-time data pipelines increases rapidly. The data platform is built on top of several distributed systems, and due to the inherent nature of these systems, it is inevitable that these workloads run into fa

Kafka 97

Kafka Big Data Data Machine Learning 97

KDnuggets

JANUARY 13, 2022

Any modern company of any significant size around the world has a data science department, and a data engineer at one company might have the same responsibilities as a marketing scientist at another company. Data science jobs are not well-labeled, so make sure to cast a wide net.

Data Science 160

Data Science Data Engineering Data Engineer Data 160

DataKitchen

JANUARY 10, 2022

Cambridge, Mass. – June 16, 2021. Today, DataKitchen announced the release of the latest book in its groundbreaking DataOps series, Recipes for DataOps Success: The Complete Guide to An Enterprise DataOps Transformation. This book follows on the heels of its successful precursor, The DataOps Cookbook , which has been downloaded more than 14,000 times and counting.

Programming 69

Programming Certification Pharmaceutical Data Analytics 69

Wolt

JANUARY 13, 2022

Data Platforms are an essential part of modern businesses. They enable reporting, low friction decision making, and if used correctly, can power very advanced data products in a compliant and traceable manner. Let us take you from the role of data at Wolt, through the data journey we’ve had so far and finish with a peek into what the future of this discipline may look like.

Data 52

Data Data Engineering Data Engineer Engineering 52

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

ProjectPro

JANUARY 13, 2022

Have you heard of the Inshorts smartphone app? It is a cutting-edge news app that embodies news stories into a concise 60-word summary. Applications like Inshorts allow you to avoid reading long articles by generating a clear and concise summary. An average individual uses Google Search more than three times a day. Thanks to Featured Snippets, or Knowledge Panels, you receive better results for your search queries.

Python 52

Python Algorithm Deep Learning Data Science 52

KDnuggets

JANUARY 12, 2022

Using one or two performance metrics seems sufficient to claim that your ML model is good — chances are that it’s not.

Machine Learning 160

Machine Learning 160

Teradata

JANUARY 13, 2022

2021 was the year of expanding 5G coverage, building out 5G use cases and the start of the migration to 5G Stand Alone. What will the year 2022 bring to the Telco industry?

Telecommunication 52

Telecommunication Building 52

Grouparoo

JANUARY 11, 2022

When it comes to storing large volumes of data, a simple database will be impractical due to the processing and throughput inefficiencies that emerge when managing and accessing big data. This article looks at the options available for storing and processing big data, which is too large for conventional databases to handle. There are two main options available, a data lake and a data warehouse.

Data Lake 52

Data Lake Data Warehouse Unstructured Data Raw Data 52

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

ProjectPro

JANUARY 11, 2022

“Machine Learning” is one of the most trending buzzwords. It is predominant in every industry sector as it empowers various organizations with innovative solutions to automate and increase the efficacy of products by reducing human intervention. You might have heard about the applications of weather forecasting, spam classification, or stock price prediction applications, so what exactly do these applications use ?

Machine Learning 52

Machine Learning Algorithm Datasets Food 52

KDnuggets

JANUARY 11, 2022

Finding the data you need is hard. So why not fake it?

Datasets 160

Datasets IT Data Data Science 160

Elder Research

JANUARY 10, 2022

The post The Customer is Always Wrong – Along with the Rest of Us appeared first on Elder Research.

52

52

RudderStack

JANUARY 11, 2022

In this update we cover our latest Destination UI feature, our new VDM for Klaviyo, new SDKs and destination integrations, and more.

40

40

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineer

ProjectPro

JANUARY 10, 2022

Time series data can be defined as a sequence of data points that need to be seen with respect to the time stamp for each sample. Data samples are indexed by the timestamps or are highly dependent on them in time series. Data for weather forecasting, stock price prediction, user subscriptions, or sales patterns are some examples of time series data.

Python 52

Python Building Machine Learning Datasets 52

KDnuggets

JANUARY 10, 2022

The author shares their experience of almost winning the competition and the things they have learned from the failures. Learn more about the WiDS Datathon and tips on winning the next challenge.

Data Science 160

Data Science Data 160

KDnuggets

JANUARY 14, 2022

Open source Redis is being increasingly used in Machine Learning, but running it on Colab is different compared to on your local machine or with Docker. Read on for a 2-step tutorial on how to do it.

Machine Learning 159

Machine Learning IT Data Science Data 159

KDnuggets

JANUARY 11, 2022

Learn how to build feedforward neural networks that are interpretable by design using PyTorch.

Designing 156

Designing Building Machine Learning 156

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

KDnuggets

JANUARY 14, 2022

Read the second article in this series on Transfer Learning, and learn how to apply it to Image Recognition and Natural Language Processing.

Process 156

Process IT Machine Learning 156

KDnuggets

JANUARY 12, 2022

It’s crucial to investigate the reasons why data science teams require context, consistency, and secure collaboration of their data to ensure data science success. Let's quickly examine each of these requirements so that we can better understand what data science success moving forward may look like.

Data Science 151

Data Science Data 151

KDnuggets

JANUARY 12, 2022

These are some of the most critical questions that I think are important to consider when selecting the next job.

Data Science 142

Data Science Data 142

KDnuggets

JANUARY 12, 2022

The new Online MS in Business Analytics for Managers from Simon Business School is the latest advancement in analytically rigorous, leadership-focused education designed to help managers and aspiring managers prepare for the future of business-wherever it may lead. Applications are being accepted now and the first 14-month class will begin August 2022.

Management 137

Management Education Designing IT 137

Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage

When tasked with building a fundamentally new product line with deeper insights than previously achievable for a high-value client, Ben Epstein and his team faced a significant challenge: how to harness LLMs to produce consistent, high-accuracy outputs at scale. In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation m

Software Engineering

KDnuggets

JANUARY 10, 2022

Also: Why are More Developers Using Python for Their Machine Learning Projects?; 3 Tools to Track and Visualize the Execution of Your Python Code; SQL Interview Questions for Experienced Professionals; Deliver a Killer Presentation in Data Science Interviews.

Machine Learning 147

Machine Learning Python Data Science SQL 147

KDnuggets

JANUARY 10, 2022

At the end of the day, the value a data scientist provides to an organization lies in their ability to apply data to real-world use cases.

Data Science 154

Data Science Data 154

KDnuggets

JANUARY 12, 2022

Is Data Science a Dying Career?; Why Do Machine Learning Models Die In Silence?; SQL Interview Questions for Experienced Professionals; Deliver a Killer Presentation in Data Science Interviews; What is Transfer Learning?

Machine Learning 151

Machine Learning Data Science SQL Data 151