KDnuggets

MARCH 8, 2022

Some exceptional female researchers and engineers are working on projects to make the world a better place with the help of AI, data science, and machine learning.

Machine Learning 160

Machine Learning Data Science Project Engineering 160

Confluent

MARCH 9, 2022

Imagine your team wants to design a data streaming architecture and you’re in charge of creating the prototype. Within a few minutes, you provision a fully managed Apache Kafka® cluster […].

Kafka 126

Kafka Cloud Architecture Designing 126

Cloudera

MARCH 10, 2022

As an official sponsor of International Women’s Da y, Cloudera is excited to celebrate Women’s History Month and International Women’s Day, and to take up the mantle of this year’s theme #BreakTheBias. . Even in industries where women are underrepresented, like tech, women have made a lot of progress. Progress over many decades has slowly transformed the workplace into an environment where women’s strengths are recognized and valued.

Big Data 118

Big Data BI Data Project 118

Data Engineering Podcast

MARCH 5, 2022

Summary Databases are an important component of application architectures, but they are often difficult to work with. HarperDB was created with the core goal of being a developer friendly database engine. In the process they ended up creating a scalable distributed engine that works across edge and datacenter environments to support a variety of novel use cases.

PostgreSQL 100

PostgreSQL MongoDB Software Engineering Software Engineer 100

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline

KDnuggets

MARCH 9, 2022

Four-day conference offers hundreds of learning and development opportunities in AI, ML, DL, robotics, data science and high performance computing for developers at all levels.

Data Science 159

Data Science Data 159

Teradata

MARCH 11, 2022

This getting started guide describes ‘high-level’ Teradata Vantage connectivity options with the Microsoft Azure Services. Find out more.

98

98

Data Engineering Podcast

MARCH 5, 2022

Summary When you think about selecting a database engine for your project you typically consider options focused on serving multiple concurrent users. Sometimes what you really need is an embedded database that is blazing fast for single user workloads. DuckDB is an in-process database engine optimized for OLAP applications to speed up your analytical queries that meets you where you are, whether that’s Python, R, Java, even the web.

Database 100

Database Data Lake Java Data Engineering 100

KDnuggets

MARCH 7, 2022

Learn these to take any data science project idea from brainstorm to deployment.

Data Science 159

Data Science Project Data 159

Confluent

MARCH 10, 2022

Decentralized architectures continue to flourish as engineering teams look to unlock the potential of their people and systems. From Git, to microservices, to cryptocurrencies, these designs look to decentralization as […].

Architecture 81

Architecture Data Designing Engineering 81

Cloudera

MARCH 11, 2022

With the launch of CDP Public Cloud 7.2.14, Cloudera Streams Messaging for Data Hub deployments has gotten some powerful new features! In this release , the Streams Messaging templates in Data Hub will come with Apache Kafka 2.8 and Cruise Control 2.5 providing new core features and fixes. KConnect has been added and gains additional capabilities with new connectors and Stateless Apache NiFi capabilities which can run NiFi Flows as connectors.

Cloud 107

Cloud Kafka Utilities Database 107

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

Teradata

MARCH 8, 2022

In honor of Women's History Month & Int'l Women's Day, we are spotlighting Hillary Ashton, Teradata's Chief Product Officer, as she looks back on her career and gives advice to young women in tech.

59

59

KDnuggets

MARCH 10, 2022

Good quality data becomes imperative and a basic building block of an ML pipeline. The ML model can only be as good as its training data.

Machine Learning 158

Machine Learning Data Building IT 158

Confluent

MARCH 8, 2022

If you’re reading this, it’s likely because you are leveraging (or considering) Apache Kafka® in your organization—especially as it has become the de facto standard for data streaming. Adopted by […].

Kafka 64

Kafka Management Data IT 64

Cloudera

MARCH 7, 2022

March 8 marks International Women’s Day and as we celebrate the accomplishments of dynamic women across the world, I sat across from one such Clouderan, Vicki Zingiris, Director of Value-Based Services. We discussed important initiatives at Cloudera, the influence that Martial Arts has had on how she leads, collaborates, and mentors, and concluded with some valuable advice for women in the workforce. .

Professional Services 88

Professional Services Consulting Education Programming 88

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

Rockset

MARCH 10, 2022

This is the first post in a series by Rockset's CTO Dhruba Borthakur on Designing the Next Generation of Data Systems for Real-Time Analytics. We'll be publishing more posts in the series in the near future, so subscribe to our blog so you don't miss them! Posts published so far in the series: Why Mutability Is Essential for Real-Time Data Analytics Handling Out-of-Order Data in Real-Time Analytics Applications Handling Bursty Traffic in Real-Time Analytics Applications SQL and Complex Queries A

Data Analytics 52

Data Analytics Data Warehouse MySQL Medical 52

KDnuggets

MARCH 8, 2022

Also: Calculus: The hidden building block of machine learning; Decision Tree Algorithm, Explained; Telling a Great Data Story: A Visualization Decision Tree; The Complete Collection of Data Science Cheat Sheets – Part 1.

Data Science 158

Data Science Algorithm Machine Learning Data 158

Monte Carlo

MARCH 10, 2022

For the second consecutive year, Monte Carlo was today named to the Enterprise Tech 30 (ET30), an exclusive list of the most promising companies in enterprise technology, as determined by some of the world’s top venture capitalists. Sponsored by Wing Venture Capital and Nasdaq, more than 15,000 private venture-backed companies are considered. The list is then narrowed to 10 early stage ($25 million or less raised), 10 mid stage ($25 to $100 million raised), and 10 late stage ($100 million or mor

Technology 52

Technology Data 52

Cloudera

MARCH 9, 2022

Over the past decade, Cloudera has matured to become a leading-edge technology company, supporting a diverse range of customers, across the globe. At Cloudera, we are passionate about helping our customers identify opportunities for innovation and growth, enabling them to accelerate their digital transformation, and aiding them to solve some of societies’ largest challenges.

Programming 15

Programming Technology Education Management 15

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

dbt Developer Hub

MARCH 9, 2022

Special Thanks: Emilie Schario, Matt Winkler dbt has done a great job of building an elegant, common interface between data engineers, analytics engineers, and any data-y role, by uniting our work on SQL. This unification of tools and workflows creates interoperability between what would normally be distinct teams within the data organization. I like to call this interoperability a “baton pass.

Machine Learning 52

Machine Learning Raw Data SQL Python 52

KDnuggets

MARCH 11, 2022

Share the interactive code blocks to impress your colleagues or post it on social media.

Coding 157

Coding Media Data IT 157

Monte Carlo

MARCH 10, 2022

Monte Carlo’s Barr Moses sat down with Snowflake Director of Product Management Chris Child to talk about building data platforms at scale, how awesome data teams approach data quality, the role of data observability tools in the modern data stack, and more. To put it simply, to understand modern data engineering, you need to understand Snowflake. And as your data platform becomes productized, you need to get serious about data quality.

Engineering 52

Engineering Management Raw Data Scala 52

Rockset

MARCH 8, 2022

Photo by Adil from Pexels I’ve found that every startup today fits into one of two categories: A solution that focuses mainly on enhancing a larger solution or platform. A solution that has arrived in the wake of those that have come before. The first category is a symbiotic relationship — think of the shark and remora fish. Even though there is a mutual benefit, one of the parties has a more significant dependency on the other.

MongoDB 52

MongoDB NoSQL Unstructured Data Recruitment 52

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineering

KDnuggets

MARCH 7, 2022

In this article, you will learn to export your models and use them outside a Jupyter Notebook environment. You will build a simple web application that is able to feed user input into a machine learning model, and display an output prediction to the user.

Machine Learning 150

Machine Learning Building 150

KDnuggets

MARCH 10, 2022

Read some tips on getting organized when it comes to working with data.

Data 138

Data IT 138

KDnuggets

MARCH 9, 2022

It takes time and considerable resources to collect, document, and clean data before it can be used. But there is a way to address this challenge – by using synthetic data.

Machine Learning 126

Machine Learning Data IT 126

KDnuggets

MARCH 11, 2022

To forecast costs for AI systems, it can be useful to talk about their “level” just like SAE has levels for self-driving cars. Adopting a level system can help organizations plan and prepare for AI systems that scale in complexity over time.

Systems 123

Systems IT 123

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

KDnuggets

MARCH 11, 2022

NVIDIA BlueField DPUs provide on-demand, simple and secure high-performance computing and AI services.

Cloud 123

Cloud 123

KDnuggets

MARCH 8, 2022

In the majority of companies, the executives in charge of data science and the decision-making process using data science, have little or no education or understanding in actual data science. Where does this leave you, the data scientist?

Data Science 120

Data Science Education Data Process 120

KDnuggets

MARCH 9, 2022

This article discusses 2 levels of data science learning, and the amount of time that will need to go into each. From 6 months to 4 years, this write-up covers a number of skills and how long it takes to acquire them.

Data Science 116

Data Science IT Data 116

KDnuggets

MARCH 9, 2022

Here are some lessons inspired by a recent panel the author moderated about how data scientists can help put equity into practice.

Data Science 112

Data Science Data 112

Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage

When tasked with building a fundamentally new product line with deeper insights than previously achievable for a high-value client, Ben Epstein and his team faced a significant challenge: how to harness LLMs to produce consistent, high-accuracy outputs at scale. In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation m

Software Engineer