KDnuggets

NOVEMBER 19, 2021

Can artificial intelligence be rid of bias? This is an important question, and it’s equally important that we look in the right place for the answer.

160

160

Azure Data Engineering

NOVEMBER 15, 2021

In one of the previous posts, we discussed how we can use Validation activity to design the Pipeline to wait for a scheduled time and retry. There is another way to introduce a delay in the Pipeline. Wait activity can be used to pause the execution of the Pipeline for a fixed amount of time. Sometimes, we come across scenarios where we would like the execution for the Pipeline to be Paused for some time but not cancelled.

Data 130

Data Designing 130

Cloudera

NOVEMBER 17, 2021

It’s no secret that Data Scientists have a difficult job. It feels like a lifetime ago that everyone was talking about data science as the sexiest job of the 21st century. Heck, it was so long ago that people were still meeting in person! Today, the sexy is starting to lose its shine. There’s recognition that it’s nearly impossible to find the unicorn data scientist that was the apple of every CEO’s eye in 2012.

Machine Learning 128

Machine Learning Algorithm Data Science Retail 128

Confluent

NOVEMBER 19, 2021

Imagine that you have real-time data about what’s happening in the stock market, and you want to support a large number of customized dashboards displaying the data as it comes […].

SQL 105

SQL IT Data Process 105

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

KDnuggets

NOVEMBER 19, 2021

The terms ‘data science’ and ‘machine learning’ are often used interchangeably. But while they are related, there are some glaring differences, so let’s take a look at the differences between the two disciplines, specifically as it relates to programming.

Machine Learning 160

Machine Learning Data Science Coding Programming 160

Data Engineering Podcast

NOVEMBER 14, 2021

Summary The most important gauge of success for a data platform is the level of trust in the accuracy of the information that it provides. In order to build and maintain that trust it is necessary to invest in defining, monitoring, and enforcing data quality metrics. In this episode Michael Harper advocates for proactive data quality and starting with the source, rather than being reactive and having to work backwards from when a problem is found.

Data Warehouse 100

Data Warehouse BI Data Workflow Data 100

Confluent

NOVEMBER 16, 2021

Some call it a challenge. Others call it a community. Whatever you call it, 100 Days Of Code is a bunch of fun and a great learning experience that helps […].

Coding 104

Coding Kafka IT 104

KDnuggets

NOVEMBER 19, 2021

Want to know the difference between distributed and federated learning? Read this article to find out.

Algorithm 160

Algorithm Systems 160

DataKitchen

NOVEMBER 18, 2021

For several years now, the elephant in the room has been that data and analytics projects are failing. Gartner estimated that 85% of big data projects fail. Data from New Vantage partners showed that the number of data-driven organizations has actually declined to 24% from 37% several years ago and that only 29% of organizations are achieving transformational outcomes from their data. .

Data Engineer 98

Data Engineer Data Engineering Engineering Government 98

Cloudera

NOVEMBER 16, 2021

Introduction. With the general availability of Cloudera DataFlow for the Public Cloud (CDF-PC) , our customers can now self-serve deployments of Apache NiFi data flows on Kubernetes clusters in a cost effective way providing auto scaling, resource isolation and monitoring with KPI-based alerting. You can find more information in this release announcement blog post and in this technical deep dive blog post.

Google Cloud 115

Google Cloud AWS Cloud Computing Kafka 115

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Netflix Tech

NOVEMBER 15, 2021

Martin Tingley with Wenjing Zheng , Simon Ejdemyr , Stephanie Lane , Michael Lindon , and Colin McFarland This is the fifth post in a multi-part series on how Netflix uses A/B tests to inform decisions and continuously innovate on our products. Need to catch up? Have a look at Part 1 (Decision Making at Netflix), Part 2 (What is an A/B Test?), Part 3 (False positives and statistical significance), and Part 4 (False negatives and power).

Building 86

Building Utilities Designing Coding 86

KDnuggets

NOVEMBER 18, 2021

Natural language processing research and applications are moving forward rapidly. Several trends have emerged on this progress, and point to a future of more exciting possibilities and interesting opportunities in the field.

Process 160

Process 160

Confluent

NOVEMBER 18, 2021

We’re pleased to announce ksqlDB 0.22.0! This release includes source streams and source tables as well as improved pull query (for key-range predicates) and push query performance. All of these […].

Process 80

Process 80

Cloudera

NOVEMBER 15, 2021

The word “data” is ubiquitous in narratives of the modern world. And data, the thing itself, is vital to the functioning of that world. This blog discusses quantifications, types, and implications of data. If you’ve ever wondered how much data there is in the world, what types there are and what that means for AI and businesses, then keep reading! Quantifications of data.

Unstructured Data 96

Unstructured Data Pipeline-centric Database-centric Entertainment 96

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

AltexSoft

NOVEMBER 17, 2021

If you’ve ever been to a bookstore, you probably know the dilemma of the book location. Say you’re looking for “Atlas Shrugged”, and you know it’s a mix of science fiction, mystery, and romance genres. Now, which bookshelf will you go for to find it? Should it be on the science fiction or on the romance shelf? The problem of document classification pertains to the library, information, and computer sciences.

Machine Learning 75

Machine Learning Insurance Medical Healthcare 75

KDnuggets

NOVEMBER 17, 2021

We describe types of recommender systems, more specifically, algorithms and methods for content-based systems, collaborative filtering, and hybrid systems.

Systems 160

Systems Algorithm Engineering 160

DataKitchen

NOVEMBER 19, 2021

Learn how a DataOps Process Hub enables Business Analysts to rapidly answer stakeholders' analytic questions without waiting on the centralized IT Team. The post Solve the Analytics Last-Mile Problem with a DataOps Process Hub first appeared on DataKitchen.

Process 52

Process Business Analyst IT 52

Cloudera

NOVEMBER 18, 2021

With so many impactful and innovative projects being carried out by our customers using the Cloudera platform, selecting the winners of our annual Data Impact Awards (DIA) is never an easy task. Not ones to shy away from a challenge, our expert judges have deliberated and combed through the finalist entries, identifying the customers who are leading industry change and inspiring peers with their data achievements.

Banking 82

Banking Data Lake Telecommunication Data 82

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

ProjectPro

NOVEMBER 18, 2021

Data science has been a trending buzzword in recent times. With wide applications in various sectors like healthcare, education, retail, transportation, media, and banking -data science applications are at the core of pretty much every industry out there. The possibilities are endless: analysis of frauds in the finance sector or the personalization of recommendations on eCommerce businesses.

Data Science 52

Data Science Project Food Pharmaceutical 52

KDnuggets

NOVEMBER 16, 2021

The field of computer vision has seen the development of very powerful applications leveraging machine learning. These projects will introduce you to these techniques and guide you to more advanced practice to gain a deeper appreciation for the sophistication now available.

Project 160

Project Machine Learning 160

Teradata

NOVEMBER 17, 2021

Many Teradata customers are interested in integrating Vantage with Microsoft Azure first party services. This guide will help you connect Teradata QueryGrid to Azure HDInsight.

52

52

Grouparoo

NOVEMBER 16, 2021

For organizations that manage large volumes of data, leveraging maximum value from the information buried in the data can be a challenge. Breaking silos and collating data into a coherent set of information for processing will yield business benefits. Still, this is only possible once information is in a form enabling the application of analytical techniques.

Data Mining 52

Data Mining Raw Data ETL Tools Data 52

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

ProjectPro

NOVEMBER 17, 2021

Emotions are essential, not only in personal life but in business as well. How your customers and target audience feel about your products or brand provides you with the context necessary to evaluate and improve the product, business, marketing, and communications strategy. Sentiment analysis or opinion mining helps researchers and companies extract insights from user-generated social media and web content.

Coding 52

Coding Project Entertainment Datasets 52

KDnuggets

NOVEMBER 16, 2021

Learn how to effectively communicate your work.

Data 160

Data Data Science 160

Preset

NOVEMBER 15, 2021

This post showcases how we at Preset achieve zero downtime web application deployment using Kubernetes (AWS EKS) with zero failed requests.

AWS 52

AWS 52

Grouparoo

NOVEMBER 15, 2021

Application Programming Interfaces or APIs are an integral part of modern software development and enable a wide variety of applications and workflows. Enterprises are becoming increasingly reliant on APIs to effectively connect with partners and customers. APIs come in an array of types and protocols that work great in different scenarios. In this article, we’ll examine the different types of APIs used in software development today.

Architecture 52

Architecture Database Java Designing 52

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

ProjectPro

NOVEMBER 15, 2021

If you are associated with the tech world, you would have unquestionably come across the term “Open-source.” From month-long open-source contribution programs for students to recruiters preferring candidates based on their contribution to open-source projects or tech-giants deploying open-source software in their organization, open-source projects have successfully set their mark in the industry.

Big Data 52

Big Data Project Metadata Programming Language 52

KDnuggets

NOVEMBER 17, 2021

Faker is a Python library that generates fake data to supplement or take the place of real world data. See how it can be used for data science.

Python 159

Python Data Science Data IT 159

Afterpay Tech

NOVEMBER 14, 2021

Photo by Joshua Sortino on Unsplash By: Dorien Koelemeijer Just like the cliché says, security is only as strong as the weakest link. We know that a single Identity and Access Management (IAM) misconfiguration in our AWS environment can lead to compromise of our entire cloud environment. It’s the task of the Security Team at Afterpay to manage this risk, while at the same time making sure that engineers are not slowed down in their everyday tasks.

AWS 52

AWS Cloud Engineering Coding 52

dbt Developer Hub

NOVEMBER 14, 2021

Hi there, Before I get to the goods, I just wanted to quickly flag that Coalesce is less than 3 weeks away! ? If you had to choose just ONE of the 60+ sessions on tap, consider Tristan's keynote with A16z's Martin Casado. It has two of my favorite elements: 1) Spice ?️ 2) Not-actually-about-us ? Martin and Tristan will discuss something we've all probably considered with the latest wave of innovation (and funding) in our space: Is the modern data stack just another wave in a long string of trend

Cloud 52

Cloud Engineering Project Data Engineer 52

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud