KDnuggets

NOVEMBER 26, 2021

Learn to use custom Named Entity Recognition and Relation Extraction models.

Building 160

Building 160

Azure Data Engineering

NOVEMBER 21, 2021

During some scenarios in Azure Data Factory, we may want to intentionally stop the execution of the pipeline. An example could be when we want to check the existence of a file or folder using Get Metadata activity. We may want to fail the pipeline if the file/folder does not exist. To achieve this, we could use the Fail Activity. Invoking the Fail Activity ensures that the pipeline execution will be stopped.

Metadata 130

Metadata Data Utilities Coding 130

Teradata

NOVEMBER 22, 2021

Teradata's Martin Willcox recently passed 17 years at Teradata and a quarter of a century in the industry. Here are the ten things he's learned about data analytics in those 20-odd years.

Data Analytics 111

Data Analytics Data 111

Cloudera

NOVEMBER 25, 2021

Advances in the performance and capability of Artificial Intelligence (AI) algorithms has led to a significant increase in adoption in recent years. In a February 2021 report by IDC, they estimate that world-wide revenues from AI will grow by 16.4% in 2021 to USD $327 billion. Furthermore, AI adoption is becoming increasingly widespread and not just concentrated within a small number of organisations.

Government 104

Government Insurance Algorithm Machine Learning 104

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

KDnuggets

NOVEMBER 23, 2021

Notes from Fluent Python by Luciano Ramalho.

Python 160

Python Programming 160

Data Engineering Podcast

NOVEMBER 20, 2021

Summary The technology for scaling storage and processing of data has gone through massive evolution over the past decade, leaving us with the ability to work with massive datasets at the cost of massive complexity. Nick Schrock created the Dagster framework to help tame that complexity and scale the organizational capacity for working with data. In this episode he shares the journey that he and his team at Elementl have taken to understand the state of the ecosystem and how they can provide a f

Data Warehouse 100

Data Warehouse Data Lake BI Business Intelligence 100

Cloudera

NOVEMBER 23, 2021

Concepts. What is Cloudera Operational Database (COD)? Operational Database is a relational and non-relational database built on Apache HBase and is designed to support OLTP applications, which use big data. The operational database in Cloudera Data Platform has the following components: . Apache Phoenix provides a relational model facilitating massive scalability.

Database 96

Database Non-relational Database NoSQL Government 96

KDnuggets

NOVEMBER 23, 2021

Sure, we all make mistakes -- which can be a bit more painful when we are trying to get hired -- so check out these typical errors applicants make while answering SQL questions during data science interviews.

SQL 160

SQL Data Science Data 160

Data Engineering Podcast

NOVEMBER 20, 2021

Summary One of the perennial challenges posed by data lakes is how to keep them up to date as new data is collected. With the improvements in streaming engines it is now possible to perform all of your data integration in near real time, but it can be challenging to understand the proper processing patterns to make that performant. In this episode Ori Rafael shares his experiences from Upsolver and building scalable stream processing for integrating and analyzing data, and what the tradeoffs are

Data Lake 100

Data Lake Data Integration Lambda Architecture Process 100

AltexSoft

NOVEMBER 23, 2021

Nowadays, all organizations need real-time data to make instant business decisions and bring value to their customers faster. But this data is all over the place: It lives in the cloud, on social media platforms, in operational systems, and on websites, to name a few. Not to mention that additional sources are constantly being added through new initiatives like big data analytics , cloud-first, and legacy app modernization.

Process 69

Process Data Lake Metadata Data Warehouse 69

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Cloudera

NOVEMBER 25, 2021

As data continues to grow at an exponential rate, our customers are increasingly looking to advance and scale operations through digital transformation and the cloud. These modern digital businesses are also dealing with unprecedented rates of data volume, which is exploding from terabytes to petabytes and even exabytes which could prove difficult to manage.

Amazon Web Services 96

Amazon Web Services Cloud AWS Machine Learning 96

KDnuggets

NOVEMBER 24, 2021

Maintaining a centralized data repository can simplify your business intelligence initiatives. Here are four data integration tools that can make data more valuable for modern enterprises.

Data Integration 160

Data Integration Business Intelligence Data Data Preparation 160

Teradata

NOVEMBER 22, 2021

Teradata's Martin Willcox recently passed 17 years at Teradata and a quarter of a century in the industry. Here are the ten things he's learned about data analytics in those 20-odd years.

Data Analytics 59

Data Analytics Data 59

ProjectPro

NOVEMBER 25, 2021

Although businesses have an inclination towards structured data for insight generation and decision-making, text data is one of the vital information generated from digital platforms. However, it is not straightforward to extract or derive insights from a colossal amount of text data. To mitigate this challenge, organizations are now leveraging natural language processing and machine learning techniques to extract meaningful insights from unstructured text data.

Machine Learning 52

Machine Learning Algorithm Datasets Google Cloud 52

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Cloudera

NOVEMBER 24, 2021

For underserved youth, the lack of educational opportunity can seriously hinder their development and future career prospects. Many are deprived of early childhood chances at experiencing the professional world, so a career in science, finance, IT, or marketing is a pipe dream. . Unless someone shows them it’s possible. At the Middle Tennessee and Peninsula chapters of the Boys & Girls Clubs, high school students are receiving an introduction into a new world of possibilities.

Finance 80

Finance Education Programming Designing 80

KDnuggets

NOVEMBER 22, 2021

PyTorch and TensorFlow are the two leading AI/ML Frameworks. In this article, we take a look at their on-device counterparts PyTorch Mobile and TensorFlow Lite and examine them more deeply from the perspective of someone who wishes to develop and deploy models for use on mobile platforms.

Deep Learning 160

Deep Learning 160

Pipeline Data Engineering

NOVEMBER 24, 2021

Most data professionals realise very early in their journey that accessing the knowledge that they really need to solve data engineering problems is hard to come by. The other thing they don’t necessarily see is how short-sighted a lot of courses are, and how most of the technical content they provide is going to be rendered useless in a year or two.

Data Engineer 52

Data Engineer Data Engineering Engineering Education 52

ProjectPro

NOVEMBER 24, 2021

“Is data science hard to learn?”, “Is data science a hard job?”, “Is it hard to get a data science job?” Are you a data science enthusiast who believes data science is hard and keeps thinking about such questions? Allow us to challenge your thoughts and read this blog as we will help you answer all those questions.

Data Science 52

Data Science Consulting Machine Learning Software Engineer 52

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Preset

NOVEMBER 23, 2021

In this tutorial, we'll learn how to build a metrics dashboard with Apache Superset, a modern and open-source data exploration and visualization platform. We'll also use Cube, an open-source metrics store, as the data source for Superset.

Building 52

Building Data 52

KDnuggets

NOVEMBER 23, 2021

Also: How I Redesigned over 100 ETL into ELT Data Pipelines; Where NLP is heading; Don’t Waste Time Building Your Data Science Network; Data Scientists: How to Sell Your Project and Yourself.

Data Science 160

Data Science Project Data Pipeline Data 160

RudderStack

NOVEMBER 23, 2021

In this update, we cover our new high-performance JavaScript SDK, announce a new destination integration, and highlight our Event Stream pricing promotion.

40

40

ProjectPro

NOVEMBER 23, 2021

Towards the end of the 2000s, complex neural networks and model-based deep learning saw a huge upsurge in demand with revolutionary results in the fields of computer vision and natural language processing. While reinforcement learning has been around the corner from the same time, it was overshadowed by its counterparts for decades. It first became the talk of the town when in 2016, Google Deepmind’s AlphaGo defeated the World Champion in the Chinese game of Go.

Project 52

Project Python Algorithm AWS 52

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Preset

NOVEMBER 22, 2021

This tutorial post will cover all of the steps needed to make your first code contribution to the Apache Superset project.

Coding 52

Coding Project 52

KDnuggets

NOVEMBER 25, 2021

You can call Mito into your Jupyter Environment and each edit you make will generate the equivalent Python in the code cell below.

Python 159

Python Coding Programming 159

Rock the JVM

NOVEMBER 20, 2021

Discover how Akka Streams implements backpressure, a key component of the Reactive Streams specification, in this detailed demonstration

40

40

ProjectPro

NOVEMBER 22, 2021

PySpark has exploded in popularity in recent years, and many businesses are capitalizing on its advantages by producing plenty of employment opportunities for PySpark professionals. According to the Businesswire report , the worldwide big data as a service market is estimated to grow at a CAGR of 36.9% from 2019 to 2026, reaching $61.42 billion by 2026.

Hadoop 52

Hadoop Python Datasets Metadata 52

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Cloudera

NOVEMBER 22, 2021

Introduction. In legacy analytical systems such as enterprise data warehouses, the scalability challenges of a system were primarily associated with computational scalability, i.e., the ability of a data platform to handle larger volumes of data in an agile and cost-efficient way. Open source frameworks such as Apache Impala, Apache Hive and Apache Spark offer a highly scalable programming model that is capable of processing massive volumes of structured and unstructured data by means of paralle

Government 100

Government Hadoop Data Security Cloud 100

KDnuggets

NOVEMBER 25, 2021

A classic KDnuggets Thanksgiving cartoon examines the predicament of one group of fowl Data Scientists.

Data Science 159

Data Science Data 159

Rockset

NOVEMBER 23, 2021

We built Rockset with the mission to make real-time analytics easy and affordable in the cloud. We put our users first and obsess about helping our users achieve speed, scale and simplicity in their modern real-time data stack (some of which I discuss in depth below). But we, as a team, still take performance benchmarks seriously. Because they help us communicate that performance is one of the core product values at Rockset.

MongoDB 59

MongoDB Data Ingestion SQL PostgreSQL 59

KDnuggets

NOVEMBER 22, 2021

Read some of the key things the author has learned during the infamous job seeking stage.

Data 159

Data Data Science 159

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud