Confluent

FEBRUARY 12, 2020

Apache Kafka® is often deployed alongside Elasticsearch to perform log exploration, metrics monitoring and alerting, data visualisation, and analytics. It is complementary to Elasticsearch but also overlaps in some ways, […].

Kafka 118

Kafka Data IT Process 118

Data Engineering Podcast

FEBRUARY 9, 2020

Summary Designing the structure for your data warehouse is a complex and challenging process. As businesses deal with a growing number of sources and types of information that they need to integrate, they need a data modeling strategy that provides them with flexibility and speed. Data Vault is an approach that allows for evolving a data model in place without requiring destructive transformations and massive up front design to answer valuable questions.

Data Lake 100

Data Lake Data Warehouse Hadoop NoSQL 100

Netflix Tech

FEBRUARY 13, 2020

By Aditya Mavlankar, Jan De C**k¹, Cyril Concolato, Kyle Swanson, Anush Moorthy and Anne Aaron TL; DR We need an alternative to JPEG that a) is widely supported, b) has better compression efficiency and c) has a wider feature set. We believe AV1 Image File Format (AVIF) has the potential. Using the framework we have open sourced, AVIF compression efficiency can be seen at work and compared against a whole range of image codecs that came before it.

Coding 88

Coding Bytes Datasets Media 88

Teradata

FEBRUARY 13, 2020

Teradata is a launch partner for Amazon Web Services's brand-new capability: the Elastic Block Store multi-attach feature.

Amazon Web Services 86

Amazon Web Services AWS 86

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline

Confluent

FEBRUARY 14, 2020

We are pleased to announce the release of ksqlDB 0.7.0. This release features highly available state, security enhancements for queries, a broadened range of language/data expressions, performance improvements, bug fixes, […].

Data 94

Data Process 94

Rockset

FEBRUARY 14, 2020

In this post I’m going to show you how I tracked the location of my Tesla Model 3 in real time and plotted it on a map. I walk through an end to end integration of requesting data from the car, streaming it into a Kafka Topic and using Rockset to expose the data via its API to create real time visualisations in D3. Getting started with Kafka When starting with any new tool I find it best to look around and see the art of the possible.

Kafka 40

Kafka SQL Metadata Python 40

Teradata

FEBRUARY 9, 2020

In part two of this three-part series, you’ll learn how to use Teradata's R and Python packages, tdplyr and teradataml, to run machine learning and predictive analytics in Vantage at scale.

Python 52

Python Machine Learning 52

Confluent

FEBRUARY 11, 2020

Osquery (developed by Facebook) is an open source tool used to gather audit log events from an operating system (OS). What’s unique about osquery is that it uses basic SQL […].

SQL 88

SQL Systems IT Kafka 88

Confluent

FEBRUARY 13, 2020

We talked about how easy it is to send osquery logs to the Confluent Platform in part 1. Now, we’ll consume streams of osquery logs, detect anomalous behavior using machine […].

Machine Learning 78

Machine Learning IT Process 78