Confluent

FEBRUARY 12, 2020

Apache Kafka® is often deployed alongside Elasticsearch to perform log exploration, metrics monitoring and alerting, data visualisation, and analytics. It is complementary to Elasticsearch but also overlaps in some ways, […].

Kafka 120

Kafka Data IT Process 120

Data Engineering Podcast

FEBRUARY 9, 2020

Summary Designing the structure for your data warehouse is a complex and challenging process. As businesses deal with a growing number of sources and types of information that they need to integrate, they need a data modeling strategy that provides them with flexibility and speed. Data Vault is an approach that allows for evolving a data model in place without requiring destructive transformations and massive up front design to answer valuable questions.

Data Lake 100

Data Lake Data Warehouse Hadoop NoSQL 100

Teradata

FEBRUARY 13, 2020

Teradata is a launch partner for Amazon Web Services's brand-new capability: the Elastic Block Store multi-attach feature.

Amazon Web Services 86

Amazon Web Services AWS 86

Netflix Tech

FEBRUARY 14, 2020

Essential Suite?—?Artwork Producer Assistant By: Hamid Shahid & Syed Haq Introduction Netflix continues to invest in content for a global audience with a diverse range of unique tastes and interests. Correspondingly, the member experience must also evolve to connect this global audience to the content that most appeals to each of them. Images that represent titles on Netflix (what we at Netflix call “ artwork” ) have proven to be one of the most effective ways to help our members discover th

Algorithm 77

Algorithm Designing Metadata Process 77

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Confluent

FEBRUARY 14, 2020

We are pleased to announce the release of ksqlDB 0.7.0. This release features highly available state, security enhancements for queries, a broadened range of language/data expressions, performance improvements, bug fixes, […].

Data 97

Data Process 97

Rockset

FEBRUARY 14, 2020

In this post I’m going to show you how I tracked the location of my Tesla Model 3 in real time and plotted it on a map. I walk through an end to end integration of requesting data from the car, streaming it into a Kafka Topic and using Rockset to expose the data via its API to create real time visualisations in D3. Getting started with Kafka When starting with any new tool I find it best to look around and see the art of the possible.

Kafka 40

Kafka SQL Metadata Python 40

Netflix Tech

FEBRUARY 13, 2020

By Aditya Mavlankar, Jan De C**k¹, Cyril Concolato, Kyle Swanson, Anush Moorthy and Anne Aaron TL; DR We need an alternative to JPEG that a) is widely supported, b) has better compression efficiency and c) has a wider feature set. We believe AV1 Image File Format (AVIF) has the potential. Using the framework we have open sourced, AVIF compression efficiency can be seen at work and compared against a whole range of image codecs that came before it.

Coding 89

Coding Bytes Datasets Media 89

Confluent

FEBRUARY 11, 2020

Osquery (developed by Facebook) is an open source tool used to gather audit log events from an operating system (OS). What’s unique about osquery is that it uses basic SQL […].

SQL 91

SQL Systems IT Kafka 91

Confluent

FEBRUARY 13, 2020

We talked about how easy it is to send osquery logs to the Confluent Platform in part 1. Now, we’ll consume streams of osquery logs, detect anomalous behavior using machine […].

Machine Learning 82

Machine Learning IT Process 82