Confluent
FEBRUARY 25, 2020
Fraud detection, payment systems, and stock trading platforms are only a few of many Apache Kafka® use cases that require both fast and predictable delivery of data. For example, detecting […].
Teradata
FEBRUARY 3, 2020
In 2020, various use cases enabled by marrying 5G & IoT will become possible, ushering in a generation where there is more data & transactions than ever before.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Data Engineering Podcast
FEBRUARY 24, 2020
Summary Misaligned priorities across business units can lead to tensions that drive members of the organization to build data and analytics projects without the guidance or support of engineering or IT staff. The availability of cloud platforms and managed services makes this a viable option, but can lead to downstream challenges. In this episode Sean Knapp and Charlie Crocker share their experiences of working in and with companies that have dealt with shadow IT projects and the importance of e
Netflix Tech
FEBRUARY 13, 2020
By Aditya Mavlankar, Jan De C**k¹, Cyril Concolato, Kyle Swanson, Anush Moorthy and Anne Aaron TL; DR We need an alternative to JPEG that a) is widely supported, b) has better compression efficiency and c) has a wider feature set. We believe AV1 Image File Format (AVIF) has the potential. Using the framework we have open sourced, AVIF compression efficiency can be seen at work and compared against a whole range of image codecs that came before it.
Speaker: Tamara Fingerlin, Developer Advocate
Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.
Rockset
FEBRUARY 28, 2020
Tech Preview TL;DR Join the Tech Deep Dive to learn how Rockset works with MongoDB! This is a tech preview of the MongoDB integration with Rockset to support millisecond-latency SQL queries such as joins and aggregations in real-time. Rockset builds fully mutable external indexes on any fields, including deeply nested fields in JSON documents, from your MongoDB collections.
Grouparoo
FEBRUARY 28, 2020
In a previous post , I talked about how powerful it is to make as many trips around the build/measure/learn loop as possible. This is an abstract concept that applies just as well to product development as marketing tests. As such, it is a little abstract. I thought it would be useful to go through the steps specific for marketers, show where the current pain is felt, and how Grouparoo makes things better.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Teradata
FEBRUARY 24, 2020
For the 11th consecutive year, Teradata - a leader in data analytics - has been named one of the World's Most Ethical Companies. Read more!
Data Engineering Podcast
FEBRUARY 17, 2020
Summary One of the biggest challenges in building reliable platforms for processing event pipelines is managing the underlying infrastructure. At Snowplow Analytics the complexity is compounded by the need to manage multiple instances of their platform across customer environments. In this episode Josh Beemster, the technical operations lead at Snowplow, explains how they manage automation, deployment, monitoring, scaling, and maintenance of their streaming analytics pipeline for event data.
Netflix Tech
FEBRUARY 14, 2020
Essential Suite?—?Artwork Producer Assistant By: Hamid Shahid & Syed Haq Introduction Netflix continues to invest in content for a global audience with a diverse range of unique tastes and interests. Correspondingly, the member experience must also evolve to connect this global audience to the content that most appeals to each of them. Images that represent titles on Netflix (what we at Netflix call “ artwork” ) have proven to be one of the most effective ways to help our members discover th
Rockset
FEBRUARY 14, 2020
In this post I’m going to show you how I tracked the location of my Tesla Model 3 in real time and plotted it on a map. I walk through an end to end integration of requesting data from the car, streaming it into a Kafka Topic and using Rockset to expose the data via its API to create real time visualisations in D3. Getting started with Kafka When starting with any new tool I find it best to look around and see the art of the possible.
Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage
There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.
Confluent
FEBRUARY 27, 2020
Today, I am pleased to announce the launch of Confluent Developer, the one and only portal for everything you need to get started with Apache Kafka®, Confluent Platform, and Confluent […].
Confluent
FEBRUARY 21, 2020
The world is changing fast, and keeping up can be hard. Companies must evolve their IT to stay modern, providing services that are more and more sophisticated to their customers. […].
Confluent
FEBRUARY 19, 2020
We just released Confluent Platform 5.4, which is one of our most important releases to date in terms of the features we’ve delivered to help enterprises take Apache Kafka® and […].
Confluent
FEBRUARY 6, 2020
When a company becomes overreliant on a centralized database, a world of bad things start to happen. Queries become slow, taxing an overburdened execution engine. Engineering decisions come to a […].
Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives
Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri
Confluent
FEBRUARY 12, 2020
Apache Kafka® is often deployed alongside Elasticsearch to perform log exploration, metrics monitoring and alerting, data visualisation, and analytics. It is complementary to Elasticsearch but also overlaps in some ways, […].
Data Engineering Podcast
FEBRUARY 9, 2020
Summary Designing the structure for your data warehouse is a complex and challenging process. As businesses deal with a growing number of sources and types of information that they need to integrate, they need a data modeling strategy that provides them with flexibility and speed. Data Vault is an approach that allows for evolving a data model in place without requiring destructive transformations and massive up front design to answer valuable questions.
Data Engineering Podcast
FEBRUARY 3, 2020
Summary Every business collects data in some fashion, but sometimes the true value of the collected information only comes when it is combined with other data sources. Data trusts are a legal framework for allowing businesses to collaboratively pool their data. This allows the members of the trust to increase the value of their individual repositories and gain new insights which would otherwise require substantial effort in duplicating the data owned by their peers.
Confluent
FEBRUARY 14, 2020
We are pleased to announce the release of ksqlDB 0.7.0. This release features highly available state, security enhancements for queries, a broadened range of language/data expressions, performance improvements, bug fixes, […].
Advertisement
With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you
Teradata
FEBRUARY 20, 2020
There's a trend in enterprise organizations where folks are starting to refer to user experience design as a "horizontal" function, but what does this mean?
Confluent
FEBRUARY 11, 2020
Osquery (developed by Facebook) is an open source tool used to gather audit log events from an operating system (OS). What’s unique about osquery is that it uses basic SQL […].
Confluent
FEBRUARY 4, 2020
Do you make New Year’s resolutions? The most I personally hear about them is people making a big show about how they don’t do them. And sure enough, I don’t […].
Teradata
FEBRUARY 13, 2020
Teradata is a launch partner for Amazon Web Services's brand-new capability: the Elastic Block Store multi-attach feature.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Confluent
FEBRUARY 13, 2020
We talked about how easy it is to send osquery logs to the Confluent Platform in part 1. Now, we’ll consume streams of osquery logs, detect anomalous behavior using machine […].
Netflix Tech
FEBRUARY 5, 2020
By Liwei Guo , Vivian Li , Julie Beckley , Venkatesh Selvaraj , and Jeff Watts Today we are excited to announce that Netflix has started streaming AV1 to our Android mobile app. AV1 is a high performance, royalty-free video codec that provides 20% improved compression efficiency over our VP9† encodes. AV1 is made possible by the wide-ranging industry commitment of expertise and intellectual property within the Alliance for Open Media (AOMedia), of which Netflix is a founding member.
Netflix Tech
FEBRUARY 5, 2020
By Liwei Guo , Vivian Li , Julie Beckley , Venkatesh Selvaraj , and Jeff Watts Today we are excited to announce that Netflix has started streaming AV1 to our Android mobile app. AV1 is a high performance, royalty-free video codec that provides 20% improved compression efficiency over our VP9† encodes. AV1 is made possible by the wide-ranging industry commitment of expertise and intellectual property within the Alliance for Open Media (AOMedia), of which Netflix is a founding member.
Teradata
FEBRUARY 27, 2020
Teradata is ranked as the top solution across all leading analyst studies for Data Analytics. Read more!
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
Teradata
FEBRUARY 9, 2020
In part two of this three-part series, you’ll learn how to use Teradata's R and Python packages, tdplyr and teradataml, to run machine learning and predictive analytics in Vantage at scale.
Rockset
FEBRUARY 7, 2020
In this IoT example, we examine how to enable complex analytic queries on real-time Kafka streams from connected car sensors. Understanding IoT and Connected Cars With an increasing number of data-generating sensors being embedded in all manner of smart devices and objects, there is a clear, growing need to harness and analyze IoT data. Embodying this trend is the burgeoning field of connected cars, where suitably equipped vehicles are able to communicate traffic and operating information, such
Expert insights. Personalized for you.
145 
Let's personalize your content