Confluent
JULY 15, 2020
With billions of Internet of Things (IoT) devices, achieving real-time interoperability has become a major challenge. Together, Confluent, Waterstream, and MQTT are accelerating Industry 4.0 with new Industrial IoT (IIoT) […].
Start Data Engineering
JULY 11, 2020
Intro A very common use case in data engineering is to build a ETL system for a data warehouse, to have data loaded in from multiple separate databases to enable data analysts/scientists to be able to run queries on this data, since the source databases are used by your applications and we do not want these analytic queries to affect our application performance and the source data is disconnected as shown below.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Data Engineering Podcast
JULY 13, 2020
Summary The first stage of every data pipeline is extracting the information from source systems. There are a number of platforms for managing data integration, but there is a notable lack of a robust and easy to use open source option. The Meltano project is aiming to provide a solution to that situation. In this episode, project lead Douwe Maan shares the history of how Meltano got started, the motivation for the recent shift in focus, and how it is implemented.
Teradata
JULY 14, 2020
The days are gone when defining a user experience was limited to the choice of designers. Now data plays a more important role in the design process than ever before.
Speaker: Tamara Fingerlin, Developer Advocate
Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.
Confluent
JULY 17, 2020
The first-ever virtual Kafka Summit 2020 kicks off next month in the comfort of your home office, couch, spare bedroom, living room, outbuilding, lanai, veranda, or in-home portico, featuring an […].
Netflix Tech
JULY 17, 2020
The cloud-based platform allows vendors, artists and creators to connect and collaborate on visual effects (VFX) from anywhere in the… Continue reading on Netflix TechBlog ».
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Teradata
JULY 13, 2020
Data Warehouse, Data Lake, Data Mesh, Data.What? If you are confused by the proliferation of these IT terms, allow us to shed some light.
Confluent
JULY 16, 2020
Apache Kafka® is a distributed commit log, commonly used as a multi-tenant data hub to connect diverse source systems and sink systems. Source systems can be systems or records, operational […].
Rock the JVM
JULY 11, 2020
Explore Akka Streams' powerful Graph DSL and learn how to get started quickly with our easy guide
Rockset
JULY 14, 2020
StoryFire is a social platform for content creators to share and monetize their stories and videos. Using Rockset to index data from their transactional MongoDB system , StoryFire powers complex aggregation and join queries for their social and leaderboard features. By moving read-intensive services off MongoDB to Rockset, StoryFire is able to solve two hard challenges: performance and scale.
Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage
There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.
Teradata
JULY 16, 2020
Get step-by-step instructions to set up Teradata Vantage and author AWS Glue Streaming ETL jobs to stream data into Vantage from Amazon Kinesis and visualize the data.
Zalando Engineering
JULY 16, 2020
The following guidelines and tricks help me with task management, time management, planning & prioritization, reacting to ad-hoc situations, and the sense of not having accomplished anything during the day. There is some overlap with our Remote Work Guidelines 1. My meta-advice for applying anything from this article: start with one improvement, don’t try it all at once.
Rock the JVM
JULY 11, 2020
Explore Akka Streams' powerful Graph DSL and learn how to get started quickly with our easy guide
Preset
JULY 13, 2020
Funnels are a very popular mental model for understanding data. Learn the pros and cons of different funnel data visualizations using Apache Superset.
Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives
Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri
Teradata
JULY 15, 2020
Teradata data scientists utilized Teradata technologies to develop models to accurately project the number of COVID-19 confirmations and deaths. Learn more.
Zalando Engineering
JULY 14, 2020
Challenges with our Tech Radar The Zalando Tech Radar is modelled after the Thoughtworks Technology Radar and includes a ring-based scoring for a certain technology/framework along with supplementary information about pros, cons, restrictions, usage, and lessons learned at Zalando available as a knowledge base for our teams. Since publishing, the approach and visualization engine has been used by others and also showcased at conferences as an example of how tech companies manage their technology
Expert insights. Personalized for you.
139 
Let's personalize your content