Teradata

AUGUST 11, 2020

Teradata Workload Management enables Vantage to be fully optimized for cloud & hybrid deployments & to efficiently deliver the lowest cost for enterprise analytics.

Cloud 124

Cloud Management 124

Confluent

AUGUST 14, 2020

In March, I wrote about Confluent’s commitment to our customers, employees, and community during the COVID-19 pandemic. In some respects, it’s hard to believe that only a few months have […].

105

105

Data Engineering Podcast

AUGUST 10, 2020

Summary Event based data is a rich source of information for analytics, unless none of the event structures are consistent. The team at Iteratively are building a platform to manage the end to end flow of collaboration around what events are needed, how to structure the attributes, and how they are captured. In this episode founders Patrick Thompson and Ondrej Hrebicek discuss the problems that they have experienced as a result of inconsistent event schemas, how the Iteratively platform integrat

Data Collection 100

Data Collection Data Engineer Data Engineering Education 100

Netflix Tech

AUGUST 10, 2020

by Mariana Afonso , Anush Moorthy , Liwei Guo , Lishan Zhu , Anne Aaron Netflix has been one of the pioneers of streaming video-on-demand content?—?we announced our intention to stream video over 13 years ago, in January 2007?—?and have only increased both our device and content reach since then. Given the global nature of the service and Netflix’s commitment to creating a service that members enjoy, it is not surprising that we support a wide variety of streaming devices, from set-top-boxes and

Accessibility 97

Accessibility Accessible IT 97

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline

Cloudera

AUGUST 14, 2020

One of the key challenges of building an enterprise-class robust scalable storage system is to validate the system under duress and failing system components. This includes, but is not limited to: failed networks, failed or failing disks, arbitrary delays in the network or IO path, network partitions, and unresponsive systems. Apache Ozone fault injection framework is designed to validate Ozone under heavy stress and failed or failing system components.

Hadoop 96

Hadoop Bytes Metadata Programming Language 96

Confluent

AUGUST 13, 2020

Multithreading is “the ability of a central processing unit (CPU) (or a single core in a multi-core processor) to provide multiple threads of execution concurrently, supported by the operating system.” […].

Kafka 105

Kafka Process Systems Java 105

Netflix Tech

AUGUST 11, 2020

Jeffrey Wong , Colin McFarland Every Netflix data scientist, whether their background is from biology, psychology, physics, economics, math, statistics, or biostatistics, has made meaningful contributions to the way Netflix analyzes causal effects. Scientists from these fields have made many advancements in causal effects research in the past few decades, spanning instrumental variables, forest methods, heterogeneous effects, time-dynamic effects, quantile effects, and much more.

Algorithm 95

Algorithm Software Engineering Software Engineer Datasets 95

FreshBI

AUGUST 11, 2020

So, what is a Power BI Template App? A Power BI Template App is a published Power BI solution that can be used by any company that has the data platform for which the Template App was created. Can you imagine picking your entire Power BI Solution off the shelf - one crafted for your specific business needs and your specific data structure. Power BI Template Apps are designed to be such an out-of-the-box solution and this blog post is an example of such for a Power BI Solution for Stripe.

BI 52

BI Banking Education Datasets 52

Confluent

AUGUST 11, 2020

Building a Cloud ETL Pipeline on Confluent Cloud shows you how to build and deploy a data pipeline entirely in the cloud. However, not all databases can be in the […].

Database 100

Database Kafka Data Pipeline Cloud 100

Teradata

AUGUST 13, 2020

Vantage on Azure provides enterprise-grade real-time business intelligence through a comprehensive solution that combines analytics, data lakes, & data warehouse technologies.

Data Lake 52

Data Lake Business Intelligence Cloud Data Warehouse 52

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

Netflix Tech

AUGUST 13, 2020

By Andrei U., Seth Katz , Janak Ramachandran , Jeff Butsch , Peter Lau , Ram Vaithilingam , and Greg Burrell Our Telltale Vision An alert fires and you get paged in the middle of the night. A metric crossed a threshold. You’re half awake and wondering, “Is there really a problem or is this just an alert that needs tuning? When was the last time somebody adjusted our alert thresholds?

Algorithm 87

Algorithm Entertainment AWS Machine Learning 87

Rockset

AUGUST 11, 2020

It was the winter of 2007 when I logged into my newly created Facebook account for the very first time and I was amazed to see Facebook immediately show me three of my friends with whom I had lost touch since elementary school. One of them was working in London in a multinational bank, the other one was an engineer at Google in their Silicon Valley office office and the third one was running a restaurant in my town of Guwahati, a sleepy town on the India-Myanmar border.

Hadoop 52

Hadoop Banking Datasets Analytics Application 52

Confluent

AUGUST 12, 2020

When operating cloud infrastructure, “time is money” is more than a cliché—it is interpreted literally as every processing second stacks up on the monthly bill. ksqlDB strives to reduce these […].

Cloud 80

Cloud Process IT 80

Teradata

AUGUST 10, 2020

In part 1 of this 3 part series on reducing conflict between business & IT to accelerate innovation, we focus on enabling flexibility for tools, languages & libraries.

IT 52

IT 52

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

Rock the JVM

AUGUST 9, 2020

Harness the full power of Scala's type system: let the compiler infer complex type relationships for you at compile time

Scala 52

Scala Programming Systems 52

RandomTrees

AUGUST 8, 2020

By Anshul Ghogre Introduction Apache NiFiis designed to automate the flow of data between software systems. It is based on the “NiagaraFiles” software previously developed by the NSA, it supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic. Apache Kafka is used for building real-time data pipelines and streaming apps.

Kafka 52

Kafka BI Data Pipeline Coding 52

Rockset

AUGUST 13, 2020

Originally published on July 8, 2020 Yesterday I read an analyst report that the serverless architecture market will be $21B by 2025. I also recently met with Alex DeBrie, author of the DynamoDB book and enjoyed learning about his serverless philosophy. He wrote a great post about the key factors for choosing serverless databases here , and we had a fascinating conversation about serverless indexing systems that complement them.

NoSQL 40

NoSQL BI Data Lake Architecture 40

Preset

AUGUST 13, 2020

Summary of Superset 0.

Coding 40

Coding 40

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

Rock the JVM

AUGUST 9, 2020

Harness the full power of Scala's type system: let the compiler infer complex type relationships for you at compile time

Scala 52

Scala Programming Systems 52

Rockset

AUGUST 10, 2020

From business communications and financial transactions to trip planning and activity tracking, much of our lives run through smartphones today. eGoGames will help you add competitive esports to that list. As the first European esports platform for mobile devices, eGoGames offers head-to-head, league, and tournament competition for skill-based mobile games.

BI 40

BI AWS SQL Data 40

Rockset

AUGUST 10, 2020

You may hear the phrase that the world is moving from batch to real-time a lot. While traditional “business intelligence” has come a long way in the past 20 years, the world of real-time analytics is still in its early days. Traditional BI had its Renaissance moments with the advent of Big Data technologies such as Hadoop, and then cloud data lakes and warehouses have brought everyone to the Modern era.

BI 40

BI Hadoop SQL Data Lake 40

Preset

AUGUST 10, 2020

Learn why Nielsen migrated to Superset for visualization and dashboards.

Accessibility 40

Accessibility Accessible Data Analytics Data 40

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineering