Teradata

SEPTEMBER 8, 2020

The fragility of your supply chain existed long before COVID-19 brought it into sharp relief. Discover the secret to true supply chain resilience.

IT 105

IT 105

Cloudera

SEPTEMBER 10, 2020

Digital transformation has been on the agenda for a long time, but the sudden need to respond to the unprecedented challenges of 2020, has meant the buzzword has become an executable reality for many enterprises. I recently came across a KPMG report that revealed that 80% of executives are increasing investments on emerging technologies now, to drive higher realized value in the future.

IT 104

IT Data Lake Cloud Healthcare 104

Data Engineering Podcast

SEPTEMBER 7, 2020

Summary Databases are limited in scope to the information that they directly contain. For analytical use cases you often want to combine data across multiple sources and storage locations. This frequently requires cumbersome and time-consuming data integration. To address this problem Martin Traverso and his colleagues at Facebook built the Presto distributed query engine.

Architecture 100

Architecture Data Architecture SQL Engineering 100

Netflix Tech

SEPTEMBER 8, 2020

How we migrated our Android endpoints out of a monolith into a new microservice by Rohan Dhruva , Ed Ballot As Android developers, we usually have the luxury of treating our backends as magic boxes running in the cloud, faithfully returning us JSON. At Netflix, we have adopted the Backend for Frontend (BFF) pattern : instead of having one general purpose “backend API”, we have one backend per client (Android/iOS/TV/web).

Java 96

Java Metadata Coding Architecture 96

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline

Teradata

SEPTEMBER 7, 2020

The latest release of Teradata Vantage with Native Object Store enables companies to not only drive down costs by leveraging object store technologies, but also improve manageability and drive business insights with the power of Vantage.

Technology 90

Technology Management 90

Cloudera

SEPTEMBER 10, 2020

Cloudera has been named a Leader in The Forrester Wave : Notebook-Based Predictive Analytics and Machine Learning, Q3 2020. At Cloudera, we are committed to always staying at the forefront of data and analytics innovation — enabling enterprises to more optimally work with data to deliver analytic results across the business quickly and securely. For enterprise machine learning teams, this means having the right platform, tools, and processes that streamline end-to-end ML to tackle once-impossibl

Machine Learning 95

Machine Learning Government Cloud Data Science 95

Grouparoo

SEPTEMBER 10, 2020

Martech Challenges in 2020 In the process of starting Grouparoo, we interviewed a hundred people who work in Marketing at various levels and roles. They spanned levels from independent contributors to executives and covered a wide range of marketing disciplines including Marketing Ops, Marketing Automation, Product Marketing, and more. Across our interviews, we heard about a diversity of experiences, but we heard a few common themes: Marketing’s scope is increasing Marketing is becoming more and

Data Warehouse 52

Data Warehouse Engineering SQL Technology 52

Teradata

SEPTEMBER 9, 2020

Learn why a data-centric organization requires an objective approach to manage and integrate its data.

Data 59

Data Management IT 59

Cloudera

SEPTEMBER 11, 2020

The ongoing disruption to critical supply chains in both the manufacturing and retail space has seen businesses having to respond quickly, turning to data, analytics, and new technologies to better predict and manage ‘real-time’ business disruptions. . To find out more about how COVID-19 has impacted the manufacturing and retail industries Vijay Raja, Director of Industry & Solutions Marketing at Cloudera sat down for a round-table discussion with Michael Ger , Managing Director of Manufactu

Manufacturing 92

Manufacturing Retail Food Entertainment 92

Rock the JVM

SEPTEMBER 9, 2020

Deploying Scala Code to AWS Lambda Is a Breeze: Discover Our Step-by-Step Tutorial to Guide You Through the Process

Scala 52

Scala AWS Coding Process 52

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

Data Science Blog: Data Engineering

SEPTEMBER 9, 2020

Data is centric in testing of several applications because data is critical to organizations. Businesses are becoming more data-driven, and hence it is imperative that as Automation Test developers, the value of the test-data is understood and completely harnessed during Test Automation development. The test-data involved in both Manual/Automation testing encompasses the test-data inputs, test-data outputs, and the test-data flow.

Data Management 52

Data Management Database-centric Management PostgreSQL 52

Teradata

SEPTEMBER 9, 2020

Teradata’s legacy of success is based upon three building blocks: People – Technology --Partnership. Learn how how a small piece of that legacy began and grew.

Technology 52

Technology Building 52

Cloudera

SEPTEMBER 9, 2020

This blog post will present a simple “hello world” kind of example on how to get data that is stored in S3 indexed and served by an Apache Solr service hosted in a Data Discovery and Exploration cluster in CDP. For the curious: DDE is a pre-templeted Solr-optimized cluster deployment option in CDP, and recently released in tech preview. We will only cover AWS and S3 environments in this blog.

AWS 84

AWS Data Unstructured Data Hadoop 84

Confluent

SEPTEMBER 9, 2020

Confluent Operator allows you to deploy and manage Confluent Platform as a cloud-native, stateful container application on Kubernetes and OpenShift. The automation provided by Kubernetes, Operator, and Helm greatly simplifies […].

AWS 52

AWS Cloud Management Kafka 52

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

Rock the JVM

SEPTEMBER 8, 2020

Explore how to impose constraints on values at compile time using the Refined library

Scala 52

Scala 52

RandomTrees

SEPTEMBER 7, 2020

Databricks and its role in Data Prep for AI solutions Databricks is a buzzword in Data Science. It is so due to a lot of reasons. In order to work with massive amounts of data in petabytes or even more, Apache Spark is widely used. Apache Spark is an open-source, fast cluster computing system and a highly popular framework for big data analysis. This framework processes the data in parallel that helps to boost the performance.

Project 52

Project Scala Machine Learning BI 52

Cloudera

SEPTEMBER 8, 2020

Technology like IoT, edge computing and 5G are changing the face of CSPs. Communication Service Providers (CSPs) are in the middle of a data-driven transformation. The current scale and pace of change in the Telecommunications sector is being driven by the rapid evolution of new technologies like the Internet of Things (IoT), 5G, advanced data analytics, and edge computing.

Telecommunication 70

Telecommunication Big Data Data Analytics Technology 70

Confluent

SEPTEMBER 10, 2020

Users of messaging technologies such as JMS and AMQP often use message prioritization so that messages can be processed in a different order based on their importance. It doesn’t take […].

Kafka 45

Kafka Technology Process IT 45

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

Zalando Engineering

SEPTEMBER 7, 2020

We spoke about his professional journey within Zalando, the evolution of Merchant Operations, and the engineering culture within the company. The interview was initially conducted for Zalando’s External Talent Community. Boris, let’s go back to the start. What attracted you to Zalando in the first place? The main reason for my attraction to Zalando was how quickly the company was able to adapt to change.

Engineering 40

Engineering Software Engineer Software Engineering Technology 40

RandomTrees

SEPTEMBER 6, 2020

Snowflake Overview A data warehouse is a critical part of any business organization. Lot of cloud-based data warehouses are available in the market today, out of which let us focus on Snowflake. Snowflake is an analytical data warehouse that is provided as Software-as-a-Service (SaaS). Built on new SQL database engine, it provides a unique architecture designed for the cloud.

Cloud Storage 52

Cloud Storage Data Ingestion Data Cleanse Data Warehouse 52

Cloudera

SEPTEMBER 11, 2020

In this blog post, we are going to take a look at some of the OpDB related security features of a CDP Private Cloud Base deployment. We are going to talk about encryption, authentication and authorization. . Data-at-rest encryption. Transparent data-at-rest encryption is available through the Transparent Data Encryption (TDE) feature in HDFS. . TDE provides the following features: Transparent, end-to-end encryption of data.

Database 69

Database Big Data Ecosystem Transportation Accessible 69

Cloudera

SEPTEMBER 9, 2020

In our last two posts, we talked with Deloitte’s Marc Beierschoder and Martin Mannion respectively about the requirement organizations have to deploy their data and analytics , quickly, into a hybrid environment. On top of that, there is the fundamental aspect of consistent security and governance of your enterprise data cloud and need for multiple users with different requirements to access data flexibly.

Cloud 66

Cloud Building Government Data 66

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineering