Data Engineering Podcast

MARCH 25, 2019

Summary Data integration is one of the most challenging aspects of any data platform, especially as the variety of data sources and formats grow. Enterprise organizations feel this acutely due to the silos that occur naturally across business units. The CluedIn team experienced this issue first-hand in their previous roles, leading them to build a business aimed at building a managed data fabric for the enterprise.

Building 100

Building Data Lake Machine Learning Kafka 100

Confluent

MARCH 26, 2019

Apache-Kafka ® -based applications stand out for their ability to decouple producers and consumers using an event log as an intermediate layer. One result of this is that producers and consumers don’t know about each other, as there is no direct communication between them. This enables choreographed service collaborations, where many components can subscribe to events stored in the event log and react to them asynchronously.

Kafka 111

Kafka Transportation Metadata Consulting 111

Netflix Tech

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can I run a check myself to understand what data is behind this metric?

Building 89

Building Metadata Transportation Data Ingestion 89

Uber Engineering

MARCH 28, 2019

Every day, Uber users around the world initiate customer support tickets through our Customer Obsession Platform. To ensure a seamless user experience, each of those tickets must be matched with an agent who speaks the user’s language and who … The post Improving the User Experience with Uber’s Customer Obsession Ticket Routing Workflow and Orchestration Engine appeared first on Uber Engineering Blog.

Engineering 82

Engineering Architecture Process 82

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Cloudera

MARCH 26, 2019

Roughly a quarter of Cloudera’s customers have clusters on public cloud, with a majority of them on AWS. These customers often look for cloud infrastructure best practices guidance as they venture into AWS cloud resources for the first time. Some of the questions asked include: How many AMIs do I need? Should I use EBS or S3 for storage? Many of these questions are answered in the Cloudera on AWS reference architecture guide.

AWS 77

AWS Cloud Architecture Management 77

Confluent

MARCH 28, 2019

Imagine a fire hose that spews out trillions of gallons of water every day, and part of your job is to withstand every drop coming out of it. This is what it is like to visualize the message throughput of Apache Kafka ®. At Confluent, we want to help developers understand how to think about event streaming and the opportunities it can create. Educating people on what an event stream looks like is a daunting task.

Kafka 79

Kafka Aggregated Data Engineering Media 79

Rockset

MARCH 27, 2019

You have complex, semi-structured data—nested JSON or XML, for instance, containing mixed types, sparse fields, and null values. It's messy, you don't understand how it's structured, and new fields appear every so often. The application you're implementing needs to analyze this data, combining it with other datasets, to return live metrics and recommended actions.

Raw Data 40

Raw Data SQL NoSQL Datasets 40

Cloudera

MARCH 28, 2019

Picture the scene: a hopeful homebuyer sits in the almost deserted lobby of a high street bank, waiting for the appointment she booked with the mortgage consultant a week ago – a week ago! It annoys her that she has had to come to a branch she has not visited for years, all because she could not work out how to apply for a home loan on the bank’s website.

Management 74

Management Banking Consulting Machine Learning 74

Confluent

MARCH 28, 2019

Imagine a fire hose that spews out trillions of gallons of water every day, and part of your job is to withstand every drop coming out of it. This is what it is like to visualize the message throughput of Apache Kafka ®. At Confluent, we want to help developers understand how to think about event streaming and the opportunities it can create. Educating people on what an event stream looks like is a daunting task.

Kafka 74

Kafka Aggregated Data Engineering Media 74

Teradata

MARCH 28, 2019

Today, we share insights from a diverse group of women at Teradata and their advice to other women entering roles in technology.

Technology 69

Technology 69

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Zalando Engineering

MARCH 27, 2019

Introducing Rust in an Enterprise Environment Discovery Sometime in 2013, on the internet I stumbled over a new programming language called Rust. Taking a look at the language, I was impressed by its high level features. At that time I was a backend Scala developer with a.Net background. When examining Rust, I found most of the features I used every day like Pattern Matching, the “New Type Pattern” and a “Scala like” Iterator API.

Scala 40

Scala Programming Language Technology Utilities 40

Cloudera

MARCH 27, 2019

Cloudera’s vision of delivering Edge to AI solutions using the Enterprise Data Cloud will enable enterprises to transform dramatically. In today’s digitally connected enterprises, data originates from the edge, streams into the data center, lands in an Enterprise Data Cloud for downstream processing including Machine Learning and then serves back to the edge for real-time prediction and action.

Management 72

Management Data Ingestion Machine Learning Java 72

Cloudera

MARCH 25, 2019

This post was originally published on the Cloudera Fast Forward Labs blog. . In recent years, machine learning technologies – especially deep learning – have made breakthroughs which have turned science fiction into reality. Autonomous cars are almost possible, and machines can comprehend language. These technical advances are unprecedented, but they hinge on the availability of vast amounts of data.

Machine Learning 63

Machine Learning Deep Learning Medical Computer Science 63