Data Engineering Podcast

MARCH 31, 2019

Summary Analytics projects fail all the time, resulting in lost opportunities and wasted resources. There are a number of factors that contribute to that failure and not all of them are under our control. However, many of them are and as data engineers we can help to keep our projects on the path to success. Eugene Khazin is the CEO of PrimeTSR where he is tasked with rescuing floundering analytics efforts and ensuring that they provide value to the business.

Project 100

Project IT Media Data Warehouse 100

Confluent

MARCH 26, 2019

Apache-Kafka ® -based applications stand out for their ability to decouple producers and consumers using an event log as an intermediate layer. One result of this is that producers and consumers don’t know about each other, as there is no direct communication between them. This enables choreographed service collaborations, where many components can subscribe to events stored in the event log and react to them asynchronously.

Kafka 111

Kafka Transportation Metadata Consulting 111

Uber Engineering

MARCH 8, 2019

First introduced by Uber in November 2018, Peloton , a unified resource scheduler, manages resources across distinct workloads, combining separate compute clusters. Peloton is designed for web-scale companies like Uber with millions of containers and tens of thousands of nodes. … The post Open Sourcing Peloton, Uber’s Unified Resource Scheduler appeared first on Uber Engineering Blog.

Engineering 109

Engineering Designing Management Architecture 109

Netflix Tech

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can I run a check myself to understand what data is behind this metric?

Building 89

Building Metadata Transportation Data Ingestion 89

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Cloudera

MARCH 26, 2019

Roughly a quarter of Cloudera’s customers have clusters on public cloud, with a majority of them on AWS. These customers often look for cloud infrastructure best practices guidance as they venture into AWS cloud resources for the first time. Some of the questions asked include: How many AMIs do I need? Should I use EBS or S3 for storage? Many of these questions are answered in the Cloudera on AWS reference architecture guide.

AWS 77

AWS Cloud Architecture Management 77

Teradata

MARCH 19, 2019

Teradata is thrilled to be named one the of the World’s Most Ethical Companies, for the tenth consecutive year.

99

99

Confluent

MARCH 12, 2019

On the heels of part one in this blog series, Spring for Apache Kafka – Part 1: Error Handling, Message Conversion and Transaction Support , here in part two we’ll focus on another project that enhances the developer experience when building streaming applications on Kafka: Spring Cloud Stream. We will cover the following in this post: Overview of Spring Cloud Stream and its programming model.

Kafka 111

Kafka Cloud Programming Coding 111

Uber Engineering

MARCH 7, 2019

Reliability engineering teams at Uber build the tools, libraries, and infrastructure that enable engineers to operate our thousands of microservices reliably at scale. At its essence, reliability engineering boils down to actively preventing outages that affect the mean time between … The post Using Machine Learning to Ensure the Capacity Safety of Individual Microservices appeared first on Uber Engineering Blog.

Machine Learning 105

Machine Learning Engineering Building IT 105

Netflix Tech

MARCH 12, 2019

Author: Andy Glover Since releasing Spinnaker to the open source community in 2015 , the platform has flourished with the addition of new cloud providers, triggers, pipeline stages, and much more. Myriad new features, improvements, and innovations have been added by an ever growing, actively engaged community. Each new innovation has been a step towards an even better Continuous Delivery platform that facilitates rapid, reliable, safe delivery of flexible assets to pluggable deployment targets.

Government 82

Government Cloud Project Cloud Computing 82

Cloudera

MARCH 28, 2019

Picture the scene: a hopeful homebuyer sits in the almost deserted lobby of a high street bank, waiting for the appointment she booked with the mortgage consultant a week ago – a week ago! It annoys her that she has had to come to a branch she has not visited for years, all because she could not work out how to apply for a home loan on the bank’s website.

Management 74

Management Banking Consulting Machine Learning 74

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Teradata

MARCH 14, 2019

Rob Armstrong uses the metaphor of turbulence when flying, to explains how businesses can prepare themselves for, and respond to, previously unforeseen challenges.

74

74

Data Engineering Podcast

MARCH 18, 2019

Summary Delivering a data analytics project on time and with accurate information is critical to the success of any business. DataOps is a set of practices to increase the probability of success by creating value early and often, and using feedback loops to keep your project on course. In this episode Chris Bergh, head chef of Data Kitchen, explains how DataOps differs from DevOps, how the industry has begun adopting DataOps, and how to adopt an agile approach to building your data platform.

Machine Learning 100

Machine Learning Data Lake Manufacturing Big Data 100

Confluent

MARCH 19, 2019

When should you use an API? When should you use an event? Most contemporary software architectures are some mix of these two approaches. I will attempt to articulate in layman’s terms what an event-driven architecture (EDA) is and contrast it with service-oriented architecture (SOA). In essence, this is an attempt to differentiate and/or associate APIs with events.

Architecture 110

Architecture Telecommunication Transportation Database 110

Uber Engineering

MARCH 28, 2019

Every day, Uber users around the world initiate customer support tickets through our Customer Obsession Platform. To ensure a seamless user experience, each of those tickets must be matched with an agent who speaks the user’s language and who … The post Improving the User Experience with Uber’s Customer Obsession Ticket Routing Workflow and Orchestration Engine appeared first on Uber Engineering Blog.

Engineering 82

Engineering Architecture Process 82

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Netflix Tech

MARCH 5, 2019

Netflix’s engineering culture is predicated on Freedom & Responsibility, the idea that everyone (and every team) at Netflix is entrusted with a core responsibility and they are free to operate with freedom to satisfy their mission. This freedom allows teams and individuals to move fast to deliver on innovation and feel responsible for quality and robustness of their delivery.

Cloud 73

Cloud Building Amazon Web Services Metadata 73

Cloudera

MARCH 27, 2019

Cloudera’s vision of delivering Edge to AI solutions using the Enterprise Data Cloud will enable enterprises to transform dramatically. In today’s digitally connected enterprises, data originates from the edge, streams into the data center, lands in an Enterprise Data Cloud for downstream processing including Machine Learning and then serves back to the edge for real-time prediction and action.

Management 72

Management Data Ingestion Machine Learning Java 72

Teradata

MARCH 18, 2019

Chris Twogood explains Pervasive Data Intelligence and why it's important to large-scale enterprise businesses.

Data 91

Data IT 91

Data Engineering Podcast

MARCH 4, 2019

Summary Customer analytics is a problem domain that has given rise to its own industry. In order to gain a full understanding of what your users are doing and how best to serve them you may need to send data to multiple services, each with their own tracking code or APIs. To simplify this process and allow your non-engineering employees to gain access to the information they need to do their jobs Segment provides a single interface for capturing data and routing it to all of the places that you

Google Cloud 100

Google Cloud Kafka Media Data Warehouse 100

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems

Confluent

MARCH 20, 2019

Back in May 2017, we laid out why we believe that Kafka Streams is better off without a concept of watermarks or triggers , and instead opts for a continuous refinement model. This article explains how we are fundamentally sticking with this model, while also opening the door for use cases that are incompatible with continuous refinement. By continuous refinement , I mean that Kafka Streams emits new results whenever records are updated.

Kafka 106

Kafka Programming Process Designing 106

Rockset

MARCH 21, 2019

When we started Rockset, we envisioned building a powerful cloud data management system that was really easy to use. Making the data stack simpler is fundamental to making data usable by developers and data scientists. Simplifying the Data Stack To that end, we incorporated user-friendly features that alleviate the pain we personally experienced as data practitioners.

SQL 52

SQL Data Management Management Engineering 52

Netflix Tech

MARCH 21, 2019

by Astha Singhal (Netflix Application Security) As Netflix continues to create entertainment people love, the security team continues to keep our members, partners, and employees secure. The security research community has partnered with us to improve the security of the Netflix service for the past few years through our responsible disclosure and bug bounty programs.

Entertainment 66

Entertainment Programming Accessible Accessibility 66

Cloudera

MARCH 13, 2019

The Data Heroes initiative is one of the ways that we recognize customers who achieve outstanding results with Cloudera technologies. The Data Visionary, Data Scientist, Data Architect, and HCC Community Champion awards are given out to organizations transforming their businesses through Big Data. Data Heroes design modern data architectures that work across hybrid and multi-cloud, and solve complex data management and analytic use cases that span from the Edge to AI. .

Data Architect 69

Data Architect Banking Utilities Deep Learning 69

Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network

In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.

Project

Teradata

MARCH 14, 2019

Rob Armstrong uses the metaphor of turbulence when flying, to explains how businesses can prepare themselves for, and respond to, previously unforeseen challenges.

67

67

InData Labs

MARCH 13, 2019

Social media has evolved into the main method of communicating ideas, sharing experience, brand stories, and building communities. The user engagement with ads on Facebook has tripled in the last 2 years, as Hootsuite reports. So far, more than 60% of users discover brands and goods on Instagram, employ such apps as Like2Buy that allows. Запись Brand Identity Issues: How Does Logo Detection Work for Effective Marketing Campaign?

Media 52

Media Building Data Engineering Data Engineer 52

Confluent

MARCH 4, 2019

Cloud is one of the key drivers for innovation. Innovative companies experiment with data to come up with something useful. It usually starts with the opening of a firehose that continuously broadcasts tons of events before they start mining it to create music out of simply noise. Today, companies from all around the world are witnessing an explosion of event generation coming from everywhere, including their own internal systems.

Cloud 89

Cloud Banking Kafka NoSQL 89

Pandora Engineering

MARCH 6, 2019

It’s 7 o’clock. I tug my blanket tightly over my face, hoping to whisk the morning away. The sound of my favorite playlist soothes me to a gentle rise. I smile. The alarm worked. Months of hard work collaborating with Google™, including many days at each others’ offices, came to life in that moment. Just a week earlier, we announced the release of the Pandora integration with the Clock app from Google.

Media 52

Media Architecture Engineering Algorithm 52

Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL

Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.

Government

Advancing Analytics: Data Engineering

MARCH 5, 2019

Today is my first day back in the office after attending SQLBits in Manchester last week. SQLBits is the UK's largest Microsoft Data Platform conference. What makes this event special is that it is organised for the community, by the community and is not for profit - All the proceeds go in to funding the event and in particular, the awesome Friday night party.

Deep Learning 52

Deep Learning Big Data Machine Learning Data Science 52

Cloudera

MARCH 25, 2019

This post was originally published on the Cloudera Fast Forward Labs blog. . In recent years, machine learning technologies – especially deep learning – have made breakthroughs which have turned science fiction into reality. Autonomous cars are almost possible, and machines can comprehend language. These technical advances are unprecedented, but they hinge on the availability of vast amounts of data.

Machine Learning 63

Machine Learning Deep Learning Medical Computer Science 63

Teradata

MARCH 24, 2019

Carrie Ballinger explores the capabilities of Teradata’s Adaptive Optimizer and how it builds better query plans for faster answers to analytic queries.

Building 65

Building IT 65

Domino Data Lab: Data Engineering

MARCH 2, 2019

Paco Nathan covers recent research on data infrastructure as well as adoption of machine learning and AI in the enterprise.

Machine Learning 52

Machine Learning Deep Learning Data Data Science 52

Advertisement

Entity Resolution Sometimes referred to as data matching or fuzzy matching, entity resolution, is critical for data quality, analytics, graph visualization and AI. Learn what entity resolution is, why it matters, how it works and its benefits. Advanced entity resolution using AI is crucial because it efficiently and easily solves many of today’s data quality and analytics problems.

IT