Cloudera

JUNE 1, 2021

Today marks the beginning of an exciting new chapter for Cloudera. Cloudera will become a private company with the flexibility and resources to accelerate product innovation, cloud transformation and customer growth. Cloudera will benefit from the operating capabilities, capital support and expertise of Clayton, Dubilier & Rice (CD&R) and KKR – two of the most experienced and successful global investment firms in the world recognized for supporting the growth strategies of the businesses

Cloud 144

Cloud Big Data Data Lake Finance 144

Uber Engineering

JUNE 8, 2021

As a recap from the last article , Uber’s API Gateway provides an interface and acts as a single point of access for all of our back-end services to expose features and data to Mobile and 3rd party partners. Two … The post Scaling of Uber’s API gateway appeared first on Uber Engineering Blog.

Engineering 139

Engineering Accessibility Accessible Architecture 139

Confluent

JUNE 11, 2021

Managing Apache Kafka® clusters can be tricky sometimes. To solve this problem, Confluent Control Center helps you easily manage and monitor your clusters and interact with other Confluent components, such […].

Kafka 138

Kafka Management 138

Netflix Tech

JUNE 7, 2021

By Alok Tiagi , Hariharan Ananthakrishnan , Ivan Porto Carrero and Keerti Lakshminarayan Netflix has developed a network observability sidecar called Flow Exporter that uses eBPF tracepoints to capture TCP flows at near real time. At much less than 1% of CPU and memory on the instance, this highly performant sidecar provides flow data at scale for network insight.

Transportation 131

Transportation AWS Cloud Metadata 131

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Start Data Engineering

JUNE 25, 2021

Introduction Objective Setup Pre-requisites Project 1. ETL Code 2. Test 3. Scheduler 4. Presentation 4.1. Formatting, Linting, and Type checks 4.2. Architecture Diagram 4.3. README.md 5. Adding Dashboard to your Profile Future Work Tear down infra Conclusion Further Reading References Introduction Building a data project for your portfolio is hard. Getting hiring managers to read through your Github code is even harder.

Project 130

Project Designing Management Portfolio 130

AltexSoft

JUNE 29, 2021

In a world fueled by disruptive technologies, no wonder businesses heavily rely on machine learning. For example, Netflix takes advantage of ML algorithms to personalize and recommend movies for clients, saving the tech giant billions. Google, in turn, uses the Google Neural Machine Translation (GNMT) system, powered by ML, reducing error rates by up to 60 percent.

Machine Learning 105

Machine Learning Engineering Algorithm Programming Language 105

Uber Engineering

JUNE 22, 2021

Introduction. Uber relies on a containerized microservice architecture. Our need for computational resources has grown significantly over the years, as a consequence of business’ growth. It is an important goal now to increase the efficiency of our computing resources. Broadly … The post Efficient and Reliable Compute Cluster Management at Scale appeared first on Uber Engineering Blog.

Management 110

Management Architecture Engineering IT 110

Confluent

JUNE 29, 2021

Making changes to a database schema is a natural part of software development. Often, it’s important to carefully manage the timing of changes and keep track of them over time. […].

Management 129

Management Database Process 129

Data Engineering Podcast

JUNE 28, 2021

Summary While the overall concept of timeseries data is uniform, its usage and applications are far from it. One of the most demanding applications of timeseries data is for application and server monitoring due to the problem of high cardinality. In his quest to build a generalized platform for managing timeseries Paul Dix keeps getting pulled back into the monitoring arena.

Data Analysis 100

Data Analysis Scala Data Warehouse Kafka 100

Teradata

JUNE 10, 2021

Open Banking platforms require high-reliability, seamless interfaces and should be driven by efficiency. Read more.

Banking 85

Banking Designing 85

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

AltexSoft

JUNE 7, 2021

Hadoop and Spark are the two most popular platforms for Big Data processing. They both enable you to deal with huge collections of data no matter its format — from Excel tables to user feedback on websites to images and video files. But which one of the celebrities should you entrust your information assets to? To come to the right decision, we need to divide this big question into several smaller ones — namely: What is Hadoop?

Big Data Tools 98

Big Data Tools Hadoop Big Data Database-centric 98

Cloudera

JUNE 7, 2021

Cloudera has been named as a Strong Performer in the Forrester Wave for Streaming Analytics, Q2 2021. We are excited to be recognized in this wave at, what we consider to be, such a strong position. We are proud to have been named as one of “ The 14 providers that matter most ” in streaming analytics. The report states that richness of analytics, development tool options and near-effortless scalability are what streaming analytics customers should look for in a provider. .

Kafka 105

Kafka Data Ingestion Cloud Architecture 105

Uber Engineering

JUNE 15, 2021

Introduction to Flaky Tests. Unit testing forms the bedrock of any Continuous Integration (CI) system. It warns software engineers of bugs in newly-implemented code and regressions in existing code, before it is merged. This ensures increased software reliability. It also … The post Handling Flaky Unit Tests in Java appeared first on Uber Engineering Blog.

Java 105

Java Software Engineer Software Engineering Coding 105

Confluent

JUNE 23, 2021

Al data til folket (all data to the people) is a compelling proposition in an enterprise context. Yet the ability to quickly address integration challenges and deliver data to those […].

Architecture 128

Architecture Data 128

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Data Engineering Podcast

JUNE 25, 2021

Summary Data Engineering is a broad and constantly evolving topic, which makes it difficult to teach in a concise and effective manner. Despite that, Daniel Molnar and Peter Fabian started the Pipeline Academy to do exactly that. In this episode they reflect on the lessons that they learned while teaching the first cohort of their bootcamp how to be effective data engineers.

Data Engineer 100

Data Engineer Data Engineering Engineering BI 100

Netflix Tech

JUNE 22, 2021

A Script Authoring Specification By: Bhanu Srikanth, Andy Swan, Casey Wilms, Patrick Pearson The Art of Dubbing and Subtitling Dubbing and subtitling are inherently creative processes. At Netflix, we strive to make shows as joyful to watch in every language as in the original language, whether a member watches with original or dubbed audio, closed captions, forced narratives, subtitles or any combination they prefer.

Metadata 73

Metadata Technology Designing Process 73

AltexSoft

JUNE 14, 2021

In today’s society, insurers can no longer ignore the mounting expectations of customers. Clients now expect insurers to provide different levels of personalization that are fast, adaptable, and up to date. That is why some insurers have gone further to provide insurance and risk management services that can be adjusted and rewritten in real-time depending on the changing risk in the consumer’s life.

Insurance 88

Insurance Medical Machine Learning Data Collection 88

Cloudera

JUNE 14, 2021

How to optimize an enterprise data architecture with private cloud and multiple public cloud options? As the inexorable drive to cloud continues, telecommunications service providers (CSPs) around the world – often laggards in adopting disruptive technologies – are embracing virtualization. Not only that, but service providers have been deploying their own clouds, some developing IaaS offerings, and partnering with cloud native content providers like Netflix and Spotify to enhance core telco bun

Telecommunication 101

Telecommunication Cloud Finance Government 101

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

ProjectPro

JUNE 22, 2021

This blog will help you master the fundamentals of classification machine learning algorithms with their pros and cons. You will also explore some exciting machine learning project ideas that implement different types of classification algorithms. So, without much ado, let's dive in. Imagine that the pandemic is over and today is a weekday. All the schools, colleges, and offices are open, and you should reach your institution by 8 A.M.

Machine Learning 72

Machine Learning Algorithm Datasets Project 72

Confluent

JUNE 1, 2021

One of the canonical examples of streaming data is tracking location data over time. Whether it’s ride-sharing vehicles, the position of trains on the rail network, or tracking airplanes waking […].

Data 118

Data Process 118

Data Engineering Podcast

JUNE 22, 2021

Summary The database is the core of any system because it holds the data that drives your entire experience. We spend countless hours designing the data model, updating engine versions, and tuning performance. But how confident are you that you have configured it to be as performant as possible, given the dozens of parameters and how they interact with each other?

Database 100

Database MySQL PostgreSQL Data Warehouse 100

DataKitchen

JUNE 15, 2021

The post Recipes for DataOps Success: The Complete Guide to an Enterprise DataOps Transformation first appeared on DataKitchen.

64

64

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Netflix Tech

JUNE 25, 2021

By Gim Mahasintunan on behalf of Data Platform Engineering. Supporting a rapidly growing base of engineers of varied backgrounds using different data stores can be challenging in any organization. Netflix’s internal teams strive to provide leverage by investing in easy-to-use tooling that streamlines the user experience and incorporates best practices.

Data 63

Data Accessibility Accessible Designing 63

Cloudera

JUNE 3, 2021

Cloudera Support’s cluster validations proactively identify known problem signatures contained in customers’ diagnostic data with the goal of increasing cluster health, performance, and overall stability. Cluster validations are included in a customer’s enterprise subscription at no additional cost. All customers with access to the Support case portal will also be able to take advantage of cluster validations.

Programming 100

Programming Consulting Designing Accessibility 100

Teradata

JUNE 30, 2021

As banks learn to adjust to the changes enforced by the COVID pandemic, the attention of customers, regulators & shareholders is returning to another global crisis – climate change.

Banking 59

Banking Data 59

Confluent

JUNE 3, 2021

What if I told you there is a query your database can’t answer? That would probably surprise you. With decades of effort behind them, databases are one of the most […].

Database 95

Database Process 95

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Data Engineering Podcast

JUNE 17, 2021

Summary Working with unstructured data has typically been a motivation for a data lake. The challenge is imposing enough order on the platform to make it useful. Kirk Marple has spent years working with data systems and the media industry, which inspired him to build a platform for automatically organizing your unstructured assets to make them more valuable.

Unstructured Data 100

Unstructured Data Data Warehouse Metadata Media 100

DataKitchen

JUNE 25, 2021

In this five-module course, Mike Lampa & Chris Bergh teach data professionals to plan their organization's DataOps program for low errors & fast deployment. The post Standing Up a DataOps Program for Practitioners first appeared on DataKitchen.

Programming 59

Programming Data 59

AltexSoft

JUNE 10, 2021

In the 1980s, American Airlines and its former president Robert Crandall started a revolution in airline pricing. Crandall is famous for many airline innovations that we use today, such as inventing the first frequent flier program and contributing to route optimization and central reservation system adoption. But he also pioneered yield management — the set of price optimization strategies that preceded revenue management.

Insurance 59

Insurance Algorithm Machine Learning Systems 59

Cloudera

JUNE 10, 2021

Apache Spark is a very popular analytics engine used for large-scale data processing. It is widely used for many big data applications and use cases. CDP Operational Database Experience Experience (COD) is a CDP Public Cloud service that lets you create and manage operational database instances and it is powered by Apache HBase and Apache Phoenix. .

Database 98

Database Data Engineer Data Engineering Big Data 98

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems