Uber Engineering

JULY 23, 2020

Introduction. Recently there has been substantial discussion around the downsides of service oriented architectures and microservice architectures in particular. While only a few years ago, many people readily adopted microservice architectures due to the numerous benefits they provide such as … The post Introducing Domain-Oriented Microservice Architecture appeared first on Uber Engineering Blog.

Architecture 145

Architecture Engineering 145

Teradata

JULY 19, 2020

To help our customers navigate the world's new normal, our teams have created a business-centric, execution-focused tool – we call it the Resiliency Dashboard.

Data 142

Data IT 142

Start Data Engineering

JULY 18, 2020

RDS AWS RDS is a managed service provided by AWS to run a relational database. We will see how to setup a postgres instance using AWS RDS. Log in to your AWS account. Go to Services -> RDS Click on Create Database, In the Create Database prompt, choose Standard Create option with PostgreSQL as engine type. In the Template section choose Free Tier and type in a DB Identifier, Master username and Master password.

PostgreSQL 130

PostgreSQL AWS Relational Database Database 130

Confluent

JULY 20, 2020

While preparing for the launch of Confluent Cloud ksqlDB, the ksqlDB Team built a system of metrics and monitoring that enabled insight into the experience of operating ksqlDB, the associated […].

Cloud 101

Cloud Process Systems 101

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Data Engineering Podcast

JULY 20, 2020

Summary Wind energy is an important component of an ecologically friendly power system, but there are a number of variables that can affect the overall efficiency of the turbines. Michael Tegtmeier founded Turbit Systems to help operators of wind farms identify and correct problems that contribute to suboptimal power outputs. In this episode he shares the story of how he got started working with wind energy, the system that he has built to collect data from the individual turbines, and how he is

Systems 100

Systems Machine Learning Manufacturing Algorithm 100

Netflix Tech

JULY 20, 2020

Stanislav Kirdey , William High Imagine having to go through 2.5GB of log entries from a failed software build?—?3 million lines?—?to search for a bug or a regression that happened on line 1M. It’s probably not even doable manually! However, one smart approach to make it tractable might be to diff the lines against a recent successful build, with the hope that the bug produces unusual lines in the logs.

Machine Learning 75

Machine Learning Algorithm Data Science Building 75

Confluent

JULY 21, 2020

Why data privacy for Apache Kafka®? As companies seek to leverage all forms of data for competitive advantage, there is a growing regulatory and reputational risk that calls for the […].

Kafka 95

Kafka Data Programming 95

Teradata

JULY 20, 2020

Models are crucial in creating business value and need to be handled with care. That's why models should be treated like data. Learn more.

Data 72

Data 72

Rockset

JULY 23, 2020

Read performance is crucial for databases. If it takes too long to read a record from a database, this can stall the request for data from the client application, which could result in unexpected behavior and adversely impact user experience. For these reasons, the read operation on your database should last no more than a fraction of a second. There are a number of ways to improve database read performance, though not all of these methods will work for every type of application.

MongoDB 52

MongoDB Database Project SQL 52

Start Data Engineering

JULY 18, 2020

Given Source database connection details (endpoint, port, username, password and database name) Source table to replicate destination schema name run frequency can be set to 10min We are assuming the destination data warehouse is already setup in stitch. Steps Log into your stitch account. here Click on Add Integration button on your dashboard. Choose PostgreSQL option as the integration in the next page.

Data Warehouse 100

Data Warehouse Database PostgreSQL Data 100

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Confluent

JULY 24, 2020

This blog post talks about the recent improvements on exactly-once semantics (EOS) to make it simpler to use and more resilient. EOS was first released in Apache Kafka® 0.11 and […].

Kafka 75

Kafka IT 75

Teradata

JULY 23, 2020

Without a data strategy, led from the top and encompassing the whole bank, tech investments run the risk of becoming expensive pet projects, driven by fads & siloed thinking.

Banking 52

Banking Data Project 52

Grouparoo

JULY 22, 2020

At Grouparoo, our front-end website is built using React and Next.js. Next.js is an excellent tool made by Vercel that handles all the hard parts of making a React app for you - Routing, Server-side Rendering, Page Hydration and more. It includes a simple starting place to build your routes and pages, based on the file system. If you want a /about page, just make an /pages/about.tsx file!

Coding 52

Coding Project Building Process 52

Rockset

JULY 21, 2020

Lessons from scaling facebook's online data infrastructure There are 3 growth numbers that stand out when I look back at the hyper-growth years of facebook from 2007 until 2015, when I was managing facebook's online data infrastructure team: user growth, team growth and infrastructure growth. Facebook’s user base grew from ~50 million monthly active users to a billion and half during that time, which is about a 30x growth.

Data Architecture 40

Data Architecture Architecture Building Data 40

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Confluent

JULY 23, 2020

This blog post applies to ksqlDB version 0.8.1 and later. Keeping a datacenter up and running is no walk in the park. It’s a job that involves mind-boggling amounts of […].

Data 71

Data Process 71

Teradata

JULY 21, 2020

Soon, retailers will need to run millions of queries everyday just to compete. But scaling to these levels needs an enterprise-wide strategy to overcome current barriers.

Retail 52

Retail Data 52

Preset

JULY 20, 2020

Diving deep into time-series bar charts in Superset.

40

40

Confluent

JULY 22, 2020

You may already know that Confluent Cloud is available across AWS, Azure, and Google Cloud, allowing you to access the amazing stack built by Confluent including a battle-tested version of […].

AWS 59

AWS Cloud Google Cloud Kafka 59

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence