Start Data Engineering

SEPTEMBER 26, 2020

Table of Contents Table of Contents Introduction Project description and requirements Infrastructure overview Apache Flink Apache Kafka Design Detect fraudulent accounts Log account actions Prerequisites Code Defining dependencies Inheritance Server logs generator Defining data flow in Apache Flink Create a streaming environment Creating a consumer to read events from Apache Kafka Detecting fraud and generating alert events Writing server logs to a PostgreSQL DB Fraud detection logic Open proces

Data Engineering 130

Data Engineering Data Engineer Project Engineering 130

Confluent

SEPTEMBER 29, 2020

ksqlDB, the event streaming database, is becoming one of the most popular ways to work with Apache Kafka®. Every day, we answer many questions about the project, but here’s a […].

Process 145

Process Kafka Database Project 145

Cloudera

SEPTEMBER 28, 2020

Cloudera delivers an enterprise data cloud that enables companies to build end-to-end data pipelines for hybrid cloud, spanning edge devices to public or private cloud, with integrated security and governance underpinning it to protect customers data. Cloudera has found that customers have spent many years investing in their big data assets and want to continue to build on that investment by moving towards a more modern architecture that helps leverage the multiple form factors.

Cloud 132

Cloud Kafka Professional Services Metadata 132

Data Engineering Podcast

SEPTEMBER 28, 2020

Summary Kafka has become a de facto standard interface for building decoupled systems and working with streaming data. Despite its widespread popularity, there are numerous accounts of the difficulty that operators face in keeping it reliable and performant, or trying to scale an installation. To make the benefits of the Kafka ecosystem more accessible and reduce the operational burden, Alexander Gallego and his team at Vectorized created the Red Panda engine.

Kafka 100

Kafka Big Data BI Data Engineering 100

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Teradata

SEPTEMBER 29, 2020

In part 1 of this 2 part topic, we will define some of the commonly used (& misused) terms in the business continuity space & help you navigate what they mean to your organization.

52

52

Confluent

OCTOBER 1, 2020

Each month, we’ve announced a set of Confluent features organized around what we think are the key foundational traits of cloud-native data systems as part of Project Metamorphosis. Data systems […].

Project 143

Project Cloud Systems Data 143

Rock the JVM

SEPTEMBER 28, 2020

This article delves into Scala 3's advanced trait functionalities, building on our previous explorations of the language's new features

Scala 52

Scala Building 52

Teradata

SEPTEMBER 29, 2020

In part 1 of this 3 part series, we will define some of the commonly used (& misused) terms in the business continuity space & help you navigate what they mean to your organization.

52

52

Confluent

OCTOBER 2, 2020

Stream processing applications, including streaming ETL pipelines, materialized caches, and event-driven microservices, are made easy with ksqlDB. Until recently, your options for interacting with ksqlDB were limited to its command-line […].

Java 122

Java Process IT 122

Cloudera

SEPTEMBER 29, 2020

Performance is one of the key, if not the most important deciding criterion, in choosing a Cloud Data Warehouse service. In today’s fast changing world, enterprises have to make data driven decisions quickly and for that they rely heavily on their data warehouse service. . In this blog post, we compare Cloudera Data Warehouse (CDW) on Cloudera Data Platform (CDP) using Apache Hive-LLAP to Microsoft HDInsight (also powered by Apache Hive-LLAP) on Azure using the TPC-DS 2.9 benchmark.

Data Warehouse 120

Data Warehouse Cloud Storage Metadata Cloud 120

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Monte Carlo

OCTOBER 1, 2020

As companies increasingly rely on data to power decision making and drive innovation, it’s important that this data is timely, accurate, and reliable. When you consider that only a small fraction of the over 7.5 septillion (7,700,000,000, 000,000,000,000) GB of data generated worldwide every day is usable, keeping tabs on what data assets are important has only gotten harder.

Data Warehouse 40

Data Warehouse Datasets Data Validation Machine Learning 40

Teradata

SEPTEMBER 27, 2020

Marketing might be the best place to start operationalizing a bank-wide data strategy. But, to be effective, the CMO needs to dissolve data silos & create a model for data orchestration.

Banking 52

Banking Data 52

Confluent

SEPTEMBER 30, 2020

The ksqlDB team is pleased to announce ksqlDB 0.12.0. This release continues to improve upon the usability of ksqlDB and aims to reduce administration time. Highlights include query upgrades, which […].

Process 98

Process 98

Cloudera

SEPTEMBER 30, 2020

In our previous blog post we introduced Cloudera Data Visualization in Cloudera Data Warehouse (CDW) available in tech preview, in CDP Public Cloud. This blog will help you get started with Cloudera Data Visualization, so you can start building interesting and powerful applications on all types of data. Before you start. Make sure that. You have a CDP account set up (for instance, you may use our trial experience ).

Data Warehouse 72

Data Warehouse Database Data Cloud 72

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Rockset

OCTOBER 1, 2020

Users interact with services in real-time. They login to websites, like and share posts, purchase goods and even converse, all in real-time. So why is it that whenever you have a problem when using a service and you reach a customer support representative, that they never seem to know who you are or what you’ve been doing recently? This is likely because they haven’t built a customer 360 profile and if they have, it certainly isn’t real-time.

MongoDB 40

MongoDB Kafka Building Data Warehouse 40

Teradata

SEPTEMBER 30, 2020

Industry 4.0 has promised productivity gains, but has not yet delivered. A large part of this has to do with the challenge of deploying analytics at scale. Find out more.

52

52

Rock the JVM

OCTOBER 1, 2020

Explore opaque type aliases in Scala 3 and learn how to define new types with zero overhead

Scala 52

Scala 52

Cloudera

SEPTEMBER 29, 2020

Meet Ali Bajwa , Director of Partner Solutions Engineering at Cloudera. For the past 6 years, Ali has been front and center in many partner field deployments, training, and discussions; he is a rockstar in the Cloudera Partner Ecosystem! We hope this interview helps you get to know the afterhours Ali. If you get a chance, follow Ali on twitter! @abajwa_hdp.

Engineering 59

Engineering Food Programming Technology 59

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems

Preset

OCTOBER 1, 2020

Build a beautiful Slack dashboard using open source tools Meltano and Superset. Part 2 of 3.

Building 40

Building 40

FreshBI

OCTOBER 1, 2020

Spotlight: The PowerBI Service Lately we have been getting a lot of questions surrounding licensing and release strategy in PowerBI. This guide should serve as an internal, quick reference manual. The following is a list of topics covered in this guide, each containing a summary of how it works and what the use case is. Licensing PowerBI Desktop / Free Who uses this?

BI 52

BI Datasets Finance Accessible 52

Rock the JVM

SEPTEMBER 28, 2020

Scala 3 Introduces Enums: A Major Update with Significant Implications

Scala 52

Scala 52