Start Data Engineering

JUNE 25, 2021

Introduction Objective Setup Pre-requisites Project 1. ETL Code 2. Test 3. Scheduler 4. Presentation 4.1. Formatting, Linting, and Type checks 4.2. Architecture Diagram 4.3. README.md 5. Adding Dashboard to your Profile Future Work Tear down infra Conclusion Further Reading References Introduction Building a data project for your portfolio is hard. Getting hiring managers to read through your Github code is even harder.

Project 130

Project Designing Management Portfolio 130

Confluent

JUNE 23, 2021

Al data til folket (all data to the people) is a compelling proposition in an enterprise context. Yet the ability to quickly address integration challenges and deliver data to those […].

Architecture 128

Architecture Data 128

Data Engineering Podcast

JUNE 25, 2021

Summary Data Engineering is a broad and constantly evolving topic, which makes it difficult to teach in a concise and effective manner. Despite that, Daniel Molnar and Peter Fabian started the Pipeline Academy to do exactly that. In this episode they reflect on the lessons that they learned while teaching the first cohort of their bootcamp how to be effective data engineers.

Data Engineer 100

Data Engineer Data Engineering Engineering BI 100

Uber Engineering

JUNE 22, 2021

Introduction. Uber relies on a containerized microservice architecture. Our need for computational resources has grown significantly over the years, as a consequence of business’ growth. It is an important goal now to increase the efficiency of our computing resources. Broadly … The post Efficient and Reliable Compute Cluster Management at Scale appeared first on Uber Engineering Blog.

Management 110

Management Architecture Engineering IT 110

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Cloudera

JUNE 25, 2021

Introduction. Many Cloudera customers are making the transition from being completely on-prem to cloud by either backing up their data in the cloud, or running multi-functional analytics on CDP Public cloud in AWS or Azure. The Replication Manager service facilitates both disaster recovery and data migration across different environments. Using easy-to-define policies, Replication Manager solves one of the biggest barriers for the customers in their cloud adoption journey by allowing them to mov

Cloud 73

Cloud Data Lake Cloud Storage Metadata 73

Confluent

JUNE 24, 2021

Today, Confluent became a publicly traded company. This is a big milestone in the short life of our company. To the employees, customers, partners, investors, and the larger developer community […].

94

94

ProjectPro

JUNE 22, 2021

This blog will help you master the fundamentals of classification machine learning algorithms with their pros and cons. You will also explore some exciting machine learning project ideas that implement different types of classification algorithms. So, without much ado, let's dive in. Imagine that the pandemic is over and today is a weekday. All the schools, colleges, and offices are open, and you should reach your institution by 8 A.M.

Machine Learning 72

Machine Learning Algorithm Datasets Project 72

Netflix Tech

JUNE 22, 2021

A Script Authoring Specification By: Bhanu Srikanth, Andy Swan, Casey Wilms, Patrick Pearson The Art of Dubbing and Subtitling Dubbing and subtitling are inherently creative processes. At Netflix, we strive to make shows as joyful to watch in every language as in the original language, whether a member watches with original or dubbed audio, closed captions, forced narratives, subtitles or any combination they prefer.

Metadata 71

Metadata Technology Designing Process 71

Confluent

JUNE 25, 2021

I’m excited to announce that Confluent was presented with the Databricks ISV Momentum Award at the Databricks Partner Executive Summit last month. This award is given to the partner whose […].

64

64

Teradata

JUNE 20, 2021

Open Banking is re-shaping the landscape of financial services and introducing new types of risks extending beyond data security. Secure open banking is everyone’s responsibility.

Banking 59

Banking Data Security Data 59

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

DataKitchen

JUNE 25, 2021

In this five-module course, Mike Lampa & Chris Bergh teach data professionals to plan their organization's DataOps program for low errors & fast deployment. The post Standing Up a DataOps Program for Practitioners first appeared on DataKitchen.

Programming 59

Programming Data 59

Netflix Tech

JUNE 25, 2021

By Gim Mahasintunan on behalf of Data Platform Engineering. Supporting a rapidly growing base of engineers of varied backgrounds using different data stores can be challenging in any organization. Netflix’s internal teams strive to provide leverage by investing in easy-to-use tooling that streamlines the user experience and incorporates best practices.

Data 62

Data Accessible Accessibility Designing 62

Confluent

JUNE 22, 2021

To the developer or architect seeking to provide their business with as much value as possible, what is the best way to start working with data in motion? Choosing Apache […].

Building 59

Building Cloud MongoDB Data 59

Cloudera

JUNE 24, 2021

CDP Operational Database Experience (COD) is a PaaS offering on the Cloudera Data Platform (CDP). COD enables you to create a new operational database with a few clicks and auto-scales based on your workload. Behind the scenes, COD automatically manages cluster deployment and configuration, reducing overheads related to setting up new database instances.

Database 55

Database Data Lake Java Cloud 55

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Monte Carlo

JUNE 24, 2021

Monte Carlo , the data reliability company, today announced two new strategic hires to its leadership team: Daniel Day , Head of Marketing, and Jordan Van Horn , Head of Revenue. With experience leading award-winning go-to-market teams at Snowflake and Segment, Day and Van Horn share a deep expertise in the data industry and will help Monte Carlo meet the growing demands as the industry leader in Data Observability.

High Quality Data 52

High Quality Data Data Management IT 52

Zalando Engineering

JUNE 23, 2021

Introduction In our previous post about Technology Choices at Zalando we spoke about a few problems with scaling technology selection in Tech companies. Since then, we have focused on the remaining categories of the Tech Radar beyond languages and the Tech Radar contribution process. Now, we'd like to reflect on our lessons learned, which you can use when designing technology selection processes.

Technology 52

Technology Data Lake PostgreSQL AWS 52

DataKitchen

JUNE 23, 2021

Cognizant's JP Thakur & DataKitchen's Chris Bergh discuss how DataOps sets the foundation for Data Modernization initiatives enabling continuous data & insight. The post The Role of DataOps in Data Modernization first appeared on DataKitchen.

Data 52

Data 52

ProjectPro

JUNE 19, 2021

If you have heard of Machine Learning and Deep Learning, you must have also heard about cost (error or loss) functions. But, even if you haven't, fret not! The cost function, in simple words, is a way of measuring the performance of a Machine Learning model by attributing a cost to every 'mistake' or wrong prediction that the model makes. But as we know from personal experience, one can gain little by simply knowing that a mistake has occurred or how many, for that matter, if you have no clue ho

Algorithm 52

Algorithm IT Machine Learning Deep Learning 52

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems

Teradata

JUNE 22, 2021

When undertaking the expense & time to execute a proof of concept on your journey to the cloud, make sure the efforts are well defined & drive an actionable outcome.

Cloud 52

Cloud 52

Rock the JVM

JUNE 23, 2021

Explore ZIO's unique fiber model for concurrency: see how it stands out from other effect libraries in the Scala ecosystem

Scala 52

Scala IT 52

RudderStack

JUNE 22, 2021

RudderStack product news update includes the latest security features- Mobile SDK Distribution, New Spreadsheet, Database Destinations, and RudderStack video content.

Database 40

Database 40

Preset

JUNE 25, 2021

Analyzing data that’s frequently updated? A comprehensive walkthrough of how to analyze and visualize ClickHouse data in Superset.

Data 40

Data 40

Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network

In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.

Project

Teradata

JUNE 24, 2021

The connected cloud data warehouse is fundamental to Data Mesh implementation in large and complex organisations. Find out why.

Data Warehouse 52

Data Warehouse Cloud Data 52

FreshBI

JUNE 23, 2021

It’s a jungle out there Back in the day- when I was stuck on a DAX problem, I used to toggle through the IntelliSense in PowerBI one letter at a time. I’ve learned much since then and in this blog I’d like to share my experience with using NORM.DIST in Dax. A: ABS ACOS ACOSH … B: BETA.DIST BETA.INV BLANK Etc…. Hours wasted. Mistakes were made A MUCH better use of my time would have been reviewing quality solutions to real world problems.

BI 52

BI Datasets Computer Science Utilities 52

RudderStack

JUNE 24, 2021

This blog talks in detail about RudderStack's product vision- what you can expect from RudderStack in the coming quarters in terms of features and experience.

40

40

ProjectPro

JUNE 24, 2021

According to Gartner , organizations can suffer a financial loss of up to 15 million dollars for the poor quality of data. As per McKinsey , 47% of organizations believe that data analytics has impacted the market in their respective industries. According to Forbes , in 2012 only 12% of Fortune 1000 companies reported having a CDO (Chief Data Officer).

Data Analytics 40

Data Analytics Project Insurance Hadoop 40

Speaker: Evelyn Chou

Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.

Business Intelligence

Cloudera

JUNE 23, 2021

Climate change is a challenge for insurers in some obvious ways, such as stronger and more frequent natural disasters. Yet there are also more subtle risks to monitor, including changes to insured assets, risks, and exposures. Climate impacts the production quality and quantity of insured consumable goods, their location, and their supply chains. Climate change can also impact the insurance carrier as an enterprise itself—similarly to cyber risks, insurers underwrite cyber risks for their custom

Insurance 52

Insurance Portfolio Food Banking 52

DataKitchen

JUNE 24, 2021

In May 2021 at the CDO & Data Leaders Global Summit, DataKitchen sat down with the following data leaders to learn how to use DataOps to drive agility and business value. Kurt Zimmer, Head of Data Engineering for Data Enablement at AstraZeneca. Ryan Chapin, Former Executive Manager, Advanced Additive Design, Chief Product and Portfolio Manager, GE Aviation.

Pipeline-centric 84

Pipeline-centric Manufacturing Education Data Cleanse 84