article thumbnail

Top 20 Data Engineering Project Ideas [With Source Code]

Analytics Vidhya

This article presents the top 20 data engineering project ideas with their source code. Whether you’re […] The post Top 20 Data Engineering Project Ideas [With Source Code] appeared first on Analytics Vidhya. Aspiring data engineers often seek real-world projects to gain hands-on experience and showcase their expertise.

article thumbnail

How to Normalize Relational Databases With SQL Code?

Analytics Vidhya

So, we are […] The post How to Normalize Relational Databases With SQL Code? If a corrupted, unorganized, or redundant database is used, the results of the analysis may become inconsistent and highly misleading. appeared first on Analytics Vidhya.

article thumbnail

How to Develop Serverless Code Using Azure Functions?

Analytics Vidhya

Introduction Azure Functions is a serverless computing service provided by Azure that provides users a platform to write code without having to provision or manage infrastructure in response to a variety of events. Azure functions allow developers […] The post How to Develop Serverless Code Using Azure Functions?

Coding 237
article thumbnail

How to test PySpark code with pytest

Start Data Engineering

Ensure the code’s logic is working as expected with tests 2.1. Set context, run code, check results & clean up 2.2.2. Introduction 2. Test types for data pipelines 2.2. pytest: A powerful Python library for testing 2.2.1. Tests are identified by their name 2.2.3. Use fixture to create fake data for testing 2.2.4.

Coding 208
article thumbnail

3 Challenges of Building Complex Dashboards with Open Source Components

Speaker: Ryan MacCarrigan, Founding Principal, LeanStudio

Many product teams use charting components and open source code libraries to get dashboards and reporting functionality quickly. But what happens when you have a growing user base and additional feature requests?

article thumbnail

An explosion in software engineers using AI coding tools?

The Pragmatic Engineer

GitHub surveyed 500 developers in the US for a sense of how they use AI coding tools. I examine the results and add context on how the survey was conducted.

article thumbnail

Getting Addicted to Coding

KDnuggets

Check out this guide to coding for unmotivated students.

Coding 137
article thumbnail

Apache Airflow® Best Practices for ETL and ELT Pipelines

ELT and decision making criteria on choosing your pipeline architecture strategy Key DAG writing best practices like setting automatic retries, testing and scaling DAGs, and avoiding top-level DAG code An overview of Airflow features that can elevate your ETL and ELT pipelines, including dynamic task mapping, data-driven scheduling, and custom XCom (..)

article thumbnail

Monetizing Analytics Features: Why Data Visualizations Will Never Be Enough

Think your customers will pay more for data visualizations in your application? Five years ago they may have. But today, dashboards and visualizations have become table stakes. Discover which features will differentiate your application and maximize the ROI of your embedded analytics. Brought to you by Logi Analytics.

article thumbnail

New Study: 2018 State of Embedded Analytics Report

Why do some embedded analytics projects succeed while others fail? We surveyed 500+ application teams embedding analytics to find out which analytics features actually move the needle. Read the 6th annual State of Embedded Analytics Report to discover new best practices. Brought to you by Logi Analytics.

article thumbnail

Digital Adoption Platform Fundamentals: The Key To Product Innovation And Elevated CX

Speaker: Pulkit Agrawal

DAPs are no-code solutions that integrate seamlessly with various technologies, and by fostering user adoption, these platforms help organizations maximize the potential of their tech investments.