Python - Data Engineering Digest

AWS Lambdas – Python vs Rust. Performance and Cost Savings.

Confessions of a Data Guy

FEBRUARY 26, 2023

Someone on Linkedin recently brought up the point that companies could save gobs of money by swapping out AWS Python lambdas for Rust ones. While it raised the ire of many a Python Data Engineer, I thought it sounded like a great idea. At least it’s an excuse to […] The post AWS Lambdas – Python vs Rust.

AWS

AWS Python Data Engineer Data Engineering

Challenges You Will Face When Parsing PDFs With Python – How To Parse PDFs With Python

Seattle Data Guy

NOVEMBER 19, 2024

Most of us will turn to Python and our trusty list of Python libraries and start plugging away. Of course, there are many challenges… Read more The post Challenges You Will Face When Parsing PDFs With Python – How To Parse PDFs With Python appeared first on Seattle Data Guy.

Python

Python Data Data Science Data Engineer

Isolated Python Environments using Docker

Analytics Vidhya

FEBRUARY 6, 2023

Introduction While working with multiple projects, there are chances of issues with versions of packages in python; for example, a project needs a new version of a package, and another requires a different version. Sometimes the python version itself changes from project to project.

Python

Python Project Management Data Engineer

Building cost effective data pipelines with Python & DuckDB

Start Data Engineering

MAY 28, 2024

KISS: DuckDB + Python = easy to debug and quick to develop 4. Cost calculation: DuckDB + Ephemeral VMs = dirt cheap data processing 4.3. Processing data less than 100GB? Use DuckDB 4.4. Distributed systems are scalable, resilient to failures, & designed for high availability 4.5.

Data Pipeline

Data Pipeline Python Building Data

Apache Airflow® Crash Course: From 0 to Running your Pipeline in the Cloud

Airflow enables you to define workflows as Python code, allowing for dynamic and scalable pipelines suitable to any use case from ETL/ELT to running ML/AI operations in production. With over 30 million monthly downloads, Apache Airflow is the tool of choice for programmatically authoring, scheduling, and monitoring data pipelines.

Cloud

What are Data Access Object and Data Transfer Object in Python?

Analytics Vidhya

FEBRUARY 6, 2023

This ensures easy […] The post What are Data Access Object and Data Transfer Object in Python? Especially while working with databases, it is often considered a good practice to follow a design pattern. appeared first on Analytics Vidhya.

Accessibility

Accessibility Accessible Python Database

7 Python Libraries Every Data Engineer Should Know

KDnuggets

APRIL 25, 2024

Here’s a list of Python libraries you’ll find super helpful. Interested in switching to data engineering?

Python

Python Data Engineer Data Engineering Engineering

7 Free Online Python REPLs

KDnuggets

SEPTEMBER 24, 2024

Running Python code directly in your browser is incredibly convenient, eliminating the need for Python environment setup and allowing instant code execution without dependency or hardware concerns. I am a strong advocate of using a cloud-based IDE for working with data, machine learning, and learning Python as a beginner.

Python

Python Machine Learning Coding Programming

Seamless SQL And Python Transformations For Data Engineers And Analysts With SQLMesh

Data Engineering Podcast

JUNE 25, 2023

__init__ covers the Python language, its community, and the innovative ways it is being used. __init__ covers the Python language, its community, and the innovative ways it is being used. Closing Announcements Thank you for listening! Don't forget to check out our other shows. Closing Announcements Thank you for listening!

Data Engineer

Data Engineer Data Engineering Python Engineering

5 Free Python Courses for Data Science Beginners

KDnuggets

MAY 31, 2024

Are you a data science beginner looking to learn Python? Start learning today with these 5 free courses.

Data Science

Data Science Python Data

FastAPI Tutorial: Build APIs with Python in Minutes

KDnuggets

JUNE 13, 2024

Want to build APIs with Python? Learn how to do so using FastAPI with this step-by-step tutorial.

Python

Python Building

10 GitHub Repositories to Master Python

KDnuggets

APRIL 10, 2024

Learn Python through tutorials, blogs, books, project work, and exercises. Access all of it on GitHub for free and join a supportive open-source community.

Python

Python Accessible Accessibility Project

5 Simple Steps to Automate Data Cleaning with Python

KDnuggets

MAY 3, 2024

Automate your data cleaning process with a practical 5-step pipeline in Python, ideal for beginners.

Python

Python Data Process

11 Python Magic Methods Every Programmer Should Know

KDnuggets

NOVEMBER 28, 2023

Want to support the behavior of built-in functions and method calls in your Python classes? Magic methods in Python let you do just that! So let’s uncover the method behind the magic.

Python

10 Python Libraries Every Data Scientist Should Know

KDnuggets

AUGUST 12, 2024

Check out these Python libraries for data science that you can't do without. Want to take the next step in your journey to becoming a data scientist?

Python

Python Data Science Data

Lesser-Known Python Functions That Are Super Useful

KDnuggets

APRIL 2, 2025

Go beyond the basics by adding these cool and useful Python functions to your programming toolbox.

Python

Python Programming

10 Built-In Python Modules Every Data Engineer Should Know

KDnuggets

SEPTEMBER 2, 2024

Check out this round-up of built-in Python modules that'll come in handy for data engineering tasks. Interested in data engineering?

Python

Python Data Engineer Data Engineering Engineering

A New Python Package Manager

KDnuggets

NOVEMBER 14, 2024

Manage Python projects, run scripts and tools, handle dependencies, and install packages—all with the uv tool.

Python

Python Management Project

3 Ways of Building Python Projects using GPT-4o

KDnuggets

AUGUST 7, 2024

Learn about essential AI tools that can help you develop Python projects faster and with fewer bugs using natural language.

Python

Python Project Building

Is Python OOP the Devil? Or Savior?

Confessions of a Data Guy

JUNE 3, 2024

Nothing will raise the hackles on the backs of hairy and pale programmers who’ve been stuck in their mom’s basement for a decade like bringing up OOP (Object Oriented Programming), especially in the context of Python. appeared first on Confessions of a Data Guy.

Python

Python Programming IT Data

Mastering f-strings in Python

KDnuggets

NOVEMBER 7, 2024

Discover how to leverage Python's f-strings (formatted string literals) to write cleaner, more efficient, and more readable code.

Python

Python Coding

Free Harvard Course: Introduction to AI with Python

KDnuggets

DECEMBER 18, 2023

Looking for a great course to learn Artificial Intelligence with Python? Check out this free course from Harvard University.

Python

Tips for Handling Large Datasets in Python

KDnuggets

NOVEMBER 29, 2024

Here are some tips to make working with such large datasets in Python simpler. Working with large datasets is common but challenging.

Datasets

Datasets Python

Convert Bytes to String in Python: A Tutorial for Beginners

KDnuggets

JULY 15, 2024

Strings are common built-in data types in Python. Let’s learn how to convert bytes to string in Python. But sometimes, you may need to work with bytes instead.

Bytes

Bytes Python Data

11 Python Libraries Every AI Engineer Should Know

KDnuggets

FEBRUARY 27, 2025

Here are Python libraries and frameworks you cant miss! Looking to build your AI engineer toolkit in 2025?

Python

Python Engineering Building

10 Python Libraries Every Developer Should Know

KDnuggets

DECEMBER 2, 2024

In this article, we’ll go over Python libraries for tasks like logging, unit testing, data handling, and more — each with features that can simplify your application development.

Python

Python Data

Partial Functions in Python: A Guide for Developers

KDnuggets

SEPTEMBER 19, 2024

In Python, functions often require multiple arguments, and you may find yourself repeatedly passing the same values for certain parameters. Python’s built-in functools module allows you to create partial functions. This is where partial functions can help.

Python

Beginner’s Guide to Machine Learning with Python

KDnuggets

JUNE 4, 2024

Master the Fundamentals of Predictive Modeling with Python: An In-Depth Guide to Machine Learning Algorithms and Sci-kit Learn Implementation.

Machine Learning

Machine Learning Python Algorithm

Mastering Python for Data Science: Beyond the Basics

KDnuggets

MARCH 28, 2024

This article serves as a detailed guide on how to master advanced Python techniques for data science. It covers topics such as efficient data manipulation with Pandas, parallel processing with Python, and how to turn models into web services.

Data Science

Data Science Python Data Process

5 Python Tips for Data Efficiency and Speed

KDnuggets

AUGUST 7, 2024

Want to write better Python code? Get one step closer with this tutorial on writing maintainable, faster, and memory-efficient Python code.

Python

Python Coding Data

5 Free Courses to Master Python for Data Science

KDnuggets

FEBRUARY 7, 2024

Want to learn Python to kickstart your career in data? Here are five free courses to help you master Python for data science.

Data Science

Data Science Python Data

Beginner’s Guide to Building LLM Apps with Python

KDnuggets

JUNE 6, 2024

In this article, you will be impacted by the knowledge you need to start building LLM apps with Python programming language.

Python

Python Building Programming Language Programming

A Gentle Introduction to Rust for Python Programmers

KDnuggets

JANUARY 15, 2025

Python programmers will find Rust's syntax familiar but with more control over memory and performance. Rust is a systems programming language that offers high performance and safety.

Python

Python Programming Language Programming Systems

Pip Install YOU: A Beginner’s Guide to Creating Your Python Library

KDnuggets

AUGUST 22, 2024

Have you ever wanted to create your library in Python? Well, it’s achievable and surprisingly quite easy!

Python

10 Python Libraries Every Data Analyst Should Know

KDnuggets

NOVEMBER 19, 2024

Here's a list of Python libraries you cannot do without. Interested in data analytics?

Python

Python Data Analytics Data

Building Interactive Data Science Applications with Python

KDnuggets

OCTOBER 25, 2024

Using Python to build engaging and interactive applications where users can pass in an input, get and feedback and make use of multimedia elements such as images, videos, and audio.

Python

Python Building Data Science Data

3 Simple Ways to Merge Python Dictionaries

KDnuggets

SEPTEMBER 17, 2024

When working with dictionaries in Python, you’ll sometimes have to merge them into a single dictionary for further processing. In this tutorial, we'll go over three common methods to merge Python dictionaries. Note: You can find.

Python

Python Process

Replacing Pandas with Polars. A Practical Guide.

Confessions of a Data Guy

JANUARY 19, 2023

For those using Python, it’s probably one of the […] The post Replacing Pandas with Polars. I haven’t used Pandas in many a year, decades, or whatever. We’ve all been there, done that. Pandas I mean. I would dare say it’s a rite of passage for most data folk. A Practical Guide.

Python

Python IT Data Big Data

Don’t Manage Your Python Environments, Just Use Docker Containers

KDnuggets

JANUARY 28, 2025

Python environment management can sometimes give you that awful feeling in the pit of your stomach. So don't do it: just use Docker containers.

Python

Python Management AWS IT

How I Would Learn Python in 2025 (If I Could Start Over)

KDnuggets

JANUARY 13, 2025

Ive been programming with Python for over 6 years now. But if I could start over, heres how Id go about learning Python in 2025.

Python

Python Programming

Building Command Line Apps in Python with Click

KDnuggets

OCTOBER 4, 2024

Have you ever wondered how you can easily create command-line applications in Python? Gather yourself up because that is what I am going to cover today.

Python

Python Building

Flask Python: A Comprehensive Guide to Building Web Applications

Edureka

JANUARY 21, 2025

One of the most popular choices among developers is Flask, a Python framework that is both lightweight and flexible. This blog will explain a core web framework, go over the basics of Python and Flask, discuss its uses, show how popular it is, compare it to Django, and give you a general idea of the pros and cons of using Flask.

Python

Python Building Certification Database

Python in Finance: Real Time Data Streaming within Jupyter Notebook

KDnuggets

FEBRUARY 20, 2024

This guide covers dynamic visualizations, a Python for quant finance use case, and Bollinger Bands analysis with live data. Learn a modern approach to stream real-time data in Jupyter Notebook.

Finance

Finance Python Data

7 Python Projects to Boost Your Data Science Portfolio

KDnuggets

NOVEMBER 5, 2024

Enhance your data science portfolio with these seven engaging Python projects that demonstrate essential programming and software engineering skills.

Portfolio

Portfolio Data Science Python Project

AWS Lambdas – Python vs Rust. Performance and Cost Savings.

Challenges You Will Face When Parsing PDFs With Python – How To Parse PDFs With Python

Trending Sources

Isolated Python Environments using Docker

Building cost effective data pipelines with Python & DuckDB

Apache Airflow® Crash Course: From 0 to Running your Pipeline in the Cloud

What are Data Access Object and Data Transfer Object in Python?

7 Python Libraries Every Data Engineer Should Know

7 Free Online Python REPLs

Seamless SQL And Python Transformations For Data Engineers And Analysts With SQLMesh

5 Free Python Courses for Data Science Beginners

FastAPI Tutorial: Build APIs with Python in Minutes

10 GitHub Repositories to Master Python

5 Simple Steps to Automate Data Cleaning with Python

11 Python Magic Methods Every Programmer Should Know

10 Python Libraries Every Data Scientist Should Know

Lesser-Known Python Functions That Are Super Useful

10 Built-In Python Modules Every Data Engineer Should Know

A New Python Package Manager

3 Ways of Building Python Projects using GPT-4o

Is Python OOP the Devil? Or Savior?

Mastering f-strings in Python

Free Harvard Course: Introduction to AI with Python

Tips for Handling Large Datasets in Python

Convert Bytes to String in Python: A Tutorial for Beginners

11 Python Libraries Every AI Engineer Should Know

10 Python Libraries Every Developer Should Know

Partial Functions in Python: A Guide for Developers

Beginner’s Guide to Machine Learning with Python

Mastering Python for Data Science: Beyond the Basics

5 Python Tips for Data Efficiency and Speed

5 Free Courses to Master Python for Data Science

Beginner’s Guide to Building LLM Apps with Python

A Gentle Introduction to Rust for Python Programmers

Pip Install YOU: A Beginner’s Guide to Creating Your Python Library

10 Python Libraries Every Data Analyst Should Know

Building Interactive Data Science Applications with Python

3 Simple Ways to Merge Python Dictionaries

Replacing Pandas with Polars. A Practical Guide.

Don’t Manage Your Python Environments, Just Use Docker Containers

How I Would Learn Python in 2025 (If I Could Start Over)

Building Command Line Apps in Python with Click

Flask Python: A Comprehensive Guide to Building Web Applications

Python in Finance: Real Time Data Streaming within Jupyter Notebook

7 Python Projects to Boost Your Data Science Portfolio

Stay Connected