Analytics Vidhya

FEBRUARY 6, 2023

This ensures easy […] The post What are Data Access Object and Data Transfer Object in Python? Especially while working with databases, it is often considered a good practice to follow a design pattern. appeared first on Analytics Vidhya.

Accessibility 234

Accessibility Accessible Python Database 234

Snowflake

APRIL 17, 2024

Yet while SQL applications have long served as the gateway to access and manage data, Python has become the language of choice for most data teams, creating a disconnect. Recognizing this shift, Snowflake is taking a Python-first approach to bridge the gap and help users leverage the power of both worlds.

Data Pipeline 112

Data Pipeline Python Data Engineering Data Engineer 112

Engineering at Meta

FEBRUARY 12, 2024

By now you’re already aware that Python 3.12 Learn how and why they built these new features for Python and how they worked with and engaged with the Python community. The post Meta loves Python appeared first on Engineering at Meta. has been released. Send us feedback on Instagram , Threads , or X.

Python 106

Python Software Engineering Software Engineer Engineering 106

Analytics Vidhya

JUNE 20, 2023

Traditional data engineering solutions, such as Apache Airflow, have played an important role in orchestrating and controlling data operations in order to tackle these difficulties.

Data Process 217

Data Process Data Engineering Data Engineer Process 217

Data Engineering Podcast

APRIL 14, 2024

The default association with the term "database" is relational engines, but non-relational engines are also used quite widely. In this episode Oren Eini, CEO and creator of RavenDB, explores the nuances of relational vs. non-relational engines, and the strategies for designing a non-relational database.

Non-relational Database 147

Non-relational Database Relational Database Database Designing 147

Seattle Data Guy

NOVEMBER 19, 2024

Most of us will turn to Python and our trusty list of Python libraries and start plugging away. Of course, there are many challenges… Read more The post Challenges You Will Face When Parsing PDFs With Python – How To Parse PDFs With Python appeared first on Seattle Data Guy.

Python 130

Python Data Data Science Data Engineering 130

Confessions of a Data Guy

JUNE 3, 2024

Nothing will raise the hackles on the backs of hairy and pale programmers who’ve been stuck in their mom’s basement for a decade like bringing up OOP (Object Oriented Programming), especially in the context of Python. appeared first on Confessions of a Data Guy.

Python 130

Python Programming IT Data 130

Snowflake

OCTOBER 23, 2023

One of our goals at Snowflake is to ensure we continue to deliver a best-in-class platform for Python developers. Snowflake customers are already harnessing the power of Python through Snowpark , a set of runtimes and libraries that securely deploy and process non-SQL code directly in Snowflake.

Python 141

Python Cloud Data Science Machine Learning 141

Cloudyard

SEPTEMBER 2, 2024

Read Time: 1 Minute, 36 Second Snowflake’s support for Python stored procedures allows data engineers and scientists to leverage Python’s vast ecosystem directly within Snowflake. This capability enables advanced analytics, custom data processing, and seamless integration of Python libraries.

Python 97

Python Utilities Coding Data Engineering 97

Confessions of a Data Guy

JANUARY 19, 2023

For those using Python, it’s probably one of the […] The post Replacing Pandas with Polars. I haven’t used Pandas in many a year, decades, or whatever. We’ve all been there, done that. Pandas I mean. I would dare say it’s a rite of passage for most data folk. A Practical Guide.

Python 361

Python IT Data Big Data 361

KDnuggets

NOVEMBER 5, 2024

Enhance your data science portfolio with these seven engaging Python projects that demonstrate essential programming and software engineering skills.

Portfolio 122

Portfolio Data Science Python Project 122

Data Engineering Podcast

JUNE 30, 2024

Petr shares his journey from being an engineer to founding Synq, emphasizing the importance of treating data systems with the same rigor as engineering systems. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex.

Pipeline-centric 147

Pipeline-centric Engineering Data Lake High Quality Data 147

Seattle Data Guy

MAY 26, 2024

Many data engineers and analysts don’t realize how valuable the knowledge they have is. They’ve spent hours upon hours learning SQL, Python, how to properly analyze data, build data warehouses, and understand the differences between eight different ETL solutions.

Consulting 130

Consulting Engineering Data Warehouse SQL 130

Analytics Vidhya

FEBRUARY 20, 2023

This blog is a tutorial for building intuitive frontend interfaces for Machine Learning models using two popular open-source libraries […] The post Streamlit vs Gradio – A Guide to Building Dashboards in Python appeared first on Analytics Vidhya.

Python 152

Python Building Machine Learning Business Intelligence 152

Data Engineering Podcast

FEBRUARY 5, 2023

In that time there have been a number of generational shifts in how data engineering is done. __init__ covers the Python language, its community, and the innovative ways it is being used. In that time there have been a number of generational shifts in how data engineering is done. Closing Announcements Thank you for listening!

Data Engineering 147

Data Engineering Data Engineer Engineering PostgreSQL 147

Confessions of a Data Guy

JUNE 11, 2024

One of the things I love about Python is its flexibility and huge community, a community that puts out a never-ending stream of useful packages for the average Software Engineer.

Python 100

Python Building Software Engineering Software Engineer 100

Data Engineering Podcast

MAY 18, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat. As someone who listens to the Data Engineering Podcast, you know that the road from tool selection to production readiness is anything but smooth or straight.

Building 278

Building Data Lake High Quality Data Business Intelligence 278

KDnuggets

OCTOBER 24, 2022

Preprocessing data for machine learning models is a core general skill for any Data Scientist or Machine Learning Engineer. Follow this guide using Pandas and Scikit-learn to improve your techniques and make sure your data leads to the best possible outcome.

Python 160

Python Machine Learning Data Engineering 160

Christophe Blefari

JANUARY 20, 2024

Learn data engineering, all the references ( credits ) This is a special edition of the Data News. But right now I'm in holidays finishing a hiking week in Corsica 🥾 So I wrote this special edition about: how to learn data engineering in 2024. The idea is to create a living reference about Data Engineering.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

The Pragmatic Engineer

OCTOBER 17, 2024

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. Python, Angular, SSR, SQLite, DuckDB, Cockroach DB, and many others. Tech stack. Benchmarking tools.

Cloud 277

Cloud AWS Metadata Cloud Computing 277

Data Engineering Weekly

JULY 21, 2024

Editor’s Note: A New Series on Data Engineering Tools Evaluation There are plenty of data tools and vendors in the industry. Data Engineering Weekly is launching a new series on software evaluation focused on data engineering to better guide data engineering leaders in evaluating data tools. Stay tuned!!!

Data Engineering 118

Data Engineering Data Engineer Engineering Generalist 118

Start Data Engineering

JANUARY 12, 2023

Singleton, & Object pool patterns Python helpers 1. Introduction Sample project Code design patterns 1. Functional design 2. Factory pattern 3. Strategy pattern 4. Dataclass 3. Context Managers 4. Testing with pytest 5.

Designing 147

Designing Coding Python Data Pipeline 147

Data Engineering Podcast

NOVEMBER 12, 2023

Generative AI has accelerated the ability of developer tools to provide useful suggestions that speed up the work of engineers. Tabnine is one of the main platforms offering an AI powered assistant for software engineers. Python, Java, C++, etc.) With Materialize, you can! How does that translate into computer languages?

Software Engineering 147

Software Engineering Software Engineer Engineering Data Lake 147

Start Data Engineering

JULY 1, 2024

Code is an interface to the execution engine 3. How to choose the execution engine and the coding interface 3.1. Chose execution engine based on your workload 3.1.1. Types of execution engine 3.1.2. Criteria to chose your execution engine 3.2. Introduction 2. Types of coding interfaces 3.2.2.

SQL 130

SQL Python Coding Engineering 130

Confessions of a Data Guy

FEBRUARY 25, 2024

I love to write Rust … but I deploy Python. Even when I know I […] The post Why I Love Rust, but Deploy Python appeared first on Confessions of a Data Guy. I’m not sure if others have this same problem, maybe they are lucky, they get to build in their favorite language 24/7, it’s their tool of choice.

Python 113

Python Building Data IT 113

Data Engineering Weekly

JULY 28, 2024

link] Chip Huyan: Building A Generative AI Platform We can’t deny that Gen-AI is becoming an integral part of product strategy, pushing the need for platform engineering. The blog Prompt Engineering for a Better SQL Code Generation With LLMs is a pretty good guide on applying prompt engineering to improve productivity.

Data Engineering 110

Data Engineering Data Engineer Engineering Database-centric 110

KDnuggets

MAY 16, 2024

This guide introduces some key techniques in the feature engineering process and provides practical examples in Python.

Engineering 103

Engineering Python Process Machine Learning 103

Analytics Vidhya

APRIL 17, 2023

Introduction Kedro is an open-source Python framework for creating reproducible, maintainable, and modular data science code. It uses best practices of software engineering to build production-ready data science pipelines. This article will give you a glimpse of Kedro framework using news classification tasks.

Data Science 221

Data Science Software Engineering Software Engineer Machine Learning 221

Data Engineering Podcast

FEBRUARY 18, 2024

In this episode Dain Sundstrom, CTO of Starburst, explains how the combination of the Trino query engine and the Iceberg table format offer the ease of use and execution speed of data warehouses with the infinite storage and scalability of data lakes. Go to dataengineeringpodcast.com/dagster today to get started. Your first 30 days are free!

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Data Engineering Podcast

JULY 2, 2023

Summary Feature engineering is a crucial aspect of the machine learning workflow. In this episode Razi Raziuddin shares how data engineering teams can support the machine learning workflow through the development and support of systems that empower data scientists and ML engineers to build and maintain their own features.

Machine Learning 130

Machine Learning Data Engineering Data Engineer Engineering 130

KDnuggets

AUGUST 16, 2024

Explore essential tools and skills for AI engineers: Python, R, big data frameworks, and cloud services essential for building and optimizing AI systems.

Engineering 113

Engineering Big Data Python Cloud 113

Confessions of a Data Guy

AUGUST 1, 2024

The post Daft: Distributed Dataframes with Python. appeared first on Confessions of a Data Guy.

Python 100

Python Data Big Data Data Engineering 100

Engineering at Meta

JULY 16, 2024

The key to developer velocity across AI lies in minimizing time to first batch (TTFB) for machine learning (ML) engineers. For example, during the rollout of the open source Python Cinder runtime , AI Lab was used to yield a 2x increase on original TTFB improvements, reducing TTFB by up to 40%. Example TTFB measurement with components.

Machine Learning 114

Machine Learning Engineering Python Coding 114

Knowledge Hut

SEPTEMBER 24, 2024

If you're considering pursuing a career as a software engineer, it's an exciting field with lots of potential for growth and opportunity. But becoming a software engineer requires more than having the right degree and technical skills. Who is a Software Engineer? So, what happened?

Software Engineering 98

Software Engineering Software Engineer Engineering Amazon Web Services 98

Seattle Data Guy

FEBRUARY 11, 2023

Apache Airflow is a very popular tool that data engineers rely on. Why do data engineers like Airflow? What are… Read more The post What Is Apache Airflow – Data Engineering Consulting appeared first on Seattle Data Guy. Also, what does Apache Airflow event do? What is a DAG? Why do people use Apache Airflow?

Consulting 147

Consulting Data Engineering Data Engineer Engineering 147

databricks

NOVEMBER 7, 2023

have brought an exciting feature to the table: Python user-defined table functions (UDTFs). Apache Spark™ 3.5 and Databricks Runtime 14.0 In this blog p.

Python 119

Python Data Engineering Data Engineer Engineering 119

Confessions of a Data Guy

APRIL 16, 2023

You might think […] The post DuckDB vs Polars for Data Engineering. I haven’t seen this since Databricks and Snowflake first came out and started throwing mud at each other. appeared first on Confessions of a Data Guy.

Data Engineering 147

Data Engineering Data Engineer Engineering SQL 147