Data Engineering Podcast

FEBRUARY 18, 2024

In this episode Dain Sundstrom, CTO of Starburst, explains how the combination of the Trino query engine and the Iceberg table format offer the ease of use and execution speed of data warehouses with the infinite storage and scalability of data lakes. Data lakes are notoriously complex. Your first 30 days are free!

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Snowflake

OCTOBER 23, 2023

One of our goals at Snowflake is to ensure we continue to deliver a best-in-class platform for Python developers. Snowflake customers are already harnessing the power of Python through Snowpark , a set of runtimes and libraries that securely deploy and process non-SQL code directly in Snowflake.

Python 141

Python Cloud Data Science Machine Learning 141

Analytics Vidhya

MARCH 5, 2023

Introduction NumPy is an open-source library in python and a must-learn if you want to enter the data science ecosystem. It is the library underpinning other important libraries such as Pandas, matplotlib, Scipy, scikit-learn, etc.

Python 269

Python Data Science Programming IT 269

KDnuggets

APRIL 19, 2024

The collection includes free courses on Python, SQL, Data Analytics, Business Intelligence, Data Engineering, Machine Learning, Deep Learning, Generative AI, and MLOps.

Data Science 155

Data Science Deep Learning Business Intelligence Machine Learning 155

Jesse Anderson

DECEMBER 12, 2022

As clearly evident as that sounds now, my writing about needing data engineering went heavily against the grain of everything that was written at the time. DJ Patil coined the term Data Scientist in 2008. For the majority of companies, that was the only title working on data problems at scale.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Data Engineering Weekly

JULY 21, 2024

Editor’s Note: A New Series on Data Engineering Tools Evaluation There are plenty of data tools and vendors in the industry. Data Engineering Weekly is launching a new series on software evaluation focused on data engineering to better guide data engineering leaders in evaluating data tools.

Data Engineering 119

Data Engineering Data Engineer Engineering Generalist 119

Analytics Vidhya

APRIL 17, 2023

Introduction Kedro is an open-source Python framework for creating reproducible, maintainable, and modular data science code. It uses best practices of software engineering to build production-ready data science pipelines.

Data Science 221

Data Science Software Engineer Software Engineering Machine Learning 221

Data Engineering Weekly

JULY 28, 2024

Switching from Apache Spark to Ray improves compact 12X larger datasets than Apache Spark, improves cost efficiency by 91%, and processes 13X more data per hour. What are the different roles inside the data engineering functions, and what should their ratio be? A key highlight for me is the following features from Maestro.

Data Engineering 111

Data Engineering Data Engineer Engineering Database-centric 111

Cloudera

JANUARY 6, 2021

Python is used extensively among Data Engineers and Data Scientists to solve all sorts of problems from ETL/ELT pipelines to building machine learning models. Apache HBase is an effective data storage system for many workflows but accessing this data specifically through Python can be a struggle.

Machine Learning 134

Machine Learning Data Science Database Building 134

Knowledge Hut

SEPTEMBER 6, 2023

Nowadays, I often hear people saying they aspire to become data scientists or they want to work with data, but they don’t know the path to do so. I myself have faced this problem and data science certifications come as a rescue for this problem. What is Data Science Certification?

Data Science 97

Data Science Certification Business Analyst Machine Learning 97

Knowledge Hut

DECEMBER 26, 2023

Of course, handling such huge amounts of data and using them to extract data-driven insights for any business is not an easy task; and this is where Data Science comes into the picture. You can execute this by learning data science with python and working on real projects.

Data Science 97

Data Science BI Business Intelligence Machine Learning 97

KDnuggets

JANUARY 16, 2023

3 Free Machine Learning Courses for Beginners • The Complete Machine Learning Study Roadmap •Markdown Cheat Sheet • Learn Data Science From These GitHub Repositories • 7 Essential Cheat Sheets for Data Engineering • Scikit-Learn Cheat Sheet for Machine Learning • 7 Super Cheat Sheets You Need To Ace Machine Learning Interview.

Data Science 108

Data Science Portfolio Python Machine Learning 108

Knowledge Hut

JANUARY 18, 2024

I’ve often noticed that people use terms like Data Science and Artificial Intelligence ( AI ) interchangeably. The key connection between Data Science and AI is data. Understanding  Data Science course eligibility  can help you understand more about Data Science. What is Data Science?

Data Science 98

Data Science Deep Learning Business Analyst Data Mining 98

Towards Data Science

OCTOBER 21, 2023

Advanced ETL techniques for beginners Continue reading on Towards Data Science »

Data Engineering 81

Data Engineering Data Engineer Python Engineering 81

Team Data Science

JANUARY 8, 2021

The purpose of this post is to expose you to the skills needed as a data engineer; now let’s look into them Understand the fundamental skill Recently, functions of computer engineering have become more important in organizations that are handling vast volumes of data, including data in diverse formats.

Data Engineering 130

Data Engineering Data Engineer Engineering Unstructured Data 130

Towards Data Science

MAY 22, 2023

Solving data preparation tasks with ChatGPT Photo by Ricardo Gomez Angel on Unsplash Data engineering makes up a large part of the data science process. In CRISP-DM this process stage is called “data preparation”. It comprises tasks such as data ingestion, data transformation and data quality assurance.

Data Engineering 89

Data Engineering Data Engineer Engineering Data Science 89

Seattle Data Guy

NOVEMBER 19, 2024

Most of us will turn to Python and our trusty list of Python libraries and start plugging away. Of course, there are many challenges… Read more The post Challenges You Will Face When Parsing PDFs With Python – How To Parse PDFs With Python appeared first on Seattle Data Guy.

Python 130

Python Data Data Science Data Engineering 130

Knowledge Hut

NOVEMBER 27, 2023

The Data Science learning path is a collective set of curated courses that comprise a learning plan for achieving the required skills for the data scientist role. While the time limit to complete the learning path to become a data scientist can expect 8-9 months to get through all Data Science courses.

Data Science 97

Data Science Healthcare Machine Learning Algorithm 97

Knowledge Hut

JANUARY 18, 2024

Most organizations are still in the initial stages of learning how to apply data science to gain business benefits and healthy returns. What is Data Science? Data science is a collaborative field that deals with the study of data using various tools and methods. Who Is a Full-Stack Data Scientist?

Data Science 96

Data Science Computer Science Programming Language Machine Learning 96

Data Engineering Podcast

JANUARY 30, 2022

Summary Pandas is a powerful tool for cleaning, transforming, manipulating, or enriching data, among many other potential uses. As a result it has become a standard tool for data engineers for a wide range of applications. What are the main tasks that you have seen Pandas used for in a data engineering context?

Data Engineering 100

Data Engineering Data Engineer Engineering Python 100

Knowledge Hut

JANUARY 18, 2024

Per the BLS, the expected growth rate of job vacancies for data scientists and software engineers is around 22% by 2030. Although both Data Science and Software Engineering domains focus on math, code, data, etc., Is mastering data science beneficial or building software is a better career option?

Software Engineer 97

Software Engineer Software Engineering Data Science Engineering 97

KDnuggets

AUGUST 28, 2023

This concise primer walks through setting up a Python data science environment using Docker, covering creating a Dockerfile, building an image, running a container, sharing and deploying images, and pushing to Docker Hub.

Data Science 105

Data Science Python Data Building 105

Knowledge Hut

MAY 3, 2023

With companies increasingly relying on data-driven insights to make informed decisions, there has never been a greater need for skilled specialists who can manage and evaluate vast amounts of data. The roles of data analyst and data engineer have emerged as two of the most in-demand professions in today's job market.

Data Engineering 98

Data Engineering Data Engineer Engineering Data Cleanse 98

Knowledge Hut

DECEMBER 29, 2023

The market for analytics is flourishing, as is the usage of the phrase Data Science. Professionals from a variety of disciplines use data in their day-to-day operations and feel the need to understand cutting-edge technology to get maximum insights from the data, therefore contributing to the growth of the organization.

Data Science 98

Data Science Data Mining Deep Learning Programming Language 98

Knowledge Hut

JUNE 26, 2023

Welcome to the world of data engineering, where the power of big data unfolds. If you're aspiring to be a data engineer and seeking to showcase your skills or gain hands-on experience, you've landed in the right spot. What are Data Engineering Projects?

Data Engineering 98

Data Engineering Data Engineer Coding Project 98

Data Engineering Podcast

SEPTEMBER 17, 2023

Summary A significant amount of time in data engineering is dedicated to building connections and semantic meaning around pieces of information. Linked data technologies provide a means of tightly coupling metadata with raw information. __init__ covers the Python language, its community, and the innovative ways it is being used.

Building 189

Building SQL BI Python 189

AltexSoft

OCTOBER 30, 2021

If you’re an executive who has a hard time understanding the underlying processes of data science and get confused with terminology, keep reading. We will try to answer your questions and explain how two critical data jobs are different and where they overlap. Data science vs data engineering.

Data Engineering 98

Data Engineering Data Engineer Engineering Machine Learning 98

Knowledge Hut

NOVEMBER 30, 2023

Are you interested in knowing how to become a data scientist with no experience  but not sure how to go about it? Here you will learn how to get your first data science job. To get a data science job without prior experience, you can first pursue o n line courses in Data Science.

Data Science 98

Data Science Portfolio Programming Language Programming 98

Cloudera

SEPTEMBER 17, 2020

For enterprise organizations, managing and operationalizing increasingly complex data across the business has presented a significant challenge for staying competitive in analytic and data science driven markets. CDP data lifecycle integration and SDX security and governance.

Data Pipeline 105

Data Pipeline Data Engineering Data Engineer Engineering 105

Data Engineering Podcast

JULY 17, 2022

Summary Data engineering is a large and growing subject, with new technologies, specializations, and "best practices" emerging at an accelerating pace. Their SDKs make event streaming from any app or website easy, and their state-of-the-art reverse ETL pipelines enable you to send enriched data to any cloud tool.

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

FEBRUARY 25, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Data lakes are notoriously complex. Go to dataengineeringpodcast.com/dagster today to get started.

Database 162

Database Technology Data Lake High Quality Data 162

Team Data Science

NOVEMBER 4, 2020

Week 1: 10/9/20 - 10/16/20 In my quest to further improve my overall data science skills, I pulled the trigger on October 9th, 2020, and enrolled in a Data Engineering boot camp lead by Andreas Kretz. I have a background in Aerospace Engineering and have been in the industry for close to 15 years now.

Data Science 130

Data Science Machine Learning Programming Language Business Intelligence 130

Data Engineering Podcast

MARCH 3, 2024

Colleen Tartow has worked across all stages of the data lifecycle, and in this episode she shares her hard-earned wisdom about how to conduct an AI program for your organization. Data lakes are notoriously complex. __init__ covers the Python language, its community, and the innovative ways it is being used.

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

KDnuggets

DECEMBER 16, 2021

Most viewed KDnuggets stories in 2021 focused on Data Scientists vs Data Engineers; How to become a Data Scientist; Increase income with Data Science; Stunning visualizations using python; and more.

Data Science 83

Data Science Data Engineering Data Engineer Engineering 83

Towards Data Science

MAY 11, 2023

or general-purpose languages (Python, JavaScript). Whereas the author illustrates his examples using JavaScript and Java, this article attempts to demonstrate the ideas in Python. Following along the article, you’ll find simple code snippets in Python that illustrate how each principle can be adhered to or broken.

Programming 87

Programming Python Data Schemas Java 87

Data Engineering Podcast

OCTOBER 15, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Process 182

Process Building SQL BI 182

Data Engineering Weekly

AUGUST 4, 2024

Take Astro (the fully managed Airflow solution) for a test drive today and unlock a suite of features designed to simplify, optimize, and scale your data pipelines. Hudi-rs project is an interesting one that provides an abstraction to bring part of Hudi data to DuckDB for interactive analytics.

Data Engineering 81

Data Engineering Data Engineer Engineering Data 81