Data Engineering Podcast

FEBRUARY 5, 2023

In that time there have been a number of generational shifts in how data engineering is done. Go to [dataengineeringpodcast.com/materialize]([link] Support Data Engineering Podcast Summary This podcast started almost exactly six years ago, and the technology landscape was much different than it is now.

Data Engineering 147

Data Engineering Data Engineer Engineering PostgreSQL 147

KDnuggets

JULY 13, 2022

Linear Algebra for Data Science; 10 Modern Data Engineering Tools; Python String Processing Cheatsheet; Simple Salary Guide for Tech Experts 2022; 16 Essential DVC Commands for Data Science.

Data Science 127

Data Science Data Engineering Data Engineer Engineering 127

Jesse Anderson

JANUARY 29, 2024

The premier of my latest talk covering The State of Data Engineering. This starts with data warehousing and goes into data science. I finish off by showing how data engineering can avoid the same fate as data warehousing and data science.

Data Engineering 130

Data Engineering Data Engineer Engineering Data Science 130

Data Engineering Weekly

JULY 21, 2024

Editor’s Note: A New Series on Data Engineering Tools Evaluation There are plenty of data tools and vendors in the industry. Data Engineering Weekly is launching a new series on software evaluation focused on data engineering to better guide data engineering leaders in evaluating data tools.

Data Engineering 119

Data Engineering Data Engineer Engineering Generalist 119

KDnuggets

JANUARY 17, 2022

In this article, I will describe three of the most promising career options within the data industry? — data analytics, data science, and data engineering.

Data Engineering 147

Data Engineering Data Engineer Engineering Data 147

Jesse Anderson

DECEMBER 12, 2022

Apache Spark came in 2009 and gave a unified batch and streaming engine. Apache Flink came in 2011 and gave us our first real streaming engine. Apache Kafka came in 2011 and gave the industry a much better way to move real-time data. DJ Patil coined the term Data Scientist in 2008. We lacked a scalable pub/sub system.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Analytics Vidhya

APRIL 24, 2023

Introduction South Africa is not an exception as data science-driven economic change sweeps the world. The nation is seeing an increase in demand for qualified data science workers as a result of its booming IT sector and developing data-driven industries.

Data Science 228

Data Science Programming Data IT 228

Data Engineering Weekly

JULY 28, 2024

link] Chip Huyan: Building A Generative AI Platform We can’t deny that Gen-AI is becoming an integral part of product strategy, pushing the need for platform engineering. Switching from Apache Spark to Ray improves compact 12X larger datasets than Apache Spark, improves cost efficiency by 91%, and processes 13X more data per hour.

Data Engineering 111

Data Engineering Data Engineer Engineering Database-centric 111

Analytics Vidhya

APRIL 17, 2023

These sessions will cover everything from conversational intelligence to people analytics covering topics like […] The post Ace Your Data Science Skills with DataHour Sessions appeared first on Analytics Vidhya.

Data Science 224

Data Science Data Technology Deep Learning 224

Analytics Vidhya

MAY 22, 2023

Introduction Join us in this interview as Sumeet shares his background, journey as a former Data Scientist to a software engineer, and learn the captivating aspects of his current job. He provides insights into the future of data science and software engineering and offers valuable advice for career transitioners.

Software Engineer 46

Software Engineer Software Engineering Engineering Data Science 46

Seattle Data Guy

MAY 20, 2023

Starting new data engineering projects can be challenging. Data engineers can get stuck on finding the right data for their data engineering project or picking the right tools.

Data Engineering 147

Data Engineering Data Engineer Project Engineering 147

KDnuggets

DECEMBER 9, 2022

As more people are entering the field of Data Science and more companies are hiring for data-centric roles, what type of jobs are currently in highest demand?

Engineering 160

Engineering Data Engineering Data Engineer Data Science 160

Data Engineering Podcast

FEBRUARY 18, 2024

In this episode Dain Sundstrom, CTO of Starburst, explains how the combination of the Trino query engine and the Iceberg table format offer the ease of use and execution speed of data warehouses with the infinite storage and scalability of data lakes. Data lakes are notoriously complex. Your first 30 days are free!

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

KDnuggets

FEBRUARY 28, 2024

Begin your data professional journey from the basics of statistics to building a production-grade AI application.

Machine Learning 160

Machine Learning Data Science Data Engineering Data Engineer 160

Analytics Vidhya

JUNE 12, 2023

Introduction Meet Tajinder, a seasoned Senior Data Scientist and ML Engineer who has excelled in the rapidly evolving field of data science. Tajinder’s passion for unraveling hidden patterns in complex datasets has driven impactful outcomes, transforming raw data into actionable intelligence.

Machine Learning 228

Machine Learning Engineering Raw Data Data Science 228

KDnuggets

FEBRUARY 27, 2023

5 SQL Visualization Tools for Data Engineers • Free TensorFlow 2.0 Complete Course • The Importance of Probability in Data Science • 4 Ways to Rename Pandas Columns • 5 Statistical Paradoxes Data Scientists Should Know

SQL 137

SQL Data Engineering Data Engineer Engineering 137

KDnuggets

DECEMBER 27, 2023

Discover a collection of top courses to launch your dream career or master a new skill, all for free!

Machine Learning 152

Machine Learning Data Science Data Engineering Data Engineer 152

Confessions of a Data Guy

DECEMBER 29, 2022

Data engineering is a vital field within the realm of data science that focuses on the practical aspects of collecting, storing, and processing large amounts of data. appeared first on Confessions of a Data Guy. Here it is.

Data Engineering 130

Data Engineering Data Engineer Engineering IT 130

KDnuggets

NOVEMBER 7, 2023

No, they’re not the same jobs! Learn what responsibilities, skills, and tools used make them different. Then, choose the right career path for you.

Data Science 157

Data Science Data Engineering Data Engineer Engineering 157

KDnuggets

AUGUST 18, 2022

If you’re considering a career in data science, it’s important to understand how these two fields differ, and which one might be more appropriate for someone with your skills and interests.

Data Engineering 159

Data Engineering Data Engineer Engineering Data Science 159

The Pragmatic Engineer

JANUARY 12, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one and a half out of eight topics in today’s subscriber-only issue, Inside Pollen's Transparent Compensation Data. Here are a few details from the data points, focusing on software engineering compensation.

Software Engineer 187

Software Engineer Software Engineering Engineering Raw Data 187

Towards Data Science

MAY 13, 2024

Engineers, scientists, and analysts have the potential to greatly reduce carbon emissions by introducing sustainable, efficient, and… Continue reading on Towards Data Science »

Data Engineering 97

Data Engineering Data Engineer Engineering Data Science 97

Knowledge Hut

JANUARY 18, 2024

Speaking of job vacancies, the two careers have high demands till date and in upcoming years are Data Scientist and a Software Engineer. Per the BLS, the expected growth rate of job vacancies for data scientists and software engineers is around 22% by 2030. What is Data Science? What is Software Engineering?

Software Engineer 97

Software Engineer Software Engineering Data Science Engineering 97

KDnuggets

DECEMBER 5, 2022

The Complete Data Engineering Study Roadmap • How to Select Rows and Columns in Pandas Using [ ],loc, iloc,at and.iat • Top 10 Data Science Myths Busted • What is Chebychev’s Theorem and How Does it Apply to Data Science? Scikit-learn for Machine Learning Cheatsheet.

Data Engineering 116

Data Engineering Data Engineer Engineering Data Science 116

Snowflake

APRIL 17, 2024

This traditional SQL-centric approach often challenged data engineers working in a Python environment, requiring context-switching and limiting the full potential of Python’s rich libraries and frameworks. This allows your applications to handle large data sets and complex workflows efficiently.

Data Pipeline 113

Data Pipeline Python Data Engineering Data Engineer 113

Netflix Tech

DECEMBER 14, 2023

Engineers from across the company came together to share best practices on everything from Data Processing Patterns to Building Reliable Data Pipelines. The result was a series of talks which we are now sharing with the rest of the Data Engineering community!

Data Engineering 106

Data Engineering Data Engineer Engineering Metadata 106

Analytics Vidhya

JUNE 12, 2023

Introduction Meet Tajinder, a seasoned Senior Data Scientist and ML Engineer who has excelled in the rapidly evolving field of data science. Tajinder’s passion for unraveling hidden patterns in complex datasets has driven impactful outcomes, transforming raw data into actionable intelligence.

Machine Learning 183

Machine Learning Engineering Raw Data Data Science 183

Knowledge Hut

MAY 3, 2023

With companies increasingly relying on data-driven insights to make informed decisions, there has never been a greater need for skilled specialists who can manage and evaluate vast amounts of data. The roles of data analyst and data engineer have emerged as two of the most in-demand professions in today's job market.

Data Engineering 98

Data Engineering Data Engineer Engineering Data Cleanse 98

Towards Data Science

FEBRUARY 10, 2024

Advanced techniques to process and load data efficiently Continue reading on Towards Data Science »

Data Engineering 89

Data Engineering Data Engineer Engineering Data Science 89

Towards Data Science

JANUARY 6, 2024

Knowledge and skills for successful data leadership Continue reading on Towards Data Science »

Data Engineering 94

Data Engineering Data Engineer Engineering Data Science 94

KDnuggets

FEBRUARY 14, 2023

Docker is dependency management on steroids, helping to ensure both reproducibility and collaboration, making it an important tool for data science. Our latest cheat sheet serves as a handy Docker reference. Check it out now!

Data Science 156

Data Science Data Management IT 156

Analytics Vidhya

FEBRUARY 6, 2023

Introduction The demand for data to feed machine learning models, data science research, and time-sensitive insights is higher than ever thus, processing the data becomes complex. To make these processes efficient, data pipelines are necessary.

Amazon Web Services 218

Amazon Web Services Data Pipeline Machine Learning Data Science 218

Knowledge Hut

JUNE 26, 2023

Welcome to the world of data engineering, where the power of big data unfolds. If you're aspiring to be a data engineer and seeking to showcase your skills or gain hands-on experience, you've landed in the right spot. What are Data Engineering Projects?

Data Engineering 98

Data Engineering Data Engineer Coding Project 98

KDnuggets

MAY 19, 2022

Read the best books on Programming, Statistics, Data Engineering, Web Scraping, Data Analytics, Business Intelligence, Data Applications, Data Management, Big Data, and Cloud Architecture.

Data Science 160

Data Science Business Intelligence Big Data Architecture 160

KDnuggets

APRIL 19, 2024

The collection includes free courses on Python, SQL, Data Analytics, Business Intelligence, Data Engineering, Machine Learning, Deep Learning, Generative AI, and MLOps.

Data Science 155

Data Science Deep Learning Business Intelligence Machine Learning 155

Data Engineering Weekly

AUGUST 4, 2024

Take Astro (the fully managed Airflow solution) for a test drive today and unlock a suite of features designed to simplify, optimize, and scale your data pipelines. The Archmage thrift proxy layer to hide the infrastructure and enable easier version upgrades is a good engineering practice that I would recommend for any platform team.

Data Engineering 81

Data Engineering Data Engineer Engineering Data 81

KDnuggets

MARCH 29, 2023

The second part covers the list of Machine Learning, Deep Learning, Computer Vision, Natural Language Processing, Data Engineering, and MLOps.

Data Science 160

Data Science Deep Learning Machine Learning Data Engineering 160