Data and SQL - Data Engineering Digest

How to Normalize Relational Databases With SQL Code?

Analytics Vidhya

FEBRUARY 27, 2023

Introduction Data is the new oil in this century. The database is the major element of a data science project. So, we are […] The post How to Normalize Relational Databases With SQL Code? So, we are […] The post How to Normalize Relational Databases With SQL Code? appeared first on Analytics Vidhya.

Relational Database

Relational Database Database SQL Coding

Step-by-Step Roadmap to Learn SQL in 2023

Analytics Vidhya

FEBRUARY 28, 2023

Introduction Structured Query Language is a powerful language to manage and manipulate data stored in databases. SQL is widely used in the field of data science and is considered an essential skill to have if you work with data.

SQL

SQL Relational Database Data Science Database

Webinars

Apache Airflow®: The Ultimate Guide to DAG Writing

MORE WEBINARS

SQL Injection: The Cyber Attack Hiding in Your Database

Analytics Vidhya

FEBRUARY 2, 2023

Introduction SQL injection is an attack in which a malicious user can insert arbitrary SQL code into a web application’s query, allowing them to gain unauthorized access to a database. We can use this to steal sensitive information or make unauthorized changes to the data stored in the database.

Database

Database SQL Coding Accessible

Tackling Real Time Streaming Data With SQL Using RisingWave

Data Engineering Podcast

FEBRUARY 4, 2024

Summary Stream processing systems have long been built with a code-first design, adding SQL as a layer on top of the existing framework. In this episode Yingjun Wu explains how it is architected to power analytical workflows on continuous data flows, and the challenges of making it responsive and scalable.

SQL

SQL Data Lake High Quality Data Data Pipeline

Top SQL Queries for Data Scientists

KDnuggets

MAY 31, 2024

SQL seems like a data science underdog compared to Python and R. I’ll show you here how you can use it as a data scientist. However, it’s far from it.

SQL

SQL Data Science Python Data

Polars – Laziness and SQL Context.

Confessions of a Data Guy

MAY 7, 2023

I mean you have a problem if you could use Polars Python, and you find yourself using […] The post Polars – Laziness and SQL Context. appeared first on Confessions of a Data Guy. It’s gotten so bad, I’ve started to use it in my Rust code on the side, Polars that is.

SQL

SQL Python Coding IT

Enable stakeholder data access with Text-to-SQL RAGs

Start Data Engineering

MAY 21, 2024

Enabling Stakeholder data access with RAGs 3.1. Loading: Read raw data and convert them into LlamaIndex data structures 3.2.1. Read data from structured and unstructured sources 3.2.2. Transform data into LlamaIndex data structures 3.3. Indexing: Generate & store numerical representation of your data 3.

Accessibility

Accessibility Accessible SQL Raw Data

4 Useful Intermediate SQL Queries for Data Science

KDnuggets

DECEMBER 6, 2022

SQL is the essential language for developers, engineers, and data professionals. Intermediate knowledge in SQL gives you an edge in your data science career.

SQL

SQL Data Science Data Engineering

Why SQL Will Remain the Data Scientist’s Best Friend

KDnuggets

JULY 15, 2022

Machine learning, big data analytics or AI may steal the headlines, but if you want to hone a smart, strategic skill that can elevate your career, look no further than SQL.

SQL

SQL Big Data Machine Learning Data Analytics

5 Free SQL Courses for Data Science Beginners

KDnuggets

APRIL 8, 2024

Are you looking to make a career in data science? Start by learning SQL with these free courses.

SQL

SQL Data Science Data

5 Free Courses to Master SQL for Data Science

KDnuggets

MARCH 6, 2024

Want to become an SQL expert? Check out these free courses to learn and hone your SQL skills for data science.

SQL

SQL Data Science Data

Data News — Week 24.11

Christophe Blefari

MARCH 15, 2024

With yato you give a folder with SQL queries and it guesses the DAG and runs the queries in the right order. Saying mainly that " Sora is a tool to extend creativity " Last point Mira has been mocked and criticised online because as a CTO she wasn't able to say on which public / licensed data Sora has been trained on.

Metadata

Metadata Datasets Data Data Warehouse

Seamless SQL And Python Transformations For Data Engineers And Analysts With SQLMesh

Data Engineering Podcast

JUNE 25, 2023

Summary Data transformation is a key activity for all of the organizational roles that interact with data. Because of its importance and outsized impact on what is possible for downstream data consumers it is critical that everyone is able to collaborate seamlessly. Can you describe what SQLMesh is and the story behind it?

Data Engineering

Data Engineering Data Engineer Python Engineering

5 Free University Courses to Learn Databases and SQL

KDnuggets

MARCH 5, 2024

Looking to learn SQL and databases to level up your data science skills? Learn SQL, database internals, and much more with these free university courses.

SQL

SQL Database Data Science Data

Why SQL is THE Language to Learn for Data Science

KDnuggets

OCTOBER 12, 2023

SQL is the essential data science language due to its universal database accessibility, efficient data cleaning capabilities, seamless integration with other languages, and requirement for most data science jobs.

Data Science

Data Science SQL Database Data

Using Pandas and SQL Together for Data Analysis

KDnuggets

NOVEMBER 12, 2024

In this tutorial, we’ll explore when and how SQL functionality can be integrated within the Pandas framework, as well as its limitations.

SQL

SQL Data Analysis Data IT

25 SQL tips to level up your data engineering skills

Start Data Engineering

OCTOBER 17, 2024

Introduction Setup SQL tips 1. Handy functions for common data processing scenarios 1.1. STRUCT data types are sorted based on their keys from left to right 1.4. Need to filter on WINDOW function without CTE/Subquery use QUALIFY 1.2. Need the first/last row in a partition, use DISTINCT ON 1.3.

SQL

SQL Data Engineering Data Engineer Engineering

Using SQL with Python: SQLAlchemy and Pandas

KDnuggets

JUNE 12, 2024

A simple tutorial on how to connect to databases, execute SQL queries, and analyze and visualize data.

SQL

SQL Python Database Data

5 Tips for Improving SQL Query Performance

KDnuggets

JULY 31, 2024

If you work in data, you’ll write SQL queries all the time. So how do you write efficient SQL queries that are optimized for performance? This tutorial will help you with just that.

SQL

SQL Data

Using Trino And Iceberg As The Foundation Of Your Data Lakehouse

Data Engineering Podcast

FEBRUARY 18, 2024

Summary A data lakehouse is intended to combine the benefits of data lakes (cost effective, scalable storage and compute) and data warehouses (user friendly SQL interface). Data lakes are notoriously complex. Join in with the event for the global data community, Data Council Austin.

Data Lake

Data Lake High Quality Data Data Warehouse Google Cloud

Boost Your Data Science Skills: The Essential SQL Certifications You Need

KDnuggets

MARCH 13, 2024

If you are a data scientist who works with large amounts of data and hasn’t learned SQL yet - now might be the time.

SQL

SQL Data Science Certification Data

7 Advanced SQL Techniques for Data Manipulation in Data Science

KDnuggets

NOVEMBER 20, 2024

Can SQL be used for advanced data manipulation in data science? It sure can with these seven techniques.

Data Science

Data Science SQL Data IT

Databricks SQL Year in Review (Part II): SQL Programming Features

databricks

JANUARY 31, 2024

Welcome to the blog series covering product advancements in 2023 for Databricks SQL, the serverless data warehouse from Databricks. This is part 2.

SQL

SQL Programming Data Warehouse Data

Data Cleaning in SQL: How To Prepare Messy Data for Analysis

KDnuggets

JANUARY 5, 2024

Want to clean your messy data so you can start analyzing it with SQL? Learn how to handle missing values, duplicate records, outliers, and much more.

SQL

SQL Data IT

Introducing the New SQL Editor

databricks

OCTOBER 14, 2024

Over the last few years, we've seen tremendous growth and adoption of Databricks SQL , our intelligent data warehouse purpose-built on the Data.

SQL

SQL Data Warehouse Data

Ingest data from SQL Server, Salesforce, and Workday with LakeFlow Connect

databricks

JULY 31, 2024

We’re excited to announce the Public Preview of LakeFlow Connect for SQL Server, Salesforce, and Workday. These ingestion connectors enable simple and efficient.

SQL

SQL Data Data Engineering Data Engineer

SQL Group By and Partition By Scenarios: When and How to Combine Data in Data Science

KDnuggets

JANUARY 15, 2024

Learn the generic scenarios and techniques of grouping and aggregating data, partitioning and ranking data in SQL, which will be very helpful in reporting requirements.

SQL

SQL Aggregated Data Data Science Data

How to Get Up and Running with SQL – A List of Free Learning Resources

KDnuggets

OCTOBER 7, 2022

We have compiled a list of the top free resources to help new data practitioners learn SQL. These include free online courses and resources to get the most out of your SQL skills.

SQL

SQL Data

An Overview Of The Sate Of Data Orchestration In An Increasingly Complex Data Ecosystem

Data Engineering Podcast

SEPTEMBER 10, 2023

Summary Data systems are inherently complex and often require integration of multiple technologies. This offers a single location for managing visibility and error handling so that data platform engineers can manage complexity. With Materialize, you can!

BI

BI SQL Machine Learning Data

Why You Should Learn SQL in 2024

KDnuggets

JUNE 26, 2024

Learning SQL in 2024 is essential as it remains the most in-demand skill for data professionals, enabling efficient management and analysis of large datasets.

SQL

SQL Datasets Management IT

Alternatives to SSIS(SQL Server Integration Services) – How To Migrate Away From SSIS

Seattle Data Guy

FEBRUARY 26, 2024

SQL Server Integration Services (SSIS) comes with a lot of functionality useful for extracting, transforming, and loading data. It can also play important roles in application development and other projects. But SSIS is far from the only platform that can provide these services.

SQL

SQL Project Data Consulting

KDnuggets News, September 13: Getting Started with SQL in 5 Steps • Introduction to Databases in Data Science

KDnuggets

SEPTEMBER 13, 2023

Getting Started with SQL in 5 Steps • Introduction to Databases in Data Science • Time 100 AI: The Most Influential?

Data Science

Data Science Database SQL Data

SQL or Python for Data Transformations?

Start Data Engineering

JULY 1, 2024

Introduction 2. Code is an interface to the execution engine 3. How to choose the execution engine and the coding interface 3.1. Chose execution engine based on your workload 3.1.1. Types of execution engine 3.1.2. Criteria to chose your execution engine 3.2. Chose coding interface for people who will maintain the pipeline 3.2.1.

SQL

SQL Python Coding Engineering

Back to Basics Week 2: Database, SQL, Data Management and Statistical Concepts

KDnuggets

NOVEMBER 13, 2023

This week, we delve into the vital world of Databases, SQL, Data Management, and Statistical Concepts in Data Science. Welcome back to Week 2 of KDnuggets’ "Back to Basics" series.

Database

Database SQL Data Management Management

Building Linked Data Products With JSON-LD

Data Engineering Podcast

SEPTEMBER 17, 2023

Summary A significant amount of time in data engineering is dedicated to building connections and semantic meaning around pieces of information. Linked data technologies provide a means of tightly coupling metadata with raw information. With Materialize, you can! Go to dataengineeringpodcast.com/materialize today to get 2 weeks free!

Building

Building SQL BI Python

How to get started with dbt

Christophe Blefari

MARCH 1, 2023

dbt Core is an open-source framework that helps you organise data warehouse SQL transformation. dbt was born out of the analysis that more and more companies were switching from on-premise Hadoop data infrastructure to cloud data warehouses. This switch has been lead by modern data stack vision. Enter the ELT.

Data Warehouse

Data Warehouse SQL Metadata Raw Data

How to use nested data types effectively in SQL

Start Data Engineering

OCTOBER 14, 2024

Code & Data 3. Using nested data types effectively 3.1. Using nested data types in data processing 3.3.1. STRUCT enables more straightforward data schema and data access 3.3.2. Nested data types can be sorted 3.3.3. Introduction 2. Use STRUCT for one-to-one & hierarchical relationships 3.2.

SQL

SQL Data Schemas Data Coding

5 Simple Steps Series: Master Python, SQL, Scikit-learn, PyTorch & Google Cloud

KDnuggets

NOVEMBER 2, 2023

Dive into KDnuggets Back to Basics: Getting Started in 5 Steps series to help you master Python, SQL, Scikit-learn, PyTorch, and Google Cloud Platform.

Google Cloud

Google Cloud SQL Python Cloud

Troubleshooting Kafka In Production

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. Data lakes are notoriously complex.

Kafka

Kafka Data Lake High Quality Data SQL

Datafusion SQL CLI – Look Ma, I made a new ETL tool.

Confessions of a Data Guy

DECEMBER 21, 2023

Thinking about and contemplating life and data engineering … something flitted across my […] The post Datafusion SQL CLI – Look Ma, I made a new ETL tool. appeared first on Confessions of a Data Guy.

ETL Tools

ETL Tools SQL Data Engineering Data Engineer

Collection of Guides on Mastering SQL, Python, Data Cleaning, Data Wrangling, and Exploratory Data Analysis

KDnuggets

MARCH 20, 2024

Are you curious about what it takes to become a professional data scientist? By following these guides, you can transform yourself into a skilled data scientist and unlock endless career opportunities. Look no further!

Data Analysis

Data Analysis SQL Python Data

dbt multi-project collaboration

Christophe Blefari

OCTOBER 19, 2023

cross-project dependencies ( credits ) Over the last few years, dbt has become a de facto standard enabling companies to collaborate easily on data transformations. With dbt, you can apply software engineering practices to SQL development. Managing your SQL patrimony has never been easier.

Project

Project Finance SQL Government

How to Normalize Relational Databases With SQL Code?

Step-by-Step Roadmap to Learn SQL in 2023

Webinars

Trending Sources

Top 5 SQL Interview Questions With Implementation

Webinars

SQL Injection: The Cyber Attack Hiding in Your Database

Tackling Real Time Streaming Data With SQL Using RisingWave

Top 5 SQL Interview Questions

Top SQL Queries for Data Scientists

Polars – Laziness and SQL Context.

Enable stakeholder data access with Text-to-SQL RAGs

4 Useful Intermediate SQL Queries for Data Science

Why SQL Will Remain the Data Scientist’s Best Friend

5 Free SQL Courses for Data Science Beginners

5 Free Courses to Master SQL for Data Science

Data News — Week 24.11

Seamless SQL And Python Transformations For Data Engineers And Analysts With SQLMesh

5 Free University Courses to Learn Databases and SQL

Why SQL is THE Language to Learn for Data Science

Using Pandas and SQL Together for Data Analysis

25 SQL tips to level up your data engineering skills

Using SQL with Python: SQLAlchemy and Pandas

5 Tips for Improving SQL Query Performance

Using Trino And Iceberg As The Foundation Of Your Data Lakehouse

Boost Your Data Science Skills: The Essential SQL Certifications You Need

7 Advanced SQL Techniques for Data Manipulation in Data Science

Databricks SQL Year in Review (Part II): SQL Programming Features

Data Cleaning in SQL: How To Prepare Messy Data for Analysis

Introducing the New SQL Editor

Ingest data from SQL Server, Salesforce, and Workday with LakeFlow Connect

SQL Group By and Partition By Scenarios: When and How to Combine Data in Data Science

How to Get Up and Running with SQL – A List of Free Learning Resources

An Overview Of The Sate Of Data Orchestration In An Increasingly Complex Data Ecosystem

Why You Should Learn SQL in 2024

Alternatives to SSIS(SQL Server Integration Services) – How To Migrate Away From SSIS

KDnuggets News, September 13: Getting Started with SQL in 5 Steps • Introduction to Databases in Data Science

SQL or Python for Data Transformations?

Back to Basics Week 2: Database, SQL, Data Management and Statistical Concepts

Building Linked Data Products With JSON-LD

How to get started with dbt

How to use nested data types effectively in SQL

5 Simple Steps Series: Master Python, SQL, Scikit-learn, PyTorch & Google Cloud

Troubleshooting Kafka In Production

Datafusion SQL CLI – Look Ma, I made a new ETL tool.

Collection of Guides on Mastering SQL, Python, Data Cleaning, Data Wrangling, and Exploratory Data Analysis

dbt multi-project collaboration

Stay Connected