Remove Cloud Storage Remove Data Preparation Remove Lambda Architecture
article thumbnail

20+ Data Engineering Projects for Beginners with Source Code

ProjectPro

Then, the Yelp dataset downloaded in JSON format is connected to Cloud SDK, following connections to Cloud storage which is then connected with Cloud Composer. Cloud composer and PubSub outputs are Apache Beam and connected to Google Dataflow. There are three stages in this real-world data engineering project.