Welcome to the "Uber_data_engineering_project" repository! This open-source project is dedicated to exploring and performing data analytics on Uber data using a variety of cutting-edge tools and technologies. Our goal is to provide a comprehensive analysis using Google Cloud Platform (GCP), Python, Mage Data Pipeline Tool, BigQuery, and Looker Studio.
The project focuses on leveraging advanced data engineering techniques to analyze Uber data. We utilize GCP Storage, Python, Compute Instance, Mage Data Pipeline Tool, BigQuery, and Looker Studio to create a robust and efficient data analytics pipeline.
Our architecture incorporates various technologies to ensure a seamless flow of data processing. The key components include Google Storage, Compute Instance, BigQuery, Looker Studio, and the modern data pipeline tool provided by Mage AI.
- Programming Language - Python
Google Cloud Platform
- Google Storage
- Compute Instance
- BigQuery
- Looker Studio
Modern Data Pipeine Tool - https://www.mage.ai/
Contibute to this open source project - https://github.com/mage-ai/mage-ai
TLC Trip Record Data Yellow and green taxi trip records include fields capturing pick-up and drop-off dates/times, pick-up and drop-off locations, trip distances, itemized fares, rate types, payment types, and driver-reported passenger counts.
More info about dataset can be found here:
- Website - https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page
- Data Dictionary - https://www.nyc.gov/assets/tlc/downloads/pdf/data_dictionary_trip_records_yellow.pdf
Credit: Special thanks to @darshilparmar for providing the Uber dataset.