CO2 vs. Temperature Exercise (Databricks)

This repository contains exercises in Databricks that ingests Global Temperature and Global Temperature By Country data from Kaggle and CO2 Emissions data from OWID and transforms it. The goal of this exercise is to teach some basics about data wrangling and Spark with respect to real world questions.

• Which countries are worse-hit (higher temperature anomalies)?
• Which countries are the biggest emitters?
• What are some attempts of ranking “biggest polluters” in a sensible way?

Data Sources

In order to answer some of the questions of the exercise, we picked open-source data from Open World in Data (OWID) and Kaggle.

The specific datasets:

• CO2 Emissions (2020).csv (OWID)
• GlobalLandTemperaturesByCountry (Kaggle)
• GlobalTemperatures.csv (Kaggle)

Data Sources (Modified!)

Since the point of this exercise is to learn how to work with data and the datasets from OWID and Kaggle are both too clean and curated, a set of dirtied data is provided.

They can be found at:

• EmissionsByCountry.csv
• GlobalTemperatures.csv
• TemperaturesByCountry.csv

Prerequisites

• Basic knowledge of Python
• Basic knowledge of Spark
• Databricks Account

Exercises

1. Data Ingestion
2. Data Transformation
3. Data Visualisation
4. Grand Finale

Contributing

Some hints

• It's easiest to make changes to the Solutions notebooks, then clone it, remove the code blocks clearly marked solutions
• When uploading new datasets from a Mac, you might have to run zip -d input-data.zip __MACOSX/\* after zipping so that this hidden file gets removed