Add documentation site with readthedocs (#10)

Uses mkdocs to compile.
Uses readthedocs to host site.

Uses material on top of mdocs for prettier UI

.github/CODEOWNERS

@@ -1,6 +1,6 @@
 # These owners will be the default owners for everything in
 # the repo.
-*       @14renus @gcouairon
+*       @INRIA/geoarches-owners
 
 /src/geoarches/backbones @gcouairon
 /src/geoarches/lightning_modules @gcouairon

.pre-commit-config.yaml

@@ -21,4 +21,4 @@ repos:
   rev: v2.1.0
   hooks:
     - id: codespell
-      args: ["--skip=*.ipynb", "--ignore-words-list=hist"]
+      args: ["--skip=*.ipynb", "--ignore-words-list=hist", "-w"]

.readthedocs.yaml

@@ -0,0 +1,25 @@
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the OS, Python version, and other tools you might need
+build:
+  os: ubuntu-24.04
+  tools:
+    python: "3.11"
+  jobs:
+    post_install:
+      # Install poetry
+      # https://python-poetry.org/docs/#installing-manually
+      - pip install poetry
+      # Install dependencies with 'docs' dependency group
+      # https://python-poetry.org/docs/managing-dependencies/#dependency-groups
+      # VIRTUAL_ENV needs to be set manually for now.
+      # See https://github.com/readthedocs/readthedocs.org/pull/11152/
+      - VIRTUAL_ENV=$READTHEDOCS_VIRTUALENV_PATH poetry install --with docs
+
+# Build documentation with mkdocs.
+mkdocs:
+  configuration: mkdocs.yaml

README.md

@@ -82,7 +82,7 @@ for MOD in $MODELS; do
     wget -O modelstore/$MOD/config.yaml $src/${MOD}_config.yaml 
 done
 ```
-You can follow instructions in [`documentation/archesweather-tutorial.ipynb`](documentation/archesweather-tutorial.ipynb) to load the models and run inference with them. See [`documentation/archesweathergen_pipeline.md`](documentation/archesweathergen_pipeline.md) to run training.
+You can follow instructions in [`docs/archesweather/run.ipynb`](docs/archesweather/run.ipynb) to load the models and run inference with them. See [`docs/archesweather/train.md`](docs/archesweather/train.md) to run training.
 
 ### Downloading ERA5 statistics
 To compute brier score on ERA5 (needed to instantiate ArchesWeather models for inference or training), you will need to download ERA5 quantiles:

documentation/archesweather-tutorial.ipynb → docs/archesweather/run.ipynb

@@ -1,5 +1,13 @@
 {
  "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "7daf94f9",
+   "metadata": {},
+   "source": [
+    "# Load and run models"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,
@@ -35,7 +43,7 @@
     }
    ],
    "source": [
-    "cd .."
+    "cd ../.."
    ]
   },
   {
@@ -57,6 +65,14 @@
     "from geoarches.lightning_modules import load_module"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "b0ffcb86",
+   "metadata": {},
+   "source": [
+    "## Load ArchesWeather"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 5,
@@ -114,6 +130,14 @@
     ")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "63617be4",
+   "metadata": {},
+   "source": [
+    "## Era5 Dataloader"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 7,
@@ -217,6 +241,14 @@
     "plt.imshow(ds[0][\"state\"][\"level\"][0, 7], cmap=\"terrain\")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "1426177b",
+   "metadata": {},
+   "source": [
+    "## Run ArchesWeather"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 10,
@@ -337,7 +369,7 @@
    "id": "3965140a-6bc4-47ed-b848-ebbce83f2ba5",
    "metadata": {},
    "source": [
-    "# Loading and running ArchesWeatherGen"
+    "## Loading and running ArchesWeatherGen"
    ]
   },
   {

documentation/archesweathergen_pipeline.md → docs/archesweather/train.md

@@ -1,4 +1,4 @@
-# pipeline for training ArchesWeatherGen
+# Pipeline to train ArchesWeather and ArchesWeatherGen
 
 This is the full training pipeline if you want to re-train ArchesWeatherGen from scratch. We assume that you have installed the geoarches package and downloaded the data with the script.
 

docs/contributing/bug.md

@@ -0,0 +1,11 @@
+## Report a bug
+
+If you think you have found a bug, you can help us by submitting an issue in our public [issue tracker](https://github.com/INRIA/geoarches/issues).
+
+Please add the `bug` label for easy tracking.
+
+## Request a feature
+
+You can request a feature in our public [issue tracker](https://github.com/INRIA/geoarches/issues).
+
+Please add the `enhancement` label for easy tracking. You can also add other labels as you see fit.

docs/contributing/contribute.md

@@ -0,0 +1,112 @@
+
+# Contribute to geoarches
+We welcome contributions to the codebase such as adding:
+- Data
+    - geospatial datasets beyond ERA5 (both download and dataloaders)
+    - formats beyond xarray (netcdf, zarr, etc) such as csv
+    - other storage stypes such as cloud storage
+- Modeling
+    - model architecture backbones
+    - training schemes beyond diffusion ddpm and flow-matching
+- Visualization
+    - better support for plotting metrics
+    - tools for visualization of geospatial data
+
+## Setup
+
+We suggest you [create a fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo) of the [geoarches repo](https://github.com/INRIA/geoarches).
+
+Then clone your fork from GitHub:
+
+```
+git clone git@github.com:<username>/geoarches.git
+```
+
+Follow [installation](../getting_started/installation.md) to install dependencies.
+
+You can make changes on your own `dev` branch(s). This way you are not blocked by development on the `main` branch, but can still contribute to the `main` branch and can still incorporate updates from other team members.
+
+Create a `dev` branch from the `main` branch of geoarches to start making changes.
+
+```shell
+cd geoarches
+git checkout main
+git checkout -b dev_<name>
+```
+
+## Local testing
+
+Every piece of code will need a corresponding test file under `tests/`.
+
+You can make sure tests still pass by running:
+
+```sh
+pytest tests/
+```
+
+## Code format
+
+We recommend reading [Google Style Python Guide](https://google.github.io/styleguide/pyguide.html) for tips of writing readable code.
+
+We also require you to run these commands before committing:
+```sh
+ruff check --fix
+ruff format
+codespell -w
+```
+
+### Optional: Automatically check format on commit
+
+You can set up automatic checks on ready-to-commit code using `pre-commit`.
+
+Run in the `geoarches/` repo:
+```
+pre-commit install
+```
+
+Now, pre-commit will run automatically on `git commit`!
+
+## Code reviews
+
+When your code is ready, push the changes to your `dev` branch and make a [pull request](https://github.com/INRIA/geoarches/pulls) on Github. You will only be able to merge with the `main` branch, once you receive approval from a maintainer.
+
+## Pull code updates
+
+When the `main` branch of geoarches gets updated, and you want to incorporate changes.
+This is important for both:
+- Allowing you to take advantage of new code.
+- Preemptively resolving any merge conflicts before merge requests.
+
+The following steps will help you pull the changes from main and then apply your changes on top.
+1. Either commit or stash your changes on your dev branch:
+    ```sh
+    git stash push -m "message"
+    ```
+
+2. Pull new changes into local main branch:
+    ```sh
+    git checkout main
+    git pull origin main
+    ```
+
+3. Rebase your changes on top of new commits from main branch:
+    ```sh
+    git checkout dev_<name>
+    git rebase main
+    ```
+
+    Resolve merge conflicts if needed. You can always decide to abort to undo the rebase completely:
+    ```sh
+    git rebase –abort
+    ```
+
+5. If you ran `git stash` in step 1, you can now apply your stashed changes on top.
+    ```sh
+    git stash pop
+    ```
+
+    Resolve merge conflicts if needed. To undo applying the stash:
+    ```sh
+    git reset --merge
+    ```
+    This will discard stashed changes, but stash contents won't be lost and can be re-applied later.

docs/getting_started/installation.md

@@ -0,0 +1,60 @@
+# Installation
+
+## Install poetry
+
+We use poetry for package dependencies. Use pipx to install poetry:
+
+```sh
+brew install pipx
+pipx install poetry
+```
+
+## Environment
+
+Create an environment or activate the environment you are already using.
+
+```sh
+conda create --name weather python=3.11
+conda activate weather
+```
+
+Move into the git repo and install dependencies:
+```sh
+cd geoarches
+poetry install
+```
+
+Poetry, by default, installs the geoarches package in editable mode.
+Editable mode allows you to make changes to the geoarches code locally, and these changes will automatically be reflected in your code that depends on it.
+
+## Useful directories
+
+We recommend making the following symlinks in the codebase folder:
+```sh
+ln -s /path/to/data/ data             # Store data for training and evaluation.
+ln -s /path/to/models/ modelstore     # Store model checkpoints and model hydra configs.
+ln -s /path/to/evaluation/ evalstore  # Store intermediate model outputs for computing metrics.
+ln -s /path/to/wandb/ wandblogs       # Store Wandb logs.
+```
+If you want to store models and data in your working directory, or can also simply create regular folders.
+
+## Downloading ArchesWeather and ArchesWeatherGen
+Use following the script to download the 4 deterministic models (archesweather-m-...) and generative model (archesweathergen).
+
+```sh
+src="https://huggingface.co/gcouairon/ArchesWeather/resolve/main"
+MODELS="archesweather-m-seed0 archesweather-m-seed1 archesweather-m-skip-seed0 archesweather-m-skip-seed1 archesweathergen"
+for MOD in $MODELS; do
+    mkdir -p modelstore/$MOD/checkpoints
+    wget -O modelstore/$MOD/checkpoints/checkpoint.ckpt $src/${MOD}_checkpoint.ckpt
+    wget -O modelstore/$MOD/config.yaml $src/${MOD}_config.yaml 
+done
+```
+You can follow instructions in [`archesweather/tutorial.ipynb`](archesweather/tutorial.ipynb) to load the models and run inference with them. See [`archesweathergen/pipeline.md`](archesweather/pipeline.md) to run training.
+
+## Downloading ERA5 statistics
+To compute brier score on ERA5 (needed to instantiate ArchesWeather models for inference or training), you will need to download ERA5 quantiles:
+```sh
+src="https://huggingface.co/gcouairon/ArchesWeather/resolve/main"
+wget -O geoarches/stats/era5-quantiles-2016_2022.nc $src/era5-quantiles-2016_2022.nc
+```