diff --git a/docs/Dockerfile b/docs/Dockerfile
index bbcad23ff..4080e33fe 100644
--- a/docs/Dockerfile
+++ b/docs/Dockerfile
@@ -4,6 +4,7 @@ WORKDIR /docs
RUN ["cargo", "install", "mdbook", "--vers", "^0.4", "--locked"]
RUN ["cargo", "install", "mdbook-alerts"]
+RUN ["cargo", "install", "mdbook-toc"]
COPY . .
VOLUME ["/docs"]
diff --git a/docs/book.toml b/docs/book.toml
index a29a001d8..8e7b20006 100644
--- a/docs/book.toml
+++ b/docs/book.toml
@@ -11,3 +11,7 @@ git-repository-url = "https://github.com/asuc-octo/berkeleytime/tree/gql/docs"
edit-url-template = "https://github.com/asuc-octo/berkeleytime/tree/gql/docs/{path}"
[preprocessor.alerts]
+
+[preprocessor.toc]
+command = "mdbook-toc"
+renderer = ["html"]
diff --git a/docs/src/README.md b/docs/src/README.md
index 2b83f4af9..e30fd7f36 100644
--- a/docs/src/README.md
+++ b/docs/src/README.md
@@ -20,26 +20,29 @@ Using Docker allows us to build the docs without downloading dependencies on our
# Ensure you are on the latest commit
git pull
-# Build the container
+# Build the container (only needed once every time docs/Dockerfile changes!)
docker build --target docs-dev --tag="docs:dev" ./docs
# Run the container
docker run --publish 3000:3000 --volume ./docs:/docs "docs:dev"
```
-The docs should be available at `http://localhost:3000/`. To change the port to port `XXXX`, modify the last command:
+The docs should be available at `http://localhost:3000/` with live reload. To change the port to port `XXXX`, modify the last command:
```sh
# Run the container and publish the docs to http://localhost:XXXX/
docker run --publish XXXX:3000 --volume ./docs:/docs "docs:dev"
```
+To kill the container, you can use the Docker Desktop UI or run `docker kill [container id]`. You can find the container ID from `docker ps`.
+
#### Without Containerization
-To build and view the docs locally, `mdBook` must be installed by following the guide [here](https://rust-lang.github.io/mdBook/guide/installation.html#build-from-source-using-rust). It is necessary to install Rust locally as there is a dependency that is installed with `cargo`. Thus, it is highly recommended to [build mdbook from Rust](https://rust-lang.github.io/mdBook/guide/installation.html#build-from-source-using-rust).
+To build and view the docs locally, `mdBook` must be installed by following the guide [here](https://rust-lang.github.io/mdBook/guide/installation.html#build-from-source-using-rust). It is necessary to install Rust locally as there are dependencies that are installed with `cargo`. Thus, it is highly recommended to [build mdbook from Rust](https://rust-lang.github.io/mdBook/guide/installation.html#build-from-source-using-rust).
```sh
-# Install mdbook-alerts dependency with cargo
+# Install mdbook preprocessors with cargo
cargo install mdbook-alerts
+cargo install mdbook-toc
# ./berkeleytime
# Ensure you are on the latest commit
diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md
index cd2dcc6b2..8c48789d3 100644
--- a/docs/src/SUMMARY.md
+++ b/docs/src/SUMMARY.md
@@ -21,9 +21,8 @@
- [Infrastructure](./core/infrastructure/README.md)
- [Onboarding](./core/infrastructure/onboarding.md)
- - [Architecture](./core/infrastructure/architecture.md)
- - [Kubernetes & Helm](./core/infrastructure/kubernetes-helm.md)
- - [CI/CD](./core/infrastructure/cicd.md)
+ - [CI/CD Workflow](./core/infrastructure/cicd-workflow.md)
+ - [Runbooks](./core/infrastructure/runbooks.md)
---
diff --git a/docs/src/core/infrastructure/architecture.md b/docs/src/core/infrastructure/architecture.md
deleted file mode 100644
index b81b82205..000000000
--- a/docs/src/core/infrastructure/architecture.md
+++ /dev/null
@@ -1,35 +0,0 @@
-# Architecture
-
-Berkeleytime uses a fairly simple microservices architecture—we decouple only a few application components into separate services. Below is a high-level diagram of the current architecture (switch to a light viewing mode to see arrows).
-
- 
-
-
-Note that, other than the application services developed by us, all other services are well-known and have large communities. These services have many tutorials, guides, and issues already created online, streamlining the setup and debugging processes.
-
-## An HTTP Request's Life
-
-To better understand the roles of each component in the Berkeleytime architecture, we describe the lifecycle of an HTTP request from a user's action.
-
-1. An HTTP request starts from a user's browser. For example, when a user visits `https://berkeleytime.com`, a `GET` request is sent to `hozer-51`.[^1]
-
-2. Once the request reaches `hozer-51`, it is first encountered by `hozer-51`'s Kubernetes cluster load balancer, a [MetalLB](https://metallb.io/) instance, which balances external traffic into the cluster across nodes.[^2]
-
-3. Next, the request reaches the [reverse proxy](https://www.cloudflare.com/learning/cdn/glossary/reverse-proxy/), an [nginx](https://nginx.org/) instance, which forwards HTTP requests to either the [backend](../backend) or [frontend](../frontend/) service based on the URL of the request.
- - Requests with URLs matching `https://berkeleytime.com/api/*` are forwarded to the backend service.
- - All other requests are forwarded to the frontend service.
-
-4. The request is processed by one of the services.
- - The backend service may interact with the MongoDB database or the Redis cache while processing the request.[^3]
-
-5. Finally, an HTTP response is sent back through the system to the user's machine.
-
-
-[^1]: More specifically, the user's machine first requests a DNS record of `berkeleytime.com` from a DNS server, which should return `hozer-51`'s IP address. After the user's machine knows the `hozer-51` IP address, the `GET` request is sent.
-
-[^2]: Currently, we only have one node: `hozer-51`.
-
-[^3]: Requests sent from the backend to the database or cache are *not* necessarily HTTP requests.
diff --git a/docs/src/core/infrastructure/cicd.md b/docs/src/core/infrastructure/cicd-workflow.md
similarity index 100%
rename from docs/src/core/infrastructure/cicd.md
rename to docs/src/core/infrastructure/cicd-workflow.md
diff --git a/docs/src/core/infrastructure/kubernetes-helm.md b/docs/src/core/infrastructure/kubernetes-helm.md
deleted file mode 100644
index 9711cf9a8..000000000
--- a/docs/src/core/infrastructure/kubernetes-helm.md
+++ /dev/null
@@ -1,158 +0,0 @@
-# Kubernetes & Helm
-
-[Kubernetes](https://kubernetes.io/) is a container orchestrator, a fundamental piece of our microservice architecture. It is a complex system with many different components. Fortunately, the documentation is decently well-written. The [concepts page](https://kubernetes.io/docs/concepts/) is a good place to start. The [glossary](https://kubernetes.io/docs/reference/glossary/?core-object=true&fundamental=true&networking=true) is also a good place to review common jargon.
-
-[Helm](https://helm.sh/) is a package manager for Kubernetes. It allows us to build Kubernetes resources that are easily configurable and reusable. For simplicity, we try to keep all of our Kubernetes resources defined with helm, as opposed to some being defined with raw resource definitions and some with helm charts.
-
-## Useful Commands
-
-> [!TIP]
-> On `hozer-51`, `k` is an alias for `kubectl` and `h` is an alias for `helm`.
-
-> [!IMPORTANT]
-> The default namespace has been set as `bt`.
-
-- `k get pods`
-
- View all running pods.
-
-
- Example Output
-
- ```bash
- root@hozer-51:~ k get pods
- NAME READY STATUS RESTARTS AGE
- bt-cert-manager-996dd87d8-2xlbs 1/1 Running 0 36d
- bt-cert-manager-cainjector-68d9974ddf-b7w84 1/1 Running 0 36d
- bt-cert-manager-webhook-599cc5679-v9v4n 1/1 Running 0 36d
- bt-dev-mongo-mongodb-56967b78d5-dwsk4 1/1 Running 0 14d
- bt-dev-redis-master-0 1/1 Running 0 90d
- bt-ingress-nginx-controller-788dfdc68d-zw5cf 1/1 Running 0 36d
- bt-metallb-controller-6c6ccbb4bb-xjjm2 1/1 Running 0 36d
- bt-metallb-speaker-7lqvj 4/4 Running 13 (91d ago) 264d
- bt-prod-app-backend-68f984687f-bbst2 1/1 Running 0 36d
- bt-prod-app-backend-68f984687f-mdbsj 1/1 Running 0 36d
- bt-prod-app-frontend-6985468b46-c89fq 1/1 Running 0 36d
- bt-prod-app-frontend-6985468b46-djxs6 1/1 Running 0 36d
- bt-prod-app-updater-28856160-dn99p 0/1 Completed 0 2d5h
- bt-prod-app-updater-28857600-4z8pw 0/1 Completed 0 29h
- bt-prod-app-updater-28859040-jmzkd 0/1 Completed 0 5h33m
- bt-prod-mongo-mongodb-b6b66fd69-h764l 1/1 Running 1 (14d ago) 36d
- bt-prod-redis-master-0 1/1 Running 0 90d
- bt-sealed-secrets-7cb5587d77-576r5 1/1 Running 0 36d
- bt-stage-app-backend-77759c7b94-7rg5q 1/1 Running 0 6d7h
- bt-stage-app-backend-77759c7b94-kkpbf 1/1 Running 0 6d7h
- bt-stage-app-frontend-bc997f75b-9mrhn 1/1 Running 0 6d7h
- bt-stage-app-frontend-bc997f75b-gsnkx 1/1 Running 0 6d7h
- bt-stage-app-updater-28856160-94lgw 0/1 Completed 0 2d5h
- bt-stage-app-updater-28857600-grv9r 0/1 Completed 0 29h
- bt-stage-app-updater-28859040-b2mjn 0/1 Completed 0 5h33m
- bt-stage-mongo-mongodb-996b5c9d8-ws5bm 1/1 Running 0 13d
- bt-stage-redis-master-0 1/1 Running 0 90d
- ```
-
-
-
-
-- `k get pods -l env=[dev|stage|prod]`
-
- View all running pods in a specified environment.
-
-- `k logs [pod name]`
-
- View logs of a pod. You can get a pod's name with `k get pods`. Include a `-f` flag to follow logs, which will stream logs into your terminal.
-
-- `k describe pod [pod name]`
-
- View a description of a pod. Useful for when pod is failing to startup, thus not showing any logs.
-
-- `k exec -it [pod name] -- [command]`
-
- Execute a command inside a pod. The command can be `bash`, which will start a shell inside the pod and allow for more commands.
-
-- `k get deploy`
-
- View all running deployments.
-
-
-
- Example Output
-
- ```bash
- root@hozer-51:~ k get deploy
- NAME READY UP-TO-DATE AVAILABLE AGE
- bt-cert-manager 1/1 1 1 90d
- bt-cert-manager-cainjector 1/1 1 1 90d
- bt-cert-manager-webhook 1/1 1 1 90d
- bt-dev-mongo-mongodb 1/1 1 1 14d
- bt-ingress-nginx-controller 1/1 1 1 267d
- bt-metallb-controller 1/1 1 1 264d
- bt-prod-app-backend 2/2 2 2 40d
- bt-prod-app-frontend 2/2 2 2 40d
- bt-prod-mongo-mongodb 1/1 1 1 90d
- bt-sealed-secrets 1/1 1 1 267d
- bt-stage-app-backend 2/2 2 2 36d
- bt-stage-app-frontend 2/2 2 2 36d
- bt-stage-mongo-mongodb 1/1 1 1 14d
- ```
-
-
-
-- `k get deploy -l env=[dev|stage|prod]`
-
- View all running deployments in a specified environment.
-
-- `k describe deploy [deploy name]`
-
- View a description of a deploy. Useful for when the deploy's pods are failing to startup, thus not showing any logs.
-
-- `k rollout restart deploy/[deploy name]`
-
- Manually restart a deployment.
-
-- `h list`
-
- List helm chart installations.
-
-
-
- Example Output
-
- ```bash
- root@hozer-51:~ h list
- NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
- bt-base bt 1 2024-08-16 02:39:08.530680512 +0000 UTC deployed bt-base-0.1.0 2.0.0-alpha
- bt-cert-manager bt 1 2024-08-15 09:09:57.055544133 +0000 UTC deployed cert-manager-v1.14.1 v1.14.1
- bt-dev-mongo bt 1 2024-10-30 19:39:12.342638847 +0000 UTC deployed bt-mongo-0.1.0 2.0.0-alpha
- bt-dev-redis bt 1 2024-08-15 22:48:23.811538319 +0000 UTC deployed bt-redis-0.1.0 2.0.0-alpha
- bt-ingress-nginx bt 2 2024-02-20 06:54:22.749755461 +0000 UTC deployed ingress-nginx-4.9.1 1.9.6
- bt-metallb bt 1 2024-02-23 22:15:39.949979855 +0000 UTC deployed metallb-0.14.3 v0.14.3
- bt-prod-app bt 1 2024-10-05 00:38:19.570732559 +0000 UTC deployed bt-app-0.1.0 2.0.0-alpha
- bt-prod-mongo bt 1 2024-08-15 22:49:24.829163584 +0000 UTC deployed bt-mongo-0.1.0 2.0.0-alpha
- bt-prod-redis bt 1 2024-08-15 22:49:30.646137811 +0000 UTC deployed bt-redis-0.1.0 2.0.0-alpha
- bt-sealed-secrets bt 1 2024-02-20 06:31:59.188302177 +0000 UTC deployed sealed-secrets-2.15.0 0.26.0
- bt-stage-app bt 1 2024-10-09 03:17:21.69782594 +0000 UTC deployed bt-app-0.1.0 2.0.0-alpha
- bt-stage-mongo bt 1 2024-10-31 05:20:36.995251245 +0000 UTC deployed bt-mongo-0.1.0 2.0.0-alpha
- bt-stage-redis bt 1 2024-08-15 22:48:39.561033896 +0000 UTC deployed bt-redis-0.1.0 2.0.0-alpha
- ```
-
-
-
-- `h list --short | grep "^bt-dev-app" | xargs -L1 h uninstall`
-
- Uninstalls all development environment deploys. Specifically, list then filters for helm charts with prefix `bt-dev-app`, then uninstalls them all. As of November 14, 2024, there is no limit on the number of dev deploys. There is a noticeable amount of lag when there exceeds about 8 dev deploys.
-
-- `helm list --all-namespaces --all | grep 'uninstalling' | awk '{print $1}' | xargs -I {} helm delete --no-hooks {}`
-
- Force uninstalls all helm charts in "uninstalling" state.
-
-- `k create job --from cronjob/[cronjob name] [job name] -n bt`
-
- Creates a job from a cronjob. This is useful if you want to manually run the datapuller cronjob.
- Uninstalls all development environment deploys. Specifically, list then filters for helm charts with prefix `bt-dev-app`, then uninstalls them all.
-
-## Common Errors
-
-When deploying a [sealed secret](https://github.com/bitnami-labs/sealed-secrets) and a `no key could decrypt secret` error is seen in `kubectl describe sealedsecret/*`, there are two possible solutions:
-- Solution #1: During creating of sealed secret, make sure the (unsealed) secret is created in the correct namespace.
-- Solution #2: Use the tag `--scope=namespace-wide` when running `kubeseal` to allow renaming of the sealed secret. More details on [the kubeseal documentation](https://github.com/bitnami-labs/sealed-secrets?tab=readme-ov-file#scopes).
diff --git a/docs/src/core/infrastructure/onboarding.md b/docs/src/core/infrastructure/onboarding.md
index 960cfd713..4c994677a 100644
--- a/docs/src/core/infrastructure/onboarding.md
+++ b/docs/src/core/infrastructure/onboarding.md
@@ -1,7 +1,11 @@
# Onboarding
+
+
+## SSH Setup
+
> [!WARNING]
-> Running commands in `hozer-51` can break production! Please continue with caution.
+> This onboarding step is not necessary for local development. As running commands in `hozer-51` can break production, please continue with caution.
The Berkeleytime website is hosted on a machine supplied by [the OCF](https://www.ocf.berkeley.edu/). This machine will be referenced as `hozer-51` in these docs. SSH allows us to connect to `hozer-51` with a shell terminal, allowing us to infra-related tasks.
@@ -36,3 +40,105 @@ This guide assumes basic experience with SSH.
ssh hozer-51
# as opposed to root@hozer-51.ocf.berkeley.edu
```
+
+## Architecture
+
+Berkeleytime uses a fairly simple microservices architecture—we decouple only a few application components into separate services. Below is a high-level diagram of the current architecture (switch to a light viewing mode to see arrows).
+
+
+
+
+Note that, other than the application services developed by us, all other services are well-known and have large communities. These services have many tutorials, guides, and issues already created online, streamlining the setup and debugging processes.
+
+### An HTTP Request's Life
+
+To better understand the roles of each component in the Berkeleytime architecture, we describe the lifecycle of an HTTP request from a user's action.
+
+1. An HTTP request starts from a user's browser. For example, when a user visits `https://berkeleytime.com`, a `GET` request is sent to `hozer-51`.[^1]
+
+2. Once the request reaches `hozer-51`, it is first encountered by `hozer-51`'s Kubernetes cluster load balancer, a [MetalLB](https://metallb.io/) instance, which balances external traffic into the cluster across nodes.[^2]
+
+3. Next, the request reaches the [reverse proxy](https://www.cloudflare.com/learning/cdn/glossary/reverse-proxy/), an [nginx](https://nginx.org/) instance, which forwards HTTP requests to either a [backend](../backend) or [frontend](../frontend/) service based on the URL of the request
+ - Requests with URLs matching `https://berkeleytime.com/api/*` are forwarded to the backend service.
+ - All other requests are forwarded to the frontend service.
+
+ The nginx instance is also responsible for load balancing between the backend/frontend replicas. Currently, there are two of each in all deployment environments.
+
+4. The request is processed by one of the services.
+ - The backend service may interact with the MongoDB database or the Redis cache while processing the request.[^3]
+
+5. Finally, an HTTP response is sent back through the system to the user's machine.
+
+
+[^1]: More specifically, the user's machine first requests a DNS record of `berkeleytime.com` from a DNS server, which should return `hozer-51`'s IP address. After the user's machine knows the `hozer-51` IP address, the `GET` request is sent.
+
+[^2]: Currently, we only have one node: `hozer-51`.
+
+[^3]: Requests sent from the backend to the database or cache are *not* necessarily HTTP requests.
+
+## Kubernetes & Helm
+
+[Kubernetes](https://kubernetes.io/) is a container orchestrator that serves as the foundation of our infrastructure. It provides a simple deployment interface. To get started with Kubernetes, here are a few resources:
+- The [concepts page](https://kubernetes.io/docs/concepts/) is a good place to start.
+- The [glossary](https://kubernetes.io/docs/reference/glossary/?core-object=true&fundamental=true&networking=true) is also a good place to glance over common jargon.
+
+[Helm](https://helm.sh/) is a package manager for Kubernetes that provides an abstraction over the Kubernetes interface for deploying groups of components called "charts". In addition, it allows us to install pre-made charts, useful for deploying services that we don't develop.
+
+### Useful Commands
+
+This is an uncomprehensive list of commands that can be executed in `hozer-51`, useful for debugging.
+
+> [!TIP]
+> On `hozer-51`, `k` is an alias for `kubectl` and `h` is an alias for `helm`.
+
+> [!IMPORTANT]
+> The default namespace has been set as `bt`.
+
+#### Pods
+
+- `k get pods`
+
+ View all running pods.
+
+- `k get pods -l env=[dev|stage|prod]`
+
+ View all running pods in a specified environment.
+
+- `k logs [pod name]`
+
+ View logs of a pod. You can get a pod's name with `k get pods`. Include a `-f` flag to follow logs, which will stream logs into your terminal.
+
+- `k describe pod [pod name]`
+
+ View a description of a pod. Useful for when pod is failing to startup, thus not showing any logs.
+
+- `k exec -it [pod name] -- [command]`
+
+ Execute a command inside a pod. The command can be `bash`, which will start a shell inside the pod and allow for more commands.
+
+#### Deployments
+
+- `k get deploy`
+
+ View all running deployments.
+
+- `k get deploy -l env=[dev|stage|prod]`
+
+ View all running deployments in a specified environment.
+
+- `k describe deploy [deploy name]`
+
+ View a description of a deploy. Useful for when the deploy's pods are failing to startup, thus not showing any logs.
+
+- `k rollout restart deploy/[deploy name]`
+
+ Manually restart a deployment.
+
+#### Helm Charts
+
+- `h list`
+
+ List helm chart releases. A release is an installed instance of a chart.
diff --git a/docs/src/core/infrastructure/runbooks.md b/docs/src/core/infrastructure/runbooks.md
new file mode 100644
index 000000000..716b7f29f
--- /dev/null
+++ b/docs/src/core/infrastructure/runbooks.md
@@ -0,0 +1,61 @@
+# Runbooks
+
+
+
+## Manually run `datapuller`
+
+1. First, list all cronjob instances:
+
+ ```sh
+ k get cronjob
+ ```
+
+2. Then, create a job from the specific cronjob:
+
+ ```sh
+ k create job --from cronjob/[cronjob name] [job name]
+ ```
+
+ For example:
+ ```sh
+ k create job --from cronjob/bt-prod-datapuller-courses bt-prod-datapuller-courses-manual-01
+ ```
+
+## Uninstall ALL development helm releases
+
+```sh
+h list --short | grep "^bt-dev-app" | xargs -L1 h uninstall
+```
+
+Development deployments are limited by CI/CD. However, if for some reason the limit is bypassed, this is a quick command to uninstall all helm releases starting with `bt-dev-app`.
+
+## Force uninstall ALL helm charts in "uninstalling" state
+
+```sh
+helm list --all-namespaces --all | grep 'uninstalling' | awk '{print $1}' | xargs -I {} helm delete --no-hooks {}
+```
+
+Sometimes, releases will be stuck in an `uninstalling` state. This command quickly force uninstalls all such stuck helm releases.
+
+## New sealed secret deployment
+
+1. SSH into `hozer-51`.
+2. Create a new secret manifest with the key-value pairs and save into `my_secret.yaml`:
+
+ ```sh
+ k create secret generic my_secret -n bt --dry-run=client --output=yaml \
+ --from-literal=key1=value1 \
+ --from-literal=key2=value2 > my_secret.yaml
+ ```
+
+3. Create a sealed secret from the previously created manifest:
+
+ ```sh
+ kubeseal --secret-file my_secret.yaml --sealed-secret-file my_sealed_secret.yaml
+ ```
+
+ If the name of the secret might change across installations, add `--scope=namespace-wide` to the `kubeseal` command. For example, `bt-dev-secret` and `bt-prod-secret` are different names. Deployment without `--scope=namespace-wide` will cause a `no key could decrypt secret` error. More details on [the kubeseal documentation](https://github.com/bitnami-labs/sealed-secrets?tab=readme-ov-file#scopes).
+
+4. The newly create sealed secret encrypts the key-value pairs, allowing it to be safely pushed to GitHub.
+
+Steps 2 and 3 are derived from [the sealed-secrets docs](https://github.com/bitnami-labs/sealed-secrets?tab=readme-ov-file#usage).
diff --git a/docs/src/getting-started/deployment-with-cicd.md b/docs/src/getting-started/deployment-with-cicd.md
index 020fe1637..e7d43fa75 100644
--- a/docs/src/getting-started/deployment-with-cicd.md
+++ b/docs/src/getting-started/deployment-with-cicd.md
@@ -1,6 +1,9 @@
# Deployment with CI/CD
The deployment process is different for [development](#development), [staging](#staging), and [production](#production) environments.
+- **Development**: Best for short-term deployments to simulate a production environment as closely as possible. Useful for deploying feature branches before merging into `master`.
+- **Staging**: The last "testing" environment to catch bugs before reaching production. Reserved for the latest commit on `master`.
+- **Production**: User facing website! Changes being pushed to production should be thoroughly tested on a developer's local machine and in development and staging environments.
## Development