Formatted files

docs/source/admin_guide/awss3curl.md

@@ -2,10 +2,12 @@
 
 In some situations, users may wish to upload content to S3 or download content from an S3 bucket. For example, when attempting [manual backups of QHub's data](./backup.md).
 
-In many cases, the most straightforward way to access AWS S3 buckets is by installing and using AWS's command-line tool. But in some situations - for example, to back up the JupyterHub SQLite database - it may be difficult to install AWS' CLI tools due to being in a restricted container environment. In that situation, it is possible to fall back on AWS' basic REST API and use HTTPS requests directly instead. (Ultimately, the AWS CLI is simply a wrapper around those REST APIs.)
+In many cases, the most straightforward way to access AWS S3 buckets is by installing and using AWS's command-line tool. But in some situations - for example, to back up the
+JupyterHub SQLite database - it may be difficult to install AWS' CLI tools due to being in a restricted container environment. In that situation, it is possible to fall back on
+AWS' basic REST API and use HTTPS requests directly instead. (Ultimately, the AWS CLI is simply a wrapper around those REST APIs.)
 
-This document describes how to use `curl` commands to interface with S3 directly, specifically in the case of [uploading a backup of JupyterHub's SQLite database](./backup.md) from a restricted pod to S3
-(or restoring it from a backup from S3).
+This document describes how to use `curl` commands to interface with S3 directly, specifically in the case of [uploading a backup of JupyterHub's SQLite database](./backup.md) from
+a restricted pod to S3 (or restoring it from a backup from S3).
 
 ## Common settings
 
@@ -67,6 +69,7 @@ curl -H "Host: s3-${region}.amazonaws.com" \
      https://s3-${region}.amazonaws.com/${bucket}/${s3_file} -o $output_file
 ```
 
+______________________________________________________________________
 
----
-Inspired by [this article on how to use curl to upload files to was s3](https://www.gyanblog.com/aws/how-upload-aws-s3-curl/) and [this StackOverflow answer on how to access was s3 buckets](https://stackoverflow.com/a/57516606/2792760).
+Inspired by [this article on how to use curl to upload files to was s3](https://www.gyanblog.com/aws/how-upload-aws-s3-curl/) and
+[this StackOverflow answer on how to access was s3 buckets](https://stackoverflow.com/a/57516606/2792760).

docs/source/admin_guide/backup.md

@@ -2,7 +2,8 @@
 
 Your cloud provider may have native ways to backup your Kubernetes cluster and volumes.
 
-This guide describes how you would manually obtain the data you need to repopulate your QHub if your cluster is lost and you wish to start it up again from the `qhub-config.yaml` file.
+This guide describes how you would manually obtain the data you need to repopulate your QHub if your cluster is lost and you wish to start it up again from the `qhub-config.yaml`
+file.
 
 There are three main locations that you need to backup:
 
@@ -15,15 +16,15 @@ There are three main locations that you need to backup:
 This amounts to:
 
 - Tarballing the /home directory
-- Saving to block storage [s3, google cloud storage, etc]
+- Saving to block storage \[s3, google cloud storage, etc\]
 - Downloading and untaring to new cluster
 
 This specific guide shows how to do this on an AWS cluster and upload to AWS S3.
 
 ### Pre-requisites
 
-- [Install kubectl](<https://kubernetes.io/docs/tasks/tools/>)
-- [Install AWS command-line tool](<https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html>)
+- [Install kubectl](https://kubernetes.io/docs/tasks/tools/)
+- [Install AWS command-line tool](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
 
 ### Kubectl configuration
 
@@ -41,7 +42,8 @@ aws eks update-kubeconfig  --region us-west-2 --name <relevant-name>
 
 ### Pod deployment
 
-With `kubectl` configured, the next step will be to deploy the pod that allows you to access the cluster files. First, save the following pod specification to a file named `pod.yaml`:
+With `kubectl` configured, the next step will be to deploy the pod that allows you to access the cluster files. First, save the following pod specification to a file named
+`pod.yaml`:
 
 ```yaml
 kind: Pod
@@ -109,7 +111,8 @@ cd /data
 tar -cvf <custom_name>.tar .
 ```
 
-The preferred naming scheme includes a year-month-day, example `2021-04-23_home_backup.tar`. You can utilize multi-backups through this step. This step takes several minutes depending on the size of the home directories.
+The preferred naming scheme includes a year-month-day, example `2021-04-23_home_backup.tar`. You can utilize multi-backups through this step. This step takes several minutes
+depending on the size of the home directories.
 
 ### Upload to block storage
 
@@ -152,22 +155,26 @@ The file permissions for the default tar is same as the original files.
 > QHub v0.4: If restoring your NFS as part of the upgrade you must also run some extra commands, immediately after extracting from the tar file.
 >
 > Previous versions contained the `shared` folder within `home`. From `0.4.0` both `shared` and `home` directories are at the same level with respect to the QHub filesystem:
+>
 > ```shell
 > cd /data
 > cp -r home/shared/* shared/
 > rm -rf home/shared/
 > ```
+>
 > And then:
+>
 > ```shell
 > chown -R 1000:100 /data/home/*
 > chown -R 1000:100 /data/shared/*
 > ```
-> From QHUb v0.4. all users will have the same `uid`.
 >
+> From QHUb v0.4. all users will have the same `uid`.
 
 ### Google cloud provider
 
-To use the Google Cloud provider, install the [gsutil](https://cloud.google.com/storage/docs/gsutil_install) CLI instead of the AWS CLI. Otherwise, the instructions are the same as for AWS above, other than when working with S3. Here are the commands to access Google Spaces instead of S3 for copy/download of the backup:
+To use the Google Cloud provider, install the [gsutil](https://cloud.google.com/storage/docs/gsutil_install) CLI instead of the AWS CLI. Otherwise, the instructions are the same as
+for AWS above, other than when working with S3. Here are the commands to access Google Spaces instead of S3 for copy/download of the backup:
 
 ```shell
 cd /data
@@ -182,18 +189,20 @@ gsutil cp gs://<your_bucket_name>/backups/2021-04-23.tar .
 Instructions will be similar to those for AWS above, but use Digital Ocean spaces instead of S3. This guide explains installation of the command-line tool:
 <https://www.digitalocean.com/community/tutorials/how-to-migrate-from-amazon-s3-to-digitalocean-spaces-with-rclone>
 
-
 ## JupyterHub Database
 
 The JupyterHub database will mostly be recreated whenever you start a new cluster, but should be backed up to save Dashboard configurations.
 
-You want to do something very similar to the NFS backup, above - this time you need to back up just one file located in the  PersistentVolume `hub-db-dir`.
+You want to do something very similar to the NFS backup, above - this time you need to back up just one file located in the PersistentVolume `hub-db-dir`.
 
-First, you might think you can just make a new `pod.yaml` file, this time specifying `claimName: "hub-db-dir"` instead of `claimName: "jupyterhub-dev-share"`. However, `hub-db-dir` is 'Read Write Once' - the 'Once' meaning it can only be mounted to one pod at a time, but the JupyterHub pod will already have this mounted! So the same approach will not work here.
+First, you might think you can just make a new `pod.yaml` file, this time specifying `claimName: "hub-db-dir"` instead of `claimName: "jupyterhub-dev-share"`. However, `hub-db-dir`
+is 'Read Write Once' - the 'Once' meaning it can only be mounted to one pod at a time, but the JupyterHub pod will already have this mounted! So the same approach will not work
+here.
 
 Instead of mounting to a new 'debugger pod' you have to access the JupyterHub pod directly using the `kubectl` CLI.
 
 Look up the JupyterHub pod:
+
 ```bash
 kubectl get pods -n dev
 ```
@@ -210,7 +219,8 @@ There is no need to TAR anything up since the only file required to be backed up
 
 ### Backing up JupyterHub DB
 
-Now we just need to upload the file to S3. You might want to [install the AWS CLI tool](#installations) as we did before, however, as the Hub container is a rather restricted environment the recommended approach is to upload files to AWS S3 buckets using curl.
+Now we just need to upload the file to S3. You might want to [install the AWS CLI tool](#installations) as we did before, however, as the Hub container is a rather restricted
+environment the recommended approach is to upload files to AWS S3 buckets using curl.
 
 For more details please refer to the [using curl to access AWS S3 buckets](./awss3curl.md) documentation.
 
@@ -219,6 +229,7 @@ For more details please refer to the [using curl to access AWS S3 buckets](./aws
 You will need to overwrite the file `/srv/jupyterhub/jupyterhub.sqlite` based on the version backed up to S3.
 
 You should restart the pod:
+
 ```bash
 kubectl delete -n dev pod hub-765c9488d6-8z4nj
 ```
@@ -227,7 +238,8 @@ As for uploads, [you may need to use curl to download items from an AWS S3 bucke
 
 ## Keycloak user/group database
 
-QHub provides a simple script to export the important user/group database. Your new QHub cluster will recreate a lot of Keycloak config (including new Keycloak clients which will have new secrets), so only the high-level Group and User info is exported.
+QHub provides a simple script to export the important user/group database. Your new QHub cluster will recreate a lot of Keycloak config (including new Keycloak clients which will
+have new secrets), so only the high-level Group and User info is exported.
 
 If you have a heavily customized Keycloak configuration, some details may be omitted in this export.
 

docs/source/admin_guide/breaking-upgrade.md

@@ -15,7 +15,8 @@ Please always check the [release notes](../dev_guide/changelog.md) for more deta
 
 > The rest of this guide assumes you are upgrading from version v0.3.14 (or earlier) to v0.4.
 
-You may be deploying QHub based on a local configuration file, or you may be using CI/CD workflows in GitHub or GitLab. Either way, you will need to locate a copy of your `qhub-config.yaml` configuration file to upgrade it (and commit back to your git repo in the CI/CD case).
+You may be deploying QHub based on a local configuration file, or you may be using CI/CD workflows in GitHub or GitLab. Either way, you will need to locate a copy of your
+`qhub-config.yaml` configuration file to upgrade it (and commit back to your git repo in the CI/CD case).
 
 For CI/CD deployments, you will need to `git clone <repo URL>` into a folder on your local machine if you haven't done so already.
 
@@ -65,11 +66,13 @@ qhub upgrade -c qhub-config.yaml
 
 This will output a newer version of `qhub-config.yaml` that's compatible with the new version of `qhub`. The process outputs a list of changes it has made.
 
-The `upgrade` command creates a copy of the original unmodified config file (`qhub-config.yaml.old.backup`) as well as a JSON file (`qhub-users-import.json`) used to import existing users into Keycloak.
+The `upgrade` command creates a copy of the original unmodified config file (`qhub-config.yaml.old.backup`) as well as a JSON file (`qhub-users-import.json`) used to import
+existing users into Keycloak.
 
 ## 5. Rename the Project and Increase Kubernetes version
 
-You need to rename the project to avoid clashes with the existing (old) cluster which would otherwise already own resources based on the names that the new cluster will attempt to use.
+You need to rename the project to avoid clashes with the existing (old) cluster which would otherwise already own resources based on the names that the new cluster will attempt to
+use.
 
 The domain should remain as the preferred main one that was always in use previously.
 
@@ -87,7 +90,8 @@ project_name: myqhubnew
 domain: qhub.myproj.com
 ```
 
-> It is also a good time to upgrade your version of Kubernetes. Look for the `kubernetes_version` field within the cloud provider section of the `qhub-config.yaml` file and increase it to the latest.
+> It is also a good time to upgrade your version of Kubernetes. Look for the `kubernetes_version` field within the cloud provider section of the `qhub-config.yaml` file and
+> increase it to the latest.
 
 ## 6. Redeploy QHub
 
@@ -97,25 +101,30 @@ You will now have a `qhub-config.yaml` file that you can deploy.
 qhub deploy -c qhub-config.yaml
 ```
 
-At this point you will see an error message saying that deployment is prevented due to the `prevent_deploy` setting in your YAML file. This is a safeguard to ensure that you only proceed if you are aware of possible breaking changes in the current upgrade.
+At this point you will see an error message saying that deployment is prevented due to the `prevent_deploy` setting in your YAML file. This is a safeguard to ensure that you only
+proceed if you are aware of possible breaking changes in the current upgrade.
 
 Make sure to **backup your data** as described in the [backup section of the documentation](./backup.md).
 
-Only after backing up your data proceed to remove the `prevent_deploy: true` line in the `qhub-config.yaml` file. This `prevent_deploy` functionality is there as a safeguard. Please only remove it if you understand why it was there in the first place - as a way to stop users blindly upgrading without realising they absolutely needed to backup their data first so that it can be restored into a completely new cluster.
+Only after backing up your data proceed to remove the `prevent_deploy: true` line in the `qhub-config.yaml` file. This `prevent_deploy` functionality is there as a safeguard.
+Please only remove it if you understand why it was there in the first place - as a way to stop users blindly upgrading without realising they absolutely needed to backup their data
+first so that it can be restored into a completely new cluster.
 
 Run the `qhub deploy -c qhub-config.yaml` command again and it should get further this time.
 
 ## 7. CI/CD: render and commit to git
 
-For CI/CD (GitHub/GitLab) workflows, as well as generating the updated `qhub-config.yaml` files as above, you will also need to regenerate the workflow files based on the latest `qhub` version's templates.
+For CI/CD (GitHub/GitLab) workflows, as well as generating the updated `qhub-config.yaml` files as above, you will also need to regenerate the workflow files based on the latest
+`qhub` version's templates.
 
 With the newly upgraded `qhub-config.yaml` file, run:
 
 ```shell
 qhub render -c qhub-config.yaml
 ```
 
-(Note that `qhub deploy` would have performed this render step too, but will also immediately redeploy your QHub instance. Run the render command alone if you are now working separately in your repo and don't want to redeploy.)
+(Note that `qhub deploy` would have performed this render step too, but will also immediately redeploy your QHub instance. Run the render command alone if you are now working
+separately in your repo and don't want to redeploy.)
 
 Commit all the files (`qhub-config.yaml` and GitHub/GitLab workflow files) back to the remote repo. All files need to be committed together in the same commit. For example:
 
@@ -133,8 +142,8 @@ If your QHub deployment relies on Auth0 or GitHub for authentication, please upd
 
 2. Select the "Regular Web Application" with the name of your deployment.
 
-3. Under the "Application URIs" section, paste the new OAuth callback URL in the "Allowed Callback URLs" text block.
-   The URL should be `https://{your-qhub-domain}/auth/realms/qhub/broker/auth0/endpoint`, replacing `{your-qhub-domain}` with your literal domain of course.
+3. Under the "Application URIs" section, paste the new OAuth callback URL in the "Allowed Callback URLs" text block. The URL should be
+   `https://{your-qhub-domain}/auth/realms/qhub/broker/auth0/endpoint`, replacing `{your-qhub-domain}` with your literal domain of course.
 
 </details>
 
@@ -144,8 +153,8 @@ If your QHub deployment relies on Auth0 or GitHub for authentication, please upd
 
 2. Click "OAuth Apps" and then click the app representing your QHub instance.
 
-3. Under "Authorization callback URL", paste the new GitHub callback URL.
-   The URL should be `https://{your-qhub-domain}/auth/realms/qhub/broker/github/endpoint`, replacing `{your-qhub-domain}` with your literal domain of course.
+3. Under "Authorization callback URL", paste the new GitHub callback URL. The URL should be `https://{your-qhub-domain}/auth/realms/qhub/broker/github/endpoint`, replacing
+   `{your-qhub-domain}` with your literal domain of course.
 
 </details>
 
@@ -166,9 +175,9 @@ The last two steps are to:
 
 For more details on this process, visit the [Keycloak docs section](../installation/login.md).
 
-
 ## Known versions that require re-deployment
 
-Version `v0.3.11` on AWS has an error with the Kubernetes config map. See [this GitHub discussion related to AWS K8s config maps](https://github.com/Quansight/qhub/discussions/841) for more details.
+Version `v0.3.11` on AWS has an error with the Kubernetes config map. See [this GitHub discussion related to AWS K8s config maps](https://github.com/Quansight/qhub/discussions/841)
+for more details.
 
 Version `v0.4`.

docs/source/admin_guide/clearml.md

@@ -54,8 +54,7 @@ This is especially useful for accessing ClearML programmatically.
 
 ## Overrides
 
-Addition helm chart variables may want to be overridden. For this an
-override hook is provided where you can specify anything with the
+Addition helm chart variables may want to be overridden. For this an override hook is provided where you can specify anything with the
 [values.yaml](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml).
 
 ```yaml