From ed55ccaed6e0627448bfcf7dfacb707a28566fe8 Mon Sep 17 00:00:00 2001 From: iameskild Date: Thu, 15 Jul 2021 17:17:42 -0700 Subject: [PATCH 1/7] Update /installation docs --- docs/source/installation/configuration.md | 172 +++++++++++++--------- docs/source/installation/management.md | 13 +- docs/source/installation/setup.md | 54 +++---- docs/source/installation/usage.md | 41 +++--- 4 files changed, 153 insertions(+), 127 deletions(-) diff --git a/docs/source/installation/configuration.md b/docs/source/installation/configuration.md index 9ba019d04b..974a5bdc27 100644 --- a/docs/source/installation/configuration.md +++ b/docs/source/installation/configuration.md @@ -1,12 +1,7 @@ # Configuration -The configuration file is split into several sections. In this page, -we detail the requirements necessary for the YAML configuration -file. The configuration file is always validated by a [pydantic -schema](https://pydantic-docs.helpmanual.io/) in -[qhub/schema.py](https://github.com/Quansight/qhub/blob/main/qhub/schema.py). For -the ultimate source of truth visit this file though these docs should -be accurate. +This is a page dedicated to the `qhub-config.yaml` file, the file that `qhub` uses to deploy and redeploy changes to your infrastructure. The `qhub-config.yaml` configuration file is split into several sections and in this page, we detail the requirements necessary for this YAML-formatted configuration file. In the [Usage](usage.md) section we covered how you can auto-generate this file using `qhub init` (and properly set options/flags and environment variables). +> NOTE: The configuration file is always validated by a [pydantic schema](https://pydantic-docs.helpmanual.io/) in [qhub/schema.py](https://github.com/Quansight/qhub/blob/main/qhub/schema.py) ## General @@ -18,19 +13,23 @@ domain: "do.qhub.dev" # top level URL exposure to monitor JupyterLab ``` - `project_name`: should be compatible with the Cloud provider naming - convention. Generally only use `A-Z`, `a-z`, `-`, and `_`. + convention. Generally only use `A-Z`, `a-z`, `-`, and `_` + (see [Project Naming Conventions](usage.md#project-naming-convention) for more details). - `namespace`: is used in combination with `project_name` to label resources. In addition `namespace` also determines the `namespace` - that used when deploying kubernetes resources for qhub. Has a - default value of `dev`. + that used when deploying kubernetes resources for qhub. + - Default value: `dev` - - `provider` possible values are `do` for DigitalOcean, `aws` for - Amazon AWS, `gcp` for Google Could Provider, `azure` for Microsoft - Azure, and `local` for a local or existing kubernetes deployment. + - `provider`: possible values are + - `do` for DigitalOcean + - `aws` for Amazon AWS + - `gcp` for Google Could Provider + - `azure` for Microsoft Azure + - `local` for a local or existing kubernetes deployment - `domain`: is the top level URL to put JupyterLab and future - services under such a monitoring. For example `qhub.dev` + services under (such a monitoring). For example `qhub.dev` would be the domain for JupyterHub to be exposed under. ## Continuous Integration and Continuous Deployment @@ -38,14 +37,13 @@ domain: "do.qhub.dev" # top level URL exposure to monitor JupyterLab `ci_cd`: is optional and specifies the continuous integration and continuous deployment framework to use. QHub uses infrastructure as code to allow developers and users of QHub to request change to the -environment via PRs which then get approved by administration. You may -configure the branch that github-actions watches for pull requests and -commits. Current allowable values are `gitlab-ci`, `github-actions`, -and not specifying the key `ci_cd`. +environment via pull requests (PRs) which then get approved by administration. +You may configure CI/CD process to watch for pull-requests or commits on +specific branches. Currently CI/CD can be setup for either GitHub Actions or GitLab CI. ```yaml ci_cd: - type: github-actions + type: gitlab-ci branch: main before_script: - echo "running commands before ci completes" @@ -58,22 +56,22 @@ ci_cd: - `branch`: branch to use to commit `qhub render` changes to - `before_script`: optional script to run before CI starts QHub infrastructure deployment. This is useful in cases that additional - setup is required for Qhub to deploy the resources. Only supported + setup is required for QHub to deploy the resources. Only supported on `gitlab-ci` at the moment. - `after_script`: optional script to run after CI ends QHub infrastructure deployment. This is useful in cases to notify resources of successful QHub deployment. Only supported on `gitlab-ci` at the moment. -If `ci_cd` is not supplied no CI/CD will be auto-generated. However, -we advise that having infrastructure as code allows teams to more -quickly modify their QHub deployment often allowing developers and -data sciences to request the changes to be approved by an +If `ci_cd` is not supplied, no CI/CD will be auto-generated. However, +we advise that employing an infrastructure-as-code approach allows teams +to more quickly modify their QHub deployment, allowing developers and +data sciencist to request the changes and approved by an administrator. ## Certificate -By default, to simplify initial deployment `QHub` uses traefik to +By default, to simplify initial deployment `qhub` uses traefik to create a self-signed certificate. In order to create a certificate that is signed so that web browsers do not throw errors we currently support [Let's Encrypt](https://letsencrypt.org/). @@ -83,15 +81,15 @@ certificate: type: self-signed ``` -To use Let's Encrypt you must specify an email address that let's -encrypt will associate the generated certificate with and whether to +To use Let's Encrypt you must specify an email address that Let's +Encrypt will associate the generated certificate with and whether to use the [staging server](https://acme-staging-v02.api.letsencrypt.org/directory) or [production server](https://acme-v02.api.letsencrypt.org/directory). In general -you should use the production server, as seen below. Please note that -let's encrypt heavily rate limits their production endpoint and -provisioning https certificates can often fail due to this limit. +you should use the production server, as seen below. +> NOTE: Let's Encrypt heavily rate limits their production endpoint and +> provisioning https certificates can often fail due to this limit. ```yaml certificate: @@ -100,10 +98,7 @@ certificate: acme_server: https://acme-v02.api.letsencrypt.org/directory ``` -You may also supply a custom self-signed certificate and secret -key. Note that the kubernetes default namespace that QHub uses is -`dev` if not specified. Otherwise, it will be your `namespace` defined -in the `qhub-config.yaml`. +You may also supply a custom self-signed certificate and secret key. ```yaml certificate: @@ -119,11 +114,18 @@ kubectl create secret tls \ --namespace= \ --cert=path/to/cert/file --key=path/to/key/file ``` +> NOTE: the default kubernetes namespace that QHub uses is `dev`, however you can change the `namespace` key in the `qhub-config.yaml`. ## Security -This section is for configuring security relating to the QHub -deployment. +This section walks through security and user authentication as it +relates to QHub deployments. There are a few different ways to handle +user authentication: +- Auth0 +- GitHub +- Password +- Custom OAuth + ```yaml security: authentication: @@ -178,12 +180,16 @@ authentication. #### Auth0 Based Authentication [Auth0](https://auth0.com/#!) can be used for authentication. While it -is not free there is a reasonable free tier that allows deployment of -QHub clusters on many different social providers, passwordless, and -email based authentication. QHub has command line options with running -`qhub init` which allow for automation of creation of the application -via `--auth-provider=auth0 --auth-auto-provision`. However for most -users this may not be the most convenient solution. Here are docs on +is not free, there is a reasonable free tier that allows deployment of +QHub clusters using many different social providers, passwordless, and +email based authentication methods. + +QHub has command line options with running `qhub init` which allow for +automation of creation of the application via: +`--auth-provider=auth0 --auth-auto-provision`. + +However for most users this may not be the most convenient solution. +Here are docs on [creating an Auth0 Application](https://auth0.com/docs/applications). Make sure to select `Regular Web Application`. Important to note is the `auth0_subdomain` @@ -191,6 +197,8 @@ field which must be only the `.auth0.com`. So for the following `qhub-dev.auth0.com` the subdomain would be `qhub-dev`. Note that all the usernames will be the email addresses of users (not usernames). +> NOTE: This is a different and distinct step from one outlined in the [Setup](setup.md#auth0) +stage. ```yaml security: @@ -206,9 +214,9 @@ security: #### GitHub Based Authentication -Github has instructions for [creating OAuth +GitHub has instructions for [creating OAuth applications](https://docs.github.com/en/developers/apps/creating-an-oauth-app). Note -that QHub usernames will their GitHub usernames. +that QHub usernames will be their GitHub usernames. ```yaml security: @@ -226,7 +234,7 @@ For Password based authentication. Note that users will require a `password` field that can be generated via the following command: `python -c "import bcrypt; print(bcrypt.hashpw(b'', bcrypt.gensalt()).decode('utf-8'))"`. Make sure to replace -`` with whatever password you are wanting. +`` with whatever secure password you want. ```yaml security: @@ -262,8 +270,8 @@ security: `users` and `groups` allows one to provision UNIX permissions to each user. Any user is assigned a `uid`, `primary_group`, and optionally -any number of `secondary_groups`. Note that `uid` and `gid` fields -must be unique and are required. +any number of `secondary_groups`. +> NOTE: the `uid` and `gid` fields must be unique and are required. ```yaml security: @@ -294,7 +302,7 @@ written for the user. `1000`, it is recommended to start with high User ID numbers e.g. `10000000`. `ids` technically supports 2 billion `ids`. -### Admin Group +#### Admin Group The admin group has special significance. If a user's `primary_group` is admin they will be able to access the jupyterhub admin page. The @@ -303,29 +311,29 @@ user's server and impersonate them. ## Provider Infrastructure -Finally, comes the Kubernetes infrastructure deployment. Although -quite similar, each provider has a different configuration. +Finally, the Kubernetes infrastructure deployment. Although +quite similar, each provider has a slightly different configuration. The following configuration sets up a kubernetes deployment with autoscaling node groups. Depending on the cloud provider there might be restrictions, which are detailed on each section. For any of the providers (besides local), adding a node group is as -easy as the following, which adds a `high-memory` group: +easy as the following: which adds a `high-memory` group: ```yaml : node_groups: ... high-memory: - instance: "s-2vcpu-4gb" + instance: "s-2vcpu-4gb" # name of cloud provider instance type min_nodes: 1 max_nodes: 50 ... ``` -> For each provider details such as **instance names**, **availability zones**, -and **Kubernetes versions** will be DIFFERENT. [duplicated info] +> NOTE: For each provider, details such as **instance names**, **availability zones**, +and **Kubernetes versions** will be DIFFERENT. ### Providers @@ -338,12 +346,9 @@ Digital Ocean does not have accelerator/gpu support. Digital Ocean is a great default choice for tying out QHub. Below is the recommended setup. -> Note: DigitalOcean regularly updates Kubernetes versions hence, the -> field `kubernetes_version` will most likely have to be changed. -> [See available instance types for -> DigitalOcean](https://www.digitalocean.com/docs/droplets/). If you -> used `qhub init` this version will automatically be compute for you -> Do not copy the version you see bellow +> NOTE: DigitalOcean regularly updates Kubernetes versions hence, the field `kubernetes_version` will most likely have to be changed. [See available instance types for DigitalOcean](https://www.digitalocean.com/docs/droplets/). +> If you used `qhub init` this version will automatically be computed for you. +> Do not copy the version you see below. To see available instance types refer to [Digital Ocean Instance Types](https://www.digitalocean.com/docs/droplets/). Additionally the @@ -501,7 +506,7 @@ Default images are to the default image run if not specified in a profile (described in the next section). The `jupyterhub` key controls the jupyterhub image run. These control the docker image used to run JupyterHub, the default JupyterLab image, the default Dask worker -image, and dask gateway docker image. +image, and Dask Gateway docker image. ```yaml default_images: @@ -515,8 +520,11 @@ default_images: Control the amount of storage allocated to shared filesystems. -> Note: when the storage size is changed, in most providers it will +> NOTE 1: when the storage size is changed, for most providers it will > automatically delete (!) the previous storage place. +> NOTE 2: changing the storage size on an AWS deployment after the initial +> deployment can be especially tricky so it might be worthwhile padding +> these storage sizes. ```yaml storage: @@ -527,7 +535,7 @@ storage: ## Profiles Profiles are used to control the JupyterLab user instances and -Dask-workers provided by Dask-gateway. +Dask workers provided by Dask Gateway. ```yaml profiles: @@ -580,7 +588,7 @@ Finally, we allow for configuration of the Dask workers. In general, similar to the JupyterLab instances you only need to configuration the cores and memory. -When configuring the memory and cpus for profiles there are some +When configuring the memory and CPUs for profiles there are some important considerations to make. Two important terms to understand are: - `limit`: the absolute max memory that a given pod can consume. If a process within the pod consumes more than the `limit` memory the @@ -593,14 +601,34 @@ important considerations to make. Two important terms to understand are: ocean](https://docs.digitalocean.com/products/kubernetes/#allocatable-memory) which is generally applicable to other clouds. -For example if a node is 8 GB of ram and 2 cpu you should +For example if a node has 8 GB of ram and 2 CPUs you should guarantee/schedule roughly 75% and follow the digital ocean guide -linked above. E.g. 1.5 cpu guarantee and 5.5 GB guaranteed. +linked above. E.g. 1.5 CPU guarantee and 5.5 GB guaranteed. + +#### Dask Scheduler + +In a few instances, the Dask worker node-group might be running on quite a large +instance, perhaps with 8 CPUs and 32 GB of memory (or more). When this is the case, you +might also want to increase the resource levels associated with the Dask Scheduler. + +```yaml +dask_worker: + "Huge Worker": + worker_cores_limit: 7 + worker_cores: 6 + worker_memory_limit: 30G + worker_memory: 28G + scheduler_cores_limit: 7 + scheduler_cores: 6 + scheduler_memory_limit: 30G + scheduler_memory: 28G + image: "quansight/qhub-dask-worker:v||QHUB_VERSION||" +``` ### Limiting profiles to specific users and groups Sometimes on a select set of users should have access to specific -resources e.g. gpus, high memory nodes etc. QHub has support for +resources e.g. GPUs, high memory nodes etc. QHub has support for limiting resources. ```yaml @@ -656,8 +684,8 @@ profiles: JupyterHub can be customized since QHub uses [Quansight/qhub-jupyterhub-theme](https://github.com/quansight/qhub-jupyterhub-theme). Available -theme options. Note that if you want to change the logo it must be an -accessible url to the logo. +theme options. +> NOTE: if you want to change the logo it must be an accessible URL to the logo. ```yaml theme: @@ -727,8 +755,6 @@ configuration for the environment to appear. # Full Configuration Example -Everything in the configuration is set besides [???] - ```yaml project_name: do-jupyterhub provider: do @@ -862,7 +888,7 @@ environments: - python=3.7 - ipykernel - ipywidgets - - qhub-dask==0.3.12 + - qhub-dask==||QHUB_VERSION|| - numpy - numba - pandas @@ -876,7 +902,7 @@ environments: - python=3.7 - ipykernel - ipywidgets - - qhub-dask==0.3.12 + - qhub-dask==||QHUB_VERSION|| - numpy - numba - pandas diff --git a/docs/source/installation/management.md b/docs/source/installation/management.md index 90736650d7..96c2bc57c9 100644 --- a/docs/source/installation/management.md +++ b/docs/source/installation/management.md @@ -3,10 +3,10 @@ ## Add users to QHub One of the first things you might want to do is to **add new users** -to your QHub. +to your QHub deployment. -Any type of supported authorization from auth0 can be used as a -username. Below is an example configuration of 2 users: +Below we have added two users (by their username in this case). +> NOTE: Auth0 authorization uses the user's email. ```yaml joeuser@example: @@ -60,9 +60,4 @@ called `shared/.userskel`. Any files in there will be copied to a user's own home folder whenever they start a new JupyterLab session. Existing files with the same name will not be overwritten. Admin users are defined as members of the admin group as -specified in your qhub-config.yaml file. - -## Monitor your QHub deployment - -TODO: add instruction on how to install and use K9s for monitoring the -system deployment. +specified in your qhub-config.yaml file. \ No newline at end of file diff --git a/docs/source/installation/setup.md b/docs/source/installation/setup.md index 981f0e13d8..bb27a91f3d 100644 --- a/docs/source/installation/setup.md +++ b/docs/source/installation/setup.md @@ -1,12 +1,12 @@ # Setup Initialization QHub handles the initial setup and management of configurable data -science environments, allowing to users to attain seamless deployment -with Github Actions. +science environments, allowing users to deploy seamlessly +using Github Actions. -QHub can be installed on a bare-metal server using HPC, on a CLoud -provider or even locally for testing purposes. Analyse the options -below to discover what option suits best your specific use case. +QHub can be installed on a bare-metal server using HPC, on a Cloud +provider or even locally for testing purposes. Review the options +below to discover which option best suits your needs. ## Local Deployment or Existing Kubernetes Cluster @@ -38,21 +38,21 @@ The [QHub HPC](https://hpc.qhub.dev/en/latest/) should be your choice if: ## Kubernetes Deployment -The Kubernetes version of QHub is considered to be the default +The Kubernetes deployment of QHub is considered to be the default option. If you are not sure which option to choose, try this one. It -is suitable to most use cases, specially when: -- You require a scalable infrastructure +is suitable for most use cases, especially if: +- You require scalable infrastructure - You aim to have a production environment with GitOps enabled by default The QHub version requires a choice of [Cloud -provider](#cloud-provider), [authentication (using Auth0, GitHub, or +provider](#cloud-provider), [authentication (using Auth0, GitHub, custom OAuth provider, or password based)](#authentication), [domain -registration](#domain-registry), and CI provider (GitHub Actions). +registration](#domain-registry), and CI provider (GitHub Actions, GitLab CI). These services require global [environment variables](https://linuxize.com/post/how-to-set-and-list-environment-variables-in-linux/) -that once set up will trigger QHub's automatic deploy using GitHub -Actions. +that once set up, will trigger QHub's automatic deploy using your +CI/CD platform of choice. To find and set the environment variables, follow the steps described on the subsections below. @@ -68,9 +68,10 @@ knowledge about the specific provider required either. QHub supports [Azure](#microsoft-azure). To deploy QHub, all access keys require fairly wide permissions to -create all the resources. Hence, once the Cloud provider has been -chosen, follow the steps below and set the environment variables as -specified with **owner/admin** level permissions. +create all the necessary cloud resources. Hence, once the Cloud +provider has been chosen, follow the steps below and set the +environment variables as specified with **owner/admin** level +permissions. For more details on configuration for each Cloud provider, check the How-To Guides section of the documentation. @@ -106,8 +107,6 @@ Set the required environment variables as specified below: export DIGITALOCEAN_TOKEN="" # API token required to generate resources export SPACES_ACCESS_KEY_ID="" # public access key for access spaces export SPACES_SECRET_ACCESS_KEY="" # the private key for access spaces -export AWS_ACCESS_KEY_ID="" # set this variable with the same value as `SPACES_ACCESS_KEY_ID` -export AWS_SECRET_ACCESS_KEY="" # set this variable identical to `SPACES_SECRET_ACCESS_KEY` ``` @@ -149,6 +148,7 @@ export ARM_TENANT_ID="" # field available under `Azure Active Director > NOTE 2: [Tenant ID](https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-how-to-find-tenant) > values can be also found using PowerShell and CLI. + ### Authentication @@ -163,7 +163,7 @@ application](https://auth0.com/docs/applications/set-up-an-application/register- - Click on the `Applications` button on the left - Select `Create Application` > `Machine to Machine Applications` > `Auth0 Management API` from the dropdown menu - Next, click `All` next to `Select all` and click `Authorize` -- Set the variable `AUTH0_CLIENT_ID` equal to the `Cliend ID` string, and do the same for the `Client secret` by running the command below. +- Set the variable `AUTH0_CLIENT_ID` equal to the `Client ID` string, and do the same for the `Client secret` by running the command below. The following environment variables must be set: @@ -173,7 +173,7 @@ The following environment variables must be set: corner) appended with `.auth0.com`, for example: ```bash -export AUTH_DOMAIN="qhub-test.auth0.com" # in case the account was called 'qhub-test' +export AUTH_DOMAIN="qhub-test.auth0.com" # in this example the account is 'qhub-test' ``` #### GitHub @@ -198,15 +198,17 @@ to be on the safe side enable all permissions. Finally, you will need to have a domain name for hosting QHub. This domain will be where your application will be exposed. -Currently, QHub only supports CloudFlare for automatic DNS +Currently, QHub only supports CloudFlare for **automatic** DNS registration. If an alternate DNS provider is desired, change the `--dns-provider` flag from `cloudflare` to `none` on the `qhub deploy` -command. The deployment then will be paused when it asks for an IP -address (or CNAME, if using AWS) and prompt to register the desired -URL. Setting a DNS record heavily depends on the provider thus it is -not possible to have detailed docs on how to create a record on your -provider. Googling `setting record on ` -should yield good results on doing it for your specific provider. +command. During the initial deployment, the deployment script will +pause as it asks for an IP address (or CNAME, if using AWS) and will +prompt you to register the desired URL. How to set a DNS record heavily +depends on the provider thus it is not possible to have detailed docs +on how to create a record on your provider. + +Googling `setting record on ` should yield +good results on doing it for your specific provider. #### Cloudflare diff --git a/docs/source/installation/usage.md b/docs/source/installation/usage.md index d0a2383cee..96de917cf6 100644 --- a/docs/source/installation/usage.md +++ b/docs/source/installation/usage.md @@ -2,13 +2,14 @@ ## Cloud Deployment -Once all environment variables have been set, you will be able to run -commands on your terminal to set QHub's deployment. +Great, you've gone through the `qhub` [Installation](installation.md) and [Setup Initialization](setup.md) steps, +and have ensured that all the necessary environment variables have been properly set, it is time to deploy QHub +from your terminal. ### Initialize configuration There are several ways to generate your configuration file. You can -type your commands according to the terminal prompts, or you can set +type the commands when prompted by terminal, or you can set it all automatically from the start. In any case, we advise you to start by creating a new project folder. Here, we will name the new folder `qhub-test`. @@ -21,38 +22,40 @@ mkdir qhub-test && cd qhub-test #### Fully automated deployment -To generate a fully automated deployment, on your terminal run: +To generate a fully automated configuration file, on your terminal run: ```shell qhub init aws --project projectname --domain qhub.dev --ci-provider github-actions --auth-provider auth0 --auth-auto-provision --repository github.com/quansight/project-name --repository-auto-provision ``` - -The command above will generate the `qhub-config.yaml` config file -with an infrastructure deployed on `aws`, named `projectname`, where -the domain will be `qhub.dev`. The deployment -will use `github-actions` as the continuous integration (CI) provider, -automatically provisioned and authenticated by `auth0`, initialized on -GitHub under the URL `github.com/quansight/projectname`. - There are several **optional** (yet highly recommended) flags that allow to configure the deployment: - `aws` indicates that the project will be deployed on the Amazon AWS Cloud provider. + Optional flags are: `gcp`, `do` and `azure`. - `--project`: the name of the project is required to be a string compliant with the Cloud provider recommendations. For - more details see official Cloud provider docs on naming policies and check our docs on [naming convention](#project-naming-convention). + more details see official Cloud provider docs on naming policies and see below on the [project naming convention](#project-naming-convention). - `--domain`: base domain for your cluster. This pattern is also applicable if you are setting your own DNS through a different provider. - + `jupyter.qhub.dev` is the domain registered on CloudFlare. In case you chose not to use Cloudflare, skip this flag. + + `qhub.dev` is the domain registered on CloudFlare. If you chose not to use Cloudflare, skip this flag. - `--ci-provider`: specifies what provider to use for CI/CD. Currently, supports GitHub Actions, GitLab CI, or none. - `--auth-provider`: This will set configuration file to use the specified provider for authentication. - `--auth-auto-provision`: This will automatically create and configure an application using OAuth. - `--repository`: Repository name that will be used to store the Infrastructure-as-Code on GitHub. - `--repository-auto-provision`: Sets the secrets for the GitHub repository used for CI/CD actions. +The command above will generate the `qhub-config.yaml` config file +with an infrastructure deployed on `aws`, named `projectname`, where +the domain will be `qhub.dev`. The deployment +will use `github-actions` as the continuous integration (CI) provider, +automatically provisioned and authenticated by `auth0`, initialized on +GitHub under the URL `github.com/quansight/projectname`. + +If employing an infrastructure-as-code approach, this is where you would make the desired infrastructure changes +including adding users, changing Dask worker instance type and much more. Once you're happy with your changes you would redeploy those changes using using GitHub Actions. For more details on on the `qhub-config.yaml` please see [Configuration](configuration.md) + ##### Project Naming Convention In order to successfully deploy QHub, there are some project naming conventions which need to be followed. For starters, -make sure your name is compatible with the specific one for your chosen Cloud provider. In addition, QHub `projectname` +make sure your project name is compatible with the specifics of your chosen Cloud provider. In addition, QHub `projectname` should also obey to the following format requirements: + letters from A to Z (upper and lower case) and numbers; + Special characters are **NOT** allowed; @@ -63,7 +66,7 @@ should also obey to the following format requirements: ### Deploy QHub -Finally, we can deploy QHub with: +Finally, with the `qhub-config.yaml` created, we can deploy QHub for the first time: ```shell qhub deploy -c qhub-config.yaml --dns-provider cloudflare --dns-auto-provision @@ -82,7 +85,7 @@ This will create the following folder structure: └── terraform-state # required by terraform to securely store the state of the deployment ``` -The terminal will prompt to press `[enter]` to check auth credentials +The terminal will prompt you to press `[enter]` to check auth credentials (which were added by the `qhub init` command). That will trigger the deployment which will take around 10 minutes to complete. @@ -127,12 +130,12 @@ git push origin main ``` Once the files are in GitHub, all CI/CD changes will be triggered by -commits to main, and deployed via GitHub actions. Since the +commits to main, and deployed via GitHub Actions. Since the infrastructure state is reflected in the repository, this workflow allows for team members to submit pull requests that can be reviewed before modifying the infrastructure, easing the maintenance process. -To automatically deploy: +To automatically deploy (and to keep track of changes more effectively): - make changes to the `qhub-config.yaml` file on a new branch. - create a pull request (PR) to main. - Trigger the deployment by merging the PR. All changes will be From b78af7c498670030a93e40a3f793c678ee33c247 Mon Sep 17 00:00:00 2001 From: iameskild Date: Wed, 13 Oct 2021 07:46:15 -0700 Subject: [PATCH 2/7] Update /user_guide docs --- docs/source/user_guide/code_server.md | 14 +++---- docs/source/user_guide/getting_started.md | 47 +++++++---------------- 2 files changed, 20 insertions(+), 41 deletions(-) diff --git a/docs/source/user_guide/code_server.md b/docs/source/user_guide/code_server.md index 4f8655e0bc..324ef59d45 100644 --- a/docs/source/user_guide/code_server.md +++ b/docs/source/user_guide/code_server.md @@ -1,16 +1,14 @@ # In Browser VSCode -Code Sever is a packaging of VS Code in the browser. Within QHub we -have packaged Code Server such that ever user's jupyterlab has a full +Code Server is a packaging of VS Code in the browser. Within QHub we +have packaged Code Server such that every user's JupyterLab has a full featured code editor. This editor will have access to all the same -files that your regular jupyterlab session has access to. To launch -`Code Server` click on the `VS Code IDE` icon within the jupyterlab -launcher seen below. +files that your regular JupyterLab session has access to. To launch +`Code Server` click on the `VS Code IDE` icon from the Launcher screen, see below. ![QHub Kernel Selection](../images/qhub_kernel_selection.png) -A new tab will be opened with VS Code and access to all your files. VS -Code state will be saved between sessions so feel free to add -extensions etc. that will enhance your experience. +A new VS Code tab will be opened, and from here you can still access +all the files. The VS Code state will be saved between sessions so feel free to add extensions, plugins, etc. to enhance your user-experience. ![VSCode in browser](../images/qhub_vscode.png) diff --git a/docs/source/user_guide/getting_started.md b/docs/source/user_guide/getting_started.md index 75ec495cdf..c1442225a0 100644 --- a/docs/source/user_guide/getting_started.md +++ b/docs/source/user_guide/getting_started.md @@ -1,51 +1,32 @@ # Log into QHub -This guide aims to explain to newcomers how to log in to QHub. Your -own organization's QHub is likely to have a slighly different -procedure for the authentication due to the many authentication -providers that QHub can integrate into. +This guide aims to give a basic overview of the QHub login process. Your +organization's QHub will likely have a slighly different authentication +process due to the many authentication providers that QHub can integrate with. -Get the url of your QHub cluster. For this example we will use +Get the URL of your QHub cluster. For this example we will use `https://training.qhub.dev`. -![Qhub login screen](../images/qhub_login_screen.png) +![QHub login screen](../images/qhub_login_screen.png) -Upon visiting the website you will be prompted by a login page similar -to the login page shown above. The login process will greatly differ -between authentication providers. Qhub supports LDAP, OAuth2, -Passwordless Auth, password based authentication and many others (any -that jupyterhub supports) meaning that it is hard to detail the exact -login process. +Once on the site, you will be prompted by a login, similar to the login page shown above. The login process will differ greatly between authentication providers. QHub supports LDAP, OAuth2, passwordless authentication, password-based authentication and many others. Any authentication process +that JupyterHub supports, QHub also supports. This makes it a little challenging to detail the exact login process. ![QHub select profile](../images/qhub_select_profile.png) -Once authenticated the user will be prompted with a set of profiles +Once authenticated, the user will be prompted with a set of profiles that are available for the authenticated user to use. Your given selections will likely differ from the image shown. The customized -profiles will give you access to fixed resources e.g. 2 cpu, 8 GB ram, -and 1 dedicated gpu. All of this is configured by the -administrator. Click `start` once the appropriate profile has been -chosen. The cluster may take several minutes to launch. This is due to -Qhub using autoscaling behind the scenes to reduce costs when the -cluster is idle. +profiles will give you access to fixed cloud resources e.g. 2 CPU, 8 GB RAM, +and 1 dedicated GPU, all of which is configured by your administrator. +Once an appropriate profile has been select, click `start`. At this point, your JupyterHub will be launched, a step which may take up to several minutes due to QHub use of autoscaling under the hood. Ultimately this autoscaling feature helps reduce costs when the cluster is idle. ![QHub kernel selection](../images/qhub_kernel_selection.png) -Finally once your cluster has launched you will notice a huge -selection of available python environments. These environments are +Finally once the your JupyterHub instance has been launched you will notice a selection of available python environments. These environments are configured by your administrator. -![Qhub notebook](../images/qhub_notebook.png) +![QHub notebook](../images/qhub_notebook.png) -When you click on a given kernel a jupyterlab notebook with that given -environment will launch. Note that kernels can take several seconds to -become responsive. The circle in the top right hand corner is a good -indicator at the status of the kernel. A lightning bold means that the +From the Launcher, with a double-click, you can launch a JupyterLab notebook with that given conda environment. Note that kernels can take several seconds to become responsive. The circle in the top right hand corner is a good indicator of the status of the kernel. A lightning bold means that the kernel has started but it not yet ready to run code. - - - - - - - From 45394a2556a20d7830931a0eaae97fcbb0c65aa5 Mon Sep 17 00:00:00 2001 From: iameskild Date: Fri, 15 Oct 2021 06:59:59 -0700 Subject: [PATCH 3/7] Updatest to /user_guide --- docs/source/user_guide/dashboard.md | 5 ++- docs/source/user_guide/ssh.md | 49 ++++++++++++++--------------- 2 files changed, 26 insertions(+), 28 deletions(-) diff --git a/docs/source/user_guide/dashboard.md b/docs/source/user_guide/dashboard.md index b52b3c9304..050a0ba07b 100644 --- a/docs/source/user_guide/dashboard.md +++ b/docs/source/user_guide/dashboard.md @@ -1,11 +1,10 @@ # Dashboards -QHub supports user's creating dashboards that can be shared with other +QHub encourages users to create dashboards that can be shared with other users and groups via [ContainDS Dashboards](https://cdsdashboards.readthedocs.io/en/stable/). Currently this dashboarding solution supports Panel, Bokeh, Voila, Streamlit, -and Plotly. The solution is general enough to support any webapp. For -a more detailed guide on using CDSDashboards see the +and Plotly. The solution is general purpose enough to support any web app. For a more detailed guide on using CDSDashboards, see the [documentation](https://cdsdashboards.readthedocs.io/en/stable/index.html). ![qhub dashboard notebook](../images/qhub_dashboard_notebook.png) diff --git a/docs/source/user_guide/ssh.md b/docs/source/user_guide/ssh.md index ad8af1017b..4098996337 100644 --- a/docs/source/user_guide/ssh.md +++ b/docs/source/user_guide/ssh.md @@ -1,47 +1,46 @@ # SSH/SFTP Access -Using [jupyterhub-ssh](https://github.com/yuvipanda/jupyterhub-ssh) +Using [`jupyterhub-ssh`](https://github.com/yuvipanda/jupyterhub-ssh), QHub has the ability for remotely accessing the cluster and a juypterlab environment via -[ssh](https://en.wikipedia.org/wiki/Secure_Shell). In addition user's +[`ssh`](https://en.wikipedia.org/wiki/Secure_Shell). In addition, users can easily transfter files back and forth via -[sftp](https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol). QHub -provides a secure manner for users to login and provides additional -ways to connect to the cluster. This enables using terminal based -editors e.g. emacs/vim along with the ability to automate tasks on the -cluster without requiring browser access. For more detailed docs on -using jupyterhub-ssh please refer to the +[`sftp`](https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol). QHub +provides a secure manner for users to login while also providing additional +ways to connect to the cluster. This allows users to use terminal based +editors (such as emacs or vim) to automate tasks on the +cluster without requiring browser access. For more detailed information on +using `jupyterhub-ssh`, please refer to the [documentation](https://jupyterhub-ssh.readthedocs.io/en/latest/index.html). -In order to login via ssh a user needs to generate an api token. Visit -`https:///hub/token`. Where qhub-url is the domain name of -your QHub cluster. You will be shown a screen similar to the once -shown bellow. This step is only required once to generate an api -token. Afterwards you can reuse the same token. +In order to login via `ssh` a user needs to generate an API token. Visit +`https:///hub/token`. Where `` is the domain name of +your QHub cluster. You will be shown a screen similar to the one +shown below. You need only generate the API token once and can be reused going forward; to revoke this API token, simply return to this page and click `revoke`. ![qhub api token](../images/qhub_api_token.png) -Click on `Request new API token` with a note describing the use of the -token e.g. `ssh login token`. Copy down the generate api token (in -this case ` f0b80688484a4ac79a21b38ec277ca08 `). +Click on `Request new API token` with a short description such as `ssh login token`. Copy down the generate api token (in +this case `f0b80688484a4ac79a21b38ec277ca08`). ![qhub api token generated](../images/qhub_api_token_generated.png) -Now you can login to the QHub cluster via a terminal and ssh. Note -from the api generation screen in the top right hand corner you can -see the username that you should use for login. Ssh handles usernames -with an `@` symbol weirdly thus you will need to provide the username -explicitly in ssh. Notice the `8022` port used for ssh. +You can now login to the QHub cluster via the terminal using `ssh`. Note +from the API generation screen in the top right hand corner, you can +see the username that you should use for login. `ssh` handles usernames +with an `@` symbol meaning that you will need to provide the username +explicitly when trying to connect via `ssh`. See the example below for details. +> NOTE: +> - `ssh` uses port `8022` +> - `sftp` uses port `8023` ``` -$ ssh -o User=costrouchov@quansight.com training.qhub.dev -p 8022 +$ ssh -o User=costrouchov@quansight.com training.qhub.dev -p 8022 The authenticity of host '[training.qhub.dev]:8022 ([35.223.107.201]:8022)' can't be established. RSA key fingerprint is SHA256:mKy546LpI0cbqm/IY8dQR0B5QcbEziWLjLglern5G+U. This key is not known by any other names Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '[training.qhub.dev]:8022' (RSA) to the list of known hosts. (costrouchov@quansight.com@training.qhub.dev) Password: -costrouchov@quansight.com@jupyter-costrouchov-40quansight-2ecom:~$ +costrouchov@quansight.com@jupyter-costrouchov-40quansight-2ecom:~$ ``` - -The `sftp` port is available on `8023`. From d9989a69d2ac8f69ce34ceab0219add2426bcee3 Mon Sep 17 00:00:00 2001 From: iameskild Date: Fri, 15 Oct 2021 07:02:05 -0700 Subject: [PATCH 4/7] Undo change --- docs/source/installation/setup.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/docs/source/installation/setup.md b/docs/source/installation/setup.md index bb27a91f3d..b86e452405 100644 --- a/docs/source/installation/setup.md +++ b/docs/source/installation/setup.md @@ -51,7 +51,7 @@ registration](#domain-registry), and CI provider (GitHub Actions, GitLab CI). These services require global [environment variables](https://linuxize.com/post/how-to-set-and-list-environment-variables-in-linux/) -that once set up, will trigger QHub's automatic deploy using your +that once set up, will trigger QHub's automatic deploy using your CI/CD platform of choice. To find and set the environment variables, follow the steps described @@ -68,9 +68,9 @@ knowledge about the specific provider required either. QHub supports [Azure](#microsoft-azure). To deploy QHub, all access keys require fairly wide permissions to -create all the necessary cloud resources. Hence, once the Cloud -provider has been chosen, follow the steps below and set the -environment variables as specified with **owner/admin** level +create all the necessary cloud resources. Hence, once the Cloud +provider has been chosen, follow the steps below and set the +environment variables as specified with **owner/admin** level permissions. For more details on configuration for each Cloud provider, check the @@ -107,6 +107,8 @@ Set the required environment variables as specified below: export DIGITALOCEAN_TOKEN="" # API token required to generate resources export SPACES_ACCESS_KEY_ID="" # public access key for access spaces export SPACES_SECRET_ACCESS_KEY="" # the private key for access spaces +export AWS_ACCESS_KEY_ID="" # set this variable with the same value as `SPACES_ACCESS_KEY_ID` +export AWS_SECRET_ACCESS_KEY="" # set this variable identical to `SPACES_SECRET_ACCESS_KEY` ``` @@ -201,13 +203,13 @@ domain will be where your application will be exposed. Currently, QHub only supports CloudFlare for **automatic** DNS registration. If an alternate DNS provider is desired, change the `--dns-provider` flag from `cloudflare` to `none` on the `qhub deploy` -command. During the initial deployment, the deployment script will -pause as it asks for an IP address (or CNAME, if using AWS) and will -prompt you to register the desired URL. How to set a DNS record heavily -depends on the provider thus it is not possible to have detailed docs -on how to create a record on your provider. +command. During the initial deployment, the deployment script will +pause as it asks for an IP address (or CNAME, if using AWS) and will +prompt you to register the desired URL. How to set a DNS record heavily +depends on the provider thus it is not possible to have detailed docs +on how to create a record on your provider. -Googling `setting record on ` should yield +Googling `setting record on ` should yield good results on doing it for your specific provider. #### Cloudflare From 7ac860a2892273c8a262bc75f434e2923f83af5d Mon Sep 17 00:00:00 2001 From: iameskild Date: Thu, 13 Jan 2022 11:48:04 -0800 Subject: [PATCH 5/7] Update upgrade docs --- docs/source/admin_guide/upgrade.md | 57 +++++++++++++++++++++++------- docs/source/installation/usage.md | 6 ---- 2 files changed, 44 insertions(+), 19 deletions(-) diff --git a/docs/source/admin_guide/upgrade.md b/docs/source/admin_guide/upgrade.md index 2ca2c2e73f..fd0657807c 100644 --- a/docs/source/admin_guide/upgrade.md +++ b/docs/source/admin_guide/upgrade.md @@ -4,9 +4,9 @@ Here we suppose a user would like to upgrade to a version ``, probably You may be deploying QHub based on a local configuration file, or you may be using CI/CD workflows in GitHub or GitLab. Either way, you will need to locate a copy of your `qhub-config.yaml` configuration file to upgrade it (and commit back to your git repo in the CI/CD case). -For CI/CD deployments, you will need to `git clone ` into a folder on your local machine. +For CI/CD deployments, you will need to `git clone ` into a folder on your local machine if you haven't done so already. -## Upgrade the qhub command package +## Step 1: Upgrade the `qhub` command package To install (or upgrade) your pip installation of the Python package used to manage QHub: @@ -14,31 +14,37 @@ To install (or upgrade) your pip installation of the Python package used to mana pip install --upgrade qhub ``` -The above will install the latest full version of qhub. For a specific older version use: +The above will install the latest full version of QHub. For a specific older version use: ```shell pip install --upgrade qhub== ``` -## Upgrade qhub-config.yaml file +## Step 2: Upgrade `qhub-config.yaml` file -In the folder containing your qhub configuration file, run: +In the folder containing your QHub configuration file, run: ```shell qhub upgrade -c qhub-config.yaml ``` -This will output a newer version of qhub-config.yaml that's compatible with the new version of qhub. The process will list any changes it has made. It will also tell you where it has stored a backup of the original file. +This will output a newer version of `qhub-config.yaml` that's compatible with the new version of `qhub`. The process outputs a list of changes it has made. The `upgrade` command creates a copy of the original unmodified config file (`qhub-config.yaml.old.backup`) as well as a JSON file (`qhub-users-import.json`) used to import existing users into Keycloak. -If you are deploying QHub from your local machine (not using CI/CD) then you will now have a qhub-config.yaml file that you can use to `qhub deploy -c qhub-config.yaml` through the latest version of the QHub command package. +## Step 3: Validate special customizations to `qhub-config.yaml` -## Special customizations +You may have made special customizations to your `qhub-config.yaml`, such as using your own versions of Docker images. Please check your `qhub-config.yaml` and decide if you need to update any values that would not have been changed automatically - or, for example, you may need to build new versions of your custom Docker images to match any changes in QHub's images. -You may have made special customizations to your qhub config, such as using your own versions of Docker images. Please check your qhub-config.yaml and decide if you need to update any values that would not have been changed automatically - or, for example, you may need to build new versions of your custom Docker images to match any changes in QHub's images. +## Step 4: Redeploy QHub -## Render and Commit to Git +If you are deploying QHub from your local machine (not using CI/CD) then you will now have a `qhub-config.yaml` file that you can deployed. -For CI/CD (GitHub/GitLab) workflows, then as well as generating the updated qhub-config.yaml files as above, you will also need to regenerate the workflow files based on the latest qhub version's templates. +```shell +qhub deploy -m qhub-config.yaml +``` + +### CI/CD: render and commit to git + +For CI/CD (GitHub/GitLab) workflows, then as well as generating the updated `qhub-config.yaml` files as above, you will also need to regenerate the workflow files based on the latest `qhub` version's templates. With the newly upgraded `qhub-config.yaml` file, run: @@ -46,9 +52,34 @@ With the newly upgraded `qhub-config.yaml` file, run: qhub render -c qhub-config.yaml ``` -(Note that `qhub deploy` would perform this render step too, but will also immediately redeploy your qhub.) +(Note that `qhub deploy` would perform this render step too, but will also immediately redeploy your QHub.) + +Commit all the files (`qhub-config.yaml` and GitHub/GitLab workflow files) back to the remote repo. All files need to be committed together in the same commit. + +## (Step 5: Update OAuth callback URL for Auth0) + +If your QHub deployment relies on Auth0 for authentication, please update the OAuth callback URL. + +
+ +1. Navigate to the your Auth0 tenacy homepage and from there select "Applications". + +2. Select the "Regular Web Application" with the name of your deployment. + +3. Under the "Application URIs" section, paste the new OAuth callback URL in the "Allowed Callback URLs" text block. +- The URL will take the shape: + ``` + https://{your-qhub-domain}/auth/realms/qhub/broker/auth0/endpoint + ``` + - Replace `{your-qhub-domain}` with the domain found in the `domain` section of your `qhub-config.yaml`. + +
+ +## Step 6: Import users into Keycloak + +The last two steps are to change the Keycloak `root` user password, documented [here](../installation/login.md#change-keycloak-root-password) and import existing users, documented [here](../admin_guide/backup.md#import-keycloak). -Commit all the files (qhub-config.yaml and GitHub/GitLab workflow files) back to the remote repo. All files need to be committed together in the same commit. +For more details on this process, visit the [Keycloak docs section](../installation/login.md). ## Known versions that require re-deployment diff --git a/docs/source/installation/usage.md b/docs/source/installation/usage.md index d4756cec7f..0eea24d24b 100644 --- a/docs/source/installation/usage.md +++ b/docs/source/installation/usage.md @@ -2,12 +2,6 @@ ## Cloud Deployment -Once all environment variables have been set (in [Configuration](configuration.md)), you will be able to run commands on your terminal to initialise the configuration for QHub, and then deploy it. - -### Initialize configuration - -QHub can help you create your configuration YAML file, and you can further edit it as needed. We advise you to start by creating a new project folder. Here, we will name the new folder `qhub-test`. -======= Great, you've gone through the `qhub` [Installation](installation.md) and [Setup Initialization](setup.md) steps, and have ensured that all the necessary environment variables have been properly set, it's time to deploy QHub from your terminal. From 122fd8db1499656629c0a8f313732fb60b94d63c Mon Sep 17 00:00:00 2001 From: iameskild Date: Thu, 13 Jan 2022 12:38:17 -0800 Subject: [PATCH 6/7] Add summary to details --- docs/source/admin_guide/upgrade.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/admin_guide/upgrade.md b/docs/source/admin_guide/upgrade.md index fd0657807c..1c2622874d 100644 --- a/docs/source/admin_guide/upgrade.md +++ b/docs/source/admin_guide/upgrade.md @@ -60,7 +60,7 @@ Commit all the files (`qhub-config.yaml` and GitHub/GitLab workflow files) back If your QHub deployment relies on Auth0 for authentication, please update the OAuth callback URL. -
+
Click for more detailed instructions 1. Navigate to the your Auth0 tenacy homepage and from there select "Applications". From ff161016d8422e7d9c833479d6a228f12f446c6b Mon Sep 17 00:00:00 2001 From: iameskild Date: Thu, 13 Jan 2022 13:39:29 -0800 Subject: [PATCH 7/7] Fix vale errors --- docs/source/installation/configuration.md | 6 +++--- docs/source/installation/setup.md | 4 ++-- docs/source/installation/usage.md | 6 +++--- docs/source/user_guide/getting_started.md | 9 ++++----- 4 files changed, 12 insertions(+), 13 deletions(-) diff --git a/docs/source/installation/configuration.md b/docs/source/installation/configuration.md index a402540425..4fae095222 100644 --- a/docs/source/installation/configuration.md +++ b/docs/source/installation/configuration.md @@ -225,7 +225,7 @@ security: ### Keycloak -The `security.keycloak` section allows you to specify an initial password for the `root` user (to login at `https://myqhubsite.com/auth/admin/`) to manage your Keycloak database, e.g. add users/groups. +The `security.keycloak` section allows you to specify an initial password for the `root` user (to login at `https://myqhubsite.com/auth/admin/`) to manage your Keycloak database, for example add users/groups. You should change this after deployment. Future deployments will not reset the password to any specified in the YAML file. @@ -624,7 +624,7 @@ important considerations to make. Two important terms to understand are: For example if a node has 8 GB of ram and 2 CPUs you should guarantee/schedule roughly 75% and follow the digital ocean guide -linked above. E.g. 1.5 CPU guarantee and 5.5 GB guaranteed. +linked above. For example 1.5 CPU guarantee and 5.5 GB guaranteed. #### Dask Scheduler @@ -649,7 +649,7 @@ dask_worker: ### Limiting profiles to specific users and groups Sometimes only a select set of users should have access to specific -resources e.g. GPUs, high memory nodes etc. QHub has support for +resources for example GPUs, high memory nodes etc. QHub has support for limiting resources. ```yaml diff --git a/docs/source/installation/setup.md b/docs/source/installation/setup.md index a59655900e..facbd033f5 100644 --- a/docs/source/installation/setup.md +++ b/docs/source/installation/setup.md @@ -151,7 +151,7 @@ export ARM_TENANT_ID="" # field available under `Azure Active Director ## Authentication -User identity in QHub is now managed within Keycloak which is a robust and highly flexible open source identity and access management solution. A Keycloak instance will be deployed inside your QHub. It can be configured to work with many OAuth2 identity providers, it can federate users from existing databases (such as LDAP), or it can be used as a simple database of username/passwords. +User identity in QHub is now managed within Keycloak which is a robust and highly flexible open source identity and access management solution. A Keycloak instance will be deployed inside your QHub. It can be configured to work with many OAuth 2.0 identity providers, it can federate users from existing databases (such as LDAP), or it can be used as a simple database of username/passwords. The full extent of possible configuration can't be covered here, so we provide three simple options that can be configured automatically by QHub when it sets up your new platform. These options are basic passwords, GitHub single-sign on, or Auth0 single-sign on (which in turn can be configured to allow identity to be provided by social login etc). @@ -184,7 +184,7 @@ export AUTH_DOMAIN="qhub-test.auth0.com" # in case the Tenant Name was called 'q
Click for GitHub SSO configuration details -To use GitHub as a single-sign on provider, you will need to create a new OAuth2 app. +To use GitHub as a single-sign on provider, you will need to create a new OAuth 2.0 app. No environment variables are needed for this - you will be given the relevant information and prompted for various inputs during the next stage, when you run [`qhub init`](./usage.md) if you provide the flag `--auth-provider github`. This will be covered when you reach that point in this documentation.
diff --git a/docs/source/installation/usage.md b/docs/source/installation/usage.md index 8b8dd54bc9..c3b54589b3 100644 --- a/docs/source/installation/usage.md +++ b/docs/source/installation/usage.md @@ -39,7 +39,7 @@ The command above will generate the `qhub-config.yaml` config file with an infra The deployment will use `github-actions` as the continuous integration (CI) provider, automatically provisioning a repository on GitHub under the URL `github.com/quansight/projectname` -User authentication will be by `auth0`, and an OAuth2 app will be created on Auth0 automatically. There are several flags that allow you to configure the deployment: +User authentication will be by `auth0`, and an OAuth 2.0 app will be created on Auth0 automatically. There are several flags that allow you to configure the deployment: - `aws` indicates that the project will be deployed on the Amazon AWS Cloud provider. + Optional flags are: `gcp`, `do` and `azure`. @@ -74,7 +74,7 @@ GitHub under the URL `github.com/quansight/projectname`. If employing an infrastructure-as-code approach, this is where you would make the desired infrastructure changes including adding users, changing Dask worker instance type and much more. Once you're happy with your changes you would redeploy those changes using GitHub Actions. For more details on the `qhub-config.yaml` please see [Configuration](configuration.md) -The command above will generate the `qhub-config.yaml` config file +The proceeding command will generate the `qhub-config.yaml` config file with an infrastructure deployed on `aws`, named `projectname`, where the domain will be `qhub.dev`. The deployment will use `github-actions` as the continuous integration (CI) provider, @@ -82,7 +82,7 @@ automatically provisioned and authenticated by `auth0`, initialized on GitHub under the URL `github.com/quansight/projectname`. If employing an infrastructure-as-code approach, this is where you would make the desired infrastructure changes -including adding environments, changing Dask worker instance type and much more. Once you're happy with your changes you would redeploy those changes using using GitHub Actions. For more details on on the `qhub-config.yaml` please see [Configuration](configuration.md) +including adding environments, changing Dask worker instance type and much more. Once you're happy with your changes you would redeploy those changes using GitHub Actions. For more details on the `qhub-config.yaml` please see [Configuration](configuration.md) ##### Project Naming Convention In order to successfully deploy QHub, there are some project naming conventions which need to be followed. For starters, diff --git a/docs/source/user_guide/getting_started.md b/docs/source/user_guide/getting_started.md index c1442225a0..f16c9aebe7 100644 --- a/docs/source/user_guide/getting_started.md +++ b/docs/source/user_guide/getting_started.md @@ -1,15 +1,14 @@ # Log into QHub This guide aims to give a basic overview of the QHub login process. Your -organization's QHub will likely have a slighly different authentication +organization's QHub will likely have a slightly different authentication process due to the many authentication providers that QHub can integrate with. -Get the URL of your QHub cluster. For this example we will use -`https://training.qhub.dev`. +Get the URL of your QHub cluster, for this `https://training.qhub.dev`. ![QHub login screen](../images/qhub_login_screen.png) -Once on the site, you will be prompted by a login, similar to the login page shown above. The login process will differ greatly between authentication providers. QHub supports LDAP, OAuth2, passwordless authentication, password-based authentication and many others. Any authentication process +Once on the site, you will be prompted by a login, similar to the login page shown in the proceeding image. The login process will differ greatly between authentication providers. QHub supports LDAP, OAuth 2.0, passwordless authentication, password-based authentication and many others. Any authentication process that JupyterHub supports, QHub also supports. This makes it a little challenging to detail the exact login process. ![QHub select profile](../images/qhub_select_profile.png) @@ -17,7 +16,7 @@ that JupyterHub supports, QHub also supports. This makes it a little challenging Once authenticated, the user will be prompted with a set of profiles that are available for the authenticated user to use. Your given selections will likely differ from the image shown. The customized -profiles will give you access to fixed cloud resources e.g. 2 CPU, 8 GB RAM, +profiles will give you access to fixed cloud resources for example 2 CPU, 8 GB RAM, and 1 dedicated GPU, all of which is configured by your administrator. Once an appropriate profile has been select, click `start`. At this point, your JupyterHub will be launched, a step which may take up to several minutes due to QHub use of autoscaling under the hood. Ultimately this autoscaling feature helps reduce costs when the cluster is idle.