Keycloak docs (nebari-dev#901)

* keycloak helm overrides, fix yamlify

* no default keycloak theme on clients [skip ci]

* simplify qhub init and deploy docs

* keycloak login docs

* config examples

* gitlab docs

docs/source/installation/configuration.md

@@ -1,4 +1,4 @@
-# Configuration
+# Advanced Configuration
 
 The configuration file is split into several sections. In this page,
 we detail the requirements necessary for the YAML configuration
@@ -83,8 +83,8 @@ certificate:
   type: self-signed
 ```
 
-To use Let's Encrypt you must specify an email address that let's
-encrypt will associate the generated certificate with and whether to
+To use Let's Encrypt you must specify an email address that Let's
+Encrypt will associate the generated certificate with and whether to
 use the [staging
 server](https://acme-staging-v02.api.letsencrypt.org/directory) or
 [production
@@ -100,6 +100,8 @@ certificate:
   acme_server: https://acme-v02.api.letsencrypt.org/directory
 ```
 
+Note the above snippet will already be present if you provided an `--ssl-cert-email` when you ran `qhub init`.
+
 You may also supply a custom self-signed certificate and secret
 key. Note that the kubernetes default namespace that QHub uses is
 `dev` if not specified. Otherwise, it will be your `namespace` defined
@@ -141,22 +143,6 @@ security:
       client_id: <CLIENT_ID>
       client_secret: <CLIENT_SECRET>
       oauth_callback_url: https://do.qhub.dev/hub/oauth_callback
-  users:
-    example-user:
-      uid: 1000
-      primary_group: users
-      secondary_groups:
-        - billing
-    dharhas:
-      uid: 1001
-      primary_group: admin
-  groups:
-    users:
-      gid: 100
-    admin:
-      gid: 101
-    billing:
-      gid: 102
 ```
 
 ### Omitting sensitive values
@@ -228,85 +214,71 @@ security:
 
 #### Password Based Authentication
 
-For Password based authentication. Note that users will require a
-`password` field that can be generated via the following command:
-`python -c "import bcrypt; print(bcrypt.hashpw(b'<password>',
-bcrypt.gensalt()).decode('utf-8'))"`. Make sure to replace
-`<password>` with whatever password you are wanting.
+For Password based authentication. Ultimately, this just defers to however Keycloak is configured. That's also true for GitHub/Auth0 cases, except that for the single-sign on providers the deployment will also configure those providers in Keycloak to save manual configuration. But ultimately, it is also possible to add GitHub, or Google etc, as an Identity Provider in Keycloak even if you formally select 'password' authentication in the `qhub-config.yaml` file.
 
 ```yaml
 security:
   authentication:
     type: password
-  users:
-    ...
-    <username>:
-      ...
-      password: $2b$....
 ```
 
-#### Custom Authentication
+### Keycloak
 
-You can specify arbitrary authentication via the `custom` type. All
-`config` attributes will be set as traitlets to the configured
-authentication class. The attributes will obey the type set via yaml
-(e.g. True -> will be a boolean True for Traitlets).
+The security.keycloak section allows you to specify an initial password for the `root` user (to login at https://myqhubsite.com/auth/admin/) to manage your Keycloak database, e.g. add users/groups.
 
-```yaml
+You should change this after deployment. Future deployments will not reset the password to any specified in the YAML file.
+
+It is also possible to provide overrides to the [Keycloak Helm deployment](https://github.com/codecentric/helm-charts/tree/master/charts/keycloak).
+
+```
 security:
-  authentication:
-    type: custom
-    authentication_class: "oauthenticator.google.GoogleOAuthenticator"
-    config:
-      login_service: "My Login Button"
-      oauth_callback_url: 'http[s]://[your-host]/hub/oauth_callback'
-      client_id: 'your-client-id'
-      client_secret: 'your-client-secret'
+  keycloak:
+    initial_root_password: initpasswd
+    overrides:
+      image:
+        repository: quansight/qhub-keycloak
 ```
 
-### User Management
+### User and Group Management
+
+It is still possible to specify `users` and `groups` in the YAML file - in older versions of QHub, all users had to be specified in this way.
+
+If specifying users/groups in this way, you can also manually add more users/groups in Keycloak. However, be aware that if users/groups were initially created based on entries in the YAML file, those users/groups will be destroyed if `qhub deploy` is ever run without those users/groups in the file. They may also be recreated after you delete them manually in Keycloak.
 
-`users` and `groups` allows one to provision UNIX permissions to each
-user. Any user is assigned a `uid`, `primary_group`, and optionally
-any number of `secondary_groups`. Note that `uid` and `gid` fields
-must be unique and are required.
+Any user is assigned a `primary_group` and optionally
+any number of `secondary_groups`.
 
 ```yaml
 security:
   users:
     example-user:
-      uid: 1000
       primary_group: users
       secondary_groups:
         - billing
+      password: plaintextpasswd
     dharhas:
-      uid: 1001
       primary_group: admin
   groups:
     users:
-      gid: 100
     admin:
-      gid: 101
     billing:
-      gid: 102
 ```
 
 * The `primary_group` is the group name assigned to files that are
 written for the user.
-* `groups` are a mapping of group name to group IDs. It is
-  recommended to not change the IDs assigned to groups and users
-  after creation, since it may lead to login issues.
-  > NOTE: While the demo shows IDs between `100` and
-`1000`, it is recommended to start with high User ID numbers
-e.g. `10000000`. `ids` technically supports 2 billion `ids`.
+* `groups` are a mapping of group name to an empty map (no entries are required within that map any longer).
 
-### Admin Group
+#### Admin and Users Group
 
 The admin group has special significance. If a user's `primary_group`
 is admin they will be able to access the jupyterhub admin page. The
 admin page allows a user to stop user's servers and launch a given
 user's server and impersonate them.
 
+All users must be a member of the `users` group.
+
+Both `admin` and `users` groups will be created even if not specified in the YAML file.
+
 ## Provider Infrastructure
 
 Finally, comes the Kubernetes infrastructure deployment. Although
@@ -622,6 +594,12 @@ profiles:
         - users
 ```
 
+Provided `groups:` is specified for a profile (even if no groups are listed), then that profile can also be made available to a group by adding the attribute `profiles` to the group in Keycloak. Add multiple profile (display) names to this attribute using ## as a delimiter:
+
+```
+"profiles": "Small Instance##Medium Instance"
+```
+
 ### JupyterLab Profile Node Selectors
 
 A common operation is to target jupyterlab profiles to specific node
@@ -732,6 +710,14 @@ each environment include `ipykernel`, `ipywidgets`, `qhub-dask==0.2.3`. Upon cha
 environment definition expect 1-10 minutes upon deployment of the
 configuration for the environment to appear.
 
+## qhub_version
+
+All qhub-config.yaml files must now contain a `qhub_version` field displaying the version of QHub which it is intended to be deployed with.
+
+QHub will refuse to deploy if it does not contain the same version as that of the `qhub` command.
+
+Typically, you can upgrade the qhub-config.yaml file itself using the [`qhub upgrade` command](../admin_guide/upgrade.md). This will update image numbers, plus updating qhub_version to match the installed version of `qhub`, as well as any other bespoke changes required.
+
 # Full Configuration Example
 
 Everything in the configuration is set besides [???]
@@ -749,39 +735,38 @@ certificate:
   type: self-signed
 
 security:
+  keycloak:
+    initial_root_password: initpasswd
+    overrides:
+      image:
+        repository: quansight/qhub-keycloak
+
   authentication:
     type: GitHub
     config:
       client_id: CLIENT_ID
       client_secret: CLIENT_SECRET
       oauth_callback_url: https://jupyter.do.qhub.dev/hub/oauth_callback
+
   users:
     example-user:
-      uid: 1000
       primary_group: users
       secondary_groups:
         - billing
     dharhas:
-      uid: 1001
       primary_group: admin
     tonyfast:
-      uid: 1002
       primary_group: admin
     prasunanand:
-      uid: 1003
       primary_group: admin
     aktech:
-      uid: 1004
       primary_group: users
       secondary_groups:
         - admin
   groups:
     users:
-      gid: 100
     admin:
-      gid: 101
     billing:
-      gid: 102
 
 digital_ocean:
   region: nyc3

docs/source/installation/index.md

@@ -6,6 +6,7 @@
 installation.md
 setup.md
 usage.md
+login.md
 configuration.md
 management.md
 ```

docs/source/installation/installation.md

@@ -4,7 +4,7 @@
 
 * QHub is supported on the macOS and Linux operating systems.
   > NOTE: **Currently, QHub cannot be installed on Windows**.
-* We recommend the adoption of virtual environments (`conda`, `pipenv` or `venv`) for successful usage. 
+* We recommend the adoption of virtual environments (`conda`, `pipenv` or `venv`) for successful usage.
 
 ## Install QHub CLI
 
@@ -15,12 +15,12 @@ QHub's installation can be performed by using:
 ```bash
 conda install -c conda-forge qhub
 ```
-  
+
  * or `pip` (instead):
 
 ```bash
 pip install qhub
-```  
+```
 
 Once finished, you can check QHub's version (and additional CLI args)
 by typing:
@@ -32,12 +32,12 @@ qhub --help
 If successful, the CLI output will be similar to the following:
 
 ```bash
-usage: qhub [-h] [-v] {deploy,destroy,render,init,validate,destroy} ...
+usage: qhub [-h] [-v] {deploy,render,init,validate,destroy,upgrade,keycloak} ...
 
 QHub command line
 
 positional arguments:
-  {deploy,destroy,render,init,validate,destroy}
+  {deploy,render,init,validate,destroy,upgrade,keycloak}
                         QHub - ||QHUB_VERSION||
 
 optional arguments:

docs/source/installation/login.md

@@ -0,0 +1,93 @@
+# Login
+
+Keycloak is the name of the open source user management software that is automatically deployed within QHub. It is used to store the database of all users in your QHub site, and can provide connectivity to other services such GitHub/Auth0 single-sign on.
+
+If you ran `qhub init` to create your `qhub-config.yaml` configuration file in the [Usage](usage.md) step, you will have been provided with a
+"random password for Keycloak root user".
+
+The password will also be visible in the `qhub-config.yaml` file under the security.keycloak.initial_root_password field.
+
+Note that when your QHub is first deployed, the password given at that location will be set for the root Keycloak user. If added or changed for subsequent deployments, that password value will have no effect.
+
+Please note the root Keycloak user is only able to login and manage the Keycloak identity management section of QHub. It is not a user of the wider QHub data science platform.
+
+## Adding a QHub user
+
+You will need to add a QHub user in order to login to your QHub platform. If you have chosen to use GitHub or Auth0 single-sign on, you must ensure the usernames you provide for new users in Keycloak match the usernames from GitHub/Auth0.
+
+### Add user from command line
+
+To make adding users easier for new QHub deployments, there is a QHub command that can help. It will only work if the initial_root_password for Keycloak has not yet been changed, so the value available in `qhub-config.yaml` is still correct.
+
+Run:
+```
+qhub keycloak -c qhub-config.yaml adduser bob pa55w0rd
+```
+
+This will create a user called bob with the initial password provided. Omit the password completely if you are using GitHub or Auth0.
+
+### Add user using Keycloak console
+
+To add a QHub user from the web console for Keycloak, visit:
+
+https://myqhubsite.com/auth/admin/
+
+(Switch 'myqhubsite.com' for the domain you provided for your QHub deployment.)
+
+![Root Login to Keycloak](/source/images/keycloak_master_login.png)
+
+Login using the username `root` and the password provided for the initial Keycloak root password.
+
+All QHub users will be part of the `qhub` realm (a realm is a distinct identity provider and set of users in Keycloak). Note that the root user alone is a member of the `master` realm.
+
+The qhub realm should be selected by default.
+
+Click 'Users' along the left-hand side of the page.
+
+Click the 'Add user' button and you will see the new user form:
+
+![Add User to Keycloak](/source/images/keycloak_adduser.png)
+
+Enter the name you would like for the user then click Save.
+
+Once the user has been created, you can set a password (not needed for GitHub/Auth0 login):
+
+![Set Password in Keycloak](/source/images/keycloak_user_password.png)
+
+It is best to unset the 'Temporary' on/off button so the user won't be forced to change the password on first login.
+
+## Login to QHub
+
+Your new user can now log into QHub proper (not Keycloak's admin console).
+
+Visit https://myqhubsite.com/ (or whatever domain you have chosen for your QHub).
+
+Click 'Sign in with Keycloak'.
+
+![Login to Keycloak](/source/images/keycloak_qhub_login.png)
+
+Enter the username and password you chose when you added a user to QHub above.
+
+If you chose GitHub or Auth0 login, click the 'GitHub' button to be taken to a GitHub login page and single-sign on from there.
+
+# Change Keycloak root password
+
+You should change your root password for Keycloak now that you've got things running.
+
+Back in https://myqhubsite.com/auth/admin/ you can click on the 'Root' dropdown in the top right of the screen, and select 'Manage account'.
+
+Under 'Account Security' click 'Signing In'.
+
+In the Password box, click the 'Update' button. This will guide you through entering your existing root password, and then entering a new password.
+
+From this point, the security.keycloak.initial_root_password field in `qhub-config.yaml` has no effect. If you redeploy QHub it will not reset the password back to the old one (or anything else that might be in the field in your YAML file). You can delete that line from your YAML file if you wish.
+
+# Groups
+
+Add Groups in the same Keycloak backend as you can add users - that is, login as `root` to https://myqhubsite.com/auth/admin/. Click Groups on the left-hand side.
+
+Groups named `users` and `admin` will have been created automatically by QHub. All users will be added to the `users` group automatically when you create them. You should never remove them from the `users` group as that group must contain all users.
+
+Members of `admin` group will have access to admin features within QHub.
+
+Neither `users` nor `admin` groups should ever be deleted.