RFC: Rework app configuration

[akphi]

💬 Request for Comments

The way it have configuration defined right now is well-structured, but it requires a lot of boilerplate code and leaky encapsulation.

Context and Motivation

We would like to achieve the ease of use of vscode's Vault-like settings, where there's a JSON file with key-value pairs. for example:

{
  "appName": "query",
  "env": "local",
  "sdlc.url": "http://localhost:7070/api",
  "graphManager.engine.url": "http://localhost:6060/api",
  "metadata.url": "http://localhost:9090/api",
  "documentation.url": "https://legend.finos.org",
  "extension:@finos/legend-studio": {},
  "core.__EXPERIMENTAL__something": true,
...
}

We should think about a way to do schema validation (could be very basic: number, string, enum, list, object--though we should think of how to do nested schema--as well as documentation), somewhat, we can model this validation schema like ajv and webpack's schema-utils and have a JSON schema for the config

We also need to have a way to scan/merge config as we may need to modularize this. We could also keep the current way we configure plugin/preset config. But since this is a schema, we need to allow extension to it and a mechanism to prevent duplication of keys between extension.

Lastly, we can then create a generic editor for each kind of field like vscode does, and we must have a way to read these config and reapply them to the app

Implementation Plan

• For each field, for the model, we should have a schema and a default value
• We must be able to pass the vault around to all places so everyone can read from the vault but dont make it static…, hence putting it in applicationstore is the best
• During the construction of the vault, we should read in user saved optioned from local storage annd merge in with default value
• Everything should be adjusted to read from the vault instead of subscribing to the vault. Certain vault value may require app update, need to think about this aspect abit when we define data model for each field (the biggest change now is probably the network clients)
• When people write/update some vault value, we should actually update the vault and check if the value does not match default we delete it in local storage
• We should have a flag to disable some settings to not let people edit them, maybe we can hide them. Disbaled means always use the default value, hidden means just dont show it in the settings page
• We can also have a flag to tell people which settings might require a restart
• note that the model in local storage only store the value with field ID
• We can introduce tag to group fields later, tags can have hierarchy that we define before hand, we can also have a field for data model of each for description
• We should distinguish between application config and settings. Appliacation config helps initialize some values of the app that settings can probably not set. Settings refer to a vault-like container. For example, application config contains config for SDLC server options to pick from, service registration endpoints, etc. these values cannot be updated; whereas settings contains the current value and can be changed
• Potentially application config can allow us to modify the schema of settings somewhat. For example, if we have a list of options for sdlc servers in studio, we need to modify the schema for sdlc seever to become an enum instead of a string validation
• This means the for application config, we need to have anew field called settingsOverrides that let us set some values for hidden or disabled values,admin can set these values, not the users. With this field, we can remove extensions.core in application config
• When we get value users defined in local config, we should check if it matches the schema if it doesn’t we need to use the default value
• Have a way to register more schema for the settings when we start the application, this must happen right after we install the plugins and presets as those will trigger configurations for those plugins first before we can call the method from each plugins to yield their configs. Each plugins willhave a method to yields their schema validator
• Main app will have a bucnh of method to yield its schemas validators
• When we start the app, these schema validators will read user local storage and decise the final shapes of the settings based on all things we mentioned: schema validator with their default values, user input from local storage, application config setting overrides
• Keep the schema in json format and only deserialize them as we start the application, keep an enum of fields ids
• We probably should have an schema validator for app config as well