Migrate defaultIndex attribute for config saved object

[jportner]

Resolves #133241

Overview

This PR adds a new mechanism to apply custom transform functions to the config saved object for UI settings. These transform functions are complementary to saved object migrations.

The config saved objects are versioned and isolated to each space.
When a client attempts to fetch or create a config saved object, Kibana first checks if there's already an old one that needs to be upgraded. This PR takes advantage of the existing functionality to simply call any necessary transform functions during that process. This way, we can ensure that transforms will be applied to each config after upgrading, but we don't need to add a complicated "on start" loop to search and update saved objects.

Testing

To manually test this end-to-end to verify the migration is applied:

1. Load the test data and scripts that are attached to Add enhancements for legacy URL aliases #125960
   • This will create a bunch of test data, but we are primarily interested in the index pattern ff959d40-b880-11e8-a6d9-e546fe2bba5f in the Default space, along with a legacy URL alias redirect-with-toast that points to the aforementioned ID
2. Log in with superduperuser/changeme which was created by the script and has access to modify system indices
3. Using Dev Tools, create an old config object that will be upgraded, which uses the legacy URL alias ID:

   POST .kibana/_doc/config:8.2.0
   {
     "config": {
       "buildNum": 123,
       "defaultIndex": "redirect-with-toast"
     },
     "type": "config",
     "references": [],
     "migrationVersion": {
       "config": "8.1.0"
     },
     "coreMigrationVersion": "8.2.0",
     "updated_at": "2022-06-07T13:21:34.759Z"
   }
   

4. Using Dev Tools, delete the existing config object (for 8.4.0):

   DELETE .kibana/_doc/config:8.4.0
   

5. Refresh the page to fetch the config object again, triggering the upgrade
6. Using Dev Tools, search for the config objects and verify that it was upgraded correctly:

   GET .kibana/_search
   {
     "query": {
       "term": {
         "type": {
           "value": "config"
         }
       }
     }
   }
   

   The result should look like this:

   "hits": [
     {
       "_index": ".kibana_8.4.0_001",
       "_id": "config:8.2.0",
       "_score": 3.6888795,
       "_source": {
         "config": {
           "buildNum": 123,
           "defaultIndex": "redirect-with-toast"
         },
         "type": "config",
         "references": [],
         "migrationVersion": {
           "config": "8.1.0"
         },
         "coreMigrationVersion": "8.4.0",
         "updated_at": "2022-06-07T13:21:34.759Z"
       }
     },
     {
       "_index": ".kibana_8.4.0_001",
       "_id": "config:8.4.0",
       "_score": 3.6888795,
       "_source": {
         "config": {
           "buildNum": 9007199254740991,
           "isDefaultIndexMigrated": true,
           "defaultIndex": "ff959d40-b880-11e8-a6d9-e546fe2bba5f"
         },
         "type": "config",
         "references": [],
         "migrationVersion": {
           "config": "8.1.0"
         },
         "coreMigrationVersion": "8.4.0",
         "updated_at": "2022-06-07T13:32:22.612Z"
       }
     }
   ]

---

Release note: The advanced setting for defaultIndex could be broken (not pointing to the correct index pattern) if it was set in a custom space before upgrading to the 8.0 release. The setting is now migrated correctly the first time that space is accessed by any user that has All access to the Advanced Settings feature.

[TinaHeiligers]

I love all the code comments but it begs the question of "Why do we need to explain what we're doing?". The whole config/settings situation needs an overhaul to avoid similar situations. For now, though, could we also add this information to the dev docs, please?

[jportner]

Great idea.
Looking at the docs I found a page for Advanced Settings, which mentions each setting and what it is for.
https://www.elastic.co/guide/en/kibana/8.3/advanced-options.html#defaultindex

Do you think I should add a "NOTE" admonition there? Or did you have something else in mind?

[TinaHeiligers]

I don't think we need to concern end-users with the internals of what we're doing here, it's more for internal developer guidance.
I was thinking about adding a note to the tutorial on how to register a uiSetting migration. The file itself lives here

[jportner]

Updated in c84b7c3, is this what you had in mind?

https://github.com/elastic/kibana/blob/c84b7c3c112286897a62106438d16bf8190d063e/dev_docs/tutorials/advanced_settings.mdx#creating-transforms

[lukeelmers]

Haven't gotten a chance to test locally yet, but the approach here makes sense overall

[pgayvallet]

NIT: should we add isDefaultIndexMigrated here too?

[jportner]

I don't think so, this was to address Luke's feedback in #133339 (comment), we have a separate UpgradeableConfigAttributes type that extends ConfigAttributes and specifies other fields such as isDefaultIndexMigrated.

[pgayvallet]

LGTM, thanks for doing this

[lukeelmers]

LGTM, thanks for all of the edits!

[TinaHeiligers]

LGTM too!

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: dc9b2a8
• Storybooks Preview
• Webpack Bundle Analyzer

Metrics [docs]

✅ unchanged

History

• 💚 Build #49453 succeeded a33ef05
• 💚 Build #49143 succeeded c84b7c3
• 💚 Build #48982 succeeded 8a58d18
• 💔 Build #48945 failed 110e043
• 💔 Build #48741 failed 6b6c833

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.3	Backport failed because of merge conflicts
❌	8.2	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 133339

Questions ?

Please refer to the Backport tool documentation

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[elasticmachine]

Pinging @elastic/kibana-security (Team:Security)