Unify the plugin installation for standalone and context-scoped plugi…

…ns and use context as recommendation (#698)

- Unify the plugin installation for standalone and context-scoped plugins
- Use context as a recommendation for which plugin/version to install
- Updates the tanzu plugin install and tanzu plugin sync experience
- Updates the tanzu plugin list output
- Updates the tanzu plugin describe output

User facing changes:

- Once a plugin is installed and plugin-specific commands are available, that plugin is always shown as installed and all plugin-related commands are available irrespective of whether the plugin was installed manually by the user with the tanzu plugin install command (standalone plugin) or it was installed automatically when the user created a context (context-scoped plugin).
- Before this change, context-scoped plugins were getting removed once the corresponding context was deleted or unset.
- After this change, the plugin's availability is mostly not dependent on whether a context is active or not, and even after the context gets deleted or unset, corresponding plugins are still available to the user to use.

.gitignore

@@ -30,4 +30,4 @@ coverage.txt
 test/e2e/coexistence/testresults/
 test/e2e/testresults/
 
-go.work.sum
+go.work.sum

cmd/plugin/test/main.go

@@ -45,7 +45,7 @@ func main() {
 		pluginsCmd,
 	)
 
-	installedPlugins, err := pluginsupplier.GetInstalledStandalonePlugins()
+	installedPlugins, err := pluginsupplier.GetInstalledPlugins()
 	if err != nil {
 		log.Fatal(err, "")
 	}

docs/dev/README.md

@@ -82,13 +82,14 @@ Below is the list of these variables:
 | `TANZU_CLI_ADDITIONAL_PLUGIN_DISCOVERY_IMAGES_TEST_ONLY` | Specifies test plugin repositories to use as a supplement to the production Central Repository of plugins. Ignored if `TANZU_CLI_PRIVATE_PLUGIN_DISCOVERY_IMAGES` is set. | Comma-separated list of test plugin repository URIs| |
 | `TANZU_CLI_ESSENTIALS_PLUGIN_GROUP_NAME` | Override the default name (`vmware-tanzucli/essentials`) of the Essential Plugins group.  Should not be needed. | Group name |
 | `TANZU_CLI_ESSENTIALS_PLUGIN_GROUP_VERSION` | Specify a fixed version to use for the Essential Plugins group instead of the latest.  Should not be needed. | Group version |
+| `TANZU_CLI_SKIP_CONTEXT_RECOMMENDED_PLUGIN_INSTALLATION` | Skips the auto-installation of the context recommended plugins
+on `tanzu context create` or `tanzu context use` | `1` or `true` to skip auto-installation, `0`, `false`, `""` or unset to auto-install |
 | `TANZU_CLI_INCLUDE_DEACTIVATED_PLUGINS_TEST_ONLY` | Instruct the CLI to treat deactivated plugins as if they were active | `1` or `true` to use deactivated plugin, `0`, `false`, `""` or unset not to use them |
 | `TANZU_CLI_E2E_TEST_BINARY_PATH` | Specifies the CLI binary to use for E2E tests.  Defaults to `tanzu` as found on `$PATH`. | The path including the binary to the CLI  |
 | `TANZU_CLI_PLUGIN_DB_CACHE_REFRESH_THRESHOLD_SECONDS` | Overrides the default threshold at which point the plugin inventory will be automatically refreshed.  Default: 24 hours. | Threshold in seconds |
 | `TANZU_CLI_PLUGIN_DB_CACHE_TTL_SECONDS` | Overrides the default 30 minute delay in which the plugin inventory cache is used without checking if it should be refreshed. | Delay in seconds |
-| `TANZU_CLI_PLUGIN_DISCOVERY_PATH_FOR_TANZU_CONTEXT` | Allows testing the preliminary context-scoped plugin support for a Tanzu context type. | The path portion of the URI to use for discovery of context-scoped plugins on a Tanzu context |
+| `TANZU_CLI_PLUGIN_DISCOVERY_PATH_FOR_TANZU_CONTEXT` | Allows testing the preliminary context-recommended plugin support for a Tanzu context type. | The path portion of the URI to use for discovery of context-recommended plugins on a Tanzu context |
 | `TANZU_CLI_SHOW_PLUGIN_INSTALLATION_LOGS` | Allows to print plugin installation logs during the Essential Plugins installation. |  `1` or `true` to print the logs, `0`, `false`, `""` or unset not to print them |
-| `TANZU_CLI_STANDALONE_OVER_CONTEXT_PLUGINS` | Tells the CLI to use any standalone plugins currently installed even if the same plugin is also installed as context-scoped.  This allows to test new plugin versions locally. |  `1` or `true` to use standalone plugins before context-scoped plugins, `0`, `false`, `""` or unset to prioritize context-scoped plugins |
 | `TANZU_CLI_SUPERCOLLIDER_ENVIRONMENT` | Specifies the use of the staging super collider environment instead of the production environment. | `"staging"` |
 | `TANZU_CLI_TMC_UNSTABLE_URL` | Specifies the endpoint for the TMC cluster to use in E2E tests. | The URI of the endpoint |
 | `TANZU_CONFIG` | Use a different `config.yaml` file. | Full path to the new config file |

docs/full/README.md

@@ -95,7 +95,7 @@ Commands producing meaningful output consistently provide alternative output suc
 
 The CLI is configured to use a default plugin repository. Through various commands like `tanzu plugin search`, `tanzu plugin install`, `tanzu plugin install --group <groupName>:<groupVersion>`, the CLI provides various means to discover, then securely install or update plugins to serve specific needs of the user.
 
-Certain context endpoints will require that a specific set of plugins be installed so as to enable proper interaction with said endpoints. For these, establishing a connection with these endpoints may lead to the discovery and automatic installation of additional plugins. Plugins installed through this mean of discovery are referred to as "context-scoped" plugins. To learn more about the context-scoped plugins, please check the [context-scoped plugin installation](../full/context-scoped-plugins.md) documentation.
+Certain context endpoints will require that a specific set of plugins be installed so as to enable proper interaction with said endpoints. For these, establishing a connection with these endpoints may lead to the discovery and automatic installation of additional plugins. Plugins installed through this mean of discovery are referred to as "context-recommended" plugins. To learn more about the context-recommended plugins, please check the [context-recommended plugin installation](../full/context-recommended-plugins.md) documentation.
 
 For an overview on some of these plugin lifecycle commands, see the [Quickstart Guide](../quickstart/quickstart.md).
 For more details on these commands, see the [command reference](../cli/commands/tanzu_plugin.md).

docs/full/cli-architecture.md

@@ -20,8 +20,6 @@ This document aims to provide a general overview of the Tanzu CLI architecture.
 
 The CLI is based on a plugin architecture. This architecture enables teams to build, own, and release their piece of functionality as well as enable external partners to integrate with the system.
 
-Two categories of plugins are determined based on Plugin Discovery. Standalone Plugins and Context-Scoped Plugins
-
 ## Plugin Discovery
 
 A plugin discovery points to a group of plugin artifacts that are installable by the Tanzu CLI. It uses an interface to fetch the list of available plugins, their supported versions, and how to download them.

docs/full/context-scoped-plugins.md → docs/full/context-recommended-plugins.md

@@ -1,4 +1,4 @@
-# Context-Scope Plugin Installation
+# Context Recommended Plugin Installation
 
 ## Abstract
 
@@ -13,7 +13,7 @@ When a user is working with multiple instances of a product, we want to
 automatically select the right set of plugins and plugin versions for use
 based on the active context the user is connected to.
 
-The goals of the Context-Scoped Plugin Installation feature are:
+The goals of the Context-Recommended Plugin Installation feature are:
 
 As a plugin developer, I want to,
 
@@ -42,23 +42,12 @@ Plugin availability is solely dependent on the configured discovery sources in t
 tanzu configuration file. Each discovery source points to a plugin repository
 which can contain one or more plugins.
 
-## Standalone Plugins
-
-The scope of a plugin depends on how the plugin is getting installed on the user's machine.
-Users can run the `tanzu plugin search` command to see all available plugins from
-the configured discovery sources.
-
-If the user wants to install a plugin that is not dependent on any active context and
-wants to use it with the Tanzu CLI, the user can run `tanzu plugin install <plugin-name>`
-to install the required plugin. Installing the plugin this way will make the
-plugin a standalone plugin and it will not be associated with any contexts.
-
-## Context-scoped Plugins
+## Context-Recommended Plugins
 
 As mentioned above in the abstract section, there might be a scenario when a user
 is working with multiple contexts at a time and wants to automatically select the
 right set of plugins and plugin versions based on the currently active context.
-The context-scoped plugin implementation is useful in this scenario.
+The context-recommended plugin implementation is useful in this scenario.
 
 When the CLI user creates a new context for the Tanzu CLI using the
 `tanzu context create` command, the CLI adds a context in the tanzu configuration file
@@ -68,19 +57,21 @@ Now, this newly created context can also recommend the list of plugins and their
 that are needed to be installed on the user's machine to interact with the created context.
 The Tanzu CLI automatically detects the list of recommended plugins and their versions and
 installs them as part of the `tanzu context create` or `tanzu context use` commands. Below
-is the workflow of context-scoped plugin installation:
+is the workflow of context-recommended plugin installation:
 
 - The user runs the `tanzu context create` or `tanzu context use` commands to create a new context or switch active context
 - The Tanzu CLI gets the list of recommended plugins and their version from the created context
 - The Tanzu CLI finds the plugins and their metadata in the available list of plugins generated from the configured discovery sources
 - The Tanzu CLI fetches the plugin binary for these plugins from the specified location and installs the plugin
 
-Users should understand that these plugins (installed based on a context) are
-only available when said context is active. If a user deletes the context the plugins
-installed based on the deleted context are no longer available to use with the CLI.
 If the user switches the context to a different context using the `tanzu context use` command,
 the CLI will automatically install/update the recommended plugins based on the new context.
 
+Note: Users should understand that these plugins (installed based on a recommendation from a context) are
+installed as normal plugins and will not be automatically deleted when a user deletes the context or switches
+the context to a different context. Commands associated with those plugins will remain available
+to be used but will likely throw an error if those plugins do not work with the active context.
+
 ## Plugin Recommendations from a Context
 
 This section provides more details on how a context can provide

docs/plugindev/README.md

@@ -401,20 +401,16 @@ As a plugin developer, you will want to test your plugin manually at some point.
 During the development cycle, you can easily build and install the latest version of your plugin
 as described in the section on [building a plugin](#building-a-plugin).
 
-If you are using "contexts" while doing testing of your plugin and that your plugin is context-scoped
-you may find that your new plugin version is being overwritten with an older version by the CLI.
-When a plugin is context-scoped a context may recommend a specific version of that plugin for the CLI
-to use.  In such a case, if you want to test your new version and prevent the CLI from overwriting it
-with the version recommended by the context you can enable the `TANZU_CLI_STANDALONE_OVER_CONTEXT_PLUGINS`
+While testing your plugin, interacting with a context that recommends an older version of said
+plugin might result in your plugin version being overwritten.
+In such a case, if you want to test your new version and prevent the CLI from overwriting it
+with the version recommended by the context you can set the `TANZU_CLI_SKIP_CONTEXT_RECOMMENDED_PLUGIN_INSTALLATION`
 variable by doing:
 
 ```sh
-tanzu config set env.TANZU_CLI_STANDALONE_OVER_CONTEXT_PLUGINS true
+tanzu config set env.TANZU_CLI_SKIP_CONTEXT_RECOMMENDED_PLUGIN_INSTALLATION true
 ```
 
-This variable will instruct the CLI that any installation done with `tanzu plugin install <pluginName>`
-should take precedence over any installation coming from a context.
-
 #### Automated tests using the test plugin
 
 Plugin tests can be run by installing the admin `test` plugin.

docs/quickstart/quickstart.md

@@ -324,24 +324,36 @@ Switching to another context:
 [i] Successfully installed all required plugins
 
 > tanzu plugin list
-Standalone Plugins
-  NAME                DESCRIPTION                                                        TARGET      VERSION  STATUS
-  isolated-cluster    Prepopulating images/bundle for internet-restricted environments   global      v0.28.0  installed
-  pinniped-auth       Pinniped authentication operations (usually not directly invoked)  global      v0.28.0  installed
-  management-cluster  Kubernetes management cluster operations                           kubernetes  v0.28.0  installed
-  package             Tanzu package management                                           kubernetes  v0.28.0  installed
-  secret              Tanzu secret management                                            kubernetes  v0.28.0  installed
-  telemetry           configure cluster-wide settings for vmware tanzu telemetry         kubernetes  v0.28.0  installed
+  NAME                DESCRIPTION                                                        TARGET      INSTALLED  RECOMMENDED  STATUS
+  cluster             Kubernetes cluster operations                                      kubernetes  v0.28.0    v0.28.0      installed
+  feature             Operate on features and featuregates                               kubernetes  v0.28.0    v0.28.0      installed
+  kubernetes-release  Kubernetes release operations                                      kubernetes  v0.28.0    v0.28.0      installed
+  isolated-cluster    Prepopulating images/bundle for internet-restricted environments   global      v0.28.0                 installed
+  pinniped-auth       Pinniped authentication operations (usually not directly invoked)  global      v0.28.0                 installed
+  management-cluster  Kubernetes management cluster operations                           kubernetes  v0.28.0                 installed
+  package             Tanzu package management                                           kubernetes  v0.28.0                 installed
+  secret              Tanzu secret management                                            kubernetes  v0.28.0                 installed
+  telemetry           configure cluster-wide settings for vmware tanzu telemetry         kubernetes  v0.28.0                 installed
 
-Plugins from Context:  tkg-mgmt-vc
-  NAME                DESCRIPTION                           TARGET      VERSION  STATUS
-  cluster             Kubernetes cluster operations         kubernetes  v0.28.0  installed
-  feature             Operate on features and featuregates  kubernetes  v0.28.0  installed
-  kubernetes-release  Kubernetes release operations         kubernetes  v0.28.0  installed
 ```
 
+The way this type of plugin installation works is that the context you
+create or activate can recommend a list of plugins and their versions.
+When you create or activate a context, the Tanzu CLI fetches the list of
+recommended plugins and automatically tries to install these plugins to
+your machine. A `RECOMMENDED` column is shown as part of the
+`tanzu plugin list` output that specifies which plugins are recommended
+by the active context.
+
+If for some reason the auto-installation of the recommended plugins fails while
+creating or activating a context or an existing active context starts recommending
+some newer versions of the plugins, you can check that with the `tanzu plugin list`
+output with the `STATUS` field mentioning `update needed` and a newer version in
+the `RECOMMENDED` column. You can run the `tanzu plugin sync` command
+to automatically install the recommended version of the plugins.
+
 To learn more about the plugins installed from context, please refer to
-[context-scoped plugin installation](../full/context-scoped-plugins.md).
+[context recommended plugin installation](../full/context-recommended-plugins.md).
 
 ## Notes to users of previous versions of the Tanzu CLI
 

go.mod

@@ -41,7 +41,7 @@ require (
 	github.com/vmware-tanzu/carvel-imgpkg v0.36.1
 	github.com/vmware-tanzu/carvel-ytt v0.40.0
 	github.com/vmware-tanzu/tanzu-cli/test/e2e/framework v0.0.0-00010101000000-000000000000
-	github.com/vmware-tanzu/tanzu-framework/capabilities/client v0.0.0-20230707192002-93a508a39d94
+	github.com/vmware-tanzu/tanzu-framework/capabilities/client v0.0.0-20230523145612-1c6fbba34686
 	github.com/vmware-tanzu/tanzu-plugin-runtime v1.3.0-dev.0.20240410164649-72cfcba566be
 	go.pinniped.dev v0.20.0
 	golang.org/x/mod v0.12.0

go.sum

@@ -736,8 +736,8 @@ github.com/vmware-tanzu/tanzu-framework/apis/core v0.0.0-20230419030809-7081502e
 github.com/vmware-tanzu/tanzu-framework/apis/core v0.0.0-20230419030809-7081502ebf68/go.mod h1:TmKXkKBVKBu3/7mZeHSr17RH4pUzlcAHuU7gdKYw34U=
 github.com/vmware-tanzu/tanzu-framework/apis/run v0.0.0-20230419030809-7081502ebf68 h1:DMsg44rX3xSBDCoKYU7zfJGzs9dfPkMYmM7C5fFx7Es=
 github.com/vmware-tanzu/tanzu-framework/apis/run v0.0.0-20230419030809-7081502ebf68/go.mod h1:e1Uef+Ux5BIHpYwqbeP2ZZmOzehBcez2vUEWXHe+xHE=
-github.com/vmware-tanzu/tanzu-framework/capabilities/client v0.0.0-20230707192002-93a508a39d94 h1:Bql3MJ4/eXU/pDLAA/9UBsMY7Ou8WQnkOCy8DRzSujI=
-github.com/vmware-tanzu/tanzu-framework/capabilities/client v0.0.0-20230707192002-93a508a39d94/go.mod h1:AFGOXZD4tH+KhpmtV0VjWjllXhr8y57MvOsIxTtywc4=
+github.com/vmware-tanzu/tanzu-framework/capabilities/client v0.0.0-20230523145612-1c6fbba34686 h1:VcuXqUXFxm5WDqWkzAlU/6cJXua0ozELnqD59fy7J6E=
+github.com/vmware-tanzu/tanzu-framework/capabilities/client v0.0.0-20230523145612-1c6fbba34686/go.mod h1:AFGOXZD4tH+KhpmtV0VjWjllXhr8y57MvOsIxTtywc4=
 github.com/vmware-tanzu/tanzu-plugin-runtime v1.3.0-dev.0.20240410164649-72cfcba566be h1:/rYyMeziawcrVaaq9W7Vogc9MdHsQRn0CcEqa8GwrOA=
 github.com/vmware-tanzu/tanzu-plugin-runtime v1.3.0-dev.0.20240410164649-72cfcba566be/go.mod h1:5m73y796B4EoeXZtvkq8jbQPPQXeYkLPLS2BBbxZp7o=
 github.com/xanzy/go-gitlab v0.83.0 h1:37p0MpTPNbsTMKX/JnmJtY8Ch1sFiJzVF342+RvZEGw=