[Extensions] Add Transport API hook for fetching Service Account information from security plugin

[stephen-crawford]

This issue is the Core-side of 2609

In order to implement Service Accounts (#2597), it needs to be possible to fetch service account details. To do this, OpenSearch core should be able to call the Security Plugin to provide a Service Account Authorization Header back to the Extensions Manager in core. This will let core create requests on behalf of extensions.

Inside core, a hook needs to be added that allows the API introduced in #2609 to be called when a new extension is registered with the Extension Manager. An outline of what this may look like can be found here.

The implementation process requires the addition of a ServiceAccountManager class inside server/src/main/java/org/opensearch/identity/. The ServiceAccountManager should be connected to an IdentityService which is built when (FeatureFlags.isEnabled(FeatureFlags.EXTENSIONS) in Node.java. This will turn on the IdentityService whenever the cluster launches and extensions are enabled.

The IdentityService should construct an instance of the ServiceAccountManager and then use the ServiceAccountManager to interact with the Security Plugin. The abstraction is used so that if any code is later moved from the Security Plugin into core, the entire system does not require a rework.

Inside the ServiceAccountManager class, there should be a hook to the Security Plugin API for getting a service account based on an extensionUniqueId.

An outline of possible changes can be found here. This may be helpful for referencing.

This issues completion will include the addition of classesServiceAccountManager, IdentityService, and possibly others. The complete PR should demonstrate the ability for core to call the Security Plugin's Service Account get API and retrieve a response. As a result, #2609 must be completed before this issue can be completed. For now, it is appropriate to receive a basic string from the Security Plugin. The completed PR should also include tests that verify that the response received by core is in a valid format and should also build on node start.

[MaciejMierzwa]

Hi, I think #2609 leads to the wrong link (it's some bug from the main Opensearch repo). I guess it should be opensearch-project/security#2609
Also, commenting to mention I started looking into it since I can't assign tasks to myself.

[stephen-crawford]

Hi, I think #2609 leads to the wrong link (it's some bug from the main Opensearch repo). I guess it should be opensearch-project/security#2609 Also, commenting to mention I started looking into it since I can't assign tasks to myself.

Oops, nice catch. I updated the links and assigned it to you. Let me know if you have any questions or run into any issues.

[MaciejMierzwa]

Hi @scrawfor99 created PR: #7307
For now the method getOrCreateServiceAccount() in ServiceAccountManager accepts auth string. I assumed this authentication is connected to this task: opensearch-project/security#2622 or this one: opensearch-project/security#2703

[MaciejMierzwa]

Hi @scrawfor99
I don't have permission to assign PR to you. Here's PR after merging in changes from the main: #7307

[stephen-crawford]

Hi @MaciejMierzwa, thank you! I will take a look at this today :)

[peternied]

Recapping offline conversation:

• In OpenSearch, Update InitializeExtensionRequest in ExtensionManager so the service account token can be passed
• In OpenSearch, Create a new interface off of IdentityPlugin and a new ServiceAccountManager, also provide a basic implementation for testing
• Create a new class in the security plugin that implements ServiceAccountManager that can provide the token via the UserService

[MaciejMierzwa]

Hi, pushed my changes for the security side ServiceAccountManager implementation. Could you please take a look at it @scrawfor99 or @scrawfor99. Here's PR: opensearch-project/security#2775
I don't fully understand the idea of implementing ServiceAccountManager in both repositories: core and security. I assumed that we want to implement core-side interface ServiceAccountManager, in security plugin. Then on a node startup the object would be initialized and this API would be used for core - security communication. Is this correct approach?

[stephen-crawford]

Hi @MaciejMierzwa, thank you for following up. Sorry for not being clear before. Basically, the idea is going to be to use an interface in core that is then implemented in Security Plugin. To test this there will be a simplified implementation of the interface in core and a simplified version of the interface itself in the Security plugin. This will let us do isolated testing within each code body. I have made progress towards making these changes in a PR and would welcome your thoughts as I move things forward. I will review what you have been working on here as well and after I add in the interface to the code base we can move some of what you have been working onto into the appropriate locations.

Part of the reason the details have not been entirely clear are that we have made a couple shifts in design since the initial outlining of the plans and some of these shifts were not made until after you did a significant amount of work.