Implement new extension points in IdentityPlugin and add ContextProvidingPluginSubject

[cwperks]

Description

Re-opening this previously closed PR to rebase on top of Optimized Privilege Evaluation

Companion PR in core: opensearch-project/OpenSearch#14630

This PR by itself does not add additional functionality, it simply implements the new extension points in core and introduces a new class called ContextProvidingPluginSubject which populates a header in the ThreadContext with the canonical class name of the plugin that is executing code using pluginSystemSubject.runAs(() -> { ... }). See ./gradlew integrationTest --tests SystemIndexTests for an example that verifies that a block of code run with pluginSystemSubject.runAs(() -> { ... }) has contextual info in the thread context.

• Category (Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation)

Enhancement

Issues Resolved

Related to #4439

Check List

• New functionality includes testing
• New functionality has been documented
• New Roles/Permissions have a corresponding security dashboards plugin PR
• API changes companion pull request created
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: Patch coverage is 85.00000% with 12 lines in your changes missing coverage. Please review.

Project coverage is 71.48%. Comparing base (2fa7eff) to head (2fdb4b9).
Report is 12 commits behind head on main.

Files with missing lines	Patch %	Lines
...ensearch/security/privileges/ActionPrivileges.java	59.09%	5 Missing and 4 partials ⚠️
.../org/opensearch/security/auth/UserSubjectImpl.java	90.00%	1 Missing ⚠️
...ecurity/privileges/SystemIndexAccessEvaluator.java	92.85%	0 Missing and 1 partial ⚠️
...c/main/java/org/opensearch/security/user/User.java	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4896      +/-   ##
==========================================
- Coverage   71.51%   71.48%   -0.03%     
==========================================
  Files         334      336       +2     
  Lines       22556    22625      +69     
  Branches     3589     3598       +9     
==========================================
+ Hits        16130    16174      +44     
- Misses       4636     4655      +19     
- Partials     1790     1796       +6     

Files with missing lines	Coverage Δ
.../opensearch/security/OpenSearchSecurityPlugin.java	83.92% <100.00%> (+0.36%)	⬆️
.../org/opensearch/security/auth/BackendRegistry.java	78.39% <100.00%> (+0.33%)	⬆️
...org/opensearch/security/filter/SecurityFilter.java	66.35% <100.00%> (+0.47%)	⬆️
...curity/identity/ContextProvidingPluginSubject.java	100.00% <100.00%> (ø)
...earch/security/privileges/PrivilegesEvaluator.java	73.96% <100.00%> (+0.31%)	⬆️
...g/opensearch/security/support/ConfigConstants.java	95.23% <ø> (ø)
.../org/opensearch/security/auth/UserSubjectImpl.java	90.00% <90.00%> (ø)
...ecurity/privileges/SystemIndexAccessEvaluator.java	76.92% <92.85%> (+1.72%)	⬆️
...c/main/java/org/opensearch/security/user/User.java	69.41% <0.00%> (-0.83%)	⬇️
...ensearch/security/privileges/ActionPrivileges.java	95.14% <59.09%> (-2.05%)	⬇️

... and 7 files with indirect coverage changes

[DarshitChanpura]

Thank you @cwperks. Left a few comments but generally looks good to me. Will there be a follow-up PR to clean up pluginActions matcher store once the PR in core gets merged?

[cwperks]

Thank you @cwperks. Left a few comments but generally looks good to me. Will there be a follow-up PR to clean up pluginActions matcher store once the PR in core gets merged?

Yes there would be one more PR to implement index actions (for completeness) if opensearch-project/OpenSearch#15778 is merged.

There's been a lot of discussion and I left a comment here to outline some reasons why I don't recommend system index protection to be a core feature. This gist is that the security plugin has a notion of connecting with the admin certificate that can be used as a backdoor for an operator to perform invasive actions on system indices if necessary. The admin certificate requires that a cluster has https enabled and https is provided by the security plugin. There isn't a secure way to create a backdoor for a cluster running without https.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/security/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/security/backport-2.x
# Create a new branch
git switch --create backport/backport-4896-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 1924e410f80c4b958f6ec48c9691f8ae7a0af102
# Push it to GitHub
git push --set-upstream origin backport/backport-4896-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/security/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4896-to-2.x.