Set the mapped security roles of the user so these can be used by the…

[ch-govau]

… DLS privileges evaluator. Allow security roles to be used for DLS parameter substitution. Fixes opensearch-project/security/#1568

Signed-off-by: Caitlin Harper caitlin.harper@defence.gov.au

Description

• Category (Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation)
  Bug Fix /Enhancement

• Why these changes are required?
  The changes are required so that we can map the backend roles to more appropriately named roles via the roles.mapping and these security roles can be used in the DLS privileges evaluation.

• What is the old behavior before changes and new behavior after changes?
  Old behaviour:
  security roles were not added to the user object prior to the DLS privileges evaluation

New behaviour:
mapped roles are added to the user object for DLS privileges evaluation
a new option for parameter substitution in the DLS query has been added. This is ${user.securityRoles} and is the mapped roles.

Note: there is no change to the existing ${user.roles} substitution. It remains the same.

Issues Resolved

• Resolves [BUG] Unexpected role set used for document level security.  #1568

Is this a backport? If so, please add backport PR # and/or commits #

Testing

Manual test steps:
Testing was completed for both basic auth and OIDC/jwt auth. Note that both users had read access to the index.

1. Push 2 documents into the index, one with DLS required role = ROLE1 and the other with ROLE2.
   example:

curl https://localhost:9200/myindex/_doc?pretty -u admin:$PASS -XPOST -H "Content-Type: application/json" -d @doc1.json

doc1.json

{
	"security": {
                        "exclusive": [
                            "ROLE1",
                        ],
                        "num_exclusive": 1
          },
	  "info":{
		"generalinfo": "this document should only be available to those with ROLE1"
	  }
}

2. Add the roles ROLE1, ROLE2 and the read to the roles.yml
   example:

ROLE1:
  reserved: false
  hidden: false
  cluster_permissions: []
  index_permissions: []
  tenant_permissions: []
  static: false

ROLE2:
  reserved: false
  hidden: false
  cluster_permissions: []
  index_permissions: []
  tenant_permissions: []
  static: false

myindex_read:
  reserved: false
  hidden: false
  cluster_permissions:
  - "indices:data/read/scroll/clear"
  - "indices:data/read/msearch"
  - "indices:data/write/bulk"
  - "indices:data/read/scroll"
  - "indices:data/read/mget"
  index_permissions:
  - index_patterns:
    - "myindex"
    dls: "{ \"bool\": { \"filter\": [\n   {\"terms_set\": {\"security.exclusive.keyword\": {\n\
      \ \"terms\": [${user.securityRoles}],\n \"minimum_should_match_field\": \"security.num_exclusive\"\
      \n }}}\n ]}}\n"
    fls: []
    masked_fields: []
    allowed_actions:
    - "read"

3. Add the role mappings to the backend roles in roles_mapping.yml

ROLE1:
  hosts: []
  users: []
  reserved: false
  hidden: false
  backend_roles:
  - "client-id/role-1"
  and_backend_roles: []

ROLE2:
  hosts: []
  users: []
  reserved: false
  hidden: false
  backend_roles:
  - "client-id/role-2"
  and_backend_roles: []

4. Also add the following user for the internal/basic auth testing

user1:
  hash: "<password-hash>"
  reserved: false
  backend_roles:
    - "client-id/role-1"
    - "myindex_read"
  description: "user1"

5. Note that we also have the test server set up to use keycloak and user2 has the following backend roles: "myindex_read", "client-id/role-1", "client-id/role-2".
6. Run the security config reload script

./plugins/opensearch-security/tools/securityadmin.sh" -cd "./plugins/opensearch-security/securityconfig" -icl -key "./config/kirk-key.pem" -cert "./config/kirk.pem" -cacert "./config/root-ca.pem" -nhnv

7. Test basic auth for user 1

curl https://localhost:9200/myindex/_search?pretty --insecure -u user1:user1

Expected result: This should return 1 document, doc1
8. Test jwt for user2

curl -H "Authorization: Bearer $TOKEN" https://localhost:9200/myindex/_search?pretty --insecure

Expected result: This should return 2 documents.
9. Also tested with another user that has neither ROLE1 or ROLE2 and confirm they have access to zero documents.

Documentation

This will require an update to the DLS parameter substitution documentation to include the new ${user.securityRoles}.
The update to the documentation has been submitted here: opensearch-project/documentation-website#420

Check List

• New functionality includes testing
• New functionality has been documented
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[ae-govau]

Gentle ping - our stakeholders are still keen on this. Would appreciate feedback (positive or negative) on the approach and whether anything additional is needed.

I note the failing "CodeQL" check - but I note that what it is complaining about is not relevant to (ie is not changed by) this PR.

[cliu123]

Thanks for contributing! Can you add UTs for all possible use cases?

[ch-govau]

Thanks for contributing! Can you add UTs for all possible use cases?

Can do.

[ch-govau]

Tests added, will rebase and squash when everyone is happy.

[ch-govau]

Any thoughts/comments stopping this from being merged. I'm happy to make changes if required? Also the documentation update has been completed by my colleague and is linked to the PR.

[peternied]

@opensearch-project/security can we review this PR?

[ae-govau]

Hey all, this has been sitting stagnant for a while. @ch-govau has added the tests, and @mm-govau has added the documentation in opensearch-project/documentation-website#420 which is already approved.

I think all we are waiting on is an approval here? Any attention for review for would be much appreciated. Outside of the tests, it's 10 lines of new code, so hoping it's not a hard review task.

Thanks again.

[peternied]

@cliu123 can you review this?

[cliu123]

Could you please sign all the commits?

[ch-govau]

Could you please sign all the commits?

sorry about that. have rebased and the commit is signed. Thanks so much for reviewing this. It is very much appreciated :)

[codecov-commenter]

Codecov Report

Merging #1588 (0efcaa5) into main (e638c76) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head 0efcaa5 differs from pull request most recent head f90dc0e. Consider uploading reports for the commit f90dc0e to get more accurate results

@@             Coverage Diff              @@
##               main    #1588      +/-   ##
============================================
- Coverage     62.94%   62.94%   -0.01%     
  Complexity     3261     3261              
============================================
  Files           253      253              
  Lines         18126    18133       +7     
  Branches       3258     3259       +1     
============================================
+ Hits          11410    11413       +3     
- Misses         5067     5071       +4     
  Partials       1649     1649              

Impacted Files	Coverage Δ
...earch/security/privileges/PrivilegesEvaluator.java	71.84% <100.00%> (+0.09%)	⬆️
...pensearch/security/securityconf/ConfigModelV7.java	62.63% <100.00%> (+0.42%)	⬆️
...ecurity/configuration/ConfigurationRepository.java	73.07% <0.00%> (-2.20%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e638c76...f90dc0e. Read the comment docs.

[ae-govau]

@cliu123 - is anything else needed on this PR? I think @ch-govau has addressed all comments and this can be merged after a 2nd approval. Thanks for your help.

[cliu123]

@cliu123 - is anything else needed on this PR? I think @ch-govau has addressed all comments and this can be merged after a 2nd approval. Thanks for your help.

LGTM! Thanks for the contribution!