Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature/Extension] Add oboauthcbackend registry and set up e2e endpoint testing flow #2857

Merged
Show file tree
Hide file tree
Changes from 60 commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
5582c5c
Fix build break
peternied Jun 8, 2023
8c8bb88
Make authenticator name more concise
peternied Jun 8, 2023
0f7b493
Clean up how roles are (de)encrypted
peternied Jun 8, 2023
4b5d48e
Single constructor
peternied Jun 8, 2023
baa1312
More JwtVendor cleanup - but not sure if this works?
peternied Jun 8, 2023
2cb7cae
Add OBO Authenticator into the Authc Backend list
RyanL1997 May 24, 2023
196fbea
Fix the logic of feching er/dr for rolesObject
RyanL1997 May 25, 2023
6e96de6
Fix of the OBO unit tests of Bearer header
RyanL1997 May 26, 2023
a85a3e9
Set up oboconfig
RyanL1997 Jun 2, 2023
98674b8
Adding this obo config to xcontent builder and remove unused imports
RyanL1997 Jun 7, 2023
aa4e599
Adding this obo config to xcontent builder and create obo authz heade…
RyanL1997 Jun 8, 2023
8204874
Done happy testing case
RyanL1997 Jun 8, 2023
293791e
Fix the header
RyanL1997 Jun 8, 2023
847f551
Revert the temorary fix of zstd library
RyanL1997 Jun 8, 2023
1047729
Misc cleanup
peternied Jun 8, 2023
d00e2e7
Remove bad import
peternied Jun 8, 2023
f46070f
Add endpoint and test that interactions with JwtVendor
peternied Jun 9, 2023
4896a4b
AlFix to do both decrypt and encrypt tokens
peternied Jun 12, 2023
71dd17f
Add basic parameters and update test to use them
peternied Jun 12, 2023
8c0d750
Change ConfigModelV7's imports
RyanL1997 Jun 13, 2023
b36bde0
Re-apply zstd dependency fix
RyanL1997 Jun 13, 2023
a6bc8c0
Update OSSecurityPlugin class
RyanL1997 Jun 13, 2023
4351752
Fix the optional longsupplier for current time
RyanL1997 Jun 14, 2023
2a129af
Set up the subscriber to grab signingkey and encryptionkey from confi…
RyanL1997 Jun 14, 2023
61eaee0
Set up manual testing with stronger signing key
RyanL1997 Jun 14, 2023
640bc33
Modify JwtVendor to grab mapped roles
RyanL1997 Jun 19, 2023
9148a98
Add backend roles + dcf listener for obo config + Fix the role grabbi…
RyanL1997 Jun 19, 2023
a455250
Add obo auth integ test case with endopoint + remove manual testing c…
RyanL1997 Jun 19, 2023
07aa292
Add obo auth integ test case with endopoint + remove manual testing c…
RyanL1997 Jun 19, 2023
0ed28b5
Fix Code Hygiene, remove unused imports
RyanL1997 Jun 19, 2023
26670ac
Remove some stale TODOs
RyanL1997 Jun 19, 2023
12bd3e8
Change the * imports into seprate ones in OBOAuthenticationTest
RyanL1997 Jun 19, 2023
0e590ba
Remove another unused import in jwtVendor
RyanL1997 Jun 19, 2023
d1341e2
Add license header for CreateOnBehalfOfToken + OnBehalfOfJwtAuthentic…
RyanL1997 Jun 19, 2023
9e832cf
Remove wording "extension" in OnBehalfOfJwtAuthenticationTest's testi…
RyanL1997 Jun 19, 2023
626c853
Trim the obo integ test body
RyanL1997 Jun 20, 2023
e755753
Switch to java base64 decoder in OBOAuthenticator L75
RyanL1997 Jun 20, 2023
68af23a
Add stack trace to see the error of base64 decoder
RyanL1997 Jun 20, 2023
66b52e9
Fix the formatting in DynamicConfigModelV7
RyanL1997 Jun 20, 2023
983267b
Fix the typo of minimalKeyFormat + adding a debugging statement to ch…
RyanL1997 Jun 20, 2023
4999a58
Fix the signingKeys + encryptionKeys in src/integrationTest/resources…
RyanL1997 Jun 21, 2023
d95ad8b
Fix the signingKeys + encryptionKeys in src/test/resources/config.yml…
RyanL1997 Jun 21, 2023
dcec4e9
Remove debugging statements and add obo setting in src/test/resources…
RyanL1997 Jun 21, 2023
cd2eac6
Add a debug statement to print dcm.getobosettings
RyanL1997 Jun 21, 2023
be83625
Fix the missing comma in src/test/resources/restapi/securityconfig_no…
RyanL1997 Jun 21, 2023
8836e8c
Add obo setting to test config of src/test/resources/config_anon.yml
RyanL1997 Jun 21, 2023
817436d
Add obo setting into multiple testing config files
RyanL1997 Jun 21, 2023
7cb5cb5
Add checking for obo setting's presence
RyanL1997 Jun 21, 2023
eb9f5a4
Add checking for obo setting's presence with correct status type of s…
RyanL1997 Jun 21, 2023
d448af6
Revert back those testing configs
RyanL1997 Jun 21, 2023
275294a
Remove the security role adding in noop auth backend class
RyanL1997 Jun 22, 2023
2b64b8b
Add the check of NoOpAuthbackend in the backend registry
RyanL1997 Jun 22, 2023
896e826
Fix both impersonation tests
RyanL1997 Jun 22, 2023
729329b
Trim the obo integration test and add a new test case for tempered token
RyanL1997 Jun 23, 2023
cfed763
Change into matcher for OBO authenticator.
RyanL1997 Jun 26, 2023
df18f61
Rename the create obo token action
RyanL1997 Jun 26, 2023
ed14cea
Add description in the testing config of signingKey + encryptionKey
RyanL1997 Jun 26, 2023
a29992f
Fix the token duration
RyanL1997 Jun 26, 2023
7b6221f
No backendRoles by default
RyanL1997 Jun 26, 2023
d243fe5
Change the roles claim extraction error log msg
RyanL1997 Jun 27, 2023
8540862
Resolve some comments of formatting
RyanL1997 Jun 28, 2023
b4be662
Update the issuer to represent the cluster
RyanL1997 Jun 28, 2023
8e0631f
Fix the obo token's duration issue
RyanL1997 Jun 28, 2023
144efdf
Fix unused imports
RyanL1997 Jun 28, 2023
b2f5d2b
Merge branch 'feature/extensions' into add-oboauthcbackend-registry
RyanL1997 Jul 3, 2023
ce9baa2
Fix formatting again after branch sync
RyanL1997 Jul 3, 2023
0495b50
Fix Impersonation tests again
RyanL1997 Jul 3, 2023
b8c9ba1
Add api parameter for services (aud)
RyanL1997 Jul 3, 2023
49a705b
Remove the obo config json config files
RyanL1997 Jul 5, 2023
4b7e42a
Remove the obo config in src/test/resources/config.yml
RyanL1997 Jul 5, 2023
dfa6df9
Remove the obo config in src/test/resources/restapi/securityconfig_no…
RyanL1997 Jul 5, 2023
cb47792
Add back the obo config of securityconfig_nondefault.json
RyanL1997 Jul 5, 2023
bc574c4
Make a keyUtil for checking algo
RyanL1997 Jul 6, 2023
9340e6c
Extend the keyUtil and add to OBO authbackend
RyanL1997 Jul 7, 2023
cd203b8
Apply KeyUtil to both OBO and HTTPJWT authbackend
RyanL1997 Jul 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*
* Modifications Copyright OpenSearch Contributors. See
* GitHub history for details.
*/

package org.opensearch.security.http;

import java.nio.charset.StandardCharsets;
import java.util.Base64;
import java.util.List;
import java.util.Map;

import com.carrotsearch.randomizedtesting.annotations.ThreadLeakScope;
import org.apache.hc.core5.http.Header;
import org.apache.hc.core5.http.message.BasicHeader;
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.runner.RunWith;

import org.opensearch.test.framework.OnBehalfOfConfig;
import org.opensearch.test.framework.TestSecurityConfig;
import org.opensearch.test.framework.cluster.ClusterManager;
import org.opensearch.test.framework.cluster.LocalCluster;
import org.opensearch.test.framework.cluster.TestRestClient;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.aMapWithSize;
import static org.hamcrest.Matchers.allOf;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasKey;
import static org.opensearch.test.framework.TestSecurityConfig.AuthcDomain.AUTHC_HTTPBASIC_INTERNAL;
import static org.opensearch.test.framework.TestSecurityConfig.Role.ALL_ACCESS;


@RunWith(com.carrotsearch.randomizedtesting.RandomizedRunner.class)
@ThreadLeakScope(ThreadLeakScope.Scope.NONE)
public class OnBehalfOfJwtAuthenticationTest {

public static final String POINTER_USERNAME = "/user_name";

static final TestSecurityConfig.User ADMIN_USER = new TestSecurityConfig.User("admin").roles(ALL_ACCESS);

private static final String signingKey = Base64.getEncoder().encodeToString("jwt signing key for an on behalf of token authentication backend for testing of OBO authentication".getBytes(StandardCharsets.UTF_8));
private static final String encryptionKey = Base64.getEncoder().encodeToString("encryptionKey".getBytes(StandardCharsets.UTF_8));
public static final String ADMIN_USER_NAME = "admin";
public static final String DEFAULT_PASSWORD = "secret";
public static final String OBO_TOKEN_REASON = "{\"reason\":\"Test generation\"}";
public static final String OBO_ENDPOINT_PREFIX = "_plugins/_security/api/user/onbehalfof";

@ClassRule
public static final LocalCluster cluster = new LocalCluster.Builder()
.clusterManager(ClusterManager.SINGLENODE).anonymousAuth(false)
.users(ADMIN_USER)
.nodeSettings(Map.of(
"plugins.security.allow_default_init_securityindex", true,
"plugins.security.restapi.roles_enabled", List.of("user_admin__all_access")
))
.authc(AUTHC_HTTPBASIC_INTERNAL)
.onBehalfOf(new OnBehalfOfConfig().signing_key(signingKey).encryption_key(encryptionKey))
.build();

@Test
public void shouldAuthenticateWithOBOTokenEndPoint() {
Header adminOboAuthHeader;

try (TestRestClient client = cluster.getRestClient(ADMIN_USER_NAME, DEFAULT_PASSWORD)) {

client.assertCorrectCredentials(ADMIN_USER_NAME);

TestRestClient.HttpResponse response = client.postJson(OBO_ENDPOINT_PREFIX, OBO_TOKEN_REASON);
response.assertStatusCode(200);

Map<String, Object> oboEndPointResponse = response.getBodyAs(Map.class);
assertThat(oboEndPointResponse, allOf(
aMapWithSize(3),
hasKey("user"),
hasKey("onBehalfOfToken"),
hasKey("duration")));

String encodedOboTokenStr = oboEndPointResponse.get("onBehalfOfToken").toString();

adminOboAuthHeader = new BasicHeader("Authorization", "Bearer " + encodedOboTokenStr);
}

try (TestRestClient client = cluster.getRestClient(adminOboAuthHeader)) {

TestRestClient.HttpResponse response = client.getAuthInfo();
response.assertStatusCode(200);

String username = response.getTextFromJsonBody(POINTER_USERNAME);
assertThat(username, equalTo(ADMIN_USER_NAME));
}
}

@Test
public void shouldNotAuthenticateWithATemperedOBOToken() {
RyanL1997 marked this conversation as resolved.
Show resolved Hide resolved
Header adminOboAuthHeader;

try (TestRestClient client = cluster.getRestClient(ADMIN_USER_NAME, DEFAULT_PASSWORD)) {

client.assertCorrectCredentials(ADMIN_USER_NAME);

TestRestClient.HttpResponse response = client.postJson(OBO_ENDPOINT_PREFIX, OBO_TOKEN_REASON);
response.assertStatusCode(200);

Map<String, Object> oboEndPointResponse = response.getBodyAs(Map.class);
assertThat(oboEndPointResponse, allOf(
RyanL1997 marked this conversation as resolved.
Show resolved Hide resolved
aMapWithSize(3),
hasKey("user"),
hasKey("onBehalfOfToken"),
hasKey("duration")));

String encodedOboTokenStr = oboEndPointResponse.get("onBehalfOfToken").toString();
StringBuilder stringBuilder = new StringBuilder(encodedOboTokenStr);
stringBuilder.deleteCharAt(encodedOboTokenStr.length() - 1);
String temperedOboTokenStr = stringBuilder.toString();

adminOboAuthHeader = new BasicHeader("Authorization", "Bearer " + temperedOboTokenStr);
}

try (TestRestClient client = cluster.getRestClient(adminOboAuthHeader)) {

TestRestClient.HttpResponse response = client.getAuthInfo();
response.assertStatusCode(401);
response.getBody().contains("Unauthorized");
}
}
peternied marked this conversation as resolved.
Show resolved Hide resolved
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*
*/
package org.opensearch.test.framework;

import java.io.IOException;

import org.apache.commons.lang3.StringUtils;

import org.opensearch.core.xcontent.ToXContent;
import org.opensearch.core.xcontent.ToXContentObject;
import org.opensearch.core.xcontent.XContentBuilder;

public class OnBehalfOfConfig implements ToXContentObject {
private String signing_key;
private String encryption_key;

public OnBehalfOfConfig signing_key(String signing_key) {
this.signing_key = signing_key;
return this;
}

public OnBehalfOfConfig encryption_key(String encryption_key) {
this.encryption_key = encryption_key;
return this;
}

@Override
public XContentBuilder toXContent(XContentBuilder xContentBuilder, ToXContent.Params params) throws IOException {
xContentBuilder.startObject();
xContentBuilder.field("signing_key", signing_key);
if (StringUtils.isNoneBlank(encryption_key)) {
xContentBuilder.field("encryption_key", encryption_key);
}
xContentBuilder.endObject();
return xContentBuilder;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,11 @@ public TestSecurityConfig xff(XffConfig xffConfig) {
return this;
}

public TestSecurityConfig onBehalfOf(OnBehalfOfConfig onBehalfOfConfig) {
config.onBehalfOfConfig(onBehalfOfConfig);
return this;
}

public TestSecurityConfig authc(AuthcDomain authcDomain) {
config.authc(authcDomain);
return this;
Expand Down Expand Up @@ -171,6 +176,7 @@ public static class Config implements ToXContentObject {

private Boolean doNotFailOnForbidden;
private XffConfig xffConfig;
private OnBehalfOfConfig onBehalfOfConfig;
private Map<String, AuthcDomain> authcDomainMap = new LinkedHashMap<>();

private AuthFailureListeners authFailureListeners;
Expand All @@ -191,6 +197,11 @@ public Config xffConfig(XffConfig xffConfig) {
return this;
}

public Config onBehalfOfConfig(OnBehalfOfConfig onBehalfOfConfig) {
this.onBehalfOfConfig = onBehalfOfConfig;
return this;
}

public Config authc(AuthcDomain authcDomain) {
authcDomainMap.put(authcDomain.id, authcDomain);
return this;
Expand All @@ -211,6 +222,10 @@ public XContentBuilder toXContent(XContentBuilder xContentBuilder, Params params
xContentBuilder.startObject();
xContentBuilder.startObject("dynamic");

if (onBehalfOfConfig != null) {
xContentBuilder.field("on_behalf_of", onBehalfOfConfig);
}

if (anonymousAuth || (xffConfig != null)) {
xContentBuilder.startObject("http");
xContentBuilder.field("anonymous_auth_enabled", anonymousAuth);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
import org.opensearch.test.framework.AuditConfiguration;
import org.opensearch.test.framework.AuthFailureListeners;
import org.opensearch.test.framework.AuthzDomain;
import org.opensearch.test.framework.OnBehalfOfConfig;
import org.opensearch.test.framework.RolesMapping;
import org.opensearch.test.framework.TestIndex;
import org.opensearch.test.framework.TestSecurityConfig;
Expand Down Expand Up @@ -471,6 +472,11 @@ public Builder xff(XffConfig xffConfig) {
return this;
}

public Builder onBehalfOf(OnBehalfOfConfig onBehalfOfConfig) {
testSecurityConfig.onBehalfOf(onBehalfOfConfig);
return this;
}

public Builder loadConfigurationIntoIndex(boolean loadConfigurationIntoIndex) {
this.loadConfigurationIntoIndex = loadConfigurationIntoIndex;
return this;
Expand Down
6 changes: 4 additions & 2 deletions src/integrationTest/resources/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,7 @@ config:
type: "internal"
config: {}
on_behalf_of:
signing_key: "signing key"
encryption_key: "encryption key"
# The decrypted signing key is: This is the jwt signing key for an on behalf of token authentication backend for testing of extensions
RyanL1997 marked this conversation as resolved.
Show resolved Hide resolved
# The decrypted encryption key is: encryptionKey
signing_key: "VGhpcyBpcyB0aGUgand0IHNpZ25pbmcga2V5IGZvciBhbiBvbiBiZWhhbGYgb2YgdG9rZW4gYXV0aGVudGljYXRpb24gYmFja2VuZCBmb3IgdGVzdGluZyBvZiBleHRlbnNpb25z"
RyanL1997 marked this conversation as resolved.
Show resolved Hide resolved
encryption_key: "ZW5jcnlwdGlvbktleQ=="
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@
import org.opensearch.search.query.QuerySearchResult;
import org.opensearch.security.action.configupdate.ConfigUpdateAction;
import org.opensearch.security.action.configupdate.TransportConfigUpdateAction;
import org.opensearch.security.action.onbehalf.CreateOnBehalfOfTokenAction;
import org.opensearch.security.action.whoami.TransportWhoAmIAction;
import org.opensearch.security.action.whoami.WhoAmIAction;
import org.opensearch.security.auditlog.AuditLog;
Expand All @@ -141,7 +142,6 @@
import org.opensearch.security.dlic.rest.validation.PasswordValidator;
import org.opensearch.security.filter.SecurityFilter;
import org.opensearch.security.filter.SecurityRestFilter;
import org.opensearch.security.http.HTTPOnBehalfOfJwtAuthenticator;
import org.opensearch.security.http.SecurityHttpServerTransport;
import org.opensearch.security.http.SecurityNonSslHttpServerTransport;
import org.opensearch.security.http.XFFResolver;
Expand Down Expand Up @@ -215,6 +215,7 @@ public final class OpenSearchSecurityPlugin extends OpenSearchSecuritySSLPlugin
private volatile SslExceptionHandler sslExceptionHandler;
private volatile Client localClient;
private final boolean disabled;
private volatile DynamicConfigFactory dcf;
private final List<String> demoCertHashes = new ArrayList<String>(3);
private volatile SecurityFilter sf;
private volatile IndexResolverReplacer irr;
Expand Down Expand Up @@ -477,6 +478,9 @@ public List<RestHandler> getRestHandlers(Settings settings, RestController restC
Objects.requireNonNull(cs), Objects.requireNonNull(adminDns), Objects.requireNonNull(cr)));
handlers.add(new SecurityConfigUpdateAction(settings, restController, Objects.requireNonNull(threadPool), adminDns, configPath, principalExtractor));
handlers.add(new SecurityWhoAmIAction(settings, restController, Objects.requireNonNull(threadPool), adminDns, configPath, principalExtractor));
CreateOnBehalfOfTokenAction cobot = new CreateOnBehalfOfTokenAction(settings, threadPool);
dcf.registerDCFListener(cobot);
handlers.add(cobot);
handlers.addAll(
SecurityRestApiActions.getHandler(
settings,
Expand Down Expand Up @@ -838,17 +842,13 @@ public Collection<Object> createComponents(Client localClient, ClusterService cl

securityRestHandler = new SecurityRestFilter(backendRegistry, auditLog, threadPool,
principalExtractor, settings, configPath, compatConfig);

HTTPOnBehalfOfJwtAuthenticator acInstance = new HTTPOnBehalfOfJwtAuthenticator();

final DynamicConfigFactory dcf = new DynamicConfigFactory(cr, settings, configPath, localClient, threadPool, cih);
dcf = new DynamicConfigFactory(cr, settings, configPath, localClient, threadPool, cih);
dcf.registerDCFListener(backendRegistry);
dcf.registerDCFListener(compatConfig);
dcf.registerDCFListener(irr);
dcf.registerDCFListener(xffResolver);
dcf.registerDCFListener(evaluator);
dcf.registerDCFListener(securityRestHandler);
dcf.registerDCFListener(acInstance);
if (!(auditLog instanceof NullAuditLog)) {
// Don't register if advanced modules is disabled in which case auditlog is instance of NullAuditLog
dcf.registerDCFListener(auditLog);
Expand Down Expand Up @@ -1203,7 +1203,7 @@ public static class GuiceHolder implements LifecycleComponent {

@Inject
public GuiceHolder(final RepositoriesService repositoriesService,
final TransportService remoteClusterService, IndicesService indicesService, PitService pitService, ExtensionsManager extensionsManager) {
final TransportService remoteClusterService, IndicesService indicesService, PitService pitService, ExtensionsManager extensionsManager) {
GuiceHolder.repositoriesService = repositoriesService;
GuiceHolder.remoteClusterService = remoteClusterService.getRemoteClusterService();
GuiceHolder.indicesService = indicesService;
Expand Down
Loading