Ensuring basic authentication provider in managed environments

[peterschretlen]

Managed environments like Elasticsearch Service, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK) use file realm authentication for administrative “service accounts” in effect. These service accounts are used to check and maintain the state of the cluster. File realm is used over other realms because it allows the cluster to be administered regardless of cluster state - say .security index is corrupted or if external authentication is not available.

This extends to Kibana as well - for example API calls are made to check if a cluster can be upgraded.

In Kibana the problem we occasionally encounter is the removal of the basic provider in kibana.yml. While multiple providers can be supported ( for example: https://www.elastic.co/guide/en/kibana/current/kibana-authentication.html#_saml_and_basic_authentication ), basic is sometimes removed in order to remove the login form when using SSO provider like SAML or OIDC. When this happens, customers unknowingly cut off Kibana from the management of the environment.

In managed environments we’d like to use the basic auth provider in order to give these service accounts access to Kibana, even when basic is not in the providers list. Perhaps this would allow the basic auth provider but exclude the login form, in order to support SSO case where the login form isn’t wanted.

[elasticmachine]

Pinging @elastic/kibana-security (Team:Security)

[peterschretlen]

cc @bleskes

[azasypkin]

Feels like the simplest solution would be just having something similar to Hapi's optional authentication mode for a particular route (currently we have only none or required modes):

1. Kibana receives request and tries to authenticate it
2. If it can, then this auth information will be used to query ES under the hood (e.g. when basic is enabled or when we hit this API on behalf of the already authenticated user)
3. If it cannot, there are two options:
   3.1 If authentication cannot be handled, aka none of the providers can extract auth information from the request (e.g. when basic isn't enabled, but request contains Authorization: Basic xxx) then we'll call the API handler anyway and it can just forward whatever headers request has to ES
   3.2 If authentication fails (e.g. when basic is enabled, but Authorization: Basic xxx isn't accepted) then we'll return 401.

[azasypkin]

Update: In this PR we added a way to enable and configure desired HTTP authentication mechanisms (aka authentication with Authorization HTTP header) via xpack.security.authc.http. In 7.7+ Cloud can use the following configuration to enable authentication via Authorization: Basic xxxx (without login form) irrespective to what providers are currently set in xpack.security.authc.providers:

# default is xpack.security.authc.http.schemes: [apikey], 
# so we need to preserve `apikey`.
xpack.security.authc.http.schemes: [apikey, basic] 

Would that work for the Cloud use case @peterschretlen ?

[kobelb]

Is it safe to assume that Cloud would need to add basic to the end of the xpack.security.authc.http.schemes array if the user has already specified this value?

[azasypkin]

Is it safe to assume that Cloud would need to add basic to the end of the xpack.security.authc.http.schemes array if the user has already specified this value?

If you are you asking whether Kibana would choke if we end up with duplicates in this field then the answer is it won't - it's supported scenario, we automatically de-duplicate these values.

[peterschretlen]

Would that work for the Cloud use case @peterschretlen ?

@azasypkin yes I think that will work nicely for cloud, thank you! cc @nachogiljaldo to confirm.

From what I can see by setting to xpack.security.authc.http.schemes: [apikey, basic] starting in 7.7 (the default appears to be used now) we should be able to authenticate regardless of xpack.security.authc.providers value.

[nachogiljaldo]

So, just to confirm, on Cloud, we should add:
xpack.security.authc.http.schemes: [apikey, basic]

and that would work for us now on any case, disregard SAML etc.?

[azasypkin]

xpack.security.authc.http.schemes: [apikey, basic]
and that would work for us now on any case, disregard SAML etc.?

I suppose yes, assuming I understand your use case correctly. But I'm not sure how you merge user overrides with the settings you specify in Cloud. If for arrays like xpack.security.authc.http.schemes you can just append your own entries to the start/end - then we're good.

Just to reiterate, when you configure xpack.security.authc.http.schemes: [${scheme1}, ${scheme2}] that basically means that Kibana should accept requests with Authorization ${scheme1} .... and Authorization ${scheme2} ...., no matter what user set in xpack.security.authc.providers (or deprecated xpack.security.authProviders) unless HTTP authentication is explicitly disabled with xpack.security.authc.http.enabled: false (technically we can even blacklist this setting in Cloud admin console).

[nachogiljaldo]

@azasypkin the way we merge overrides is by replacing the original node (i.e. if we default xpack.security.authc.http.schemes to [apikey,basic] and the user would override it, we would have only the last value.

I am wondering, though, whether we should whitelist it at all (which would prevent the overrides). Is it needed to override it to make SAML work?

[peterschretlen]

From what I can see we don't set any xpack.security.authc.http.* settings in cloud right now, and we do not whitelist any of these values so there is no risk of override (today).

I am wondering, though, whether we should whitelist it at all (which would prevent the overrides). Is it needed to override it to make SAML work?

Editing the value of xpack.security.authc.http.schemes is not needed for SAML, the default is sufficient. I think we want to set xpack.security.authc.http.schemes to [apikey,basic] and then explicitly blacklist settings like xpack.security.authc.http.schemes and xpack.security.authc.http.enabled, so we don't set ourselves up for the same problem in the future where management API calls can be locked out.

[azasypkin]

++ to what Peter said, we can blacklist xpack.security.authc.http completely for now. If in the future we have a use case for this setting to be whitelisted in the Cloud we can think about relaxing this restriction.

[azasypkin]

Closing this one since https://github.com/elastic/cloud/issues/52454 is resolved. Feel free to reopen if anything else is needed from the Kibana side.