Unable to call Kubernetes API from pod in cluster running on AWS EKS

[tleef]

In my pod the KUBERNETES_SERVICE_HOST var is set to 172.20.0.1 and KUBERNETES_SERVICE_PORT_HTTPS is set to 443 by default.

When I try to call the API using the FLAMEK8sBackend.K8sClient I get the following error

{:error,
 "failed GET https://172.20.0.1:443/api/v1/namespaces/vial/pods/ with {:failed_connect, [{:to_address, {~c\"172.20.0.1\", 443}}, {:inet, [:inet], {:tls_alert, {:handshake_failure, ~c\"TLS client: In state wait_cert at ssl_handshake.erl:2140 generated CLIENT ALERT: Fatal - Handshake Failure\\n {bad_cert,hostname_check_failed}\"}}}]} [{~c\"Authorization\", ~c\"Bearer <token omitted>"}]"}

As I was trouble shooting this issue, one of the things I tried was reassigning the KUBERNETES_SERVICE_HOST to https://<id-omitted>.sk1.us-west-1.eks.amazonaws.com, the endpoint I use when connecting to the API server externally. This worked!

So the issue appears to be the default KUBERNETES_SERVICE_HOST setting of 172.20.0.1 on EKS clusters.

It's easy enough to workaround this issue but I'm unsure if this is a common problem for EKS users or if I am doing something wrong.

[mruoss]

What OTP version are you running on? Would it be possible for you to upgrade to 27?

See: erlang/otp#7968

[mruoss]

But yeah. After removing req, I might have to add back the fix for older versions of OTP

[tleef]

I updated to OTP 27.0.1 and it's working now. Thank you!

[mruoss]

Reopening this as a reminder to add back the workaround for older versions removed in 1123909