Undocumented breaking change from 0.38.0 -> 0.44.0

[chrisjohnson]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

Prior to the most recent upgrade we ran consul 1.10.4 on consul-k8s helm chart 0.38.0. Our datacenter name is az-east

This consul cluster was federated (using traditional mesh gateway-based WAN federation) to another cluster -- the other cluster being the primary mi. mi is run on bare-metal VMs

We configured our az-east cluster to join the other (mi) cluster using this set of values:

server:
  extraConfig: |
    {
      "primary_datacenter": "mi",
      "primary_gateways": ["mesh-gateway-dns-name.internaldns.net:8443"]
          }
    }

After upgrading to consul-k8s 0.44.0 with consul image 1.12.6, we observed the following error in the acl-init pod logs:

cannot find auth method with name "consul-k8s-component-auth-method-az-east"

Checking this issue we deduced that there is now first-class support for specifying the primary datacenter and gateways in helm values. We also figured out that we were required to pass in the k8s API server URL. We updated our values to pass these values in as

global:
  federation: 
    enabled: true
    primaryDatacenter: mi
    primaryGateways: ["mitplatformconsulmeshgateway1.cmmint.net:8443"]
    k8sAuthMethodHost: https://foo-bar-baz.hcp.eastus.azmk8s.io:443

Expectations

We didn't know about these required changes when looking at the CHANGELOG so we proceded with the upgrade without knowing it wouldn't work. I think the CHANGELOG should have called out these changes.

[chrisjohnson]

https://github.com/hashicorp/consul-k8s/blob/v0.44.0/CHANGELOG.md#0420-april-04-2022

Looks like I'm just blind :)