Have kops-controller create headless k8s services for dns=none clusters

[rifelpet]

Fixes #16332

Like gossip clusters, dns=none clusters wont be able to resolve these DNS names which causes e2e test failures for OIDC tests. This ensures the OIDC test pods can resolve the DNS names.

Example failure: https://testgrid.k8s.io/kops-grid#kops-grid-calico-amzn2-k26

I0204 15:43:18.460415 1 log.go:198] Get "https://api.internal.e2e-e2e-kops-grid-calico-amzn2-k26.test-cncf-aws.k8s.io/.well-known/openid-configuration": dial tcp: lookup api.internal.e2e-e2e-kops-grid-calico-amzn2-k26.test-cncf-aws.k8s.io on 100.64.0.10:53: no such host

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from rifelpet. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[rifelpet]

/retest

[rifelpet]

/retest

[rifelpet]

/cc @hakman @justinsb

All of our "kops latest" grid jobs are failing because of this OIDC test
https://testgrid.k8s.io/kops-grid#kops-grid-calico-u2004-k28

[hakman]

@rifelpet I believe the fix for the issue is more on the line of changing this condition to include dns=none, instead of the headless service. What do you think?

kops/upup/models/cloudup/resources/addons/coredns.addons.k8s.io/k8s-1.12.yaml.template

Lines 79 to 84 in 3d6599b

	{{- if GossipName }}
	hosts /rootfs/etc/hosts k8s.local {
	ttl 30
	fallthrough
	}
	{{- end }}

[hakman]

@rifelpet Is this still needed?

[rifelpet]

@rifelpet Is this still needed?

To be honest, I'm not sure. This comment #12792 (comment) suggests that this was meant to replace the /etc/hosts mount, but we seem to have both in place. Ideally we can figure out if these headless services are used and remove them if not. If we do want to keep them around, I'm inclined to keep their behavior in sync with the /etc/hosts strategy which would mean merging this.

Perhaps a good topic for office hours.
/kind office-hours

[rifelpet]

in office hours we decided this isn't needed. we will remove the kops-controller headless services at the same time that we eventually remove gossip support