Support cluster-scoped resources when using watchNamespaces

[rainest]

What this PR does / why we need it:

When watchNamespaces is non-empty, create a ClusterRole that contains
cluster-scoped resources only.

Remove endpoints from the cluster resource template. They appear to have
been included by mistake.

Configure one of the CI values to use watchNamespaces.

Which issue this PR fixes

(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

• fixes New installs of v2.8.2 missing RBAC permissions for IngressClass #607

When watchNamespaces was originally developed, the only cluster-scoped resource we had was KongClusterPlugin, and KongPlugin could substitute for it with an acceptable loss of functionality. Since, CNCF has introduced IngressClass and GatewayClass, which have no namespace-scoped equivalent. We need to read these to properly implement the v1 Ingress and vAny Gateway specs.

Special notes for your reviewer:

Ready for review, but do not merge pending discussion with product and field team stakeholders in KUBE-61.

Checklist

[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

• PR is based off the current tip of the main branch.
• Changes are documented under the "Unreleased" header in CHANGELOG.md
• Commits follow the Kong commit message guidelines

[shaneutt]

👍

[jasiek-zywczak]

the behaviour of charts is (almost) correct, when we are running kong in the cluster we should not require cluster role, the problem is that chart is not disable watching the ingressClass in that case
#648

[rainest]

@jasiek-zywczak while the controller should indeed disable that lookup when disabling the controller, the plan is to continue with this change (we haven't received objections from the stakeholders we reached out to), which should resolve your issue, correct? The install would have permissions for IngressClass, and the attempts to list them would not fail. Did you have any concerns with this approach?

[jasiek-zywczak]

It is a matter of being consequent. Why disabling access to cluster plugins then? I mean, in case watch namespace is set?
And btw chart's docs precisely mention that access to cluster objects should be disabled.
When I set that I want to watch a particular namespace it means I don't want to watch, in some cases, the whole cluster.

[rainest]

Why disabling access to cluster plugins then? I mean, in case watch namespace is set?

That is a good catch--this change should indeed remove the KongClusterPlugin disable as well. I've updated this to do so and document the new behavior.

When I set that I want to watch a particular namespace it means I don't want to watch, in some cases, the whole cluster.

That will be the case for namespaced resources. The library we use to implement watchNamespaces has two modes: it can add cluster-level watches for everything or create separate watches for each namespace and a cluster-level watch for cluster-scoped resources only. It's not possible to watch cluster-scoped resources at anything other than the cluster level, so trying to impose a namespace-restricted model on them doesn't make sense.