Investigation/ potential enhancement: Test/ PoC multi-cluster ingress with Submariner

[srampal]

Ideally Submariner should support and complement Multi-cluster ingress load balancing use cases. This issue is a tracker to research and test the current functional readiness of Submariner + multi-cluster ingress in various use cases. Based on the results of this testing if gaps are found, additional issues may be needed in future.:

Why is this needed:
Upstream k8s community is proposing architectures that allow implementations of the MCS api to support and complement multi-cluster ingress load balancing via functions the Gateway Ingress api as well as Istio Ingress Gateway. We need to understand what is the status of this functionality when used with Submariner.

[srampal]

@dfarrell07 You can assign this to me .. I am working on testing this functionality.

[srampal]

Test 1

1. Tried testing this with Istio and its implementation of the K8s Gateway API on two Kind clusters. It looks like the ServiceImport BackendRef is not supported on Istio's implementation of K8s Gateway API currently so this is not ready to get tested in that manner. However I did determine that using the Istio IngressGateway API instead, one can achieve a form of multi-cluster ingress load balancing by simply using the Service as backend and using Submariner purely for L3 interconnect and with non-overlapping pod and service CIDR.

Test 2

1. A potential alternate test is to use Gateway API on Google cloud. I have requested getting a Google cloud account and will try that out once I have the account.

[srampal]

1. Retried this using istio 0.13.2 which is supposed to have the fix for istio support of gateway api including support for backendrefs that are serviceimports.

2. However ran into another issue which is caused by a Submariner limitation that is already known and has a tracking issue open. The issue is that in order to support multi-cluster ingress proxying/ load balancing, the Gateway api manifest refers to a single backendRef for service import that represents the aggregate ServiceImport and is created in the namespace of the service itself. However Submarine currently creates multiple ServiceImport objects, with separate names and also not in the same namespace as the actual service itself. Hence this limitation (which is a violation of the upstream MCS api specification) is additionally also preventing Submariner from working with Multi-Cluster ingress load balancing using the Gateway api.

3. Also added a note to that issue indicating it is blocking this scenario.

[stale]

This issue has been automatically marked as stale because it has not had activity for 60 days. It will be closed if no further activity occurs. Please make a comment if this issue/pr is still valid. Thank you for your contributions.