Submariner not working with K8s Gateway API based Multi-Cluster Ingress load balancing

[srampal]

I am testing the use of K8s gateway api along with Submariner for Multi-Cluster ingress load balancing (as part of this investigation . The approach used for testing was based on using Istio-Envoy's implementation of the Kuebrnetes Gateway API with MCS api extensions support (as suggested here) in combination with Submariner as the implementor of the MCS api. This was done in a Kind environment with 2 Submariner clusters, Istio and Istio-envoy implementation of the k8s gateway api.

What happened:
The observation was that ingress load balancing vie the Istio gateway api did not work (traffic not received by backend pods of the Ingress HttpRoute service).

What you expected to happen:
It was expected that Submariner should support Multi-cluster ingress Gateway API based L7 load balancing.

How to reproduce it (as minimally and precisely as possible):
Will occur with any implementation of K8s gateway api when combined with Submariner for the MCS api implementation.

Anything else we need to know?:
The root cause of this has been diagnosed as being due to the known existing issue namely this one. It is being written up as a separate issue since the external behavior to the user is different (the original root cause issue has nothing to do with k8s gateway api and multi-cluster ingress) so this way someone running into these symptoms can easily find and track the issue.

Environment:

• Diagnose information (use subctl diagnose all):
• Gather information (use subctl gather):
• Cloud provider or hardware configuration:
• Install tools:
• Others:

This information is not considered necessary since the issue has already been root caused to another existing issue and is being documented here for tracking purposes.

[srampal]

on a related note, there is also an upstream istio bug that prevents this from working even if the Submariner Service aggregation restriction is fixed. This has been opened as the following issue istio/istio#38529

[sridhargaddam]

Thanks for reporting the issue @srampal. As the fix needs to be done in Lighthouse component, moving it to Lighthouse repo.

[dfarrell07]

@yboaron volunteered to look in to this

[dfarrell07]

@skitt gave some background making it clear that this is critical for Submariner in the mid-term

[srampal]

Just to clarify, the root cause of this issue is the known existing issue of not aggregating ServiceImport objects.

#214

Once that issue is fixed it may also resolve this issue. Hence requesting prioritization of fix for that issue (214) .

[stale]

This issue has been automatically marked as stale because it has not had activity for 60 days. It will be closed if no further activity occurs. Please make a comment if this issue/pr is still valid. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had activity for 60 days. It will be closed if no further activity occurs. Please make a comment if this issue/pr is still valid. Thank you for your contributions.

[dfarrell07]

Discussing with the team, this still seems useful, like something we should eventually do. The gateway routing part will need to be handled in a kubeproxy/CNI eventually, and we'll need this to better be able to provide only-Lighthouse without Submariner connectivity eventually.

[stale]

This issue has been automatically marked as stale because it has not had activity for 60 days. It will be closed if no further activity occurs. Please make a comment if this issue/pr is still valid. Thank you for your contributions.

[dfarrell07]

If we end up going down this path, it'll be driven by something like service mesh support and we'll know we need to make this change at that point. For now let's close this.