kubernetes.io/cluster/<cluster-name>:shared tag on unmanaged subnets won't be removed on cluster deletion

[wyike]

/kind bug

What steps did you take and what happened:
[A clear and concise description of what the bug is.]

In BYOI https://github.com/kubernetes-sigs/cluster-api-provider-aws/blob/main/docs/book/src/topics/bring-your-own-aws-infrastructure.md case, when deleting subnets https://github.com/kubernetes-sigs/cluster-api-provider-aws/blob/main/pkg/cloud/services/network/subnets.go#L278 in ReconcileDelete cluster, it just skips the unmanaged subnets and won't remove the tags (https://github.com/kubernetes-sigs/cluster-api-provider-aws/blob/main/pkg/cloud/services/network/subnets.go#L506 kubernetes.io/cluster/<cluster-name>:shared) added for this cluster on the unmanaged subnets.

When customer have many clusters created and deleted, the tags are staled on those unmanaged subnets. Even worse, as more and more clusters create and delete, customers will hit tag limitation for these subnets in their environment. They cannot create new clusters with these unmanaged subnets and have to remove these tags manually to workaround.

What did you expect to happen:

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• Cluster-api-provider-aws version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[k8s-ci-robot]

This issue is currently awaiting triage.

If CAPA/CAPI contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[wyike]

/assign

[Ankitasw]

I have an ongoing PR for this opened. This is duplicate of #3948 and its sub-issues