Prevent installing if previous CRDs are pending to be removed

[davidcassany]

This PR checks there are no previous CRDs having a deletion time stamp set. If so it assumes these are leftovers of previous installations and refuses to install the crds chart.

For the time being it only checks on machineinventories, however it could be expanded if we consider that necessary. For now I'd keep the check to the minimum.

When trying to install crds chart if the machine inventory is already there with a deletionTimestamp set produces an error like the following:

$ helm upgrade --install -n cattle-elemental-system --create-namespace \
    elemental-operator-crds \
    oci://registry.opensuse.org/isv/rancher/elemental/pr/rancher/elemental-operator/pr-553/charts/rancher/elemental-operator-crds-chart                                                                                                                                                                                                                            
Release "elemental-operator-crds" does not exist. Installing it now.                                                                                                                                                                               
Pulled: registry.opensuse.org/isv/rancher/elemental/pr/rancher/elemental-operator/pr-553/charts/rancher/elemental-operator-crds-chart:1.4.0                                                                                                        
Digest: sha256:263aa9a6d54990923ea92aa3c6f8eb1bad4550545d45981cd043087a964d9c52                                                                                                                                                                    
Error: execution error at (elemental-operator-crds/templates/validate-no-pending-deletions.yaml:4:9): CRDs from previous installations are pending to be removed (deletionTimestamp is set). Fully deleting them before (re-)installing is required

In the particular case of #515 to fully clear the offending resource it can be done by removing the finalizers of any MachineInventory resource:

$ kubectl patch machineinventory <machineInventoryName> -n fleet-default --type merge \
    -p '{"metadata": {"finalizers": []}}'

Fixes #515

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (1f53971) 53.65% compared to head (f3a4b85) 53.65%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #553   +/-   ##
=======================================
  Coverage   53.65%   53.65%           
=======================================
  Files          39       39           
  Lines        5690     5690           
=======================================
  Hits         3053     3053           
  Misses       2368     2368           
  Partials      269      269           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[davidcassany]

@fgiudici I think this is a more reasonable solution to #515. I did not want to provide many details or messages as guessing the reason for which a previous uninstall left stuff behind might not be limited to #515 use case only. I am fine as long we prevent the installation and provide a message to review if any already existing CRD is pending to be deleted. How to proceed then could be documented somewhere else if we see fit.

[fgiudici]

Thanks David, this will catch the issue indeed.
I like not having some "auto" clean-up here, as the user will be in a position where resources (and so potentially Elemental clusters) are running and will be cleaned up as soon as the operator chart is installed: one should be aware of what is doing at this point and to have to erase manually the finalizers looks to me really requires the user to double check and be sure of what is doing. Like it, nice patch 👍🏼