Responsibility to keep VMs running?

[simplysoft]

We did observe that when VM is stopped (e.g. the PVE node restarted), the VMs remain in the stopped state.
With the current implementation, the ProxmoxMachine will have status stopped but CAPI Machine are in phase Running
On a first glance we did not find concrete information in the cluster api documentation / contract what should be done about this.

The mismatch could be addressed on 3 different layers:

• Infrastructure Layer: It is responsibility of the underlying infrastructure to ensure the VM is running. In this case proxmox is responsible and thus proxmox provider should set the autostart on boot flag. But this would only covert the boot case.
• CAPI Provider Layer: Part of a reconciliation loop, ensure that the machine is running, and if stopped, start it.
• CAPI Layer: It seems like the remediate concept of CAPI could also be one way to solve this issue. Essentially meaning that it would replace the offline machine with a new one

Would be great to hear some ideas / suggestion on how to address this in a proper way

[sp-yduck]

I think usually CAPI infra provider does not ensure machine is running. at least there is no specification for such behavior in Cluster API book so it depends on each provider I would say. also as you mentioned, CAPI MachineHealthCheck just replaces a vm to new one not keeps it running. so the responsibility to keep vms running is on infra side.