Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

suc-upgrade: allow OS upgrades on hosts with failed systemd units #902

Merged
merged 2 commits into from
Jul 6, 2023

Conversation

fgiudici
Copy link
Member

@fgiudici fgiudici commented Jul 6, 2023

Before update we check the system status as reported by systemd via the systemctl is-system-running cmd, as we want to skip in case the system is shutting down. This would prevent overlapping OS updates.
We rely on the command exit code only, which returns error (EXIT_FAILURE) when the state is anything but "running".

The available states are:

  • "initializing"
  • "starting"
  • "running"
  • "degraded"
  • "maintenance"
  • "stopping"

This patch enables OS upgrades also when the system is in the "degraded" state.
We have to prevent OS upgrades for the "stopping" state.
For the other states it could be arguable whether it makes sense to hold the OS upgrade or not: this patch maintains the previous behavior.

NOTE: fixed also the tab spacing while there. Anyway, kept in a separate commit, actual patch is pretty small.

Fixes #901

fgiudici added 2 commits July 6, 2023 13:03
keep 4 spaces for tabs

Signed-off-by: Francesco Giudici <francesco.giudici@suse.com>
Fixes rancher#901

Signed-off-by: Francesco Giudici <francesco.giudici@suse.com>
@fgiudici fgiudici requested a review from a team as a code owner July 6, 2023 11:51
Copy link
Contributor

@kkaempf kkaempf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤞🏻

@kkaempf
Copy link
Contributor

kkaempf commented Jul 6, 2023

How can we improve testing here ?

/cc @juadk @ldevulder

Copy link
Contributor

@davidcassany davidcassany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@davidcassany
Copy link
Contributor

Just for further background, these sort of checks within script they are all there to prevent concurrent upgrades. IMHO this should be handled at System Upgrade Controller level, there is a stalled issue about it here rancher/system-upgrade-controller#225 . If no concurrent upgrades are prevented at a higher level I'd say we can get rid of some the checks within the upgrade script, like the one causing the issue here.

@davidcassany davidcassany merged commit ca96c83 into rancher:main Jul 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

OS upgrades not allowed when systemctl is-system-running reports a degraded state
3 participants