Refresh metadata cache after failed package installation

[bajertom]

Fixes #3277

When testing I found out that there is different behavior in dnf5 compared to dnf4, where a mere dnf makecache would be enough, but dnf5 needs the --refresh option as well (or dnf clean metadata or dnf clean expire-cache), see https://bugzilla.redhat.com/show_bug.cgi?id=2324177

Not sure about the Exception being too generic. Should I create a new one, something like InstallationError? And notify the user through logger as to why metadata --refresh is happening?

Pull Request Checklist

• implement the feature

[bajertom]

/packit test

[happz]

@bajertom please, add a unit test for this new operation. It can be fairly trivial, and it would verify those commands work without crashing. See e.g. https://github.com/teemtee/tmt/blob/main/tests/unit/test_package_managers.py#L420 or other unit tests.

To run new unit tests, the following command should work (with dev environment activated, and let's say you'd call the test test_refresh_metadata):

$ hatch -e dev shell
$ pytest -vv --showlocals tests/unit/test_package_managers.py::test_refresh_metadata

[bajertom]

I've added unit tests covering the command. Since there was difference between cases in "Metadata C/cache C/created" in different distros but for the same package manager (yum) I've decided to just stick with "Metadata". Is this enough or should I catch a more specific part of the stdout in this case?

Same goes for apt - it depends on whether the system is up to date or not. It's either (debian test)

Hit:1 http://repolink reponame
Hit:2 http://repolink reponame
50 packages can be upgraded. Run 'apt list --upgradable' to see them.

before apt update or (ubuntu test)

Hit:1 http://repolink reponame
Hit:2 http://repolink reponame
All packages are up to date.

if the system is up to date, so I decided to go for packages as a keyword here.

However I am lost with rpm-ostree packager. On my VM I can run rpm-ostree upgrade --check, but the unit test throws this error

DEBUG    tmt:log.py:742 Run command: podman exec ec41e25298a91ed3dd829df46b889f4605f5b33862696297c3146063dd030f5a /bin/bash -c 'rpm-ostree upgrade --check'
INFO     tmt:log.py:742     cmd: rpm-ostree upgrade --check
DEBUG    tmt:log.py:742 environment
DEBUG    tmt:log.py:742 Command event: 2.91e-06 waiting for process to finish
INFO     tmt:log.py:742     err: System has not been booted with systemd as init system (PID 1). Can't operate.
INFO     tmt:log.py:742     err: Failed to connect to bus: Host is down
INFO     tmt:log.py:742     err: System has not been booted with systemd as init system (PID 1). Can't operate.
INFO     tmt:log.py:742     err: Failed to connect to bus: Host is down
INFO     tmt:log.py:742     err: error: Loading sysroot: exit status: 1
DEBUG    tmt:log.py:742 Command event: 0.1897 waiting for process completed
DEBUG    tmt:log.py:742 Command event: 0.19 waiting for stream readers
DEBUG    tmt:log.py:742 Command event: 0.1902 stdout reader done
DEBUG    tmt:log.py:742 Command event: 0.1903 stderr reader done
DEBUG    tmt:log.py:742 Command returned '1' (failure).

[happz]

I've added unit tests covering the command. Since there was difference between cases in "Metadata C/cache C/created" in different distros but for the same package manager (yum) I've decided to just stick with "Metadata". Is this enough or should I catch a more specific part of the stdout in this case?

LGTM. It's not perfect, which is true for other test cases in that file, we're fine. Success of the command itself is a good indication things went well.

However I am lost with rpm-ostree packager. On my VM I can run rpm-ostree upgrade --check, but the unit test throws this error

DEBUG    tmt:log.py:742 Run command: podman exec ec41e25298a91ed3dd829df46b889f4605f5b33862696297c3146063dd030f5a /bin/bash -c 'rpm-ostree upgrade --check'
INFO     tmt:log.py:742     cmd: rpm-ostree upgrade --check
DEBUG    tmt:log.py:742 environment
DEBUG    tmt:log.py:742 Command event: 2.91e-06 waiting for process to finish
INFO     tmt:log.py:742     err: System has not been booted with systemd as init system (PID 1). Can't operate.
INFO     tmt:log.py:742     err: Failed to connect to bus: Host is down
INFO     tmt:log.py:742     err: System has not been booted with systemd as init system (PID 1). Can't operate.
INFO     tmt:log.py:742     err: Failed to connect to bus: Host is down
INFO     tmt:log.py:742     err: error: Loading sysroot: exit status: 1
DEBUG    tmt:log.py:742 Command event: 0.1897 waiting for process completed
DEBUG    tmt:log.py:742 Command event: 0.19 waiting for stream readers
DEBUG    tmt:log.py:742 Command event: 0.1902 stdout reader done
DEBUG    tmt:log.py:742 Command event: 0.1903 stderr reader done
DEBUG    tmt:log.py:742 Command returned '1' (failure).

Never seen this one before, and there are other rpm-ostree tests where I'd expect this to appear. No idea so far.

Can you rebase the PR? That should make the patch easier to review and play with.

[happz]

@bajertom I tried to play a bit with rpm-ostree, but no luck. @thrix @KwisatzHaderach @BeeGrech any tips, or better approach?

So far I'd propose adding this, basically a no-op with a warning:

diff --git a/tests/unit/test_package_managers.py b/tests/unit/test_package_managers.py
index 0655fcea..b7434ead 100644
--- a/tests/unit/test_package_managers.py
+++ b/tests/unit/test_package_managers.py
@@ -478,6 +478,15 @@ def _parametrize_test_refresh_metadata() -> \
                 r"rpm-ostree upgrade --check", \
                 'Available'
 
+            yield pytest.param(
+                container,
+                package_manager_class,
+                r"rpm-ostree refresh-md --force", \
+                'Available',
+                marks=pytest.mark.skip(reason="refresh-md does not work with how tmt runs ostree container")
+                )
+
+
         elif package_manager_class is tmt.package_managers.apk.Apk:
             yield container, \
                 package_manager_class, \
diff --git a/tmt/package_managers/rpm_ostree.py b/tmt/package_managers/rpm_ostree.py
index 97224202..b37532c3 100644
--- a/tmt/package_managers/rpm_ostree.py
+++ b/tmt/package_managers/rpm_ostree.py
@@ -101,10 +101,20 @@ class RpmOstree(tmt.package_managers.PackageManager):
         return extra_options
 
     def refresh_metadata(self) -> CommandOutput:
-        script = ShellScript(
-            f'{self.command.to_script()} upgrade --check')
-
-        return self.guest.execute(script)
+        self.guest.warn("Metadata refresh is not supported with rpm-ostree.")
+
+        return CommandOutput(stdout=None, stderr=None)
+
+        # The following should work, but it hits some ostree issue:
+        #
+        #   System has not been booted with systemd as init system (PID 1). Can't operate.
+        #   Failed to connect to bus: Host is down
+        #   System has not been booted with systemd as init system (PID 1). Can't operate.
+        #   Failed to connect to bus: Host is down
+        #   error: Loading sysroot: exit status: 1
+        #
+        # script = ShellScript(f'{self.command.to_script()} refresh-md --force')
+        # return self.guest.execute(script)
 
     def install(
             self,

[KwisatzHaderach]

As I already mentioned in my comment thread above, you can refresh with dnf, but it would look weird

I have no problem with the above approach and don't have a better one apart from ^

[BeeGrech]

LGTM, no other ideas aside from what has been mentioned

[happz]

Thanks for the input. Let's make it a no-op for now, with a warning, and we can get back to it later. I guess the way how tmt runs ostree containers could be improved, enabling the feature eventually.

[psss]

Thanks for improving this!

[psss]

Failing test is a network glitch, merging.