core: Don't allow noent when resolving pkgcache rev #2545

jlebon · 2021-02-05T19:14:52Z

If we get here, it's that we expect the pkgcache to be there. So don't
allow ENOENT (we weren't even checking for the ENOENT case here, which
shows that this was the intent).

Related: https://bugzilla.redhat.com/show_bug.cgi?id=1925584

If we get here, it's that we expect the pkgcache to be there. So don't allow ENOENT (we weren't even checking for the ENOENT case here, which shows that this was the intent). Related: https://bugzilla.redhat.com/show_bug.cgi?id=1925584

cgwalters · 2021-02-05T19:15:39Z

/lgtm

jlebon · 2021-02-05T19:15:52Z

This alone doesn't fix https://bugzilla.redhat.com/show_bug.cgi?id=1925584 though. It just makes the error more clear. We need to figure out why it's not finding the rev.

To help debug https://bugzilla.redhat.com/show_bug.cgi?id=1925584

cgwalters · 2021-02-06T20:15:09Z

I added another commit here "Make failure to find packages fatal, add more error prefixing" - let's make this the PR to fix this bug.

cgwalters · 2021-02-08T14:22:51Z

Current theory: we're not setting the dbpath macro in all cases; currently playing with

diff --git a/src/app/libmain.cxx b/src/app/libmain.cxx
index 41a361df..2c21aac6 100644
--- a/src/app/libmain.cxx
+++ b/src/app/libmain.cxx
@@ -32,6 +32,7 @@
 
 #include "rpmostree-util.h"
 #include "rpmostree-builtins.h"
+#include "rpmostree-core.h"
 #include "rpmostree-cxxrs.h"
 #include "rpmostree-polkit-agent.h"
 #include "rpmostreemain.h"
@@ -444,6 +445,11 @@ early_main (void)
 
   setlocale (LC_ALL, "");
 
+  /* Set this early on becuase libsolv now uses it and it needs to be correct
+   * for all code in the process.
+   */
+  rpmostreecxx::set_rpm_macro_define ("_dbpath", "/" RPMOSTREE_RPMDB_LOCATION);
+
   /* We don't support /etc/dnf/dnf.conf, so tell libdnf to not look for it. The function
    * name here is misleading; it's not attached to a `DnfContext` object, but instead
    * controls a global var. And it's not just the `DnfContext` that uses it, but e.g.
diff --git a/src/libpriv/rpmostree-core.cxx b/src/libpriv/rpmostree-core.cxx
index c2cebf21..d97e9e23 100644
--- a/src/libpriv/rpmostree-core.cxx
+++ b/src/libpriv/rpmostree-core.cxx
@@ -381,17 +381,6 @@ rpmostree_context_init (RpmOstreeContext *self)
   self->enable_rofiles = TRUE;
 }
 
-static void
-set_rpm_macro_define (const char *key, const char *value)
-{
-  g_autofree char *buf = g_strconcat ("%define ", key, " ", value, NULL);
-  /* Calling expand with %define (ignoring the return
-   * value) is apparently the way to change the global
-   * macro context.
-   */
-  free (rpmExpand (buf, NULL));
-}
-
 RpmOstreeContext *
 rpmostree_context_new_system (OstreeRepo   *repo,
                               GCancellable *cancellable,
@@ -4129,11 +4118,6 @@ rpmostree_context_assemble (RpmOstreeContext      *self,
   g_auto(rpmts) ordering_ts = rpmtsCreate ();
   rpmtsSetRootDir (ordering_ts, dnf_context_get_install_root (dnfctx));
 
-  /* First for the ordering TS, set the dbpath to relative, which will also gain
-   * the root dir.
-   */
-  set_rpm_macro_define ("_dbpath", "/" RPMOSTREE_RPMDB_LOCATION);
-
   /* Determine now if we have an existing rpmdb; whether this is a layering
    * operation or a fresh root. Mostly we use this to change how we render
    * output.
@@ -4619,7 +4603,7 @@ rpmostree_context_assemble (RpmOstreeContext      *self,
     if (!break_hardlinks_at (tmprootfs_dfd, RPMOSTREE_RPMDB_LOCATION, cancellable, error))
       return FALSE;
 
-    set_rpm_macro_define ("_dbpath", rpmdb_abspath);
+    rpmostreecxx::set_rpm_macro_define ("_dbpath", rpmdb_abspath);
   }
 
   g_auto(rpmts) rpmdb_ts = rpmtsCreate ();
@@ -4694,7 +4678,7 @@ rpmostree_context_assemble (RpmOstreeContext      *self,
 
   /* And finally revert the _dbpath setting because libsolv relies on it as well
    * to find the rpmdb and RPM macros are global state. */
-  set_rpm_macro_define ("_dbpath", "/" RPMOSTREE_RPMDB_LOCATION);
+  rpmostreecxx::set_rpm_macro_define ("_dbpath", "/" RPMOSTREE_RPMDB_LOCATION);
 
   /* And now also sanity check the rpmdb */
   if (!skip_sanity_check)
diff --git a/src/libpriv/rpmostree-rpm-util.cxx b/src/libpriv/rpmostree-rpm-util.cxx
index cd62ee98..dd82f704 100644
--- a/src/libpriv/rpmostree-rpm-util.cxx
+++ b/src/libpriv/rpmostree-rpm-util.cxx
@@ -35,6 +35,24 @@
 #include <libglnx.h>
 
 #include <rpm/rpmts.h>
+#include <rpm/rpmmacro.h>
+
+namespace rpmostreecxx {
+
+void
+set_rpm_macro_define (rust::Str key, rust::Str value)
+{
+  std::ostringstream buf;
+  buf << "%define " << key << " " << value;
+  /* Calling expand with %define (ignoring the return
+   * value) is apparently the way to change the global
+   * macro context.
+   */
+  auto bufs = buf.str();
+  free (rpmExpand (bufs.c_str(), NULL));
+}
+
+} /* namespace */
 
 static inline void
 cleanup_rpmtdFreeData (rpmtd *tdp)
diff --git a/src/libpriv/rpmostree-rpm-util.h b/src/libpriv/rpmostree-rpm-util.h
index d7495307..3d8111af 100644
--- a/src/libpriv/rpmostree-rpm-util.h
+++ b/src/libpriv/rpmostree-rpm-util.h
@@ -36,6 +36,7 @@
 // C++ code
 namespace rpmostreecxx {
   std::unique_ptr<std::string> nevra_to_cache_branch(const std::string &nevra);
+  void set_rpm_macro_define (rust::Str key, rust::Str value);
 }
 
 // C code follows

cgwalters · 2021-02-08T14:57:54Z

Hmm, nope.

cgwalters · 2021-02-08T15:00:52Z

We need to figure out why it's not finding the rev.

The reason the rev went missing is the code in master isn't finding the package database, so we end up pruning the pkgcache refs. This PR makes "no packages in root" a fatal error.

jlebon · 2021-02-08T15:05:57Z

OK, I was able to reproduce https://bugzilla.redhat.com/show_bug.cgi?id=1925717 at least. I think what's happening there is that overrides finalization happens before we create the RpmOstreeContext, which sets the macro. So then, we don't see the base packages and think the overrides are inactive.

Hmm, nope.

Hmm, the problem there might be that the RC files get re-read at dnf_context_setup time. I think we need to ship a macro file that overrides the baked one and sets the _dbpath correctly?

jlebon · 2021-02-08T15:38:29Z

Currently testing

diff --git a/src/libpriv/rpmostree-rpm-util.cxx b/src/libpriv/rpmostree-rpm-util.cxx
index cd62ee98..a866faaf 100644
--- a/src/libpriv/rpmostree-rpm-util.cxx
+++ b/src/libpriv/rpmostree-rpm-util.cxx
@@ -842,6 +855,8 @@ get_sack_for_root (int               dfd,
   if (!dnf_sack_setup (sack, 0, error))
     return FALSE;

+  rpmostree_set_rpm_macro_define ("_dbpath", "/" RPMOSTREE_RPMDB_LOCATION);
+
   if (!dnf_sack_load_system_repo (sack, NULL, 0, error))
     return FALSE;

cgwalters · 2021-02-08T15:56:01Z

Ahh right, that looks likely to work.

cgwalters · 2021-02-08T15:57:27Z

OK so CI is passing here again because we have coreos/fedora-coreos-config@714faaf

So...we could merge this PR as is and make a new PR that also pulls in the new libsolv explicitly in CI?

jlebon · 2021-02-08T16:00:38Z

So...we could merge this PR as is and make a new PR that also pulls in the new libsolv explicitly in CI?

Yeah, let's do that. I'd like to see the actual error in CI (the previous failure was wiped out by a Jenkins restart).

/lgtm

openshift-ci-robot · 2021-02-08T16:00:39Z

@jlebon: you cannot LGTM your own PR.

In response to this:

So...we could merge this PR as is and make a new PR that also pulls in the new libsolv explicitly in CI?

Yeah, let's do that. I'd like to see the actual error in CI (the previous failure was wiped out by a Jenkins restart).

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jlebon · 2021-02-08T16:00:44Z

So...we could merge this PR as is and make a new PR that also pulls in the new libsolv explicitly in CI?

Yeah, let's do that. I'd like to see the actual error in CI (the previous failure was wiped out by a Jenkins restart).

/lgtm

openshift-ci-robot · 2021-02-08T16:00:45Z

@jlebon: you cannot LGTM your own PR.

In response to this:

So...we could merge this PR as is and make a new PR that also pulls in the new libsolv explicitly in CI?

Yeah, let's do that. I'd like to see the actual error in CI (the previous failure was wiped out by a Jenkins restart).

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

cgwalters · 2021-02-08T16:05:11Z

/lgtm

cgwalters · 2021-02-08T16:10:51Z

/refresh
/lgtm

openshift-ci-robot · 2021-02-08T16:10:53Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters, jlebon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [cgwalters,jlebon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

cgwalters · 2021-02-08T16:12:30Z

I think we need to ship a macro file that overrides the baked one and sets the _dbpath correctly?

This path is a bit dangerous because we don't want to affect any traditional systems or containers that just happen to have the rpm-ostree package installed. Really the end game here is changing the dbpath to /usr for all Fedora derivatives by default.

In the short term, I think we need to live with hacking in our override dynamically in memory.

Now...maybe this would be cleaner as a top-level libdnf API? Something like:
dnf_global_set_modern_rpmdb_path(); or so?

heyakyra · 2021-02-08T16:38:38Z

This alone doesn't fix https://bugzilla.redhat.com/show_bug.cgi?id=1925584 though.

Once hit, how does one get out of this error situation since upgrades fail?

jlebon · 2021-02-08T18:15:43Z

This alone doesn't fix bugzilla.redhat.com/show_bug.cgi?id=1925584 though.

Once hit, how does one get out of this error situation since upgrades fail?

For now, probably best to rpm-ostree reset -o to remove all your overrides until you upgrade to commit with the reverted libsolv.

jlebon · 2021-02-08T18:18:23Z

I think we need to ship a macro file that overrides the baked one and sets the _dbpath correctly?

This path is a bit dangerous because we don't want to affect any traditional systems or containers that just happen to have the rpm-ostree package installed. Really the end game here is changing the dbpath to /usr for all Fedora derivatives by default.

I meant more injecting the macro at compose time.

Still, I agree we do want to get this fixed by setting the rpmdb path too so there isn't a dependency on the rpm-ostree version on the compose server.

heyakyra · 2021-02-08T18:26:58Z

For now, probably best to rpm-ostree reset -o to remove all your overrides until you upgrade to commit with the reverted libsolv.

This is what I get:

$ rpm-ostree reset -o
error: Bus owner changed, aborting. This likely means the daemon crashed; check logs with `journalctl -xe`.
$ rpm-ostree reset -o
error: Could not activate remote peer.
$ rpm-ostree reset -o
error: Bus owner changed, aborting. This likely means the daemon crashed; check logs with `journalctl -xe`.
$ rpm-ostree reset -o
error: Could not activate remote peer.
$ rpm-ostree reset -o
error: Bus owner changed, aborting. This likely means the daemon crashed; check logs with `journalctl -xe`.
$ rpm-ostree reset -o
error: Could not activate remote peer.
$ rpm-ostree reset -o
error: Bus owner changed, aborting. This likely means the daemon crashed; check logs with `journalctl -xe`.
$ rpm-ostree reset -o
error: Could not activate remote peer.
$ rpm-ostree reset -o
error: Bus owner changed, aborting. This likely means the daemon crashed; check logs with `journalctl -xe`.
$ rpm-ostree reset -o
error: Could not activate remote peer.

jlebon · 2021-02-08T18:35:37Z

This is what I get:

What does journalctl -xe say?

jlebon · 2021-02-08T19:31:38Z

Ugh, I was trying to figure out why I couldn't reproduce this on git master when using make vmsync to hack on it and finally realized/remembered that make vmsync also copies libsolv from the dev env, which invalidated my testing.

So I can confirm that #2545 (comment) doesn't fix it either.

This does work though:

[root@cosa-devsh ~]# echo '%_dbpath /usr/share/rpm' > /etc/rpm/macros.rpm-ostree

And is way more foolproof. So my vote is to just ship that in our OSTree commits and then ratchet it into compose servers before dropping the libsolv pin.

cgwalters · 2021-02-08T20:05:00Z

and then ratchet it into compose servers

Won't that break e.g. coreos-assembler?

cgwalters · 2021-02-08T20:05:54Z

FWIW https://pagure.io/workstation-ostree-config/pull-request/190 is inbound to revert libsolv for Silverblue. FCOS also is reverted. That leaves Fedora IOT and other desktop spins out though for now.

jlebon · 2021-02-08T20:10:32Z

OK, added a commit in #2553 which does this! We'll see if that fixes CI.

Won't that break e.g. coreos-assembler?

Hmm, can you expand?

cgwalters · 2021-02-08T20:37:05Z

When you mentioned adding the macro, I thought you meant add it to our RPM package, which would affect both server and client side. #2553 only changes the client side, which will help a lot but I'm still concerned about the server side.

But to elaborate here, if we added the macro to our package, we'd end up affecting e.g. yum/dnf inside the coreos-assembler container.

OTOH it is kind of tempting to try switching the coreos-assembler container to use /usr/lib/sysimage/rpm to start...

Iolaum · 2021-02-13T16:43:00Z

I am also still having a problem. During researching the issue I found that one can see available commits with:

$ sudo ostree pull fedora:fedora/33/x86_64/silverblue --commit-metadata-only --depth=10
$ sudo ostree log fedora:fedora/33/x86_64/silverblue

I then rebased on latest commit, and performed rpm-ostree reset. Incidentally this is how I 'd suggest affected users upgrade to latest available versions until this is resolved. However even after all this, I am still hitting an error:

$ rpm-ostree upgrade --preview
error: Bus owner changed, aborting. This likely means the daemon crashed; check logs with `journalctl -xe`.

I then run $ journalctl -xe > ./Downloads/journalctlxe.txt to see if there was anything of interest. Given my limited understanding I think the following may be of use.

journalctl logs

Feb 13 18:19:07 laptopname rpm-ostree[2556]: Allowing active client :1.132 (uid 1000)
Feb 13 18:19:07 laptopname rpm-ostree[2556]: client(id:cli dbus:1.132 unit:vte-spawn-507b6b16-01ec-4707-8095-9f462632f70e.scope uid:1000) added; new total=2
Feb 13 18:19:08 laptopname rpm-ostree[2556]: Locked sysroot
Feb 13 18:19:08 laptopname rpm-ostree[2556]: Initiated txn AutomaticUpdateTrigger for client(id:cli dbus:1.132 unit:vte-spawn-507b6b16-01ec-4707-8095-9f462632f70e.scope uid:1000): /org/projectatomic/rpmostree1/fedora
Feb 13 18:19:08 laptopname rpm-ostree[2556]: Process [pid: 7737 uid: 1000 unit: user@1000.service] connected to transaction progress
Feb 13 18:19:10 laptopname systemd[1882]: dbus-:1.2-org.gnome.ControlCenter.SearchProvider@0.service: Succeeded.
░░ Subject: Unit succeeded
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit UNIT has successfully entered the 'dead' state.
Feb 13 18:19:11 laptopname rpm-ostree[2556]: **
Feb 13 18:19:11 laptopname rpm-ostree[2556]: OSTree:ERROR:src/libostree/ostree-repo-pull.c:1637:scan_commit_object: assertion failed: (ref)
Feb 13 18:19:11 laptopname rpm-ostree[2556]: Bail out! OSTree:ERROR:src/libostree/ostree-repo-pull.c:1637:scan_commit_object: assertion failed: (ref)
Feb 13 18:19:11 laptopname audit[2556]: ANOM_ABEND auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:install_t:s0 pid=2556 comm="pool-/usr/bin/r" exe="/usr/bin/rpm-ostree" sig=6 res=1
Feb 13 18:19:11 laptopname systemd[1]: Created slice system-systemd\x2dcoredump.slice.
░░ Subject: A start job for unit system-systemd\x2dcoredump.slice has finished successfully
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit system-systemd\x2dcoredump.slice has finished successfully.
░░ 
░░ The job identifier is 3658.
Feb 13 18:19:11 laptopname audit: BPF prog-id=52 op=LOAD
Feb 13 18:19:11 laptopname audit: BPF prog-id=53 op=LOAD
Feb 13 18:19:11 laptopname audit: BPF prog-id=54 op=LOAD
Feb 13 18:19:11 laptopname systemd[1]: Started Process Core Dump (PID 8211/UID 0).
░░ Subject: A start job for unit systemd-coredump@0-8211-0.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit systemd-coredump@0-8211-0.service has finished successfully.
░░ 
░░ The job identifier is 3657.
Feb 13 18:19:11 laptopname audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-coredump@0-8211-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=?     terminal=? res=success'
Feb 13 18:19:11 laptopname systemd-coredump[8212]: Process 2556 (rpm-ostree) of user 0 dumped core.
                                               
                                               Stack trace of thread 7745:
                                               #0  0x00007f4ab88269d5 raise (libc.so.6 + 0x3d9d5)
                                               #1  0x00007f4ab880f8a4 abort (libc.so.6 + 0x268a4)
                                               #2  0x00007f4ab8c8eb6c g_assertion_message.cold (libglib-2.0.so.0 + 0x1db6c)
                                               #3  0x00007f4ab8ce937f g_assertion_message_expr (libglib-2.0.so.0 + 0x7837f)
                                               #4  0x00007f4ab932b831 idle_worker (libostree-1.so.1 + 0x59831)
                                               #5  0x00007f4ab8cc3b1b g_idle_dispatch (libglib-2.0.so.0 + 0x52b1b)
                                               #6  0x00007f4ab8cc4a5f g_main_context_dispatch (libglib-2.0.so.0 + 0x53a5f)
                                               #7  0x00007f4ab8d16a58 g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xa5a58)
                                               #8  0x00007f4ab8cc1e33 g_main_context_iteration (libglib-2.0.so.0 + 0x50e33)
                                               #9  0x00007f4ab93332c3 ostree_repo_pull_with_options (libostree-1.so.1 + 0x612c3)
                                               #10 0x00005638f724b81c rpmostree_sysroot_upgrader_pull_base (rpm-ostree + 0xac81c)
                                               #11 0x00005638f723cdf9 _ZL26deploy_transaction_executeP22_RpmostreedTransactionP13_GCancellablePP7_GError (rpm-ostree + 0x9ddf9)
                                               #12 0x00005638f71f966f _ZL26transaction_execute_threadP6_GTaskPvS1_P13_GCancellable (rpm-ostree + 0x5a66f)
                                               #13 0x00007f4ab8eaa872 g_task_thread_pool_thread (libgio-2.0.so.0 + 0xad872)
                                               #14 0x00007f4ab8cf4f64 g_thread_pool_thread_proxy.lto_priv.0 (libglib-2.0.so.0 + 0x83f64)
                                               #15 0x00007f4ab8cf23c2 g_thread_proxy (libglib-2.0.so.0 + 0x813c2)
                                               #16 0x00007f4ab89bd3f9 start_thread (libpthread.so.0 + 0x93f9)
                                               #17 0x00007f4ab88eab53 __clone (libc.so.6 + 0x101b53)
                                               
                                               Stack trace of thread 8053:
                                               #0  0x00007f4ab88e555d syscall (libc.so.6 + 0xfc55d)
                                               #1  0x00007f4ab8d114cc g_cond_wait_until (libglib-2.0.so.0 + 0xa04cc)
                                               #2  0x00007f4ab8c95461 g_async_queue_pop_intern_unlocked (libglib-2.0.so.0 + 0x24461)
                                               #3  0x00007f4ab8c955e6 g_async_queue_timeout_pop (libglib-2.0.so.0 + 0x245e6)
                                               #4  0x00007f4ab8cf5009 g_thread_pool_thread_proxy.lto_priv.0 (libglib-2.0.so.0 + 0x84009)
                                               #5  0x00007f4ab8cf23c2 g_thread_proxy (libglib-2.0.so.0 + 0x813c2)
                                               #6  0x00007f4ab89bd3f9 start_thread (libpthread.so.0 + 0x93f9)
                                               #7  0x00007f4ab88eab53 __clone (libc.so.6 + 0x101b53)
                                               
                                               Stack trace of thread 2557:
                                               #0  0x00007f4ab88dfa5f __poll (libc.so.6 + 0xf6a5f)
                                               #1  0x00007f4ab8d169f6 g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xa59f6)
                                               #2  0x00007f4ab8cc1e33 g_main_context_iteration (libglib-2.0.so.0 + 0x50e33)
                                               #3  0x00007f4ab8cc3a51 glib_worker_main (libglib-2.0.so.0 + 0x52a51)
                                               #4  0x00007f4ab8cf23c2 g_thread_proxy (libglib-2.0.so.0 + 0x813c2)
                                               #5  0x00007f4ab89bd3f9 start_thread (libpthread.so.0 + 0x93f9)
                                               #6  0x00007f4ab88eab53 __clone (libc.so.6 + 0x101b53)
                                               
                                               Stack trace of thread 2556:
                                               #0  0x00007f4ab88dfa5f __poll (libc.so.6 + 0xf6a5f)
                                               #1  0x00007f4ab8d169f6 g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xa59f6)
                                               #2  0x00007f4ab8cc1e33 g_main_context_iteration (libglib-2.0.so.0 + 0x50e33)
                                               #3  0x00005638f71f1e8c rpmostreed_daemon_run_until_idle_exit (rpm-ostree + 0x52e8c)
                                               #4  0x00005638f71e9d39 rpmostree_builtin_start_daemon (rpm-ostree + 0x4ad39)
                                               #5  0x00005638f71e07d8 rpmostree_main (rpm-ostree + 0x417d8)
                                               #6  0x00007f4ab88111e2 __libc_start_main (libc.so.6 + 0x281e2)
                                               #7  0x00005638f71dfc2e _start (rpm-ostree + 0x40c2e)
                                               
                                               Stack trace of thread 2558:
                                               #0  0x00007f4ab88dfa5f __poll (libc.so.6 + 0xf6a5f)
                                               #1  0x00007f4ab8d169f6 g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xa59f6)
                                               #2  0x00007f4ab8cc4123 g_main_loop_run (libglib-2.0.so.0 + 0x53123)
                                               #3  0x00007f4ab8f0cf2a gdbus_shared_thread_func.lto_priv.0 (libgio-2.0.so.0 + 0x10ff2a)
                                               #4  0x00007f4ab8cf23c2 g_thread_proxy (libglib-2.0.so.0 + 0x813c2)
                                               #5  0x00007f4ab89bd3f9 start_thread (libpthread.so.0 + 0x93f9)
                                               #6  0x00007f4ab88eab53 __clone (libc.so.6 + 0x101b53)
░░ Subject: Process 2556 (rpm-ostree) dumped core
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ Documentation: man:core(5)
░░ 
░░ Process 2556 (rpm-ostree) crashed and dumped core.
░░ 
░░ This usually indicates a programming error in the crashing program and
░░ should be reported to its vendor as a bug.
Feb 13 18:19:11 laptopname systemd[1]: systemd-coredump@0-8211-0.service: Succeeded.
░░ Subject: Unit succeeded
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit systemd-coredump@0-8211-0.service has successfully entered the 'dead' state.
Feb 13 18:19:11 laptopname audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-coredump@0-8211-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=?     terminal=? res=success'
Feb 13 18:19:11 laptopname audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=rpm-ostreed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Feb 13 18:19:11 laptopname systemd[1]: rpm-ostreed.service: Main process exited, code=killed, status=6/ABRT
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ An ExecStart= process belonging to unit rpm-ostreed.service has exited.
░░ 
░░ The process' exit code is 'killed' and its exit status is 6.
Feb 13 18:19:11 laptopname polkitd[1248]: Unregistered Authentication Agent for unix-process:7737:20124 (system bus name :1.131, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Feb 13 18:19:11 laptopname systemd[1]: rpm-ostreed.service: Failed with result 'signal'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit rpm-ostreed.service has entered the 'failed' state with result 'signal'.
Feb 13 18:19:11 laptopname systemd[1]: rpm-ostreed.service: Consumed 18.235s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit rpm-ostreed.service completed and consumed the indicated resources.
Feb 13 18:19:11 laptopname audit: BPF prog-id=54 op=UNLOAD
Feb 13 18:19:11 laptopname audit: BPF prog-id=53 op=UNLOAD
Feb 13 18:19:11 laptopname audit: BPF prog-id=52 op=UNLOAD

P.S. Latest commit as of the time of posting is:

$ rpm-ostree status -v
State: idle
AutomaticUpdates: disabled
Deployments:
● ostree://fedora:d9ce789c1b1932c58508e17b5bd18721540b57dca09097b9bf26b9014b702c92
                   Version: 33.20210212.0 (2021-02-12T00:49:25Z)
                    Commit: d9ce789c1b1932c58508e17b5bd18721540b57dca09097b9bf26b9014b702c92
                            ├─ repo-0 (2020-10-19T23:27:19Z)
                            ├─ repo-1 (2021-02-12T00:16:46Z)
                            └─ repo-2 (2021-02-12T00:23:48Z)
                    Staged: no
                 StateRoot: fedora
              GPGSignature: 1 signature
                            Signature made Fri 12 Feb 2021 02:49:31 AM EET using RSA key ID 49FD77499570FF31
                            Good signature from "Fedora <fedora-33-primary@fedoraproject.org>"

Update:

I was also able to get this concise error, in case it helps:

$ rpm-ostree upgrade --preview
error: Bus owner changed, aborting. This likely means the daemon crashed; check logs with `journalctl -xe`.
$ systemctl status rpm-ostreed.service
● rpm-ostreed.service - rpm-ostree System Management Daemon
     Loaded: loaded (/usr/lib/systemd/system/rpm-ostreed.service; static)
     Active: failed (Result: signal) since Sat 2021-02-13 20:20:14 EET; 2s ago
       Docs: man:rpm-ostree(1)
    Process: 10192 ExecStart=/usr/bin/rpm-ostree start-daemon (code=killed, signal=ABRT)
   Main PID: 10192 (code=killed, signal=ABRT)
     Status: "clients=1; txn=AutomaticUpdateTrigger caller=:1.180 path=/org/projectatomic/rpmostree1/fedora"
        CPU: 3.823s

Feb 13 20:20:11 laptopname rpm-ostree[10192]: client(id:cli dbus:1.180 unit:vte-spawn-9817f6c9-6e6b-4fe0-8fdc-17117244dffe.scope uid:1000) added; new total=1
Feb 13 20:20:12 laptopname rpm-ostree[10192]: Locked sysroot
Feb 13 20:20:12 laptopname rpm-ostree[10192]: Initiated txn AutomaticUpdateTrigger for client(id:cli dbus:1.180 unit:vte-spawn-9817f6c9-6e6b-4fe0-8fdc-17117244dffe.scope uid:1000): /org/projectatomic/rpmostree1/fedora
Feb 13 20:20:12 laptopname rpm-ostree[10192]: Process [pid: 11285 uid: 1000 unit: user@1000.service] connected to transaction progress
Feb 13 20:20:13 laptopname rpm-ostree[10192]: **
Feb 13 20:20:13 laptopname rpm-ostree[10192]: OSTree:ERROR:src/libostree/ostree-repo-pull.c:1637:scan_commit_object: assertion failed: (ref)
Feb 13 20:20:13 laptopname rpm-ostree[10192]: Bail out! OSTree:ERROR:src/libostree/ostree-repo-pull.c:1637:scan_commit_object: assertion failed: (ref)
Feb 13 20:20:14 laptopname systemd[1]: rpm-ostreed.service: Main process exited, code=killed, status=6/ABRT
Feb 13 20:20:14 laptopname systemd[1]: rpm-ostreed.service: Failed with result 'signal'.
Feb 13 20:20:14 laptopname systemd[1]: rpm-ostreed.service: Consumed 3.823s CPU time.

I don't know if that 's the same issue this PR addresses or a separate one? cc @jlebon

jlebon · 2021-02-16T16:18:59Z

Feb 13 20:20:13 laptopname rpm-ostree[10192]: OSTree:ERROR:src/libostree/ostree-repo-> pull.c:1637:scan_commit_object: assertion failed: (ref)
Feb 13 20:20:13 laptopname rpm-ostree[10192]: Bail out! OSTree:ERROR:src/libostree/ostree-repo-pull.c:1637:scan_commit_object: assertion failed: (ref)

I don't know if that 's the same issue this PR addresses or a separate one? cc @jlebon

Hmm interesting. Yes, this is a different bug. Can you open a new issue in this repo?

openshift-ci-robot requested review from ashcrow and lucab February 5, 2021 19:14

openshift-ci-robot added the approved label Feb 5, 2021

openshift-ci-robot assigned cgwalters Feb 5, 2021

openshift-ci-robot added the lgtm label Feb 5, 2021

cgwalters mentioned this pull request Feb 6, 2021

libsolv-0.7.17-1.fc33 failure tracker #2548

Closed

Make failure to find packages fatal, add more error prefixing

e1cd158

To help debug https://bugzilla.redhat.com/show_bug.cgi?id=1925584

openshift-ci-robot removed the lgtm label Feb 6, 2021

jlebon closed this Feb 8, 2021

jlebon reopened this Feb 8, 2021

cgwalters added the jira for syncing to jira label Feb 8, 2021

openshift-ci-robot added the lgtm label Feb 8, 2021

openshift-merge-robot merged commit 8ddaf0b into coreos:master Feb 8, 2021

Iolaum mentioned this pull request Feb 20, 2021

rpm-ostree upgrade crashes when on checksum refspec #2603

Closed

jlebon deleted the pr/fix-loose-path branch April 23, 2023 23:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

core: Don't allow noent when resolving pkgcache rev #2545

core: Don't allow noent when resolving pkgcache rev #2545

jlebon commented Feb 5, 2021

cgwalters commented Feb 5, 2021

jlebon commented Feb 5, 2021

cgwalters commented Feb 6, 2021

cgwalters commented Feb 8, 2021

cgwalters commented Feb 8, 2021

cgwalters commented Feb 8, 2021

jlebon commented Feb 8, 2021 •

edited

Loading

jlebon commented Feb 8, 2021

cgwalters commented Feb 8, 2021

cgwalters commented Feb 8, 2021

jlebon commented Feb 8, 2021

openshift-ci-robot commented Feb 8, 2021

jlebon commented Feb 8, 2021

openshift-ci-robot commented Feb 8, 2021

cgwalters commented Feb 8, 2021

cgwalters commented Feb 8, 2021

openshift-ci-robot commented Feb 8, 2021

cgwalters commented Feb 8, 2021 •

edited

Loading

heyakyra commented Feb 8, 2021

jlebon commented Feb 8, 2021

jlebon commented Feb 8, 2021

heyakyra commented Feb 8, 2021 •

edited

Loading

jlebon commented Feb 8, 2021

jlebon commented Feb 8, 2021

cgwalters commented Feb 8, 2021

cgwalters commented Feb 8, 2021

jlebon commented Feb 8, 2021

cgwalters commented Feb 8, 2021 •

edited

Loading

Iolaum commented Feb 13, 2021 •

edited

Loading

jlebon commented Feb 16, 2021

core: Don't allow noent when resolving pkgcache rev #2545

core: Don't allow noent when resolving pkgcache rev #2545

Conversation

jlebon commented Feb 5, 2021

cgwalters commented Feb 5, 2021

jlebon commented Feb 5, 2021

cgwalters commented Feb 6, 2021

cgwalters commented Feb 8, 2021

cgwalters commented Feb 8, 2021

cgwalters commented Feb 8, 2021

jlebon commented Feb 8, 2021 • edited Loading

jlebon commented Feb 8, 2021

cgwalters commented Feb 8, 2021

cgwalters commented Feb 8, 2021

jlebon commented Feb 8, 2021

openshift-ci-robot commented Feb 8, 2021

jlebon commented Feb 8, 2021

openshift-ci-robot commented Feb 8, 2021

cgwalters commented Feb 8, 2021

cgwalters commented Feb 8, 2021

openshift-ci-robot commented Feb 8, 2021

cgwalters commented Feb 8, 2021 • edited Loading

heyakyra commented Feb 8, 2021

jlebon commented Feb 8, 2021

jlebon commented Feb 8, 2021

heyakyra commented Feb 8, 2021 • edited Loading

jlebon commented Feb 8, 2021

jlebon commented Feb 8, 2021

cgwalters commented Feb 8, 2021

cgwalters commented Feb 8, 2021

jlebon commented Feb 8, 2021

cgwalters commented Feb 8, 2021 • edited Loading

Iolaum commented Feb 13, 2021 • edited Loading

jlebon commented Feb 16, 2021

jlebon commented Feb 8, 2021 •

edited

Loading

cgwalters commented Feb 8, 2021 •

edited

Loading

heyakyra commented Feb 8, 2021 •

edited

Loading

cgwalters commented Feb 8, 2021 •

edited

Loading

Iolaum commented Feb 13, 2021 •

edited

Loading