osd/PG.cc: handle removal of pgmeta object

In 7f04700, we made the pg removal code
much more efficient. But it started marking the pgmeta object as an unexpected
onode, which in reality is expected to be removed after all the other objects.

This behavior is very easily reproducible in a vstart cluster:

ceph osd pool create test 1 1
rados -p test bench 10 write --no-cleanup
ceph osd pool delete test test  --yes-i-really-really-mean-it

Before this patch:

"do_delete_work additional unexpected onode list (new onodes has appeared
since PG removal started[#2:00000000::::head#]" seen in the OSD logs.

After this patch:

"do_delete_work removing pgmeta object #2:00000000::::head#" is seen.

Related to:https://tracker.ceph.com/issues/50466
Signed-off-by: Neha Ojha <nojha@redhat.com>

Manually applied 0e917f1, because
nautilus does not have do_delete_work.

Signed-off-by: Neha Ojha <nojha@redhat.com>

src/osd/PG.cc

@@ -7154,10 +7154,14 @@ ghobject_t PG::_delete_some(ObjectStore::Transaction *t,
       max,
       &olist,
       &next);
-    if (!olist.empty()) {
-      dout(0) << __func__ << " additional unexpected onode list"
-              <<" (new onodes has appeared since PG removal started"
-              << olist << dendl;
+    for (auto& oid : olist) {
+      if (oid == pgmeta_oid) {
+        dout(20) << __func__ << " removing pgmeta object " << oid << dendl;
+      } else {
+        dout(0) << __func__ << " additional unexpected onode"
+                <<" new onode has appeared since PG removal started"
+                << oid << dendl;
+      }
     }
   }