DLPX-78776 estat zpl may warn about missing more events than were obs…

[brad-lewis]

This script has been reporting warnings like this for awhile:

$ sudo estat zpl 60
12/14/21 - 01:23:07 UTC

 Tracing enabled... Hit Ctrl-C to end.
WARNING: probe r_zfs_read_bcc_56749 missed 7846 of 2037 events
WARNING: probe r_zfs_write_bcc_56749 missed 12893 of 3297 events

Investigations have found that the issue is in the the code to determine the spa_name from the inode. The objset field is not always set on entering zfs_write or zfs_read. In the most recent version of zfs this appears to be the case all the time. In order to get some data, the check for the pool was pulled out though this results in the rpool operations be included by default. Also there is no way to check for SYNC_ALWAYS. This seems better than no data.

Some other cleanup was done, including removing of the tracking sync vs async reads (whatever that was supposed to mean).

[sebroy]

Removing the pool selection doesn't feel like the right solution, the reason being that the root disk often has different performance characteristics than disks in the zpl; so it would be hard to understand why there might be latency outliers (is it a performance issue on domain0, or is it just what we'd expect from rpool?). In Azure, for example, the root disk is on a fully virtualized I/O controller that performs rather poorly.

Are there other possible approaches?

[brad-lewis]

Looking harder for another approach, we could check the spa name somewhere further down the call stack. For the writes, zil_log_write() should be called at least once for the normal path and the zilog structure gives access to the spa to get the name and the os to check for SYNC_ALWAYS.

I read seems to have multiple paths. Thought it might be worth discussing before further investigation.