Hash the sysroot files metadata #16

saethlin · 2024-02-10T22:17:29Z

Worst-case (filesystem on a USB drive with the cache recently purged) time to hash the whole sysroot file contents looks like ~1 second. That seems a bit steep. Worst-case time to hash just the modification time and size of every file looks like 225 ms.

That's not perfect; a modification to a file that doesn't change size or modification time will cause spurious rebuilds from saving a file in an editor without changes, and if a file write happens to not change size or modification time we may fail to rebuild. Still a huge upgrade from nothing.

RalfJung · 2024-02-11T11:30:10Z

Some experimentation on my own machine says it takes ~22 ms to hash the sysroot sources, and ~8 ms to stat everything in it.

I assume that was in a pretty fast SSD though. I wonder how it fares on slower storage...
Do you happen to have a spinning disk, e.g. a USB drive, that you can test this on?

Cargo.lock

src/lib.rs

saethlin · 2024-02-11T16:15:01Z

I assume that was in a pretty fast SSD though. I wonder how it fares on slower storage...
Do you happen to have a spinning disk, e.g. a USB drive, that you can test this on?

With a sync && echo 3 > /proc/sys/vm/drop_caches to purge all filesystem/disk caching, it's ~600 ms on fast storage, and ~1000 ms on a USB drive. After the first run, the USB drive is ~28 ms.

Trying to come up with a realistic workload, I ran MIRI_LIB_SRC=/home/ben/usb/library cargo +miri miri setup then let crater-at-home run for a few minutes. It's a rather disk-heavy workload, and I'm hoping this puts the cache into a more realistic state. That time is ~120 ms.

RalfJung · 2024-02-11T16:52:09Z

Okay, so that is noticeable, but it's a worst-case in terms of disk performance.

Still, hashing the file sizes and mtimes should almost always be sufficient, shouldn't it?

saethlin · 2024-02-11T17:16:07Z

Well, it'll surely be a lot better than what we have now, and probably better than what Cargo does. And the worst-case looks like 107 ms (NVMe) or 225 ms (USB) which is an advantage.

RalfJung

Looks good, let's just add a few for comments.

src/lib.rs

RalfJung · 2024-02-11T19:24:32Z

Thanks. :-)

RalfJung · 2024-02-23T17:45:43Z

src/lib.rs

@@ -84,6 +85,29 @@ fn make_writeable(p: &Path) -> Result<()> {
    Ok(())
 }

+/// Hash the metadata and size of every file in a directory, recursively.
+pub fn hash_recursive(path: &Path, hasher: &mut DefaultHasher) -> Result<()> {


Is there a specific reason you made this public? Doesn't seem to me like we should expose this function from this library.

Nope. I think I just didn't notice.

we properly rebuild the sysroot now when MIRI_LIB_SRC contents change Thanks to RalfJung/rustc-build-sysroot#16

saethlin force-pushed the hash-the-sysroot branch from 2b20750 to 2dd67fc Compare February 10, 2024 22:22