Drop evicted entries immediately

[tatsuya6502]

Changes

This PR makes sync and future caches to drop the value part of evicted entries immediately. It calls the flush method of crossbeam_epoch::Guard when necessary.

There is no change in the public API.

Background

moka::cht uses crossbeam-epoch's defer_unchecked method, which takes a closure as the argument, to drop evicted entries. However, to improve the throughput, crossbeam-epoch will stash these deferred functions away in thread local storage until it gets an enough number of them (62 for each thread). They do not get executed until they are pushed to the global queue.

So, if client has N threads calling Moka's write methods (insert, get_with, invalidate, etc.), crossbeam-epoch can delay dropping evicted/invalidated entries up to N * (62 - 1) entries.

This PR ensures that these deferred functions will be executed as soon as possible by calling the flush method of crossbeam_epoch::Guard in a timely manner.

[tatsuya6502]

I ran some performance tests using mokabench with ARC-S3 workload.

• Result: Moka-v0.9.3-overheads-2022-07-24-linux-x86_64.pdf
• Test environment: Linux x86_64 on Intel Core i7 12700F (20 × threads = 8 × P-cores + 4 x E-cores)

As expected, this change added some performance overheads:

1. For "Sync Segmented" tests where all processor cores were saturated (100% CPU utilization), duration of the benchmarks got ~25% or ~10% longer than the one before applying this change:
   • a. Duration was ~25% longer when the cache hit ratio was ~10%.
   • b. Duration was ~10% longer when the cache hit ratio was ~65%.
2. For "Sync Cache" and "Async Cache" tests where CPU utilization was ~60%, duration did not get longer.

I think the overheads will be acceptable for real-world workloads. Those workloads will be much lighter than both 1. and 2, so the overhead will be much smaller.

Here is a summary of when flush will be called:

	When flush will be called	Per entry overhead
A	get_with or try_get_with on a non existing key	High
B	insert over an existing key	High
C	invalidate an existing key	High
D	invalidate_all	Low (Called only once after processing many entries)
E	Batch eviction of many entries	Low (Same as above)
F	Hash table expanding or shrinking	Low (Same as above)

The above benchmark runs all flush cases except B and D. The benchmark 1-a runs A more often than 1-b due to low hit ratio. This is why 1-a had larger overhead than 1-b.

[tatsuya6502]

By some reason, this PR did not pick one last commit: b2c244d

Merging anyway.