Try turning off patching in Lucene's PFOR encoding

[mikemccand]

One difference between Lucene and Tantivy is Lucene uses the "patch" FOR, meaning the large values in a block are held out as exceptions so that the remaining values can use a smaller number of bits to encode, a tradeoff of CPU for lower storage space.

Let's try temporarily disabling the patching in Lucene, to match how Tantivy encodes, to see how much of a performance difference that is costing us?

[Tony-X]

++

Is there an easy way to disable it? Based on my code-reading, it does not seem to be configurable

[slow-J]

I forked a Test PostingsFormat, PostingsReader, PostingsWriter, codec and forutil to not do the patching for exceptions (large values).

Done a quick test. Want to re-index and confirm again.
The performance difference for Lucene latency is: -2% in COUNT, -2% in TOP_10_COUNT, -2.07% in TOP_100.
It seems to be most improving Queries of AndHighHigh and HighSloppyPhase. MedTerm and LowTerm have regressions.

Done a benchmark (m6g.4xlarge) with COUNT, TOP_10_COUNT, TOP_100 against Lucene with pfor (from previous test after d1b928c) and without the patching in the PFOR. See the test code here: slow-J/lucene@cd68926 (please let me know if there are any improvements I could have made).

To run the code in the benchmark, I ran gradlew jar and copied the core jar to my benchmark workspace and added this to build.gradle: implementation files("./libs/lucene-core-9.7.1-SNAPSHOT.jar"). I added config.setCodec(new LuceneTestCodec(LuceneTestCodec.Mode.BEST_COMPRESSION)); to BuildIndex.java.

Attaching results:
results.json.zip

[Tony-X]

Did you get a chance to check the index size impact?

[mikemccand]

I also wonder whether removal of patching is more or less impactful on Graviton3?

[slow-J]

Re-indexed again to make sure I have the right correct version built.

with patching turned on (baseline):
10.098 GiB idx

with patching turned off:
10.605 GiB idx

So turning off patching causes a +5.0208% increase in the size of the index.

I'll test Graviton3 when I get a chance.

[slow-J]

Variables

• Graviton3
• JDK 17
• Same test code as before

Candidate:
COUNT, avg: 11,108 μs
TOP_10_COUNT, avg: 18,652 μs
TOP_100, avg: 14,810 μs

Comparing to baseline results from #36

Baseline:
COUNT, avg: 10,883 μs
TOP_10_COUNT, avg: 19,539 μs
TOP_100, avg: not tested

Changes with turning off the patching in PFOR encoding:
COUNT, avg: +2.06744% latency
TOP_10_COUNT, avg: -4.53964% latency

So the improvement to TOP_10_COUNT is doubled vs Graviton 2.
COUNT on the other hand has regressed.

Attaching results.json.
results.json.zip

[mikemccand]

TOP_10_COUNT, avg: -4.53964% latency

This is quite a compelling gain.

Did you turn off patching for both doc and freq? It's odd that COUNT regressed, but it is (should be!) only decoding doc in each postings list.

[slow-J]

I turned off all patching in postings (both doc and freq I believe), see slow-J/lucene@cd68926

I will re-run the graviton3 benchmark to double check will the COUNT regression stand.

[slow-J]

Ran the Graviton 3 benchmark again.

COUNT, avg: 10,822 μs
TOP_10_COUNT, avg: 19,039 μs
TOP_100, avg: 14,914 μs

So this time compared to the control, we have
COUNT: -0.56%
TOP_10_COUNT: -2.559%

So there is some variance between benchmark runs, and no COUNT regression this time.

[slow-J]

The Lucene PR: apache/lucene#12741 has been merged! Resolving!