Reduce DeletesMerges time when softdelete enable

[luyuncheng]

Problem statement

we found when Lucene using in frequently update OR update by query scenarios. it will do many iteration in the following code:

lucene/lucene/core/src/java/org/apache/lucene/index/SoftDeletesRetentionMergePolicy.java

Lines 166 to 176 in 0c29390

	if (scorer != null) {
	DocIdSetIterator iterator = scorer.iterator();
	Bits liveDocs = reader.getLiveDocs();
	int numDeletedDocs = reader.numDeletedDocs();
	while (iterator.nextDoc() != DocIdSetIterator.NO_MORE_DOCS) {
	if (liveDocs.get(iterator.docID()) == false) {
	numDeletedDocs--;
	}
	}
	return numDeletedDocs;
	}

Because SoftDeletesRetentionMergePolicy need query with retentionQuerySupplier AND then filter the retention documents. it is time consuming to iterator docid in frequently updates scenarios

there is flame graph:

we tracing the stack:

it will be called from the stack in update documents:

lucene/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java

Lines 5891 to 5899 in 0c29390

	private void processEvents(boolean triggerMerge) throws IOException {
	if (tragedy.get() == null) {
	eventQueue.processEvents();
	}
	if (triggerMerge) {
	maybeMerge(
	getConfig().getMergePolicy(), MergeTrigger.SEGMENT_FLUSH, UNBOUNDED_MAX_MERGE_SEGMENTS);
	}
	}

and will be called from the stack in merge:

lucene/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java

Lines 2346 to 2361 in 0c29390

	switch (trigger) {
	case GET_READER:
	case COMMIT:
	spec = mergePolicy.findFullFlushMerges(trigger, segmentInfos, this);
	break;
	case ADD_INDEXES:
	throw new IllegalStateException(
	"Merges with ADD_INDEXES trigger should be "
	+ "called from within the addIndexes() API flow");
	case EXPLICIT:
	case FULL_FLUSH:
	case MERGE_FINISHED:
	case SEGMENT_FLUSH:
	case CLOSING:
	default:
	spec = mergePolicy.findMerges(trigger, segmentInfos, this);

Proposal

there is some optimize to reduce the number of calling numDeletesToMerge:

1. feat: soft delete optimize #12339 try to reduce in getSortedBySegmentSize
   when me do merge before we call: getSortedBySegmentSize, and it will duplicate calculate numDeletesToMerge

2. this pr try to reduce in findForcedDeletesMerges
   when we try to find delete size, it will duplicate calculate numDeletesToMerge

In our scenarios, numDeletesToMerge calling make the write latency strike increased, because updatePendingMerges is a synchronized method. we can reduce duplicate calculation