A Caching Split Policy For Real-Time Index Caching [LUCENE-2432]

[asfimport]

(Please note that the feature described in #3499 is required reading for this issue.)

he caching split policy builds on the real-time split policy, providing a caching mechanism that serves to buffer writes to the split index.

As one might expect, this policy delays writes to the index, holding them in the RAM-based splits (created by the real-time policy) instead. When a flush is forced (typically through a commit, but in general, through a split rule), then all un-flushed changes (splits) are written to the super-directory.

There are a couple of reasons why one may wish to use the caching split policy opposed to a RAMDirectory. The latter is primarily used to "load a disk-based index into memory", and suffers from a limitation: "the resulting RAMDirectory instance is fully independent from the original Directory (it is a complete copy). Any subsequent changes to the original Directory will not be visible in the RAMDirectory instance". Furthermore, the RAMDirectory gives little control over when changes are flushed to the disk. The caching split policy attempts to address those limitations by allowing (a) changes to the original directory (which we refer to as the super-directory) are in fact visible to the reader and (b) the user to define exactly when to flush writes to the underlying directory.

---

Migrated from LUCENE-2432 by Karthick Sankarachary
Attachments: LUCENE-2432.patch
Linked issues:

• An Anti-Merging Multi-Directory Indexing Framework [LUCENE-2425] #3499