A Sharding Split Policy For Load-Sharing Index Writes [LUCENE-2435]

[asfimport]

(Please note that the feature described in #3499 is required reading for this issue.)

The sharding split policy builds on the remoting split policy, treating each sub-index as a shard of the split index. Optionally, the super-index on which the split writer is built may also be considered to be a shard. The motivation for this policy is to allow a given index to be partitioned across multiple nodes (that define a cluster), which helps to (a) load-balance document writes across the nodes in that cluster and (b) serve search requests from a (narrower) set of sub-indices (that the split rule routes the query to).

Unlike other sharding solutions for Lucene, this approach gives users the ability to (a) pick and choose the types of directories underlying the mirror, and (b) add/remove shards on the fly.

---

Migrated from LUCENE-2435 by Karthick Sankarachary
Attachments: LUCENE-2435.patch
Linked issues:

• An Anti-Merging Multi-Directory Indexing Framework [LUCENE-2425] #3499