Allocate docIds in a way that is monotonic with some fast field

[fulmicoton]

Sorting documents by a given field can open the door to various optimization

• sort by score related metric
• group by/collapse by a sorted field
• locality
• range queries

[druzn3k]

I want to work on this (as soon as the other PR is closed, of course), could you give me some pointer for where to start?

[hwchen]

@fulmicoton can you explain a little how this would differ from the top docs by field collector?

https://docs.rs/tantivy/0.8.1/tantivy/collector/struct.TopDocsByField.html

[fulmicoton]

@hwchen Sorry my previous message was misleading. I removed it.

Right now documents get an internal document id. This document id is currenlty defined by the order in which document are added in the segment.

There are a bunch of optimization that can be done if someone choose a better order for these.

For instance, ordering by decreasing page rank makes it possible to get a nice SERP while scoring only the first Q docs.

This is a very interesting feature for many business, but let's not consider it until somebody actually has the need and want to use tantivy.

[shikhar]

Is this a dupe of #1014, which got closed with #1026? cc @PSeitz

[fulmicoton]

Correct . Thank you @shikhar