[RFC] Additional Reranker Implementations Requested

[macohen]

We want to be sure we call ourselves out as per our principles for development. Specifically, because we are releasing this plugin with AWS Kendra as the first integrated reranker, we are unfortunately violating the "A level playing field" principle. We do see the Kendra re-ranking integration as simply the first re-ranker integration and want to add more as soon as possible. In calling this out, we are also asking the community for comments here, PRs on the plugin for additional integrations if interested, as well as cooperative work to add an additional re-ranker so we can refactor the code and generalize it.

[macohen]

The background for this feature is available in #4.

[jmazanec15]

Hi @macohen one use case I can think of would be the ability to re-rank post query phase inside the cluster. For instance, a re-ranking plugin inside the cluster that uses ML Commons to host and execute model could be beneficial.

Would it be possible to make reranking final results after a query phase a plugin? So that implementers would not have to implement the action filter directly?

[macohen]

@jmazanec15 that's the intent of the search-processor so this should fit. How do users do this with ML Commons today? Can you provide some more information so we can verify. We also do have an RFC for the overall architecture of search-processor coming in January so there should be much more to discuss then.

[jmazanec15]

@macohen Currently, I am not aware of any users using ml-commons to do re-ranking today. However, I could see it being extended to this use case. ML Commons provides an extendible interface for model inference workloads. (here is the client). I imagine a re-ranker plugin could be implemented that would use a reranking model hosted with ml-commons to re-rank the top K search results.

[macohen]

Closing. Please comment on #80 if you have ideas for additional reranker implementations.