Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Support recursive graph bisection based merge policy #17054

Open
jhinch-at-atlassian-com opened this issue Jan 18, 2025 · 2 comments
Labels
enhancement Enhancement or improvement to existing feature or request Search:Performance untriaged

Comments

@jhinch-at-atlassian-com

Is your feature request related to a problem? Please describe

Lucene has support for BPReorderingMergePolicy reorganises segments on merge when possible to group related documents together. However it is currently not available within OpenSearch. Such a merge policy can be an alternative to using a strict sort order on an index to improve performance of common filters on an index by encouraging continuous doc IDs for the same field value.

Describe the solution you'd like

Allow for BPReorderingMergePolicy or equivalent to be configure via index settings. This would need to allow for control over:

  • The fields to partition on
  • The min and max doc frequency
  • Control over the minimum doc thresholds for triggering the reordering

Related component

Search:Performance

Describe alternatives you've considered

No response

Additional context

No response

@jhinch-at-atlassian-com jhinch-at-atlassian-com added enhancement Enhancement or improvement to existing feature or request untriaged labels Jan 18, 2025
@msfroh
Copy link
Collaborator

msfroh commented Jan 21, 2025

@rishabhmaurya -- You had expressed some interest in this in #12257

@rishabhmaurya
Copy link
Contributor

@msfroh yes, thanks for tagging.
I'm in support of exposing this as a setting, maybe something we can add in opensearch 3.0. I will take a look at ways of exposing it post 2.19 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Search:Performance untriaged
Projects
Status: 🆕 New
Development

No branches or pull requests

3 participants