You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Opensearch always reads terms index from disk using mmap which can impact performance significantly, especially when the terms index size is small.
Describe the solution you'd like
There is index level setting for id field only index.force_memory_id_terms_dictionary which has limitations (only id field) and buggy (same named field across indices affected). We should have cluster level setting to keep terms indices in memory on heap. Since, index level setting anyways has wider scope, this setting should be removed or scoped at cluster level than index
Describe alternatives you've considered
I have thought about and considered below alternative:
Move terms index on heap by default always - This used to be the case earlier and resulted in JVM sizing issues for many customers
Disable off heap by default and add setting to move it off heap - This is viable option though more customers prefer to have it off heap than on heap. Hence, I prefer the proposal than this alternative
The text was updated successfully, but these errors were encountered:
The performance issue was observed due to regression as part of this PR: elastic/elasticsearch#36668. It was rolled back (to mmapfs default) for couple of weeks as part of this PR: elastic/elasticsearch#38157. Finally, the issue was identified to be due to missing cfs files, and added back as part of PR: elastic/elasticsearch#38940.
Although we have the correct set of files today, should default to mmapfs and have explicit list for niofs within hybridfs
Is your feature request related to a problem? Please describe.
Opensearch always reads terms index from disk using mmap which can impact performance significantly, especially when the terms index size is small.
Describe the solution you'd like
There is index level setting for id field only
index.force_memory_id_terms_dictionary
which has limitations (only id field) and buggy (same named field across indices affected). We should have cluster level setting to keep terms indices in memory on heap. Since, index level setting anyways has wider scope, this setting should be removed or scoped at cluster level than indexDescribe alternatives you've considered
I have thought about and considered below alternative:
The text was updated successfully, but these errors were encountered: