Give users ability to set number_of_shards and number_of_replicas via datastore config #336
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… config benchmark.ini Signed-off-by: Ian Hoang <hoangia@amazon.com>
gkamat
approved these changes
Jun 16, 2023
Comment on lines
+227
to
+230
| self._config = cfg | ||
| self.script_dir = self._config.opts("node", "benchmark.root") | ||
| self._number_of_shards = self._config.opts("reporting", "datastore.number_of_shards", default_value=None, mandatory=False) | ||
| self._number_of_replicas = self._config.opts("reporting", "datastore.number_of_replicas", default_value=None, mandatory=False) |
There was a problem hiding this comment.
This is good -- it is exactly what I thought would be needed to resolve the issue.
sarthakaggarwal97
pushed a commit
to sarthakaggarwal97/opensearch-benchmark
that referenced
this pull request
Jun 28, 2023
… datastore config (opensearch-project#336) Signed-off-by: Ian Hoang <hoangia@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR allows users to set number_of_shards and number_of_replicas for
benchmark-*indices.Why are we reverting the changes of PR #321?
The changes introduced in PR #321 can potentially impact OSB users with varying datastore cluster configurations since it forces their datastore clusters to have
benchmark-metrics-*indices of 10 primary shards and 2 replicas shards. 10 primary shards and 2 replica shards are not suitable for all kinds of OpenSearch clusters. A better solution is to offer flexibility and allow users to override the defaults imposed by OpenSearch (1 primary and 0 replicas according to the documentation). With this PR, users can now do this by setting their own primary and replica shards with the benchmark.ini config. See #330 for more information.Issues Resolved
#330 #188
Testing
See issue #330 for more testing.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.