[Remote Store] Add support to create index with remote store by default #6932

linuxpi · 2023-04-02T23:02:54Z

Description

Allows clusters to be configured for all indices to be Remote Store backed by default. The user doesn't need to specify the settings for remote store and can create an index without specifying Remote Store settings. The cluster will automatically use the default settings and create the index with Remote Store enabled.

Issues Resolved

[List any issues this PR will resolve]

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2023-04-02T23:26:41Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/13246/
CommitID: 41b9ae8
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-04-02T23:55:58Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/13247/
CommitID: 065c8aa

codecov-commenter · 2023-04-02T23:56:57Z

Codecov Report

Merging #6932 (edb480b) into main (64f3123) will decrease coverage by 0.31%.
The diff coverage is 97.61%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@             Coverage Diff              @@
##               main    #6932      +/-   ##
============================================
- Coverage     71.10%   70.79%   -0.31%     
+ Complexity    59808    59514     -294     
============================================
  Files          4852     4852              
  Lines        285138   285176      +38     
  Branches      41107    41118      +11     
============================================
- Hits         202755   201902     -853     
- Misses        66040    66679     +639     
- Partials      16343    16595     +252

Impacted Files	Coverage Δ
...org/opensearch/cluster/metadata/IndexMetadata.java	`85.00% <ø> (+0.70%)`	⬆️
...h/cluster/metadata/MetadataCreateIndexService.java	`78.86% <96.66%> (+0.43%)`	⬆️
...rg/opensearch/common/settings/ClusterSettings.java	`93.18% <100.00%> (+0.68%)`	⬆️
...org/opensearch/common/settings/SettingsModule.java	`86.18% <100.00%> (ø)`
...in/java/org/opensearch/indices/IndicesService.java	`63.45% <100.00%> (-1.06%)`	⬇️

... and 472 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

github-actions · 2023-04-04T10:44:11Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/13372/
CommitID: 68667e1
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-04-04T11:19:13Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/13374/
CommitID: e8b08d4

github-actions · 2023-04-10T16:18:46Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/13757/
CommitID: 790c601
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Varun Bansal <bansvaru@amazon.com>

github-actions · 2023-04-10T17:16:49Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/13766/
CommitID: 4fef117
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Varun Bansal <bansvaru@amazon.com>

github-actions · 2023-04-10T18:03:45Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/13771/
CommitID: e02f7d3
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-04-10T18:12:01Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.cluster.allocation.AwarenessAllocationIT.testThreeZoneOneReplicaWithForceZoneValueAndLoadAwareness

URL: https://build.ci.opensearch.org/job/gradle-check/13773/
CommitID: a566cd7
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

gbbafna · 2023-04-10T14:26:11Z

distribution/src/config/opensearch.yml

+# Controls whether cluster imposes index creation only with remote store enabled
+# cluster.remote_store.enabled.force: true


add one setting for remote translog as well .

Sure, updated

gbbafna · 2023-04-10T14:27:57Z

server/src/main/java/org/opensearch/cluster/metadata/MetadataCreateIndexService.java

+                    && !INDEX_REMOTE_STORE_ENABLED_SETTING.get(requestSettings))) {
+                return;
+            }
+            String segmentStoreRepo = requestSettings.get(SETTING_REMOTE_STORE_REPOSITORY);


nit : Lets call it remoteStoreRepo

gbbafna · 2023-04-11T04:28:19Z

server/src/main/java/org/opensearch/cluster/metadata/MetadataCreateIndexService.java

+        if (FeatureFlags.isEnabled(FeatureFlags.REMOTE_STORE)
+            && FeatureFlags.isEnabled(FeatureFlags.REPLICATION_TYPE)


We need to move these settings under FEATURE_FLAGGED_CLUSTER_SETTINGS . In that case, we won't need to put these checks here and user won't be able be set it w/o enabling the feature flags first.

thanks for pointing this out. i'll update

in our case its two features we are dependent on. FEATURE_FLAGGED_CLUSTER_SETTINGS only supports adding settings which are dependent on single feature

gbbafna · 2023-04-11T04:44:07Z

server/src/main/java/org/opensearch/cluster/metadata/MetadataCreateIndexService.java

+                .put(SETTING_REMOTE_STORE_ENABLED, true)
+                .put(SETTING_REMOTE_STORE_REPOSITORY, segmentStoreRepo);
+
+            if (!Objects.equals(requestSettings.get(INDEX_REMOTE_TRANSLOG_STORE_ENABLED_SETTING.getKey()), "false")) {


we also need to check if translog_repo is also supplied . The reason being either we pick both enabled and repo from index settings . If not provided , fall back to cluster level settings.

do we not allow user to just send INDEX_REMOTE_TRANSLOG_STORE_ENABLED_SETTING as true and pick translog repo from cluster settings? I think we should allow this, let me know what you think

Signed-off-by: Varun Bansal <bansvaru@amazon.com>

github-actions · 2023-04-11T05:46:49Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/13829/
CommitID: 7115af0
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-04-11T10:49:44Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/13838/
CommitID: f501085
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-04-11T11:27:35Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/13839/
CommitID: b7e50c9
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Varun Bansal <bansvaru@amazon.com>

…riding settings Signed-off-by: Varun Bansal <bansvaru@amazon.com>

github-actions · 2023-04-11T12:18:22Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.indices.replication.SegmentReplicationIT.testPitCreatedOnReplica

URL: https://build.ci.opensearch.org/job/gradle-check/13841/
CommitID: 3151753
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

sachinpkale · 2023-04-12T03:04:56Z

server/src/main/java/org/opensearch/cluster/metadata/MetadataCreateIndexService.java

+    private static void updateRemoteStoreSettings(Settings.Builder settingsBuilder, Settings requestSettings, Settings clusterSettings) {
+        if (CLUSTER_REMOTE_STORE_ENABLED_SETTING.get(clusterSettings)) {
+            // Verify if we can create a remote store based index based on user provided settings
+            if (!canCreateRemoteStoreIndex(requestSettings)) {


nit: canCreateRemoteStoreIndex(requestSettings) == false (Applicable to all the places where we are using ! in if condition)

sachinpkale · 2023-04-12T03:12:28Z

server/src/main/java/org/opensearch/cluster/metadata/MetadataCreateIndexService.java

+                return;
+            }
+
+            settingsBuilder.put(SETTING_REPLICATION_TYPE, ReplicationType.SEGMENT).put(SETTING_REMOTE_STORE_ENABLED, true);


Why are we adding Segment Replication setting explicitly here? Are we sure that replication type will always be segment at this point?
As I understand, Segment replication also has a cluster level setting. settingsBuilder should be updated by cluster and index level setting of segment replication prior to updateRemoteStoreSettings

if there is an index level setting for segment replication then it would have been put by the user. We already have check for it.
For cluster level setting, if its set to SEGMENT, we dont have any problem. If its set to DOCUMENT then we are overriding and sending SEGMENT as replication.
I'll update to throw an error incase of cluster level settings being document

Signed-off-by: Varun Bansal <bansvaru@amazon.com>

github-actions · 2023-04-12T04:57:05Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/13917/
CommitID: edb480b

shwetathareja · 2023-04-12T04:11:10Z

distribution/src/config/opensearch.yml

+# cluster.remote_store.enabled: true
+#
+# Repository to use for segment upload while enforcing remote store for an index
+# cluster.remote_store.repository: my-repo-1


will it throw exception if remote repo is not setup and prevent index creation?

yes. we have checks during index creation

…lt (#6932) Signed-off-by: Varun Bansal <bansvaru@amazon.com> (cherry picked from commit 4d6c02d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…lt (opensearch-project#6932) Signed-off-by: Varun Bansal <bansvaru@amazon.com>

linuxpi force-pushed the remote-store-default-enable branch from 41b9ae8 to 065c8aa Compare April 2, 2023 23:27

linuxpi force-pushed the remote-store-default-enable branch from 68667e1 to e8b08d4 Compare April 4, 2023 10:49

linuxpi changed the title ~~Add support to create index with remote store by default~~ [Remote Store] Add support to create index with remote store by default Apr 4, 2023

linuxpi marked this pull request as ready for review April 4, 2023 11:20

linuxpi requested review from reta, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, gbbafna, setiah, kartg, kotwanikunal, mch2, nknize, owaiskazi19, Rishikesh1159, ryanbogan, saratvemulapalli, shwetathareja, dreamer-89, tlfeng, VachaShah and xuezhou25 as code owners April 4, 2023 11:20

Add ITs and update UTs

4fef117

Signed-off-by: Varun Bansal <bansvaru@amazon.com>

linuxpi force-pushed the remote-store-default-enable branch from 790c601 to 4fef117 Compare April 10, 2023 16:43

linuxpi added 2 commits April 10, 2023 23:03

Fix failing tests and correct error string

e02f7d3

Signed-off-by: Varun Bansal <bansvaru@amazon.com>

Fix spotless errors

a566cd7

Signed-off-by: Varun Bansal <bansvaru@amazon.com>

gbbafna requested changes Apr 11, 2023

View reviewed changes

Update based on PR feedback

7115af0

Signed-off-by: Varun Bansal <bansvaru@amazon.com>

linuxpi added 2 commits April 11, 2023 17:10

Merge branch 'main' into remote-store-default-enable

d611076

Signed-off-by: Varun Bansal <bansvaru@amazon.com>

Refactor: rename settings, change logic to provide simplicity in over…

3151753

…riding settings Signed-off-by: Varun Bansal <bansvaru@amazon.com>

linuxpi force-pushed the remote-store-default-enable branch from b7e50c9 to 3151753 Compare April 11, 2023 11:46

sachinpkale reviewed Apr 12, 2023

View reviewed changes

Throw error if default relication type is segment at cluster level

edb480b

Signed-off-by: Varun Bansal <bansvaru@amazon.com>

shwetathareja reviewed Apr 12, 2023

View reviewed changes

sachinpkale added the skip-changelog label Apr 12, 2023

sachinpkale approved these changes Apr 12, 2023

View reviewed changes

gbbafna approved these changes Apr 12, 2023

View reviewed changes

gbbafna merged commit 4d6c02d into opensearch-project:main Apr 12, 2023

gbbafna added the backport 2.x Backport to 2.x branch label Apr 12, 2023

opensearch-trigger-bot bot mentioned this pull request Apr 12, 2023

[Backport 2.x] [Remote Store] Add support to create index with remote store by default #7123

Merged

austintlee pushed a commit to austintlee/OpenSearch that referenced this pull request Apr 28, 2023

[Remote Store] Add support to create index with remote store by defau…

b0951b0

…lt (opensearch-project#6932) Signed-off-by: Varun Bansal <bansvaru@amazon.com>

linuxpi mentioned this pull request Sep 5, 2023

[Remote Cluster State] Integrate remote cluster state settings with node attrs #9756

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Remote Store] Add support to create index with remote store by default #6932

[Remote Store] Add support to create index with remote store by default #6932

linuxpi commented Apr 2, 2023 •

edited

Loading

github-actions bot commented Apr 2, 2023

github-actions bot commented Apr 2, 2023

codecov-commenter commented Apr 2, 2023 •

edited

Loading

github-actions bot commented Apr 4, 2023

github-actions bot commented Apr 4, 2023

github-actions bot commented Apr 10, 2023

github-actions bot commented Apr 10, 2023

github-actions bot commented Apr 10, 2023

github-actions bot commented Apr 10, 2023

gbbafna Apr 10, 2023

linuxpi Apr 11, 2023

gbbafna Apr 10, 2023

linuxpi Apr 11, 2023

gbbafna Apr 11, 2023

linuxpi Apr 11, 2023

linuxpi Apr 11, 2023

gbbafna Apr 11, 2023

linuxpi Apr 11, 2023

github-actions bot commented Apr 11, 2023

github-actions bot commented Apr 11, 2023

github-actions bot commented Apr 11, 2023

github-actions bot commented Apr 11, 2023

sachinpkale Apr 12, 2023

linuxpi Apr 12, 2023

sachinpkale Apr 12, 2023

linuxpi Apr 12, 2023

github-actions bot commented Apr 12, 2023

shwetathareja Apr 12, 2023

linuxpi Apr 12, 2023

		# Controls whether cluster imposes index creation only with remote store enabled
		# cluster.remote_store.enabled.force: true

		if (FeatureFlags.isEnabled(FeatureFlags.REMOTE_STORE)
		&& FeatureFlags.isEnabled(FeatureFlags.REPLICATION_TYPE)

[Remote Store] Add support to create index with remote store by default #6932

[Remote Store] Add support to create index with remote store by default #6932

Conversation

linuxpi commented Apr 2, 2023 • edited Loading

Description

Issues Resolved

Check List

github-actions bot commented Apr 2, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Apr 2, 2023

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Apr 2, 2023 • edited Loading

Codecov Report

github-actions bot commented Apr 4, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Apr 4, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Apr 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Apr 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Apr 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Apr 10, 2023

Gradle Check (Jenkins) Run Completed with:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Apr 11, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Apr 11, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Apr 11, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Apr 11, 2023

Gradle Check (Jenkins) Run Completed with:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Apr 12, 2023

Gradle Check (Jenkins) Run Completed with:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linuxpi commented Apr 2, 2023 •

edited

Loading

codecov-commenter commented Apr 2, 2023 •

edited

Loading