-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Partially resolved issue #4687 by changing the improper status code for concurrent snapshot executions #5855
Closed
baba-devv
wants to merge
203
commits into
opensearch-project:main
from
baba-devv:Mani-ExceptionChange
Closed
Changes from 2 commits
Commits
Show all changes
203 commits
Select commit
Hold shift + click to select a range
05874f3
Changed status of ConcurrentSnapshotException from 5xx -> 429(Too_Man…
baba-devv 1395a14
Merge branch 'main' of github.com:baba-devv/OpenSearch into Mani-Exce…
baba-devv 114e6bc
Merge branch 'main' of github.com:baba-devv/OpenSearch into Mani-Exce…
baba-devv dda7f85
Merge branch 'main' of github.com:baba-devv/OpenSearch into Mani-Exce…
baba-devv ed74bf2
Merge branch 'main' of github.com:baba-devv/OpenSearch into Mani-Exce…
baba-devv 439bf24
Merge branch 'main' of github.com:baba-devv/OpenSearch into Mani-Exce…
baba-devv 50da23f
Added test case for ConcurrentSnapshotException -> 429(Too_Many_Reque…
baba-devv d3cff6b
Merge branch 'main' of github.com:opensearch-project/OpenSearch into …
baba-devv 08ffb33
Added test case for ConcurrentSnapshotException -> 429(Too_Many_Reque…
baba-devv 023be3d
Added test case for ConcurrentSnapshotException -> 429(Too_Many_Reque…
baba-devv edf182f
Return 409 Conflict HTTP status instead of 500 on failure to concurre…
baba-devv b2ff95f
Return 409 Conflict HTTP status instead of 500 on failure to concurre…
baba-devv 4649e21
Return 409 Conflict HTTP status instead of 500 on failure to concurre…
baba-devv b285b62
Bump org.apache.hadoop:hadoop-minicluster from 3.3.5 to 3.3.6 in /tes…
dependabot[bot] 65f13e8
Bump com.networknt:json-schema-validator from 1.0.84 to 1.0.85 in /bu…
dependabot[bot] ec38404
Bump io.projectreactor.netty:reactor-netty-http from 1.1.7 to 1.1.8 i…
dependabot[bot] 5c82e2d
Bump org.jruby.joni:joni from 2.1.48 to 2.2.1 (#8254)
dependabot[bot] f3fe1a2
Add integ tests for remote store stats api (#8135)
linuxpi 0d6117f
[Upgrade] Lucene 9.7.0 release (#8272)
nknize 20c634e
Fix SegmentReplication flaky integ tests (#8134)
sachinpkale 5b34d80
[Remote Store] Add remote segment upload backpressure integ tests (#8…
ashking94 2e3c02c
Bump `resteasy-jackson2-provider` from 3.0.26.Final to 6.2.4.Final in…
Xtansia c20668e
Fix Apache Lucene verson for 2.9.0 release (#8290)
reta 9d58b8d
[Snapshot Interop] Add Changes in Restore Snapshot Flow for remote st…
harishbhakuni f002709
Remove minimum file cache size restriction (#8294)
kotwanikunal edb5821
Removed redundant ubuntu precommit check (#8207)
owaiskazi19 1482d07
Add request tracing framework (#7648)
suranjay 79d1779
Added release notes for 1.3.11 (#8302)
owaiskazi19 a8108f6
[Snapshot Interop] Change Version Checks from 3.0 to 2.9 in Restore S…
harishbhakuni 9b08947
[Segment Replication] Update bwc test to rely on segments for verific…
dreamer-89 31d9ea7
[Mute] SegRepUsingRemoteStoreIT.testPressureServiceStats (#8324)
nknize aa54b4e
Bump BouncyCastle from jdk15on to jdk15to18 (#8247)
willyborankin a06ab7c
[Search Pipelines] Add stats for search pipelines (#8053)
msfroh a019032
Self-organizing hash table to improve the performance of bucket aggre…
ketanv3 d322fd7
Bump com.google.api-client:google-api-client from 1.34.0 to 2.2.0 in …
Xtansia 0e8e8dc
Moving zstd out of sandbox (#7908)
sarthakaggarwal97 8d5752e
Implement buildEmptyAggregations for MultiTermsAggregator (#7089) (#7…
austintlee cf732b7
Adding the SearchPhaseResultsProcessor interface in Search Pipeline (…
navneet1v 36946f5
Handle case where IndexShard is null due to failed creation of index …
gbbafna 0dedea7
Check UTF16 string size before converting to String to avoid OOME (#7…
imRishN 1fc775e
Enable Point based optimization for custom comparators (#8168)
gashutos cf0213d
[Fix] Replace cached time with system clock in MasterService debug lo…
sandeshkr419 6cd8122
Fix HDFS fixture by excluding BouncyCastle (#8359)
willyborankin 8dbbe34
Enable Fast Double Parser in Jackson (#7909)
mgodwan e29dcd0
Update search.aggregation/370_multi_terms.yml after backport (#8358)
reta 8204813
Fix primary balance flaky test (#8366)
kotwanikunal 49490f4
Fix search.aggregation/60_empty.yml test case (fails on BWC runs) (#8…
reta 368f729
Add lower and upper bound support for appropriate variance thresholds…
kotwanikunal bae8afd
Fix version scipt for higher patch numbers (#8385)
kotwanikunal cd84c20
Avoid search_after short cutting in case of missing is specified (#8391)
gashutos 4631c58
[Remote Store] Moving stale commit deletion to async flow (#8201)
psychbot f14ed89
[Remote Store] Add method in BlobContainer to fetch the keys in sorte…
sachinpkale 92d1df2
[Remote Store] Add remote store utils useful to name metadata files (…
sachinpkale 2adb72a
Fix flaky tests in restore flow using snapshot interop (#8422)
sachinpkale 856fd2a
Adds log4j configuration for telemetry-otel plugin (#8393)
Gaganjuneja 6048031
Support extension additional settings with extension REST initializat…
cwperks 99cec3f
Update Apache HttpCore/ HttpClient and Apache HttpCore5 / HttpClient5…
reta 09e3bc9
[Remote translog] Add integration tests for primary term validation (…
ashking94 f8e9377
Adds mock implementation for TelemetryPlugin (#8357)
Gaganjuneja 612a09d
Bump org.apache.maven:maven-model from 3.9.2 to 3.9.3 (#8403)
dependabot[bot] e60af04
Bump org.apache.avro:avro from 1.11.1 to 1.11.2 in /plugins/repositor…
dependabot[bot] 673f161
Move span operation to Scope from Tracer (#8411)
suranjay 2723bbe
Bump io.projectreactor.netty:reactor-netty from 1.1.7 to 1.1.8 in /pl…
dependabot[bot] 49bfd2b
Fix flaky test. (#8441)
Rishikesh1159 22a1e4e
Support transport action names when registering NamedRoutes (#7957)
DarshitChanpura b326c06
Changing version check to 2.9 (#8416)
mingshl b55e4df
Unmute remote store test testPressureServiceStats (#8458)
kotwanikunal 6d2310e
default compression level change (#8471)
sarthakaggarwal97 e5dca8b
Add logs to debug NoSuchFileException during segments upload (#8475)
ashking94 90b40ee
[Remote Segment Store] Make metadata file immutable (#8363)
sachinpkale dfcc342
Flaky test: Add condition to ensure data correctness on primary promo…
ankitkala 01e20e8
[Segment Replication] Remove Restriction of strictly using only docre…
Rishikesh1159 238e651
[Search pipelines] Pass "adhocness" flag to processor factories (#8164)
msfroh a372de7
Update to Gradle 8.2 (#8018)
reta 3d44108
Create concept of persistent ThreadContext headers that are unstashab…
cwperks 7d3f787
Enable Partial Flat Object (#7997)
mingshl a84b673
Add Changes in Snapshot Clone Flow for remote store interoperability.…
kasundra07 bed1627
Mute testStaleCommitDeletion test cases (#8516)
sachinpkale 43f5092
Compression Levels Settings for zstd and zstd_no_dict (#8312)
sarthakaggarwal97 d88e903
[Remote Store] Update Translog Metadata file name (#8350)
gbbafna ba00c42
Bump com.google.guava:guava from 32.0.1-jre to 32.1.1-jre in /distrib…
owaiskazi19 69e1657
Bump com.google.guava:guava from 32.0.1-jre to 32.1.1-jre in /distrib…
dependabot[bot] b45c732
Bump com.google.guava:guava from 32.0.1-jre to 32.1.1-jre in /distrib…
dependabot[bot] ceffdeb
Add early termination support for concurrent segment search (#8306)
jed326 a69c485
Change extension integration testing to reflect new initialization se…
ryanbogan 0f6ab74
[Remote Store] Mute testRemoteRefreshRetryOnFailure test case (#8542)
ashking94 6af29ff
[Segment Replication] Prevent store clean up post reader close and re…
dreamer-89 5ea6547
Fix flakyness in RemoteStoreRefreshListenerIT (#8547)
ashking94 b4d3542
Fix flakyness in RemoteStoreRefreshListenerTests (#8550)
ashking94 c1f7349
[Optimization] Cluster State Update Optimization (#7853)
sandeshkr419 54ea5dd
[Remote Segment Store] Fix flaky RemoteStoreIT.testStaleCommitDeletio…
sachinpkale 7260c78
Extend existing IndexRecoveryIT for remote indexes (#8505)
ashking94 5daa21f
Bump com.google.guava:guava from 30.1.1-jre to 32.1.1-jre (#8583)
reta f26e098
Add safeguard limits for file cache during node level allocation (#8208)
kotwanikunal 9ca2fee
Bump com.netflix.nebula:gradle-info-plugin from 12.1.4 to 12.1.5 (#8568)
dependabot[bot] f7d536e
Manually update google cloud core from 2.17.0 to 2.21.0 (#8586)
stephen-crawford 32d02f9
Bump com.azure:azure-storage-blob from 12.22.2 to 12.22.3 in /plugins…
dependabot[bot] 14e866c
replace with getField (#7855)
marevol a0b42e2
Update Gradle to 8.2.1 (#8580)
reta 7184bc7
Bump com.google.jimfs:jimfs from 1.2 to 1.3.0 in /distribution/tools/…
dependabot[bot] 4b142b4
Bump schema validator from from 1.0.85 to 1.0.86 (#8573)
stephen-crawford 0d9a026
Move changelog entry (#8599)
stephen-crawford f458a65
add jdk.incubator.vector module support for JDK 20+ (#8601)
nknize 95ea499
Bump com.google.jimfs:jimfs in /distribution/tools/upgrade-cli (#8571)
dependabot[bot] 547a338
Fix painless casting bug causing opensearch to crash (#8315)
shiv0408 ffdf8b4
Disabled translog fsync in remote store path (#8288)
vikasvb90 112c8a2
[Snapshot Interop] Add Changes in Snapshot Status Flow for remote sto…
kasundra07 687e05d
Manually bump net.minidev:json-smart from 2.4.11 to 2.5.0 (#8576)
stephen-crawford 5561832
[Snapshot Interop] Keep API parameters behind remote store experiment…
harishbhakuni 411f4e9
Add Changes in Snapshot Delete Flow for remote store interoperability…
kasundra07 5829c3e
Add shard id to remote store logs (#8574)
linuxpi ced2371
Add wrapped tracer implementation (#8565)
suranjay 6c7c932
[Remote Store] Add multipart upload integration for translog and segm…
raghuvanshraj dc3eebd
[Mute] flaky RemoteStoreRefreshListenerTests (#8626)
Rishikesh1159 aaff823
Add logs for remote store metadata intermittent read failures (#8618)
linuxpi 94a68f8
Fix bug where ReplicationListeners would not complete on cancellation…
mch2 8729f3d
Bump org.apache.logging.log4j:log4j-core from 2.17.1 to 2.20.0 (#8307)
Xtansia f036f9d
[Segment Replication] Add logic back to update tracking replication c…
dreamer-89 d8bf752
[Search pipelines] Add Global Ignore_failure options for Processors (…
mingshl a2d6671
Flip SEARCH_PIPELINE feature flag for search pipeline GA (#8613)
noCharger 2b668d7
[Refactor] HPPC IntArrayList and ObjectArrayList to j.u.List (#8602)
nknize fae1780
Bump net.minidev:json-smart from 2.4.11 to 2.5.0 in /plugins/reposito…
dependabot[bot] e6a2ceb
Manually update com.google.jimfs:jimfs from 1.2 to 1.3.0 (#8585)
stephen-crawford 18fe8f4
Start replication checkpointTimers on primary before segments upload …
ankitkala b5dbd2d
Fix remote segments sync retry regression introduced in PR #7119 (#8632)
ashking94 2362844
Telemetry exporter config (#8624)
Gaganjuneja 3728768
[Remote Store] Add S3 async upload utilities and models (#7217)
raghuvanshraj bf87d32
[Refactor] StreamIO from common to core.common namespace in core lib …
nknize 01367d6
Add 2.10.0 to bwc versions. (#8670)
andrross 0433a67
[Remote Store] Integrate S3 async upload with S3BlobContainer (#7218)
raghuvanshraj b8beb4a
Added the addition of the server version and distribution as a respon…
Sam-ScottLogic 6cf52cc
CardinalityIT/NestedIT test failures with concurrent search enabled a…
sohami 9076333
Create 2.9 release notes and clear out changelog (#8682)
andrross e70da4b
Remove unused extension directory (#8674)
owaiskazi19 865f339
Remove CHANGELOG entries that have been backported (#8686)
andrross 917fac3
Changing search shard failure logging to trace from debug (#8470)
sgup432 4d46b08
Handle replication cases where the remote store metadata file is not …
ankitkala 077e6b6
Fix RemoteStoreReplicationSourceTests.testGetCheckpointMetadataEmpty.…
mch2 18b0ec7
Segment Replication - Allow search idle with no replicas (#8173)
mch2 58b322d
update grpc (#8726)
stephen-crawford 32f400a
ClusterShardLimitIT.testCreateIndexWithMaxClusterShardSetting (#8708)
RS146BIJAY 8969e2d
Fix flaky test testDropPrimaryDuringReplication. (#8715)
mch2 5140c42
Manually update commons-codec from 1.15 to 1.16.0 (#8725)
stephen-crawford 32fb067
Manually update Netflix Gradle info from 12.1.5 to 12.1.6 (#8724)
stephen-crawford 405a44d
[Refactor] Remaining HPPC to java.util collections (#8730)
nknize e0d8151
Do not trigger storeDirectory.sync in segment download flow from remo…
sachinpkale 46f72c8
Use CloseableRetryableRefreshListener to drain ongoing after refresh …
ashking94 93dc412
[Remote Segment Store] Use files from segmentInfosSnapshot in upload …
sachinpkale 421d198
[Segment Replication][Remote Store] Remove commits when remote store …
Poojita-Raj eabe73c
Remove SEARCH_PIPELINE feature flag (#8513)
noCharger 1232dc8
[Remote Store] Change remote store integ tests to use remote store fo…
sachinpkale d03f24a
Make Tracer injectable (#8696)
suranjay cdbd14d
Add integration tests for remote store restore flow (#8484)
b23dcca
Change InternalSignificantTerms to only sum shard level counts in fin…
jed326 99cdff1
Remove remote translog settings to make all remote indices translog e…
linuxpi df50665
Mute test - SegmentReplicationIndexShardTests#testNRTReplicaWithRemot…
linuxpi a966931
[Remote Segment Store] Add Lucene major version to UploadedSegmentMet…
d331b1b
fixtures: fix read-only krb5kdc composeBuild failure (#8799)
r1walz 96663cf
Exclude 'benchmarks' from codecov report (#8805)
ketanv3 3888d9f
Remove searchSegmentOrderReversed from IndexSettings (#8808)
andrross cc11f4c
Adding @sohami to OpenSearch maintainers (#8828)
Bukhtawar d50757a
Fix flaky test - org.opensearch.repositories.blobstore.BlobStoreRepos…
linuxpi 6cd229f
Bump com.netflix.nebula.ospackage-base from 11.3.0 to 11.4.0 in /dist…
dependabot[bot] 0bb6638
Manually Bump org.gradle.test-retry from 1.5.3 to 1.5.4 (#8842)
stephen-crawford 558a81d
Bump com.google.http-client:google-http-client-gson from 1.43.2 to 1.…
dependabot[bot] fdf6479
Manually bump org.apache.zookeeper:zookeeper from 3.8.1 to 3.8.2 (#8844)
stephen-crawford 1b1d440
[Refactor] MediaTypeParser to MediaTypeParserRegistry (#8636)
nknize 7e45204
Retry only specified tests (#8825)
andrross cb757ad
Fix versions for 2.x on main (#8846)
kotwanikunal 919aa7c
Fix flaky test SegmentReplicationRemoteStoreIT.testPressureServiceSta…
mch2 320eefa
OpenJDK Update (July 2023 Patch releases) (#8868)
reta 2960c66
Update Apache Lucene to 9.8.0-snapshot-4373c3b (#8668)
reta 3e91183
Create separate SourceLookup instance per segment slice in Significan…
jed326 e9a6bbe
Bump version of Hadoop dependencies to 3.3.5 (#6995)
defdec2
Extend publishXToMavenLocal explicit dependencies to all repo tasks (…
ccook-ibm efebd02
Skip testPitCreatedOnReplica IT with remote store (#8877)
dreamer-89 ac8f553
Fix testReplicaHasDiffFilesThanPrimary. (#8863)
mch2 5d2321e
Replace the deprecated IndexReader APIs with new storedFields() & ter…
luyuncheng 72e385e
Remove unnecessary refresh listeners from NRTReplicationReaderManager…
mch2 0019a14
Updated Version.java path from server to libs in version.yml (#8883)
vibrantvarun 683b895
[Remote Store] Add support to restore only unassigned shards of an in…
sachinpkale 64b3b39
Add 2.9.1 to BWC and known versions (#8923) (#8929)
reta a0e7f21
Fix flakiness in MasterServiceTests.testThrottlingForMultipleTaskType…
andrross 8b317d9
Fix SegmentReplicationIT.testReplicaHasDiffFilesThanPrimary for node-…
mch2 3fb0b3f
Configure test-retry plugin filter properly (#8933)
andrross ba6ccdb
[Refactor] MediaTypeParserRegistry to MediaTypeRegistry (#8940)
nknize 671c7cc
[Segment Replication] Use deterministic mechanism to have concurrent …
dreamer-89 2576fec
Register MediaTypes through SPI (#8938)
nknize 594627f
Remove testUpdateSegmentsWhileRefreshing flaky test failure (#8943)
dreamer-89 bfe5462
amending tests to include all codecs (#8907)
sarthakaggarwal97 91b0be2
Add metadata prefix to Remote Translog Metadata file (#8914)
gbbafna f7ece2d
[BUG] CompletionSuggestSearchIT.testSkipDuplicates is flaky (#8963)
reta dd78796
Added empty build.gradle files for arm64 and s390x because the gradle…
evankielley d492e57
Revert "amending tests to include all codecs (#8907)" (#8968)
reta d19ba04
[distribution/archives] [Linux] [x64] Provide the variant of the dist…
reta 249693b
Replace deprecated Query#rewrite(IndexReader with rewrite(IndexSearch…
nknize ac210bf
[Refactor] XContentFactory contentType introspection to MediaTypePars…
nknize 4c4d2e8
amending tests to include all codecs (#8907)
sarthakaggarwal97 77c4bcf
Revert "amending tests to include all codecs (#8907)" (#8968)
reta 08ccde4
Unmute test SegmentReplicationIndexShardTests#testNRTReplicaWithRemot…
Poojita-Raj 15d08fa
amending tests to include all codecs (#8907)
sarthakaggarwal97 7a73a7d
Revert "amending tests to include all codecs (#8907)" (#8968)
reta fd1572d
Merge branch 'main' of github.com:baba-devv/OpenSearch into Mani-Exce…
baba-devv File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks can you please also add a test?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am wondering shouldn't this be 409 (Conflict) instead of 429?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the underlying error? Is it concurrent snapshots being taken whereas this isn't supported? In that case it should definitely not be a 429, that implies the server is unable to handle traffic, not a predictable conflicting behavior - 409 seems more correct in that case."Thrown when a user tries to multiple conflicting snapshot/restore operations at the same time." -> 409
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only question I had was that since it is concurrent & not specifically conflicting, should I change that to 409 ?
@dblock @shwetathareja
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@baba-devv : Since it is concurrent, it is resulting in conflict and hence rejected by server.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@baba-devv yes in my opinion it should be 409