Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decommission/init merge #86

Open
wants to merge 288 commits into
base: main
Choose a base branch
from
Open

Decommission/init merge #86

wants to merge 288 commits into from

Conversation

imRishN
Copy link
Owner

@imRishN imRishN commented Nov 8, 2022

Description

[Describe what this change achieves]

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

reta and others added 30 commits October 3, 2022 14:19
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Fixing PIT flaky tests

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
… Jenkins build now waits for pre-commit checks to pass before attempting (opensearch-project#4660)

* Ignore all malformed objects when ignore_malformed is true (opensearch-project#4494)

Fixes a bug to not fail the entire document when "ignore_malformed" is set to true. Allowing the valid fields 
to be indexed and ignore only the malformed fields.

Signed-off-by: Hauck <joaoh14@gmail.com>
Signed-off-by: Braddy Yeoh <braddy.yeoh@ucdconnect.ie>

* Adding javadoc check to the precommi\t GitHub Action

Signed-off-by: Braddy Yeoh <braddy.yeoh@ucdconnect.ie>

* gradle check now only runs after successful gradle precommit check

Signed-off-by: Braddy Yeoh <braddy.yeoh@ucdconnect.ie>

* Added to CHANGELOG

Signed-off-by: Braddy Yeoh yeohbraddy@gmail.com
Signed-off-by: Braddy Yeoh <braddy.yeoh@ucdconnect.ie>

Signed-off-by: Hauck <joaoh14@gmail.com>
Signed-off-by: Braddy Yeoh <braddy.yeoh@ucdconnect.ie>
Signed-off-by: Braddy Yeoh yeohbraddy@gmail.com
Co-authored-by: Hauck <67768441+hauck-jvsh@users.noreply.github.com>
Co-authored-by: Braddy Yeoh <braddy.yeoh@ucdconnect.ie>
)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
…ct#4679)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* First draft for Windows precommit

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>

* Adding changelog

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
* First draft for Windows precommit

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>

* Adding changelog

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>

* Adding precommit for macos

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>

* Adding Changelog

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
…t#4683)

This test fails pretty reliably if I run it on repeat. I believe the
problem is that the test assumes the function will take longer than 2ms,
which is likely not a valid assumption in all cases. Fortunately, I can
pass in a zero duration which is guaranteed to timeout even if the
system clock does not advance at all.

Also moved the assertions out of the callback into the main test method,
otherwise the assertion error messages would get buried and the test
report would just show a timeout error.

Signed-off-by: Andrew Ross <andrross@amazon.com>

Signed-off-by: Andrew Ross <andrross@amazon.com>
…elyRunningAsRoot() (opensearch-project#4656)

* Bump jna from 5.11.0 to 5.12.1

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Include Windows OS in check for definitelyRunningAsRoot()

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Combine try/catch blocks

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Someone sniped my PR number between predicting and pushing

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Update SHAs

Signed-off-by: Daniel Widdis <widdis@gmail.com>

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
* Recommission API service level changes

Signed-off-by: pranikum <109206473+pranikum@users.noreply.github.com>
* Weighted round-robin scheduling policy for shard coordination traffic routing

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
…ject#4688)

My previous fix introduced a new race condition by making the assertions
before waiting on the latch.

Signed-off-by: Andrew Ross <andrross@amazon.com>

Signed-off-by: Andrew Ross <andrross@amazon.com>
Fixes bwc failure caused by commit
6993ac9.

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
…tailed_error disabled (opensearch-project#4669)

* Fixed misunderstanding message 'No OpenSearchException found' when detailed_error disabled

Signed-off-by: Xue Zhou <xuezhou@amazon.com>
…pensearch-project#4572)

* Update access modifier to support extensibility

Change access modifier from default to protected.
This will help to build new geo based aggregation
outside OpenSearch, by keeping GeoGrid Classes as base class.

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

* Updated CHANGELOG

Added PR details to CHANGELOG.md

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

* Rename InternalGeoGridBucket to BaseGeoGridBucket

Update class names, references and comments.

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

* Rename InternalGeoGrid to BaseGeoGrid

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

* Make GridBucket classes package-private

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

* Remove Internal prefix from Geo Grid classes

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

* Update constructor and class access modifier

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

* Update access modifier based on usage

Made classes package private if it is not used
outside the package.

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>
…oject#2768)

* [Remove] LegacyESVersion.V_7_0_* and V_7_1_* constants

Removes all usages of LegacyESVersion.V_7_0_ and LegacyESVersion.V_7_1 version
constants.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* Rebase from main

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix serialization issue with build flavor removal

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* remove stale bwc test

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* rebase and update

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* cleanup

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix failing mapper test

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
…xx tasks (opensearch-project#4696)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
…rch-project#4066)

* Bump gson from 2.9.0 to 2.9.1 in /test/fixtures/hdfs-fixture

Bumps [gson](https://github.com/google/gson) from 2.9.0 to 2.9.1.
- [Release notes](https://github.com/google/gson/releases)
- [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md)
- [Commits](google/gson@gson-parent-2.9.0...gson-parent-2.9.1)

---
updated-dependencies:
- dependency-name: com.google.code.gson:gson
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
* Always auto release the flood stage block

Removes support for using a system property to disable the automatic release of
the write block applied when a node exceeds the flood-stage watermark.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* update IAE message

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Controlling discovery for decommissioned nodes

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
…oject#4702)

Removes all usages of LegacyESVersion.V_7_2_ and LegacyESVersion.V_7_3 version
constants along with related ancient APIs.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
…oject#4704)

Removes all usages of LegacyESVersion.V_7_4_ and LegacyESVersion.V_7_5
version constants along with ancient API logic.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
…re (opensearch-project#4727)

* Bump protobuf-java from 3.21.2 to 3.21.7 in /test/fixtures/hdfs-fixture

Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.2 to 3.21.7.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](protocolbuffers/protobuf@v3.21.2...v3.21.7)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…roject#4732)

Refactors base action classes implemented by external plugins through
ActionPlugin from opensearch.internal to opensearch.api to signal extensibility
to third-party developers.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
…pensearch-project#4726)

* Bump azure-core from 1.31.0 to 1.33.0 in /plugins/repository-azure

Bumps [azure-core](https://github.com/Azure/azure-sdk-for-java) from 1.31.0 to 1.33.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-core_1.31.0...azure-core_1.33.0)

---
updated-dependencies:
- dependency-name: com.azure:azure-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…ch-project#4076)

* Bump avro from 1.11.0 to 1.11.1 in /plugins/repository-hdfs

Bumps avro from 1.11.0 to 1.11.1.

---
updated-dependencies:
- dependency-name: org.apache.avro:avro
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…ability (opensearch-project#4689)

Signed-off-by: Tianli Feng <ftianli@amazon.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
@github-actions
Copy link

Gradle Check (Jenkins) Run Completed with:

  • RESULT: null ❌
  • URL:
  • CommitID: 0a8e37a

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
@github-actions
Copy link

Gradle Check (Jenkins) Run Completed with:

  • RESULT: null ❌
  • URL:
  • CommitID: 87bf89c

gaobinlong and others added 21 commits November 30, 2022 10:30
* Add max_shard_size parameter for Shrink API

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* add change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* fix yaml test failed

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* optimize the code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* fix test failed

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* optimize changelog & code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
…ject#5283)

* Update Rest status for DecommissioningFailedException

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>

* Add tests for decommission response code

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>

* Remove unnecessary test change

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>

* Fix spotless check

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>

* Fix

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>

* Add changelog

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>

* Update changelog

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
When the new 'cancelled' field was introduced it was a miss not to
increment the version number on the mapping definitions for the .tasks
index. This commit fixes that oversight, as well as modifies the
existing backward compatiblity test to ensure that it will catch future
mistakes like this one.

Closes opensearch-project#5376

Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: sashashura <aleksandrosansan@gmail.com>

Signed-off-by: sashashura <aleksandrosansan@gmail.com>
Signed-off-by: owaiskazi19 <owaiskazi19@gmail.com>
…roject#5439)

Add conditional check on assertNull to fix flaky tests.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
)

Signed-off-by: Dhwanil Patel <dhwanip@amazon.com>
…he Mime4j 0.8.8, Apache Poi 5.2.3, Apache PdfBox 2.0.27 (opensearch-project#5448)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
…ect#5282)

* CheckpointState enhanced to support no-op replication

Signed-off-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: Bukhtawar Khan<bukhtawa@amazon.com>
…T/test {yaml=repository_s3/20_repository_permanent_credentials/Snapshot and Restore with repository-s3 using permanent credentials} flaky: randomizing basePath (opensearch-project#5482)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
)

Fixes the wildcard query to not normalize the pattern when case_insensitive is
set by the user. This is achieved by creating a new normalizedWildcardQuery
method so that query_string queries (which do not support case sensitivity) can
still normalize the pattern when the default analyzer is used; maintaining
existing behavior.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
…ject#5460)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
…ch-project#4902)" (opensearch-project#5465)

This reverts commit 8c9ca4e.

It seems that this wasn't entirely the correct way and is currently
blocking us from removing the `build.sh` from the `opensearch-build`
repository (i.e. this `build.sh` here is not yet being used).
See the discussion in opensearch-project/opensearch-build#2835 for
further details.

Signed-off-by: Ralph Ursprung <Ralph.Ursprung@avaloq.com>

Signed-off-by: Ralph Ursprung <Ralph.Ursprung@avaloq.com>
…ter backport) (opensearch-project#5503)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>

Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
* Added jackson dependency to server

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Updated CHANGELOG

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Update build.gradle files

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Add RuntimePermission to fix errors

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

Signed-off-by: Ryan Bogan <rbogan@amazon.com>
…arch-project#5491)

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

Signed-off-by: Poojita Raj <poojiraj@amazon.com>
Signed-off-by: Xue Zhou <xuezhou@amazon.com>

Signed-off-by: Xue Zhou <xuezhou@amazon.com>
…ze on Windows. (opensearch-project#5511)

PR opensearch-project#5397 skipped this test in @before block but still
frequently throws a TestCouldNotBeSkippedException.  This is caused by the after block still executing and throwing  an exception
while cleaning the directory created at the path in @before.  Moving the assumption to the individual test prevents this exception by ensuring the path exists.

Signed-off-by: Marc Handalian <handalm@amazon.com>

Signed-off-by: Marc Handalian <handalm@amazon.com>
…#5347)

* Merge first batch of feature/extensions into main

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fixed CHANGELOG

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fixed newline errors

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Renaming and CHANGELOG fixes

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Refactor extension loading into private method

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Removed skipValidation and added connectToExtensionNode method

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Remove unnecessary feature flag calls

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Renaming and exception handling

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Change latches to CompletableFuture

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Removed unnecessary validateSettingKey call

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fix azure-core dependency

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Update SHAs

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Remove unintended dependency changes

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Removed dynamic settings regitration, removed info() method, and added NoopExtensionsManager

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Add javadoc

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fixed spotless failure

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Removed NoopExtensionsManager

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Added functioning NoopExtensionsManager

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Added missing javadoc

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Remove forbiddenAPI

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fix spotless

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Change logger.info to logger.error in handleException

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fix ExtensionsManagerTests

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Removing unrelated change

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Update SHAs

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

Signed-off-by: Ryan Bogan <rbogan@amazon.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
@github-actions
Copy link

Gradle Check (Jenkins) Run Completed with:

  • RESULT: null ❌
  • URL:
  • CommitID: 4d46a29

@github-actions
Copy link

Gradle Check (Jenkins) Run Completed with:

  • RESULT: null ❌
  • URL:
  • CommitID: a159da6

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
@github-actions
Copy link

Gradle Check (Jenkins) Run Completed with:

  • RESULT: null ❌
  • URL:
  • CommitID: 6ce6628

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.