Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented cross-cluster monitor support #1404

Merged

Conversation

AWSHurneyt
Copy link
Collaborator

@AWSHurneyt AWSHurneyt commented Feb 5, 2024

Issue #, if available:
opensearch-project/alerting-dashboards-plugin#796

Description of changes:

  1. Implemented support for cross-cluster cluster metrics monitors.
  2. Implemented GetRemoteIndexes API to populate the frontend UI with details regarding the remote clusters, and indexes.

These changes are intended to enhance query, bucket, and cluster metrics monitors. As mentioned in the documentation issue (link), query and bucket monitors already support querying remote indexes; but configuring such monitors required using devtools/API commands. The enhancements in this PR allow users to configure remote using the frontend UI. In addition, these changes allow cluster metrics monitors to execute against remote clusters, which was not previously possible.

When this feature is enabled, the frontend will display a cluster selection dropdown for query, bucket, and cluster metrics monitors. The index selection dropdown will then display the indexes on the selected clusters. Alternatively for cluster metrics monitors, users can include a list of cluster name strings in the monitor input to create remote cluster metrics monitors via devtools/API commands.

Documentation PR for reference opensearch-project/documentation-website#6350

CheckList:

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…onitors.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>
…mented GetRemoteIndexes API to populate the frontend UI with details regarding the remote clusters, and indexes.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>
@AWSHurneyt
Copy link
Collaborator Author

Successfully building this PR will depend on merging the common utils changes, and publishing maven.
opensearch-project/common-utils#584

@bowenlan-amzn
Copy link
Member

@AWSHurneyt can you fix the workflow

For the description, can you add how would user use this feature? and a quick walkthrough of the implementation idea.

@@ -190,6 +191,19 @@ class AlertService(
)
}

// Including a list of triggered clusters for cluster metrics monitors
var triggeredClusters: MutableList<String>? = null
if (result is ClusterMetricsTriggerRunResult)
Copy link
Member

@eirsep eirsep Feb 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

composeQueryLevelAlert() method is now checking for ClusterMetricsTriggerRunResult?

isnt cluster metrics a separate kind of monitpr?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we not add triggered clusters metric to all monitor types' alerts?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. Cluster metrics monitors started out as query level monitors before we added additional monitor types when we introduced bucket level monitors. Their alerts were handled in a similar way up until this point, so I added this check in composeQueryLevelAlert rather than duplicate the code elsewhere.

@AWSHurneyt
Copy link
Collaborator Author

@AWSHurneyt can you fix the workflow

For the description, can you add how would user use this feature? and a quick walkthrough of the implementation idea.

@bowenlan-amzn I added some more details to the description.


private val log = LogManager.getLogger(RestGetRemoteIndexesAction::class.java)

class RestGetRemoteIndexesAction : BaseRestHandler() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

javadocs and code comments

private val log = LogManager.getLogger(RestGetRemoteIndexesAction::class.java)

class RestGetRemoteIndexesAction : BaseRestHandler() {
val ROUTE = "${AlertingPlugin.REMOTE_BASE_URI}/indexes"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indices*

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went with indexes because we had this issue a while back in which we were moving away from indices.
opensearch-project/index-management-dashboards-plugin#637

@@ -183,7 +187,8 @@ internal class AlertingPlugin : PainlessExtension, ActionPlugin, ScriptPlugin, R
RestGetWorkflowAlertsAction(),
RestGetFindingsAction(),
RestGetWorkflowAction(),
RestDeleteWorkflowAction()
RestDeleteWorkflowAction(),
RestGetRemoteIndexesAction(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indices*

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went with indexes because we had this issue a while back in which we were moving away from indices.
opensearch-project/index-management-dashboards-plugin#637

@eirsep
Copy link
Member

eirsep commented Feb 6, 2024

why are we adding support to fetch remote indices in alerting plugin? isn't that a core feature?

@AWSHurneyt
Copy link
Collaborator Author

why are we adding support to fetch remote indices in alerting plugin? isn't that a core feature?

@eirsep The GetRemoteIndexes API is used to gather details about the clusters and their indexes to display in the frontend. The frontend plugins do not have access to remoteClients to make most API calls against the remote clusters.

IndicesOptions.lenientExpand(),
true,
index
)
result.addAll(concreteIndices)
}

if (result.size == 0) {
throw IndexNotFoundException(indices[0])
Copy link
Member

@bowenlan-amzn bowenlan-amzn Feb 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel you want to do this above before result.addAll(concreteIndices)?
if concreteIndices is empty, throw exception?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this was implemented in PR #784.

@eirsep @sbcd90 do you recall if there was a particular reason why we considering throwing the exception after attempting to resolve all of the indexes? Would it be better to throw the exception if even 1 index failed to have any concrete indexes?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 now I don't understand why this code shows up here like you added this...

And adding on your question, why only shows indices[0] but not indices

client.threadPool().threadContext.stashContext().use {
scope.launch {
val singleThreadContext = newSingleThreadContext("GetRemoteIndexesActionThread")
withContext(singleThreadContext) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For your reference, this is how ISM does the user role injection

https://github.com/opensearch-project/index-management/blob/ba24f862a4b59bdac5a4def76033766346e9eea8/src/main/kotlin/org/opensearch/indexmanagement/opensearchapi/OpenSearchExtensions.kt#L280-L322

And seems QueryLevelMonitorRunner also does withClosableContext in it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah interesting. It looks like on this line ISM checks whether the user info is in the context, and injects it if not. Craig and I discussed doing something similar last week, but landed on the single thread implementation since we want to ensure the actions performed by the API take place in a specific sequence. Do you have any insight into the pros and cons of ISM's approach?
https://github.com/opensearch-project/index-management/blob/ba24f862a4b59bdac5a4def76033766346e9eea8/src/main/kotlin/org/opensearch/indexmanagement/opensearchapi/OpenSearchExtensions.kt#L300

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using ThreadContextElement is more natural to kotlin coroutine, since it's provided directly by them.
Basically what it does is to attach this element to the coroutine that will execute your code.

While force the user info in one thread, I feel it's possible that this thread may be used to execute other coroutine? then it may cause some problem.

Comment on lines +106 to +125
/**
* The [NodeClient] used by the plugin cannot execute searches against local indexes
* using format "<LOCAL_CLUSTER_NAME>:<INDEX_NAME>". That format only supports querying remote indexes.
* This function formats a list of indexes to be supplied directly to a [SearchRequest].
* @param indexes A list of index names in "<CLUSTER_NAME>:<INDEX_NAME>" format.
* @param localClusterName The name of the local cluster.
* @return A list of indexes with any remote indexes in "<CLUSTER_NAME>:<INDEX_NAME>" format,
* and any local indexes in "<INDEX_NAME>" format.
*/
@JvmStatic
fun parseIndexesForRemoteSearch(indexes: List<String>, localClusterName: String): List<String> {
return indexes.map {
var index = it
val clusterName = parseClusterName(it)
if (clusterName.isNotEmpty() && clusterName == localClusterName) {
index = parseIndexName(it)
}
index
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is confusing to me, in the end it seems only useful to user input <local cluster name>:<index> and transform it in to <index>
Instead of doing such thing, why not just let it fail?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the local cluster is named cluster-1, and a monitor input is configured with indexes ["cluster-1:index-1","cluster-2:index-2"], the node client of the local cluster will not be able to query the first index because the pattern references the local cluster name. This function will parse that list of indexes to ["index-1","cluster-2:index-2"] so it can be successfully fed into the node client.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I get this function. The confusing part is why we want to support user to use local cluster name in the input, does the other feature from UI also support that?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to cover cases where a user may do this without knowing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If user cannot do it in other place, this should also be forbidden here.
So to decide, try do a search call in a cross cluster environment with <local cluster name>:<index> on a local index, and see if that works. If so, this is ok.

val remoteMonitoringEnabled = monitorCtx.clusterService!!.clusterSettings.get(AlertingSettings.REMOTE_MONITORING_ENABLED)
logger.debug("Remote monitoring enabled: {}", remoteMonitoringEnabled)
if (remoteMonitoringEnabled) monitorCtx.triggerService!!.runClusterMetricsTrigger(monitor, trigger, triggerCtx, monitorCtx.clusterService!!)
else monitorCtx.triggerService!!.runQueryLevelTrigger(monitor, trigger, triggerCtx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, why cluster metric monitor run query level trigger?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cluster metrics monitors were initially query level monitors before bucket level introduced monitor types. If I remember correctly, we gave cluster metrics monitors a unique type around the implementation of doc level monitors. It could be nice to eventually give cluster metrics monitors their own assets like runner, and trigger; but at the time, their execution was so similar to query level, we didn't change them.

@@ -190,6 +191,19 @@ class AlertService(
)
}

// Including a list of triggered clusters for cluster metrics monitors
var triggeredClusters: MutableList<String>? = null
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider this is for experiemental, does here need a check for cluster setting for this feature?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that would be necessary, as the feature disabled, this will always be just the local cluster. But I'm open adding a setting check if we think that's best.

… a data class to an open class for inheritability.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>
Copy link
Member

@getsaurabh02 getsaurabh02 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets follow up on test failures via opening separate issues

@eirsep
Copy link
Member

eirsep commented Feb 6, 2024

have we made changes in security plugin?

@eirsep
Copy link
Member

eirsep commented Feb 6, 2024

no integ tests added to verify end to end behaviour?

@AWSHurneyt
Copy link
Collaborator Author

have we made changes in security plugin?

No, we didn't need to make any changes in the security plugin.

@AWSHurneyt
Copy link
Collaborator Author

no integ tests added to verify end to end behaviour?

This is not ideal, but so far testing has all been manual. I have implementing feature-specific tests as a fast follow-up item. For some additional context, query and bucket monitors already supported remote index querying. The major change in this PR is supporting remote calls for cluster metrics monitors.

@AWSHurneyt
Copy link
Collaborator Author

AWSHurneyt commented Feb 6, 2024

CVE PR #1405 adjusted the ktlint version, which is causing style failures upon rerunning the tests now that the PR opensearch-project/common-utils#584 changes have been published to maven. Cleaning up those errors.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>
Signed-off-by: AWSHurneyt <hurneyt@amazon.com>
@AWSHurneyt
Copy link
Collaborator Author

Lets follow up on test failures via opening separate issues

All tests are passing now. Merging in these changes.

@AWSHurneyt AWSHurneyt merged commit ea36996 into opensearch-project:main Feb 6, 2024
18 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/alerting/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/alerting/backport-2.x
# Create a new branch
git switch --create backport-1404-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 ea36996616eb91b2547b6a64274bbd9e50b1af5d
# Push it to GitHub
git push --set-upstream origin backport-1404-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/alerting/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport-1404-to-2.x.

AWSHurneyt added a commit to AWSHurneyt/OpenSearch-Alerting that referenced this pull request Feb 6, 2024
* Updated alert mappings to accommodate cross-cluster cluster metrics monitors.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Implemented support for cross-cluster cluster metrics monitors. Implemented GetRemoteIndexes API to populate the frontend UI with details regarding the remote clusters, and indexes.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Fixed a writeable test after changing QueryLevelTriggerRunResult from a data class to an open class for inheritability.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Fixed ktlint errors.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Removed changes to IndexUtils as they're only needed by doc monitors.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

---------

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

(cherry picked from commit ea36996)
Signed-off-by: AWSHurneyt <hurneyt@amazon.com>
AWSHurneyt added a commit that referenced this pull request Feb 6, 2024
* Implemented cross-cluster monitor support (#1404)

* Updated alert mappings to accommodate cross-cluster cluster metrics monitors.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Implemented support for cross-cluster cluster metrics monitors. Implemented GetRemoteIndexes API to populate the frontend UI with details regarding the remote clusters, and indexes.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Fixed a writeable test after changing QueryLevelTriggerRunResult from a data class to an open class for inheritability.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Fixed ktlint errors.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Removed changes to IndexUtils as they're only needed by doc monitors.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

---------

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

(cherry picked from commit ea36996)
Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Fixed a test.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

---------

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>
sbcd90 pushed a commit to sbcd90/alerting that referenced this pull request Mar 10, 2024
…ject#1307)

* Added 2.11.1 release notes.

* Added 2.11.1 release notes.

---------

(cherry picked from commit 06c1b8a)

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

fix workflow security tests in alerting (opensearch-project#1310) (opensearch-project#1311)

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

Increment version to 2.12.0-SNAPSHOT (opensearch-project#1239)

Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com>
Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com>

[Backport 2.x] Reference get monitor and search monitor action / request / responses from common-utils (opensearch-project#1315)

* Use get monitor action / req / resp from common-utils

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

* Dummy commit to retrigger

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

---------

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

optimize doc-level monitor execution workflow for datastreams (opensearch-project#1302) (opensearch-project#1322)

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

Update to Gradle 8.5 (opensearch-project#1369) (opensearch-project#1371)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

[Backport 2.x] Inject namedWriteableRegistry during ser/deser of SearchMonitorAction (opensearch-project#1382) (opensearch-project#1384)

* Inject namedWriteableRegistry during ser/deser of SearchMonitorAction (opensearch-project#1382)

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

* remove bin files

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

* remove core bin

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

---------

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

Don't attempt to parse workflow if it doesn't exist (opensearch-project#1346) (opensearch-project#1359)

(cherry picked from commit 733fd4e)

Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Set docData to empty string if actual is null (opensearch-project#1325) (opensearch-project#1334)

(cherry picked from commit 008e076)

Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

removed default admin credentials for alerting (opensearch-project#1399) (opensearch-project#1400)

(cherry picked from commit 3c50f7d)

Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dennis Toepker <toepkerd@amazon.com>

ipaddress lib upgrade as part of cve fix (opensearch-project#1397) (opensearch-project#1407)

(cherry picked from commit 8d59060)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Bulk index findings and sequentially invoke auto-correlations (opensearch-project#1355) (opensearch-project#1410)

* Bulk index findings and sequentially invoke auto-correlations

* Bulk index findings in batches of 10000 and make it configurable

* Addressing review comments

* Add integ tests to test bulk index findings

* Fix ktlint formatting

---------

(cherry picked from commit b561965)

Signed-off-by: Megha Goyal <goyamegh@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Add 2.12 release notes (opensearch-project#1408) (opensearch-project#1413)

* Add 2.12 release notes

* Fix release notes PR

* Add 2 more PRs

---------

(cherry picked from commit b10eaad)

Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

[Backport 2.x] Implemented cross-cluster monitor support (opensearch-project#1404) (opensearch-project#1412)

* Implemented cross-cluster monitor support (opensearch-project#1404)

* Updated alert mappings to accommodate cross-cluster cluster metrics monitors.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Implemented support for cross-cluster cluster metrics monitors. Implemented GetRemoteIndexes API to populate the frontend UI with details regarding the remote clusters, and indexes.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Fixed a writeable test after changing QueryLevelTriggerRunResult from a data class to an open class for inheritability.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Fixed ktlint errors.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Removed changes to IndexUtils as they're only needed by doc monitors.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

---------

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

(cherry picked from commit ea36996)
Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

* Fixed a test.

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

---------

Signed-off-by: AWSHurneyt <hurneyt@amazon.com>

Add publishToMavenLocal in build.sh (opensearch-project#1418) (opensearch-project#1419)

(cherry picked from commit 4cdc1d1)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

fix for MapperException[the [enabled] parameter can't be updated for the object mapping [metadata.source_to_query_index_mapping] (opensearch-project#1432) (opensearch-project#1434)

Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

bacport PRs opensearch-project#1445, opensearch-project#1430, opensearch-project#1441, opensearch-project#1435 to 2.x (opensearch-project#1452)

* Add jvm aware setting and max num docs settings for batching docs for percolate queries (opensearch-project#1435)

* add jvm aware and max docs settings for batching docs for percolate queries

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* fix stats logging

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* add queryfieldnames field in findings mapping

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

---------

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* optimize to fetch only fields relevant to doc level queries in doc level monitor instead of entire _source for each doc (opensearch-project#1441)

* optimize to fetch only fields relevant to doc level queries in doc level monitor

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* fix test for settings check

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* fix ktlint

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

---------

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* clean up doc level queries on dry run (opensearch-project#1430)

Signed-off-by: Joanne Wang <jowg@amazon.com>

* optimize sequence number calculation and reduce search requests in doc level monitor execution (opensearch-project#1445)

* optimize sequence number calculation and reduce search requests by n where n is number of shards being queried in the executino

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* fix tests

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* optimize check indices and execute to query only write index of aliases and datastreams during monitor creation

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* fix test

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* add javadoc

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

* add tests to verify seq_no calculation

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

---------

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>

---------

Signed-off-by: Surya Sashank Nistala <snistala@amazon.com>
Signed-off-by: Joanne Wang <jowg@amazon.com>
Co-authored-by: Joanne Wang <jowg@amazon.com>

[Backport 2.x] Add an _exists_ check to document level monitor queries (opensearch-project#1425) (opensearch-project#1456)

* Add an _exists_ check to document level monitor queries (opensearch-project#1425)

* clean up and add integ tests

Signed-off-by: Joanne Wang <jowg@amazon.com>

* refactored out common method and renamed test

Signed-off-by: Joanne Wang <jowg@amazon.com>

* remove _exists_ flag

Signed-off-by: Joanne Wang <jowg@amazon.com>

---------

Signed-off-by: Joanne Wang <jowg@amazon.com>

* fix integ test

Signed-off-by: Joanne Wang <jowg@amazon.com>

---------

Signed-off-by: Joanne Wang <jowg@amazon.com>

add distributed locking to jobs in alerting (opensearch-project#1403) (opensearch-project#1458)

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
@AWSHurneyt AWSHurneyt deleted the 3.0-cross-cluster-dev branch April 3, 2024 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants