Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Concurrent segment search GA and API changes #6356

Merged
merged 16 commits into from
Feb 12, 2024
Merged
83 changes: 37 additions & 46 deletions _api-reference/index-apis/stats.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,41 @@ GET /testindex/_stats
```
{% include copy-curl.html %}

#### Example request: Comma-separated list of indexes

```json
GET /testindex1,testindex2/_stats
```
{% include copy-curl.html %}

#### Example request: Wildcard expression

```json
GET /testindex*/_stats
```
{% include copy-curl.html %}

#### Example request: Specific stats

```json
GET /testindex/_stats/refresh,flush
```
{% include copy-curl.html %}

#### Example request: Expand wildcards

```json
GET /testindex*/_stats?expand_wildcards=open,hidden
```
{% include copy-curl.html %}

#### Example request: Shard-level statistics

```json
GET /testindex/_stats?level=shards
```
{% include copy-curl.html %}

#### Example response

By default, the returned statistics are aggregated in the `primaries` and `total` aggregations. The `primaries` aggregation contains statistics for the primary shards. The `total` aggregation contains statistics for both primary and replica shards. The following is an example Index Stats API response:
Expand Down Expand Up @@ -773,50 +808,6 @@ By default, the returned statistics are aggregated in the `primaries` and `total
```
</details>

#### Example request: Comma-separated list of indexes

```json
GET /testindex1,testindex2/_stats
```
{% include copy-curl.html %}

#### Example request: Wildcard expression

```json
GET /testindex*/_stats
```
{% include copy-curl.html %}

#### Example request: Specific stats

```json
GET /testindex/_stats/refresh,flush
```
{% include copy-curl.html %}

#### Example request: Expand wildcards

```json
GET /testindex*/_stats?expand_wildcards=open,hidden
```
{% include copy-curl.html %}

#### Example request: Shard-level statistics

```json
GET /testindex/_stats?level=shards
```
{% include copy-curl.html %}

## Concurrent segment search

Starting in OpenSearch 2.10, [concurrent segment search]({{site.url}}{{site.baseurl}}/search-plugins/concurrent-segment-search/) allows each shard-level request to search segments in parallel during the query phase. If you [enable the experimental concurrent segment search feature flag]({{site.url}}{{site.baseurl}}/search-plugins/concurrent-segment-search#enabling-the-feature-flag), the Index Stats API response will contain several additional fields with statistics about slices (units of work executed by a thread). These fields will be provided whether or not the cluster and index settings for concurrent segment search are enabled. For more information about slices, see [Concurrent segment search]({{site.url}}{{site.baseurl}}/search-plugins/concurrent-segment-search#searching-segments-concurrently).

The following table provides information about the added response fields.
## Response fields

|Response field | Description |
|:--- |:--- |
|`search.concurrent_avg_slice_count` |The average slice count of all search requests. This is computed as the total slice count divided by the total number of concurrent search requests. |
|`search.concurrent_query_total` |The total number of query operations that use concurrent segment search. |
|`search.concurrent_query_time_in_millis` |The total amount of time taken by all query operations that use concurrent segment search, in milliseconds. |
|`search.concurrent_query_current` |The number of currently running query operations that use concurrent segment search. |
For information about response fields, see [Nodes Stats API response fields]({{site.url}}{{site.baseurl}}/api-reference/nodes-apis/nodes-stats/#indices)
kolchfa-aws marked this conversation as resolved.
Show resolved Hide resolved
9 changes: 4 additions & 5 deletions _api-reference/nodes-apis/nodes-stats.md
Original file line number Diff line number Diff line change
Expand Up @@ -831,6 +831,10 @@ get.missing_total | Integer | The number of failed get operations.
get.missing_time_in_millis | Integer | The total time for all failed get operations, in milliseconds.
get.current | Integer | The number of get operations that are currently running.
search | Object | Statistics about the search operations for the node.
search.concurrent_avg_slice_count | Integer | The average slice count of all search requests. This is computed as the total slice count divided by the total number of concurrent search requests.
search.concurrent_query_total |Integer | The total number of query operations that use concurrent segment search.
search.concurrent_query_time_in_millis | Integer | The total amount of time taken by all query operations that use concurrent segment search, in milliseconds.
search.concurrent_query_current |Integer | The number of currently running query operations that use concurrent segment search.
search.open_contexts | Integer | The number of open search contexts.
search.query_total | Integer | The total number of shard query operations.
search.query_time_in_millis | Integer | The total amount of time for all shard query operations, in milliseconds.
Expand Down Expand Up @@ -1259,11 +1263,6 @@ Field | Field type | Description
admission_control.global_cpu_usage.transport.rejection_count.search | Integer | The total number of search rejections in the transport layer when the node CPU usage limit was breached. In this case, additional search requests are rejected until the system recovers.
admission_control.global_cpu_usage.transport.rejection_count.indexing | Integer | The total number of indexing rejections in the transport layer when the node CPU usage limit was breached. In this case, additional indexing requests are rejected until the system recovers.


## Concurrent segment search

Starting in OpenSearch 2.10, [concurrent segment search]({{site.url}}{{site.baseurl}}/search-plugins/concurrent-segment-search/) allows each shard-level request to search segments in parallel during the query phase. If you [enable the experimental concurrent segment search feature flag]({{site.url}}{{site.baseurl}}/search-plugins/concurrent-segment-search#enabling-the-feature-flag), the Nodes Stats API response will contain several additional fields with statistics about slices (units of work executed by a thread). For the descriptions of those fields, see [Index Stats API]({{site.url}}{{site.baseurl}}/api-reference/index-apis/stats#concurrent-segment-search).

## Required permissions

If you use the Security plugin, make sure you have the appropriate permissions: `cluster:monitor/nodes/stats`.
Loading