Skip to content

Commit

Permalink
[8.4] Rollover min_* conditions docs and highlight (elastic#89434) (e…
Browse files Browse the repository at this point in the history
  • Loading branch information
joegallo authored Aug 17, 2022
1 parent d4e83af commit a3e7603
Show file tree
Hide file tree
Showing 4 changed files with 107 additions and 14 deletions.
30 changes: 30 additions & 0 deletions docs/changelog/83345.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,33 @@ summary: Add min_* conditions to rollover
area: ILM+SLM
type: enhancement
issues: []
highlight:
title: Minimum conditions for the rollover API and ILM actions
body: |-
The rollover API and ILM actions now support minimum conditions for rollover.
Minimum conditions prevent rollover from occuring until they are met. That is, an index
will rollover once one or more max conditions are satisfied and all min conditions are satisfied.
As an example, the following ILM policy would roll an index over if it is at least 7 days old or
at least 100 gigabytes, but only as long as the index is not empty.
```
PUT _ilm/policy/my_policy
{
"policy": {
"phases": {
"hot": {
"actions": {
"rollover" : {
"max_age": "7d",
"max_size": "100gb",
"min_docs": 1
}
}
}
}
}
}
```
notable: true
77 changes: 68 additions & 9 deletions docs/reference/ilm/actions/ilm-rollover.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@

Phases allowed: hot.

Rolls over a target to a new index when the existing index meets one or more of the rollover conditions.
Rolls over a target to a new index when the existing index satisfies
the specified rollover conditions.

IMPORTANT: If the rollover action is used on a <<ccr-put-follow,follower index>>,
policy execution waits until the leader index rolls over (or is
Expand Down Expand Up @@ -45,8 +46,11 @@ PUT my-index-000001
[[ilm-rollover-options]]
==== Options

You must specify at least one rollover condition.
An empty rollover action is invalid.
A rollover action must specify at least one max_* condition, it may include zero
or more min_* conditions. An empty rollover action is invalid.

The index will rollover once any max_* condition is satisfied and all
min_* conditions are satisfied.

// tag::rollover-conditions[]
`max_age`::
Expand Down Expand Up @@ -90,6 +94,32 @@ replicas are ignored.
+
TIP: To see the current shard docs, use the <<cat-shards, _cat shards>> API.
The `docs` value shows the number of documents each shard.

`min_age`::
(Optional, <<time-units, time units>>)
Prevents rollover until after the minimum elapsed time from index creation is reached.
See notes on `max_age`.

`min_docs`::
(Optional, integer)
Prevents rollover until after the specified minimum number of documents is reached.
See notes on `max_docs`.

`min_size`::
(Optional, <<byte-units, byte units>>)
Prevents rollover until the index reaches a certain size.
See notes on `max_size`.

`min_primary_shard_size`::
(Optional, <<byte-units, byte units>>)
Prevents rollover until the largest primary shard in the index reaches a certain size.
See notes on `max_primary_shard_size`.

`min_primary_shard_docs`::
(Optional, integer)
Prevents rollover until the largest primary shard in the index reaches a certain number of documents.
See notes on `max_primary_shard_docs`.

// end::rollover-conditions[]

[[ilm-rollover-ex]]
Expand All @@ -109,7 +139,7 @@ PUT _ilm/policy/my_policy
"hot": {
"actions": {
"rollover" : {
"max_primary_shard_size": "50GB"
"max_primary_shard_size": "50gb"
}
}
}
Expand All @@ -132,7 +162,7 @@ PUT _ilm/policy/my_policy
"hot": {
"actions": {
"rollover" : {
"max_size": "100GB"
"max_size": "100gb"
}
}
}
Expand Down Expand Up @@ -214,8 +244,9 @@ PUT _ilm/policy/my_policy
===== Roll over using multiple conditions

When you specify multiple rollover conditions,
the index is rolled over when _any_ of the conditions are met.
This example rolls the index over if it is at least 7 days old or at least 100 gigabytes.
the index is rolled over when _any_ of the max_* and _all_ of the min_* conditions are met.
This example rolls the index over if it is at least 7 days old or at least 100 gigabytes,
but only as long as the index is not empty.

[source,console]
--------------------------------------------------
Expand All @@ -227,7 +258,35 @@ PUT _ilm/policy/my_policy
"actions": {
"rollover" : {
"max_age": "7d",
"max_size": "100GB"
"max_size": "100gb",
"min_docs": 1
}
}
}
}
}
}
--------------------------------------------------

[ilm-rollover-conditions-ex]]
===== Roll over while maintaining shard sizes

This example rolls the index over when the primary shard size is at least 50gb,
or when the index is at least 30 days old, but only as long as a primary shard is at least 1gb.
For low-volume indices, this prevents the creation of many small shards.

[source,console]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
"policy": {
"phases": {
"hot": {
"actions": {
"rollover" : {
"max_primary_shard_size": "50gb",
"max_age": "30d",
"min_primary_shard_size": "1gb"
}
}
}
Expand All @@ -254,7 +313,7 @@ PUT /_ilm/policy/rollover_policy
"hot": {
"actions": {
"rollover": {
"max_size": "50GB"
"max_size": "50gb"
}
}
},
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/ilm/index-rollover.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ On each rollover, the new index becomes the write index.
=== Automatic rollover

{ilm-init} enables you to automatically roll over to a new index based
on the index size, document count, or age. When a rollover is triggered, a new
on conditions like the index size, document count, or age. When a rollover is triggered, a new
index is created, the write alias is updated to point to the new index, and all
subsequent updates are written to the new index.

Expand Down
12 changes: 8 additions & 4 deletions docs/reference/indices/rollover-index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ include::{es-repo-dir}/indices/create-index.asciidoc[tag=index-name-reqs]

`dry_run`::
(Optional, Boolean)
If `true`, checks whether the current index matches one or more specified
If `true`, checks whether the current index satisfies the specified
`conditions` but does not perform a rollover. Defaults to `false`.

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=wait_for_active_shards]
Expand All @@ -132,10 +132,14 @@ include::{es-repo-dir}/indices/create-index.asciidoc[tag=aliases-props]
`conditions`::
(Optional, object)
Conditions for the rollover. If specified, {es} only performs the rollover if
the current index meets one or more of these conditions. If this parameter is
the current index satisfies these conditions. If this parameter is
not specified, {es} performs the rollover unconditionally.
+
IMPORTANT: To trigger a rollover, the current index must meet these conditions
If conditions are specified, at least one of them must be a max_* condition.
The index will rollover if any max_* condition is satisfied and all
min_* conditions are satisfied.
+
IMPORTANT: To trigger a rollover, the current index must satisfy these conditions
at the time of the request. {es} does not monitor the index after the API
response. To automate rollover, use {ilm-init}'s <<ilm-rollover,`rollover`>>
instead.
Expand Down Expand Up @@ -197,7 +201,7 @@ conditions were specified, this is an empty object.
====
`<condition>`::
(Boolean) The key is each condition. The value is its result. If `true`, the
index met the condition at rollover.
index met the condition.
====

[[rollover-index-api-example]]
Expand Down

0 comments on commit a3e7603

Please sign in to comment.