include tiering configuration in ttl docs (#11704)

Co-authored-by: Ivan Blinkov <ivan@ydb.tech>
ydb-platform · Jan 19, 2025 · a096daa · a096daa
1 parent 3f1aaf1
commit a096daa
Show file tree

Hide file tree

Showing 30 changed files with 935 additions and 581 deletions.
diff --git a/ydb/docs/en/core/_includes/not_allow_for_oltp_note.md b/ydb/docs/en/core/_includes/not_allow_for_oltp_note.md
@@ -0,0 +1,9 @@
+{% if oss == true and backend_name == "YDB" %}
+
+{% note warning %}
+
+{% include [OLTP_not_allow_text](not_allow_for_oltp_text.md) %}
+
+{% endnote %}
+
+{% endif %}
diff --git a/ydb/docs/en/core/_includes/not_allow_for_oltp_text.md b/ydb/docs/en/core/_includes/not_allow_for_oltp_text.md
@@ -0,0 +1 @@
+Supported only for [column-oriented](../concepts/datamodel/table.md#column-oriented-tables) tables. Support for [row-oriented](../concepts/datamodel/table.md#row-oriented-tables) tables is currently under development.
diff --git a/ydb/docs/en/core/concepts/_includes/ttl.md b/ydb/docs/en/core/concepts/_includes/ttl.md
@@ -1,26 +1,22 @@
-# Time to Live (TTL)
+# Time to Live (TTL) and eviction to external storage
 
-This section describes how the TTL mechanism works and what its limits are. It also gives examples of commands and code snippets that can be used to enable, configure, and disable TTL.
+This section describes how the TTL mechanism works and what its limits are.
 
 ## How it works {#how-it-works}
 
-{{ ydb-short-name }} allows you to specify a TTL column in both [row-oriented](../datamodel/table.md#row-oriented-tables) and [column-oriented](../datamodel/table.md#column-oriented-tables) tables. Values in TTL columns determine the table rows lifetime.
+The table's TTL is a sequence of storage tiers. Each tier contains an expression (TTL expression) and an action. When the expression is triggered, that tier is assigned to the row. When a tier is assigned to a row, the specified action is automatically performed: moving the row to external storage or deleting it. External storage is represented by the [external data source](../datamodel/external_data_source.md) object.
 
-{% note warning %}
-
-An item with the `NULL` value in the TTL column is never deleted.
-
-{% endnote %}
+{{ ydb-short-name }} allows you to specify a column (TTL column) whose values are used in TTL expressions. The expression is triggered when the specified number of seconds has passed since the time recorded in the TTL column. For rows with `NULL` value in TTL column, the expression is not triggered.
 
 The timestamp for deleting a table item is determined by the formula:
 
 ```text
-expiration_time = valueof(ttl_column) + expire_after_seconds
+eviction_time = valueof(ttl_column) + evict_after_seconds
 ```
 
 {% note info %}
 
-TTL doesn't guarantee that the item will be deleted exactly at `expiration_time`, it might happen later. If it's important to exclude logically obsolete but not yet physically deleted items from the selection, use query-level filtering.
+TTL doesn't guarantee that the item will be deleted exactly at `eviction_time`, it might happen later. If it's important to exclude logically obsolete but not yet physically deleted items from the selection, use query-level filtering.
 
 {% endnote %}
 
@@ -58,273 +54,14 @@ The *BRO* has the following properties:
   * Nanoseconds.
 
 * You can't specify multiple TTL columns.
-* You can't delete the TTL column. However, if this is required, you should first [disable TTL](#disable) for the table.
+* You can't delete the TTL column. However, if this is required, you should first [disable TTL](../../recipes/yql/ttl.md#disable) for the table.
+* Only {{ objstorage-name }} is supported as external storage.
+* The delete action can only be specified for the last tier.
 
 ## Setup {#setting}
 
 Currently, you can manage TTL settings using:
 
-* [YQL](../../yql/reference/index.md).
-* [{{ ydb-short-name }} console client](../../reference/ydb-cli/index.md).
-* {{ ydb-short-name }} {% if oss %}C++, {% endif %}Go and Python [SDK](../../reference/ydb-sdk/index.md).
-
-### Enabling TTL for an existing table {#enable-on-existent-table}
-
-In the example below, the items of the `mytable` table will be deleted an hour after the time set in the `created_at` column:
-
-{% list tabs group=tool %}
-
-- YQL
-
-  ```yql
-  ALTER TABLE `mytable` SET (TTL = Interval("PT1H") ON created_at);
-  ```
-
-- CLI
-
-  ```bash
-  $ {{ ydb-cli }} -e <endpoint> -d <database> table ttl set --column created_at --expire-after 3600 mytable
-  ```
-
-{% if oss == true %}
-
-- C++
-
-  ```c++
-  session.AlterTable(
-    "mytable",
-    TAlterTableSettings()
-      .BeginAlterTtlSettings()
-        .Set("created_at", TDuration::Hours(1))
-      .EndAlterTtlSettings()
-  );
-  ```
-
-{% endif %}
-
-- Go
-
-  ```go
-  err := session.AlterTable(ctx, "mytable",
-    options.WithSetTimeToLiveSettings(
-      options.NewTTLSettings().ColumnDateType("created_at").ExpireAfter(time.Hour),
-    ),
-  )
-  ```
-
-- Python
-
-  ```python
-  session.alter_table('mytable', set_ttl_settings=ydb.TtlSettings().with_date_type_column('created_at', 3600))
-  ```
-
-{% endlist %}
-
-{% note tip %}
-
-When setting up TTL using YQL, an `Interval` is created from a string literal in [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format with [some restrictions](../../yql/reference/builtins/basic#data-type-literals).
-
-{% endnote %}
-
-The example below shows how to use the `modified_at` column with a numeric type (`Uint32`) as a TTL column. The column value is interpreted as the number of seconds since the Unix epoch:
-
-{% list tabs group=tool %}
-
-- YQL
-
-  ```yql
-  ALTER TABLE `mytable` SET (TTL = Interval("PT1H") ON modified_at AS SECONDS);
-  ```
-
-- CLI
-
-  ```bash
-  $ {{ ydb-cli }} -e <endpoint> -d <database> table ttl set --column modified_at --expire-after 3600 --unit seconds mytable
-  ```
-
-{% if oss == true %}
-
-- C++
-
-  ```c++
-  session.AlterTable(
-    "mytable",
-    TAlterTableSettings()
-      .BeginAlterTtlSettings()
-        .Set("modified_at", TTtlSettings::EUnit::Seconds, TDuration::Hours(1))
-      .EndAlterTtlSettings()
-  );
-  ```
-
-{% endif %}
-
-- Go
-
-  ```go
-  err := session.AlterTable(ctx, "mytable",
-    options.WithSetTimeToLiveSettings(
-      options.NewTTLSettings().ColumnSeconds("modified_at").ExpireAfter(time.Hour),
-    ),
-  )
-  ```
-
-- Python
-
-  ```python
-  session.alter_table('mytable', set_ttl_settings=ydb.TtlSettings().with_value_since_unix_epoch('modified_at', UNIT_SECONDS, 3600))
-  ```
-
-{% endlist %}
-
-### Enabling TTL for a newly created table {#enable-for-new-table}
-
-For a newly created table, you can pass TTL settings along with the table description:
-
-{% list tabs group=tool %}
-
-- YQL
-
-  ```yql
-  CREATE TABLE `mytable` (
-    id Uint64,
-    expire_at Timestamp,
-    PRIMARY KEY (id)
-  ) WITH (
-    TTL = Interval("PT0S") ON expire_at
-  );
-  ```
-
-{% if oss == true %}
-
-- C++
-
-  ```c++
-  session.CreateTable(
-    "mytable",
-    TTableBuilder()
-      .AddNullableColumn("id", EPrimitiveType::Uint64)
-      .AddNullableColumn("expire_at", EPrimitiveType::Timestamp)
-      .SetPrimaryKeyColumn("id")
-      .SetTtlSettings("expire_at")
-      .Build()
-  );
-  ```
-
-{% endif %}
-
-- Go
-
-  ```go
-  err := session.CreateTable(ctx, "mytable",
-    options.WithColumn("id", types.Optional(types.TypeUint64)),
-    options.WithColumn("expire_at", types.Optional(types.TypeTimestamp)),
-    options.WithTimeToLiveSettings(
-      options.NewTTLSettings().ColumnDateType("expire_at"),
-    ),
-  )
-  ```
-
-- Python
-
-  ```python
-  session.create_table(
-    'mytable',
-    ydb.TableDescription()
-      .with_column(ydb.Column('id', ydb.OptionalType(ydb.DataType.Uint64)))
-      .with_column(ydb.Column('expire_at', ydb.OptionalType(ydb.DataType.Timestamp)))
-      .with_primary_key('id')
-      .with_ttl(ydb.TtlSettings().with_date_type_column('expire_at'))
-  )
-  ```
-
-{% endlist %}
-
-### Disabling TTL {#disable}
-
-{% list tabs group=tool %}
-
-- YQL
-
-  ```yql
-  ALTER TABLE `mytable` RESET (TTL);
-  ```
-
-- CLI
-
-  ```bash
-  $ {{ ydb-cli }} -e <endpoint> -d <database> table ttl reset mytable
-  ```
-
-{% if oss == true %}
-
-- C++
-
-  ```c++
-  session.AlterTable(
-    "mytable",
-    TAlterTableSettings()
-      .BeginAlterTtlSettings()
-        .Drop()
-      .EndAlterTtlSettings()
-  );
-  ```
-
-{% endif %}
-
-- Go
-
-  ```go
-  err := session.AlterTable(ctx, "mytable",
-    options.WithDropTimeToLive(),
-  )
-  ```
-
-- Python
-
-  ```python
-  session.alter_table('mytable', drop_ttl_settings=True)
-  ```
-
-{% endlist %}
-
-### Getting TTL settings {#describe}
-
-The current TTL settings can be obtained from the table description:
-
-{% list tabs group=tool %}
-
-- CLI
-
-  ```bash
-  $ {{ ydb-cli }} -e <endpoint> -d <database> scheme describe mytable
-  ```
-
-{% if oss == true %}
-
-- C++
-
-  ```c++
-  auto desc = session.DescribeTable("mytable").GetValueSync().GetTableDescription();
-  auto ttl = desc.GetTtlSettings();
-  ```
-
-{% endif %}
-
-- Go
-
-  ```go
-  desc, err := session.DescribeTable(ctx, "mytable")
-  if err != nil {
-    // process error
-  }
-  ttl := desc.TimeToLiveSettings
-  ```
-
-- Python
-
-  ```python
-  desc = session.describe_table('mytable')
-  ttl = desc.ttl_settings
-  ```
-
-{% endlist %}
+* [YQL](../../recipes/yql/ttl.md).
+* [{{ ydb-short-name }} console client](../../recipes/ydb-cli/ttl.md).
+* {{ ydb-short-name }} {% if oss %}C++, {% endif %}Go and Python [SDK](../../recipes/ydb-sdk/ttl.md).
diff --git a/ydb/docs/en/core/concepts/datamodel/_includes/table.md b/ydb/docs/en/core/concepts/datamodel/_includes/table.md
@@ -134,16 +134,9 @@ If there are multiple followers, their delay from the leader may vary: although
 
 | Parameter name | Type | Acceptable values | Update capability | Reset capability |
 | ------------- | --- | ------------------- | --------------------- | ------------------ |
-| `TTL` | Expression | `Interval("<literal>") ON <column> [AS <unit>]` | Yes | Yes |
+| `TTL` | Expression | `Interval("<literal>") ON <column> [AS <unit>]` or `Interval("literal1") action1, ..., Interval("literal1") action1 ON <column> [AS <unit>]` | Yes | Yes |
 
-Where `<unit>` is a unit of measurement, specified only for column with a [numeric type](../../../concepts/ttl.md#restrictions):
-
-* `SECONDS`
-* `MILLISECONDS`
-* `MICROSECONDS`
-* `NANOSECONDS`
-
-For more information about deleting expired data, see [Time to Live (TTL)](../../../concepts/ttl.md).
+Syntax of TTL value is described in the article [{#T}](../../../yql/reference/syntax/create_table/with.md#time-to-live). For more information about deleting expired data, see [Time to Live (TTL)](../../../concepts/ttl.md).
 
 ### Renaming a table {#rename}
 

diff --git a/ydb/docs/en/core/concepts/toc_i.yaml b/ydb/docs/en/core/concepts/toc_i.yaml
@@ -19,7 +19,7 @@ items:
 - name: Change Data Capture (CDC)
   href: cdc.md
   when: feature_changefeed
-- name: Time to Live (TTL)
+- name: Time to live and eviction
   href: ttl.md
 - name: Scan queries
   href: scan_query.md

diff --git a/ydb/docs/en/core/recipes/ydb-cli/index.md b/ydb/docs/en/core/recipes/ydb-cli/index.md
@@ -6,6 +6,7 @@ Table of contents:
 
 * [{#T}](convert-table-type.md)
 * [{#T}](benchmarks.md)
+* [{#T}](ttl.md)
 
 See also:
 

diff --git a/ydb/docs/en/core/recipes/ydb-cli/toc_p.yaml b/ydb/docs/en/core/recipes/ydb-cli/toc_p.yaml
@@ -2,4 +2,6 @@ items:
 - name: Convert table type
   href: convert-table-type.md
 - name: Benchmarks
-  href: benchmarks.md
+  href: benchmarks.md
+- name: Time to live (TTL)
+  href: ttl.md
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Supported only for [column-oriented](../concepts/datamodel/table.md#column-oriented-tables) tables. Support for [row-oriented](../concepts/datamodel/table.md#row-oriented-tables) tables is currently under development.