Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Add subcommands in workload-transfer #1462

Merged
merged 8 commits into from
Feb 16, 2024
Merged

[docs] Add subcommands in workload-transfer #1462

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
d69bc09
[docs] Add subcommands in workload-transfer
Jornydb Jan 31, 2024
0aea09f
Update workload-transfer
Jornydb Jan 31, 2024
b51ea92
Update workload-transfer global version
Jornydb Jan 31, 2024
8dec4a9
Update workload-transfer
Jornydb Feb 1, 2024
c4c05b3
Apply suggestions from code review
Jornydb Feb 1, 2024
7a5495e
Update workload-transfer
Jornydb Feb 1, 2024
f039fce
Update workload-transfer
Jornydb Feb 15, 2024
efdd8be
Update workload-transfer
Jornydb Feb 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 26 additions & 22 deletions ydb/docs/en/core/reference/ydb-cli/workload-transfer.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,32 +76,35 @@ Parameter name | Parameter Description | Default value
`--consumer-threads`, `-t` | Number of consumer threads | `1`
`--consumers`, `-c` | Number of consumers | `1`
`--message-size`, `-m` | Message size in bytes. It is possible to specify in KB, MB, GB by adding suffixes `K`, `M`, `G` respectively | `10240`
`--message-rate` | Target total write speed. In messages per second. Excludes the use of the `--byte-rate` | `0` parameter
`--byte-rate` | Target total write speed. In bytes per second. Excludes the use of the --message-rate parameter. It is possible to specify in KB/s, MB/s, GB/s by adding suffixes `K`, `M`, `G` respectively | `0`
`--message-rate` | Target total write speed. In messages per second. Excludes the use of the `--byte-rate` parameter | `0`
`--byte-rate` | Target total write speed. In bytes per second. Excludes the use of the `--message-rate` parameter. It is possible to specify in KB/s, MB/s, GB/s by adding suffixes `K`, `M`, `G` respectively | `0`
`--commit-period` | The period between `COMMIT` calls. In seconds | `10`
`--commit-messages` | The period between `COMMIT` calls. In number of messages | `1000000`
`--only-topic-in-tx` | Only topic partitions are forced to participate in transactions. Excludes the use of the `--only-table-in-tx` parameter | `0`
`--only-table-in-tx` | Only table shards are forced to participate in transactions. Excludes the use of the `--only-topic-in-tx` parameter | `0`

For example, the command `{{ ydb-cli }} --profile quickstart workload transfer topic-to-table run` will run a test lasting 60 seconds. The data for the first 5 seconds will not be taken into account in the work statistics. Example of console output:

```text
Window Write speed Write time Inflight Lag Lag time Read speed Full time
# msg/s MB/s percentile,ms percentile,msg percentile,msg percentile,ms msg/s MB/s percentile,ms
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 103 1 911 78 0 0 103 1 914
7 103 1 983 78 0 0 103 1 984
8 103 1 1103 80 0 0 103 1 1106
9 103 1 1003 85 0 0 103 1 1004
10 103 1 1003 86 0 0 103 1 1006
11 103 1 1015 85 0 0 103 1 1019
12 103 1 1047 91 0 0 103 1 1043
13 103 1 999 88 0 0 103 1 1003
14 103 1 1063 85 0 0 103 1 1064
15 103 1 1003 89 0 0 103 1 1008
16 103 1 999 88 0 0 103 1 1003
17 103 1 1071 78 0 0 103 1 1077
Window Write speed Write time Inflight Read speed Topic time Select time Upsert time Commit time
# msg/s MB/s percentile,ms percentile,msg msg/s MB/s percentile,ms percentile,ms percentile,ms percentile,ms
1 0 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0
6 103 1 1023 83 103 1 1025 0 0 0
7 103 1 999 78 103 1 1001 0 0 0
8 103 1 1003 93 103 1 1002 0 0 0
9 103 1 1003 88 103 1 1003 0 0 0
10 103 1 999 79 103 1 999 0 0 0
11 103 1 1119 89 0 0 0 0 0 0
12 103 1 1023 90 206 2 1028 90 223 695
13 103 1 975 84 103 1 976 0 0 0
14 103 1 1003 91 103 1 1006 0 0 0
15 103 1 1003 93 103 1 1005 0 0 0
16 103 1 1103 89 103 1 1100 0 0 0
17 103 1 1063 89 103 1 1061 0 0 0
...
```

Expand All @@ -112,7 +115,8 @@ Window Write speed Write time Inflight Lag Lag time
* `Lag` — the specified percentile of maximum number of messages waiting to be read in the statistics collection window. Messages for all batches are taken into account.
* `Lag time` — the specified percentile of message delay time in ms.
* `Read speed` — the speed of reading messages by consumers. In messages per second and in megabytes per second.
* `Full time` — the specified percentile of the time of complete processing of the message, from writing by the producer to reading by the consumer in ms.
* `Select time`, `Upsert time`, `Commit time` — the specified percentile of the execution time of Select, Insert, Commit operations in ms.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't see this columns in the table above. It may be OK, just noticed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't see this columns in the table above. It may be OK, just noticed.
✅

<!-- * `Full time` — the specified percentile of the time of complete processing of the message, from writing by the producer to reading by the consumer in ms. -->

## Removing the test environment {#clean}

Expand Down
48 changes: 26 additions & 22 deletions ydb/docs/ru/core/reference/ydb-cli/workload-transfer.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,11 @@
`--consumers`, `-c` | Количество читателей | `1`
`--message-size`, `-m` | Размер сообщения в байтах. Возможно задание в КБ, МБ, ГБ путем добавления суффиксов `K`, `M`, `G` соответственно | `10240`
`--message-rate` | Целевая суммарная скорость записи. В сообщениях в секунду. Исключает использование параметра `--byte-rate` | `0`
`--byte-rate` | Целевая суммарная скорость записи. В байтах в секунду. Исключает использование параметра --message-rate. Возможно задание в КБ/с, МБ/с, ГБ/с путем добавления суффиксов `K`, `M`, `G` соответственно | `0`
`--commit-period` | Период между вызовами `COMMIT`. В секундах | `10`
`--byte-rate` | Целевая суммарная скорость записи. В байтах в секунду. Исключает использование параметра `--message-rate`. Возможно задание в КБ/с, МБ/с, ГБ/с путем добавления суффиксов `K`, `M`, `G`, соответственно | `0`
`--commit-period` | Период между вызовами `COMMIT`. В секундах | `10`
`--commit-messages` | Период между вызовами `COMMIT`. В количестве сообщений | `1000000`
`--only-topic-in-tx` | В транзакциях принудительно участвуют только партиции топиков. Исключает использование параметра `--only-table-in-tx` | `0`
`--only-table-in-tx` | В транзакциях принудительно участвуют только шарды таблиц. Исключает использование параметра `--only-topic-in-tx` | `0`

Например, команда
```bash
Expand All @@ -91,25 +94,25 @@
запустит тест продолжительностью 60 секунд. Данные за первые 5 секунд не будут учитываться в статистике работы. Пример вывода на консоль

```text
Window Write speed Write time Inflight Lag Lag time Read speed Full time
# msg/s MB/s percentile,ms percentile,msg percentile,msg percentile,ms msg/s MB/s percentile,ms
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 103 1 911 78 0 0 103 1 914
7 103 1 983 78 0 0 103 1 984
8 103 1 1103 80 0 0 103 1 1106
9 103 1 1003 85 0 0 103 1 1004
10 103 1 1003 86 0 0 103 1 1006
11 103 1 1015 85 0 0 103 1 1019
12 103 1 1047 91 0 0 103 1 1043
13 103 1 999 88 0 0 103 1 1003
14 103 1 1063 85 0 0 103 1 1064
15 103 1 1003 89 0 0 103 1 1008
16 103 1 999 88 0 0 103 1 1003
17 103 1 1071 78 0 0 103 1 1077
Window Write speed Write time Inflight Read speed Topic time Select time Upsert time Commit time
# msg/s MB/s percentile,ms percentile,msg msg/s MB/s percentile,ms percentile,ms percentile,ms percentile,ms
1 0 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0
6 103 1 1023 83 103 1 1025 0 0 0
7 103 1 999 78 103 1 1001 0 0 0
8 103 1 1003 93 103 1 1002 0 0 0
9 103 1 1003 88 103 1 1003 0 0 0
10 103 1 999 79 103 1 999 0 0 0
11 103 1 1119 89 0 0 0 0 0 0
12 103 1 1023 90 206 2 1028 90 223 695
13 103 1 975 84 103 1 976 0 0 0
14 103 1 1003 91 103 1 1006 0 0 0
15 103 1 1003 93 103 1 1005 0 0 0
16 103 1 1103 89 103 1 1100 0 0 0
17 103 1 1063 89 103 1 1061 0 0 0
...
```

Expand All @@ -120,7 +123,8 @@ Window Write speed Write time Inflight Lag Lag time
* `Lag` — максимальное число сообщений, ожидающих чтения, в окне сбора статистики. Учитываются сообщения по всем партициям.
* `Lag time` — процентиль времени задержки сообщений в мс.
* `Read speed` — скорость чтения сообщений читателями. В сообщениях в секунду и в мегабайтах в секунду.
* `Full time` — процентиль времени полной обработки сообщения, от записи писателем до чтения читателем в мс.
* `Select time`, `Upsert time`, `Commit time` — процентиль времени выполнения операций Select, Upsert, Commit в мс.
<!-- * `Full time` — процентиль времени полной обработки сообщения, от записи писателем до чтения читателем в мс. -->

## Удаление тестового окружения {#clean}

Expand Down
Loading