Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add tx progress counters #7407

Merged
merged 11 commits into from
Aug 6, 2024
Merged

add tx progress counters #7407

merged 11 commits into from
Aug 6, 2024

Conversation

swalrus1
Copy link
Collaborator

@swalrus1 swalrus1 commented Aug 2, 2024

  • add Monitoring counters for number of transactions completing specific stages
  • refactor counters

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 10:12:01 UTC Pre-commit check for 5578d6c has started.
⚫ 2024-08-02 10:14:24 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 10:15:50 UTC Pre-commit check for 9f24faf has started.
⚫ 2024-08-02 10:16:53 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 10:15:52 UTC Pre-commit check for 9f24faf has started.
⚫ 2024-08-02 10:17:02 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 10:15:53 UTC Pre-commit check for 9f24faf has started.
⚫ 2024-08-02 10:17:43 UTC Check cancelled

@swalrus1 swalrus1 force-pushed the refactor-counters-clean-up-total-stats branch from d91bb76 to 1644ad1 Compare August 2, 2024 10:17
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 10:19:16 UTC Pre-commit check for 61fefff has started.
⚪ 2024-08-02 10:22:20 UTC Check linux-x86_64-release-clang14 is running...
⚫ 2024-08-02 10:25:20 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 10:21:31 UTC Pre-commit check for 61fefff has started.
⚪ 2024-08-02 10:24:29 UTC Check linux-x86_64-relwithdebinfo is running...
⚫ 2024-08-02 10:25:10 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 10:21:34 UTC Pre-commit check for 61fefff has started.
⚪ 2024-08-02 10:24:31 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-08-02 10:25:10 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 10:26:42 UTC Pre-commit check for 775bfe9 has started.
⚪ 2024-08-02 10:29:29 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-02 10:41:11 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 10:28:29 UTC Pre-commit check for 775bfe9 has started.
⚪ 2024-08-02 10:31:53 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-08-02 12:13:02 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 10:29:00 UTC Pre-commit check for 775bfe9 has started.
⚪ 2024-08-02 10:32:44 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-02 12:06:22 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13803 12000 0 435 1333 35

⚪ 2024-08-02 12:08:01 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-02 12:13:38 UTC ydbd size 8.1 GiB changed* by +1.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 53e1321 merge: 775bfe9 diff diff %
ydbd size 8 664 604 632 Bytes 8 664 605 664 Bytes +1.0 KiB +0.000%
ydbd stripped size 471 407 304 Bytes 471 415 112 Bytes +7.6 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation
⚫ 2024-08-02 12:13:39 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 12:15:32 UTC Pre-commit check for f8d5548 has started.
⚪ 2024-08-02 12:18:51 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-08-02 12:35:46 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 12:17:05 UTC Pre-commit check for f8d5548 has started.
⚪ 2024-08-02 12:20:08 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-02 12:27:42 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 12:17:07 UTC Pre-commit check for f8d5548 has started.
⚪ 2024-08-02 12:19:50 UTC Check linux-x86_64-relwithdebinfo is running...
⚫ 2024-08-02 12:35:51 UTC Check cancelled

ivanmorozov333
ivanmorozov333 reviewed Aug 2, 2024
View reviewed changes
ydb/core/tx/columnshard/counters/columnshard.h Outdated
@@ -66,11 +65,92 @@ class TCSInitialization: public TCommonCountersOwner {
}
};

class TCSCounters: public TCommonCountersOwner {
class TTxProgressCounters: public TCommonCountersOwner {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

в отдельный файл

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 12:39:12 UTC Pre-commit check for ef3a901 has started.
⚪ 2024-08-02 12:42:09 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-02 12:49:05 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 12:39:36 UTC Pre-commit check for ef3a901 has started.
⚪ 2024-08-02 12:42:37 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-02 15:26:02 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9643 9564 0 36 23 20

🟢 2024-08-02 15:27:09 UTC Build successful.
🟡 2024-08-02 15:27:41 UTC ydbd size 5.4 GiB changed* by +199.4 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: ca41215 merge: ef3a901 diff diff %
ydbd size 5 824 578 784 Bytes 5 824 783 016 Bytes +199.4 KiB +0.004%
ydbd stripped size 1 463 004 432 Bytes 1 463 043 120 Bytes +37.8 KiB +0.003%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 12:39:49 UTC Pre-commit check for ef3a901 has started.
⚪ 2024-08-02 12:42:37 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-02 14:42:48 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13803 12429 0 27 1334 13

⚪ 2024-08-02 14:44:22 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-02 14:52:03 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
41 (only retried tests) 8 0 23 0 10

⚪ 2024-08-02 14:53:14 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-02 15:00:24 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
34 (only retried tests) 2 0 22 0 10

🟢 2024-08-02 15:01:32 UTC Build successful.
🟢 2024-08-02 15:02:09 UTC ydbd size 8.1 GiB changed* by +6.8 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: ca41215 merge: ef3a901 diff diff %
ydbd size 8 664 088 648 Bytes 8 664 095 560 Bytes +6.8 KiB +0.000%
ydbd stripped size 471 413 320 Bytes 471 425 032 Bytes +11.4 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 15:48:34 UTC Pre-commit check for 008cb8b has started.
⚫ 2024-08-02 15:50:32 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 15:48:35 UTC Pre-commit check for 008cb8b has started.
⚫ 2024-08-02 15:51:48 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 15:48:45 UTC Pre-commit check for 008cb8b has started.
⚫ 2024-08-02 15:50:35 UTC Check cancelled

@swalrus1 swalrus1 force-pushed the refactor-counters-clean-up-total-stats branch from 2b6a46e to 3b066ed Compare August 2, 2024 15:50
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 15:53:29 UTC Pre-commit check for 3bb8608 has started.
⚪ 2024-08-02 15:56:11 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-02 16:01:23 UTC Build successful.

@swalrus1 swalrus1 marked this pull request as ready for review August 2, 2024 15:55
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 15:55:43 UTC Pre-commit check for 3bb8608 has started.
⚪ 2024-08-02 15:59:50 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-02 18:02:22 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9633 9581 0 13 25 14

🟢 2024-08-02 18:03:27 UTC Build successful.
🟡 2024-08-02 18:03:55 UTC ydbd size 5.4 GiB changed* by +198.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: e9a7b28 merge: 3bb8608 diff diff %
ydbd size 5 824 578 416 Bytes 5 824 781 872 Bytes +198.7 KiB +0.003%
ydbd stripped size 1 463 004 496 Bytes 1 463 038 960 Bytes +33.7 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 15:57:19 UTC Pre-commit check for 3bb8608 has started.
⚪ 2024-08-02 16:00:34 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-02 17:32:13 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13803 12451 0 4 1336 12

⚪ 2024-08-02 17:33:52 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-02 17:45:09 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
20 (only retried tests) 5 0 3 4 8

⚪ 2024-08-02 17:46:22 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-02 17:53:27 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15 (only retried tests) 6 0 1 0 8

🟢 2024-08-02 17:54:40 UTC Build successful.
🟢 2024-08-02 17:55:17 UTC ydbd size 8.1 GiB changed* by +18.8 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: e9a7b28 merge: 3bb8608 diff diff %
ydbd size 8 664 088 744 Bytes 8 664 108 016 Bytes +18.8 KiB +0.000%
ydbd stripped size 471 413 320 Bytes 471 425 032 Bytes +11.4 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 20:30:30 UTC Pre-commit check for a3541f2 has started.
⚪ 2024-08-02 20:33:32 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-02 20:39:43 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 20:30:41 UTC Pre-commit check for a3541f2 has started.
⚪ 2024-08-02 20:34:03 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-02 22:09:14 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13805 12454 0 2 1335 14

⚪ 2024-08-02 22:10:47 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-02 22:21:52 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
21 (only retried tests) 6 0 2 5 8

⚪ 2024-08-02 22:23:01 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-02 22:29:44 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14 (only retried tests) 6 0 0 0 8

🟢 2024-08-02 22:30:51 UTC Build successful.
🟢 2024-08-02 22:31:28 UTC ydbd size 8.1 GiB changed* by +19.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: a1823e1 merge: a3541f2 diff diff %
ydbd size 8 664 239 176 Bytes 8 664 259 024 Bytes +19.4 KiB +0.000%
ydbd stripped size 471 417 096 Bytes 471 428 744 Bytes +11.4 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024 •
edited
Loading

⚪ 2024-08-02 20:30:52 UTC Pre-commit check for a3541f2 has started.
⚪ 2024-08-02 20:33:47 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-02 22:36:47 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9638 9585 0 9 26 18

🟢 2024-08-02 22:37:51 UTC Build successful.
🟡 2024-08-02 22:38:18 UTC ydbd size 5.4 GiB changed* by +190.6 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: a1823e1 merge: a3541f2 diff diff %
ydbd size 5 824 722 240 Bytes 5 824 917 408 Bytes +190.6 KiB +0.003%
ydbd stripped size 1 463 035 152 Bytes 1 463 061 360 Bytes +25.6 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

ivanmorozov333
ivanmorozov333 previously approved these changes Aug 5, 2024
View reviewed changes
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-05 15:31:25 UTC Pre-commit check for 13ec0e8 has started.
⚪ 2024-08-05 15:34:38 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-05 15:41:14 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-06 09:46:37 UTC Pre-commit check for 13ec0e8 has started.
⚪ 2024-08-06 09:49:28 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-06 11:16:30 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13813 12453 0 4 1335 21

⚪ 2024-08-06 11:18:03 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-06 11:25:20 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
20 (only retried tests) 11 0 1 0 8

⚪ 2024-08-06 11:26:30 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-06 11:33:48 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10 (only retried tests) 2 0 0 0 8

🟢 2024-08-06 11:34:56 UTC Build successful.
🟢 2024-08-06 11:35:28 UTC ydbd size 8.1 GiB changed* by +15.6 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: e4c34ae merge: 13ec0e8 diff diff %
ydbd size 8 668 415 856 Bytes 8 668 431 872 Bytes +15.6 KiB +0.000%
ydbd stripped size 471 630 824 Bytes 471 638 376 Bytes +7.4 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-05 15:36:02 UTC Pre-commit check for 13ec0e8 has started.
⚪ 2024-08-05 15:38:59 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-05 17:52:00 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9648 9595 0 14 23 16

🟢 2024-08-05 17:53:17 UTC Build successful.
🟡 2024-08-05 17:53:50 UTC ydbd size 5.4 GiB changed* by +198.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: e4c34ae merge: 13ec0e8 diff diff %
ydbd size 5 828 208 184 Bytes 5 828 411 720 Bytes +198.8 KiB +0.003%
ydbd stripped size 1 463 788 400 Bytes 1 463 822 800 Bytes +33.6 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@swalrus1 swalrus1 merged commit 58580f8 into ydb-platform:main Aug 6, 2024
10 of 12 checks passed
@swalrus1 swalrus1 deleted the refactor-counters-clean-up-total-stats branch August 6, 2024 12:17
@StekPerepolnen StekPerepolnen mentioned this pull request Aug 8, 2024
Merged
This was referenced Aug 8, 2024
Merged
Merged
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 9, 2024
@swalrus1 @zverevgeny
add tx progress counters (ydb-platform#7407)
941eef8
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 10, 2024
@swalrus1 @zverevgeny
add tx progress counters (ydb-platform#7407)
12e7506
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 11, 2024
@swalrus1 @zverevgeny
add tx progress counters (ydb-platform#7407)
0b3cfb4
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 12, 2024
@swalrus1 @zverevgeny
add tx progress counters (ydb-platform#7407)
1e6814d
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 14, 2024
@swalrus1 @zverevgeny
add tx progress counters (ydb-platform#7407)
78abfb8
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 15, 2024
@swalrus1 @zverevgeny
add tx progress counters (ydb-platform#7407)
6ac00b5
@zverevgeny zverevgeny mentioned this pull request Sep 16, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ivanmorozov333 ivanmorozov333 ivanmorozov333 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@swalrus1 @ivanmorozov333