Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change TEvLocal actorsystem metrics #6355

Merged
merged 1 commit into from
Jul 11, 2024

Conversation

kruall
Copy link
Collaborator

@kruall kruall commented Jul 5, 2024

Changelog entry

...

Changelog category

  • Bugfix

Additional information

...

@kruall kruall added the area/actorsystem Actor System related issues label Jul 5, 2024
@kruall kruall self-assigned this Jul 5, 2024
@github-actions github-actions bot added the bugfix label Jul 5, 2024
Copy link

github-actions bot commented Jul 5, 2024

2024-07-05 17:04:54 UTC Pre-commit check for c0da302 has started.
2024-07-05 17:08:02 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-07-05 17:10:31 UTC Build failed. see the build logs.

Copy link

github-actions bot commented Jul 5, 2024

2024-07-05 17:06:07 UTC Pre-commit check for c0da302 has started.
2024-07-05 17:09:50 UTC Build linux-x86_64-release-asan is running...
🔴 2024-07-05 17:12:20 UTC Build failed. see the build logs.
🔴 2024-07-05 17:12:21 UTC Tests run skipped.

Copy link

github-actions bot commented Jul 5, 2024

2024-07-05 17:08:41 UTC Pre-commit check for c0da302 has started.
2024-07-05 17:12:01 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-07-05 17:14:53 UTC Build failed. see the build logs.
🔴 2024-07-05 17:14:55 UTC Tests run skipped.

@kruall kruall force-pushed the as/change_stats_in_local branch from 9a1d4d5 to 6b9fe41 Compare July 9, 2024 13:19
Copy link

github-actions bot commented Jul 9, 2024

2024-07-09 13:56:38 UTC Pre-commit check for f3296f5 has started.
2024-07-09 14:00:01 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-09 14:46:21 UTC Build successful.
2024-07-09 14:46:32 UTC Tests are running...
🔴 2024-07-09 18:01:42 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13946 12549 0 518 701 178

🟡 2024-07-09 18:02:34 UTC ydbd size 5.2 GiB changed* by +239.6 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 601cb7a merge: f3296f5 diff diff %
ydbd size 5 610 839 456 Bytes 5 611 084 848 Bytes +239.6 KiB +0.004%
ydbd stripped size 1 206 598 584 Bytes 1 206 635 096 Bytes +35.7 KiB +0.003%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 9, 2024

2024-07-09 14:04:08 UTC Pre-commit check for f3296f5 has started.
2024-07-09 14:07:03 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-09 14:49:54 UTC Build successful.

Copy link

github-actions bot commented Jul 9, 2024

2024-07-09 14:57:08 UTC Pre-commit check for f3296f5 has started.
2024-07-09 15:00:09 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-09 16:06:01 UTC Build successful.
2024-07-09 16:06:20 UTC Tests are running...
🔴 2024-07-09 19:37:33 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
75085 60849 0 430 13789 17

🟡 2024-07-09 19:38:28 UTC ydbd size 8.1 GiB changed* by +184.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 4e84252 merge: f3296f5 diff diff %
ydbd size 8 720 738 120 Bytes 8 720 927 336 Bytes +184.8 KiB +0.002%
ydbd stripped size 474 986 976 Bytes 475 000 384 Bytes +13.1 KiB +0.003%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

alexvru
alexvru previously approved these changes Jul 10, 2024
@@ -53,6 +63,7 @@ message TEvStatus {
optional uint64 AvailableWeight = 5;
optional NKikimrTabletBase.TMetrics ResourceMaximum = 8;
optional uint64 StartTime = 7;
optional TActorSystemInfo ActorSystemInfo = 9;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TEvStatus is sent when Local first registers in Hive, and when the tenant is altered. If we want this info to be sent regularly, I would put in TabletMetrics message.

@@ -111,6 +111,9 @@ class TLocalNodeRegistrar : public TActorBootstrapped<TLocalNodeRegistrar> {
ui64 MemLimit = 0;
double NodeUsage = 0;

TInstant LastUpdate;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name is confusing, last update of what? Also If we change it to use TabletMetrics message, there already is a SendTabletMetricsTime member

double cores = 0;
for (ui8 poolId = 0; poolId < poolStates.size(); ++poolId) {
auto &poolState = poolStates[poolId];
if (poolId != AppData()->IOPoolId) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have a setting PoolsToMonitorForUsage. I believe it should be used here. But it also excludes Batch pool by default - is there a reason you're including it here?

@@ -32,6 +32,16 @@ message TLocalConfig {
repeated NKikimrSchemeOp.TResourceProfile ResourceProfiles = 1;
}

message TActorSystemInfo {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently we send a single number TotalNodeUsage, that is the maximum over pool usages (and mem usage).
What do we expect to gain from sending this detailed info? Just seems like we will simply do the calculate usage + take maximum logic on Hive side instead.

Copy link

github-actions bot commented Jul 10, 2024

2024-07-10 13:09:41 UTC Pre-commit check for 9071e00 has started.
2024-07-10 13:13:01 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-10 13:57:18 UTC Build successful.

Copy link

github-actions bot commented Jul 10, 2024

2024-07-10 13:10:19 UTC Pre-commit check for 9071e00 has started.
2024-07-10 13:13:49 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-10 14:06:00 UTC Build successful.
2024-07-10 14:06:13 UTC Tests are running...
🔴 2024-07-10 16:11:41 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13784 13171 0 93 343 177

🟢 2024-07-10 16:12:23 UTC ydbd size 5.2 GiB changed* by +31.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: d32d5d0 merge: 9071e00 diff diff %
ydbd size 5 611 367 768 Bytes 5 611 400 224 Bytes +31.7 KiB +0.001%
ydbd stripped size 1 206 696 312 Bytes 1 206 696 088 Bytes -224 Bytes -0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 10, 2024

2024-07-10 13:10:33 UTC Pre-commit check for 9071e00 has started.
2024-07-10 13:14:00 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-10 14:18:38 UTC Build successful.
2024-07-10 14:18:48 UTC Tests are running...
🟢 2024-07-10 16:25:32 UTC Tests successful.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
74916 61468 0 0 13430 18

🟢 2024-07-10 16:26:16 UTC ydbd size 8.1 GiB changed* by +42.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: d32d5d0 merge: 9071e00 diff diff %
ydbd size 8 721 401 176 Bytes 8 721 444 608 Bytes +42.4 KiB +0.000%
ydbd stripped size 475 017 568 Bytes 475 013 280 Bytes -4.2 KiB -0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/actorsystem Actor System related issues bugfix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants