Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

count-min-sketch as index #7978

Merged
merged 9 commits into from
Aug 23, 2024
Merged

Conversation

ildar-khisambeev
Copy link
Contributor

Should be the same as #6581, but should run only relevant checks

Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 07:32:56 UTC Pre-commit check for 8e3b3c7 has started.
2024-08-19 07:36:08 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-19 10:28:04 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15569 15460 0 33 43 33

🟢 2024-08-19 10:29:24 UTC Build successful.
🟡 2024-08-19 10:29:58 UTC ydbd size 5.5 GiB changed* by +1.4 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 274bfdd merge: 8e3b3c7 diff diff %
ydbd size 5 880 316 984 Bytes 5 881 754 176 Bytes +1.4 MiB +0.024%
ydbd stripped size 1 477 405 904 Bytes 1 477 604 016 Bytes +193.5 KiB +0.013%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 07:33:04 UTC Pre-commit check for 8e3b3c7 has started.
2024-08-19 07:36:13 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-19 08:15:27 UTC Build successful.

Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 07:33:13 UTC Pre-commit check for 8e3b3c7 has started.
2024-08-19 07:36:25 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-19 09:41:48 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
77300 63645 0 16 13606 33

2024-08-19 09:49:09 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-19 09:57:28 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
74 (only retried tests) 36 0 11 0 27

2024-08-19 09:57:37 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-19 10:05:22 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
63 (only retried tests) 26 0 10 0 27

🟢 2024-08-19 10:05:30 UTC Build successful.
🔴 2024-08-19 10:06:07 UTC ydbd size 8.1 GiB changed* by +2.4 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 5522a96 merge: 8e3b3c7 diff diff %
ydbd size 8 736 888 376 Bytes 8 739 431 400 Bytes +2.4 MiB +0.029%
ydbd stripped size 475 560 200 Bytes 475 616 712 Bytes +55.2 KiB +0.012%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 20, 2024

2024-08-20 10:55:38 UTC Pre-commit check for e6332f8 has started.
2024-08-20 10:58:48 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-20 11:44:57 UTC Build successful.

Copy link

github-actions bot commented Aug 20, 2024

2024-08-20 10:56:13 UTC Pre-commit check for e6332f8 has started.
2024-08-20 10:59:32 UTC Check linux-x86_64-release-asan is running...

Copy link

github-actions bot commented Aug 20, 2024

2024-08-20 10:56:18 UTC Pre-commit check for e6332f8 has started.
2024-08-20 10:59:34 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-20 13:02:44 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
77370 63729 0 15 13609 17

2024-08-20 13:10:04 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-20 13:19:56 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
47 (only retried tests) 28 0 10 0 9

2024-08-20 13:20:05 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-20 13:26:44 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
34 (only retried tests) 16 0 10 0 8

🟢 2024-08-20 13:26:52 UTC Build successful.
🔴 2024-08-20 13:27:41 UTC ydbd size 8.1 GiB changed* by +2.4 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: d103d5d merge: e6332f8 diff diff %
ydbd size 8 748 356 272 Bytes 8 750 890 184 Bytes +2.4 MiB +0.029%
ydbd stripped size 475 797 800 Bytes 475 846 120 Bytes +47.2 KiB +0.010%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 16:03:30 UTC Pre-commit check for 13e0649 has started.
2024-08-21 16:06:51 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-21 18:10:54 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15470 15361 0 46 47 16

🟢 2024-08-21 18:12:15 UTC Build successful.
🟡 2024-08-21 18:12:45 UTC ydbd size 5.5 GiB changed* by +1.4 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 8e44a97 merge: 13e0649 diff diff %
ydbd size 5 888 643 904 Bytes 5 890 140 272 Bytes +1.4 MiB +0.025%
ydbd stripped size 1 478 666 608 Bytes 1 478 869 040 Bytes +197.7 KiB +0.014%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 16:03:35 UTC Pre-commit check for 13e0649 has started.
2024-08-21 16:06:57 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-21 16:54:26 UTC Build successful.

Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 16:04:18 UTC Pre-commit check for 13e0649 has started.
2024-08-21 16:07:33 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-21 18:10:34 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
77410 63746 0 23 13624 17

2024-08-21 18:18:01 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-21 18:26:40 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
51 (only retried tests) 32 0 10 1 8

2024-08-21 18:26:49 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-21 18:34:37 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
33 (only retried tests) 15 0 10 0 8

🟢 2024-08-21 18:34:44 UTC Build successful.
🔴 2024-08-21 18:35:20 UTC ydbd size 8.2 GiB changed* by +2.4 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 8e44a97 merge: 13e0649 diff diff %
ydbd size 8 749 629 152 Bytes 8 752 143 472 Bytes +2.4 MiB +0.029%
ydbd stripped size 475 887 432 Bytes 475 937 352 Bytes +48.8 KiB +0.010%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 10:22:54 UTC Pre-commit check for fbea2fd has started.
2024-08-22 10:26:11 UTC Check linux-x86_64-release-clang14 is running...
2024-08-22 10:32:06 UTC Check cancelled

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 10:22:58 UTC Pre-commit check for fbea2fd has started.
2024-08-22 10:26:09 UTC Check linux-x86_64-release-asan is running...
2024-08-22 10:32:07 UTC Check cancelled

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 10:23:29 UTC Pre-commit check for fbea2fd has started.
2024-08-22 10:26:36 UTC Check linux-x86_64-relwithdebinfo is running...
2024-08-22 10:32:08 UTC Check cancelled

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 10:37:47 UTC Pre-commit check for 6520b17 has started.
2024-08-22 10:41:46 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-22 10:42:01 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 10:41:55 UTC Pre-commit check for 6520b17 has started.
2024-08-22 10:46:15 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-22 10:46:31 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 10:44:15 UTC Pre-commit check for 6520b17 has started.
2024-08-22 10:47:35 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-22 10:47:49 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 11:10:04 UTC Pre-commit check for dfee346 has started.
2024-08-22 11:14:24 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-22 11:43:42 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 11:14:46 UTC Pre-commit check for dfee346 has started.
2024-08-22 11:17:56 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-22 11:50:57 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 11:17:16 UTC Pre-commit check for dfee346 has started.
2024-08-22 11:20:32 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-22 11:53:08 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 13:29:43 UTC Pre-commit check for 5d2ac72 has started.
2024-08-22 13:32:55 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-22 13:51:39 UTC Build successful.

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 13:33:26 UTC Pre-commit check for 5d2ac72 has started.
2024-08-22 13:36:51 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-22 15:19:21 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
77412 63768 0 8 13620 16

2024-08-22 15:26:53 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-22 15:34:14 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26 (only retried tests) 17 0 0 0 9

🟢 2024-08-22 15:34:22 UTC Build successful.
🔴 2024-08-22 15:34:59 UTC ydbd size 8.2 GiB changed* by +2.4 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: ba138ae merge: 5d2ac72 diff diff %
ydbd size 8 752 126 272 Bytes 8 754 640 936 Bytes +2.4 MiB +0.029%
ydbd stripped size 476 005 128 Bytes 476 050 952 Bytes +44.8 KiB +0.010%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 13:33:48 UTC Pre-commit check for 5d2ac72 has started.
2024-08-22 13:38:22 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-22 15:40:35 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15481 15353 0 43 61 24

🟢 2024-08-22 15:41:54 UTC Build successful.
🟡 2024-08-22 15:42:27 UTC ydbd size 5.5 GiB changed* by +1.4 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: ba138ae merge: 5d2ac72 diff diff %
ydbd size 5 890 308 040 Bytes 5 891 803 048 Bytes +1.4 MiB +0.025%
ydbd stripped size 1 479 110 672 Bytes 1 479 309 008 Bytes +193.7 KiB +0.013%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

sketchesByColumns.emplace(id, TCountMinSketch::Create());
}

for (const auto& [indexKey, keyPortions] : spg->GetPortionsIndex().GetPoints()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

вот тут нужно spg->GetPortions() взять и сканировать только те что Visible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

давай ты это сделаешь в отдельном акторе, передав туда spg->GetPortions() Visible отфильтрованные

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spg->GetPortions() взять и сканировать только те что Visible
Это сделал.

Рефакторинг с выносом сканирования в отдельный актор я хочу сделать в следующем PR

@@ -267,7 +281,7 @@ std::vector<TTableInfo> CreateDatabaseColumnTables(TTestEnv& env, ui8 tableCount
auto& runtime = *env.GetServer().GetRuntime();
auto sender = runtime.AllocateEdgeActor();

runtime.SimulateSleep(TDuration::Seconds(10));
runtime.SimulateSleep(TDuration::Seconds(30));
Copy link
Collaborator

@azevaykin azevaykin Aug 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Можно этот Sleep заменить на ожидание какого-то события? Иначе тесты могут мигать.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Вопрос за рамками этого PR

Copy link

github-actions bot commented Aug 23, 2024

2024-08-23 10:10:24 UTC Pre-commit check for cd7ee51 has started.
2024-08-23 10:13:37 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-23 12:04:02 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15486 15376 0 45 48 17

🟢 2024-08-23 12:05:26 UTC Build successful.
🟡 2024-08-23 12:05:53 UTC ydbd size 5.5 GiB changed* by +1.4 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 3ca30a5 merge: cd7ee51 diff diff %
ydbd size 5 890 853 720 Bytes 5 892 367 176 Bytes +1.4 MiB +0.026%
ydbd stripped size 1 479 181 488 Bytes 1 479 385 648 Bytes +199.4 KiB +0.014%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 23, 2024

2024-08-23 10:10:53 UTC Pre-commit check for cd7ee51 has started.
2024-08-23 10:14:11 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-23 10:42:18 UTC Build successful.

Copy link

github-actions bot commented Aug 23, 2024

2024-08-23 10:13:11 UTC Pre-commit check for cd7ee51 has started.
2024-08-23 10:16:23 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-23 11:42:28 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
77438 63767 0 28 13627 16

2024-08-23 11:50:13 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-23 12:03:23 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
56 (only retried tests) 21 0 24 3 8

2024-08-23 12:03:32 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-23 12:14:44 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
45 (only retried tests) 10 0 26 1 8

🟢 2024-08-23 12:14:51 UTC Build successful.
🔴 2024-08-23 12:15:28 UTC ydbd size 8.2 GiB changed* by +2.4 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 3ca30a5 merge: cd7ee51 diff diff %
ydbd size 8 753 303 584 Bytes 8 755 832 224 Bytes +2.4 MiB +0.029%
ydbd stripped size 476 022 600 Bytes 476 068 936 Bytes +45.2 KiB +0.010%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

ivanmorozov333
ivanmorozov333 previously approved these changes Aug 23, 2024
Copy link

github-actions bot commented Aug 23, 2024

2024-08-23 13:16:10 UTC Pre-commit check for d41cdba has started.
2024-08-23 13:19:21 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-23 15:02:09 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
77526 63830 0 5 13675 16

2024-08-23 15:09:41 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-23 15:19:38 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
24 (only retried tests) 15 0 0 0 9

🟢 2024-08-23 15:19:52 UTC Build successful.
🔴 2024-08-23 15:20:27 UTC ydbd size 8.2 GiB changed* by +2.4 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 6cd8814 merge: d41cdba diff diff %
ydbd size 8 760 671 008 Bytes 8 763 199 648 Bytes +2.4 MiB +0.029%
ydbd stripped size 478 027 400 Bytes 478 073 704 Bytes +45.2 KiB +0.010%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 23, 2024

2024-08-23 13:16:10 UTC Pre-commit check for d41cdba has started.
2024-08-23 13:19:24 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-23 14:03:30 UTC Build successful.

Copy link

github-actions bot commented Aug 23, 2024

2024-08-23 13:16:35 UTC Pre-commit check for d41cdba has started.
2024-08-23 13:19:47 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-23 15:19:45 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15492 15386 0 31 51 24

🟢 2024-08-23 15:21:06 UTC Build successful.
🟡 2024-08-23 15:21:38 UTC ydbd size 5.5 GiB changed* by +1.4 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6cd8814 merge: d41cdba diff diff %
ydbd size 5 896 376 120 Bytes 5 897 889 560 Bytes +1.4 MiB +0.026%
ydbd stripped size 1 481 802 320 Bytes 1 482 006 480 Bytes +199.4 KiB +0.014%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@ildar-khisambeev ildar-khisambeev merged commit 78bb468 into ydb-platform:main Aug 23, 2024
10 of 12 checks passed
@ildar-khisambeev ildar-khisambeev deleted the cms3 branch August 27, 2024 14:14
rekby pushed a commit to rekby-forks/ydb that referenced this pull request Aug 28, 2024
stanislav-shchetinin pushed a commit to stanislav-shchetinin/ydb that referenced this pull request Aug 30, 2024
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 13, 2024
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants