Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up SIMPLE scanner #12164

Merged
merged 5 commits into from
Dec 2, 2024

Conversation

ivanmorozov333
Copy link
Collaborator

  • Performance improvement

Copy link

github-actions bot commented Dec 1, 2024

2024-12-01 14:46:43 UTC Pre-commit check linux-x86_64-release-asan for ce83f08 has started.
2024-12-01 14:46:54 UTC Artifacts will be uploaded here
2024-12-01 14:49:54 UTC ya make is running...
2024-12-01 15:57:26 UTC Check cancelled

Copy link

github-actions bot commented Dec 1, 2024

2024-12-01 14:46:44 UTC Pre-commit check linux-x86_64-relwithdebinfo for ce83f08 has started.
2024-12-01 14:46:55 UTC Artifacts will be uploaded here
2024-12-01 14:49:59 UTC ya make is running...
🟡 2024-12-01 15:53:03 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
32334 29275 0 154 2788 117

2024-12-01 15:55:43 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-01 15:57:39 UTC ydbd size 2.5 GiB changed* by +690.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 5f44598 merge: ce83f08 diff diff %
ydbd size 2 691 347 568 Bytes 2 692 054 936 Bytes +690.8 KiB +0.026%
ydbd stripped size 483 332 272 Bytes 483 418 928 Bytes +84.6 KiB +0.018%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation
2024-12-01 15:57:40 UTC Check cancelled

Copy link

github-actions bot commented Dec 1, 2024

2024-12-01 16:01:46 UTC Pre-commit check linux-x86_64-relwithdebinfo for 736e5e3 has started.
2024-12-01 16:01:57 UTC Artifacts will be uploaded here
2024-12-01 16:05:00 UTC ya make is running...
🟡 2024-12-01 17:20:33 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
32334 29487 0 19 2718 110

2024-12-01 17:23:14 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-01 17:35:04 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
886 (only retried tests) 761 0 17 1 107

2024-12-01 17:35:15 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-12-01 17:46:30 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
821 (only retried tests) 697 0 17 0 107

🟢 2024-12-01 17:46:40 UTC Build successful.
🟡 2024-12-01 17:47:02 UTC ydbd size 2.5 GiB changed* by +687.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 5f44598 merge: 736e5e3 diff diff %
ydbd size 2 691 347 568 Bytes 2 692 051 768 Bytes +687.7 KiB +0.026%
ydbd stripped size 483 332 272 Bytes 483 418 480 Bytes +84.2 KiB +0.018%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 1, 2024

2024-12-01 16:01:54 UTC Pre-commit check linux-x86_64-release-asan for 736e5e3 has started.
2024-12-01 16:02:05 UTC Artifacts will be uploaded here
2024-12-01 16:05:10 UTC ya make is running...
🟡 2024-12-01 17:11:21 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14995 14897 0 43 8 47

🟢 2024-12-01 17:12:35 UTC Build successful.
🟡 2024-12-01 17:13:03 UTC ydbd size 4.9 GiB changed* by +1.4 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 5f44598 merge: 736e5e3 diff diff %
ydbd size 5 290 370 128 Bytes 5 291 809 840 Bytes +1.4 MiB +0.027%
ydbd stripped size 1 364 811 504 Bytes 1 365 031 504 Bytes +214.8 KiB +0.016%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 2, 2024

2024-12-02 07:51:43 UTC Pre-commit check linux-x86_64-relwithdebinfo for 96343d0 has started.
2024-12-02 07:51:55 UTC Artifacts will be uploaded here
2024-12-02 07:54:55 UTC ya make is running...
🟡 2024-12-02 09:01:07 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
32334 29480 0 21 2719 114

2024-12-02 09:03:51 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-02 09:15:52 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
945 (only retried tests) 822 0 14 1 108

2024-12-02 09:16:04 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-12-02 09:27:48 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
788 (only retried tests) 668 0 14 0 106

🟢 2024-12-02 09:27:58 UTC Build successful.
🟡 2024-12-02 09:28:22 UTC ydbd size 2.5 GiB changed* by +755.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 9bef86a merge: 96343d0 diff diff %
ydbd size 2 691 439 344 Bytes 2 692 213 400 Bytes +755.9 KiB +0.029%
ydbd stripped size 483 344 080 Bytes 483 442 960 Bytes +96.6 KiB +0.020%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 2, 2024

2024-12-02 07:52:18 UTC Pre-commit check linux-x86_64-release-asan for 96343d0 has started.
2024-12-02 07:52:30 UTC Artifacts will be uploaded here
2024-12-02 07:55:37 UTC ya make is running...
🟡 2024-12-02 09:12:48 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14995 14904 0 36 9 46

🟢 2024-12-02 09:13:59 UTC Build successful.
🟡 2024-12-02 09:14:29 UTC ydbd size 4.9 GiB changed* by +1.6 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 9bef86a merge: 96343d0 diff diff %
ydbd size 5 290 521 624 Bytes 5 292 161 536 Bytes +1.6 MiB +0.031%
ydbd stripped size 1 364 839 376 Bytes 1 365 106 192 Bytes +260.6 KiB +0.020%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 2, 2024

2024-12-02 09:42:08 UTC Pre-commit check linux-x86_64-release-asan for f553878 has started.
2024-12-02 09:42:21 UTC Artifacts will be uploaded here
2024-12-02 09:45:20 UTC ya make is running...
2024-12-02 10:13:35 UTC Check cancelled

Copy link

github-actions bot commented Dec 2, 2024

2024-12-02 09:44:01 UTC Pre-commit check linux-x86_64-relwithdebinfo for f553878 has started.
2024-12-02 09:44:12 UTC Artifacts will be uploaded here
2024-12-02 09:47:28 UTC ya make is running...
2024-12-02 10:13:35 UTC Check cancelled

Copy link

github-actions bot commented Dec 2, 2024

2024-12-02 10:15:14 UTC Pre-commit check linux-x86_64-relwithdebinfo for dc1a73e has started.
2024-12-02 10:16:29 UTC Artifacts will be uploaded here
2024-12-02 10:20:09 UTC ya make is running...
🟡 2024-12-02 11:21:31 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
32335 29493 0 5 2719 118

2024-12-02 11:24:17 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-02 11:35:42 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
740 (only retried tests) 633 0 1 0 106

2024-12-02 11:35:52 UTC ya make is running... (failed tests rerun, try 3)
🟢 2024-12-02 11:47:20 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
623 (only retried tests) 518 0 0 0 105

🟢 2024-12-02 11:47:29 UTC Build successful.
🟢 2024-12-02 11:47:51 UTC ydbd size 2.5 GiB changed* by +65.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: d3a36a5 merge: dc1a73e diff diff %
ydbd size 2 692 171 576 Bytes 2 692 238 176 Bytes +65.0 KiB +0.002%
ydbd stripped size 483 439 056 Bytes 483 446 224 Bytes +7.0 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 2, 2024

2024-12-02 10:17:47 UTC Pre-commit check linux-x86_64-release-asan for dc1a73e has started.
2024-12-02 10:17:59 UTC Artifacts will be uploaded here
2024-12-02 10:21:11 UTC ya make is running...
🟡 2024-12-02 11:26:33 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14996 14934 0 10 5 47

🟢 2024-12-02 11:27:46 UTC Build successful.
🟡 2024-12-02 11:28:17 UTC ydbd size 4.9 GiB changed* by +209.3 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: d3a36a5 merge: dc1a73e diff diff %
ydbd size 5 291 894 792 Bytes 5 292 109 128 Bytes +209.3 KiB +0.004%
ydbd stripped size 1 365 080 336 Bytes 1 365 100 688 Bytes +19.9 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@ivanmorozov333 ivanmorozov333 changed the title Speed up scanner Speed up SIMPLE scanner Dec 2, 2024
@ivanmorozov333 ivanmorozov333 marked this pull request as ready for review December 2, 2024 11:59
@ivanmorozov333 ivanmorozov333 requested a review from a team as a code owner December 2, 2024 11:59
@ivanmorozov333 ivanmorozov333 enabled auto-merge (squash) December 2, 2024 11:59
@ivanmorozov333 ivanmorozov333 merged commit 6fc18f8 into ydb-platform:main Dec 2, 2024
11 checks passed
@ivanmorozov333 ivanmorozov333 deleted the speed_up_scanner branch December 2, 2024 12:51
iddqdex added a commit to iddqdex/ydb that referenced this pull request Dec 11, 2024
iddqdex added a commit to iddqdex/ydb that referenced this pull request Dec 11, 2024
iddqdex added a commit to iddqdex/ydb that referenced this pull request Dec 11, 2024
iddqdex added a commit to iddqdex/ydb that referenced this pull request Dec 11, 2024
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 2, 2025
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 5, 2025
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 5, 2025
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants