Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sparsed columns are disabled by default and can be optionally enabled #9002

Merged
merged 5 commits into from
Sep 12, 2024

Conversation

aavdonkin
Copy link
Collaborator

Changelog entry

...

Changelog category

  • New feature
  • Experimental feature
  • Improvement
  • Performance improvement
  • Bugfix
  • Backward incompatible change
  • Documentation (changelog entry is not required)
  • Not for changelog (changelog entry is not required)

Additional information

...

Copy link

github-actions bot commented Sep 10, 2024

2024-09-10 13:26:48 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6ac7f19 has started.
2024-09-10 13:26:56 UTC Artifacts will be uploaded here
2024-09-10 13:30:03 UTC ya make is running...
🟡 2024-09-10 15:33:19 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40817 35354 0 5 5432 26

2024-09-10 15:37:15 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-09-10 15:47:33 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
42 (only retried tests) 22 0 1 1 18

2024-09-10 15:47:41 UTC ya make is running... (failed tests rerun, try 3)
🟢 2024-09-10 15:55:21 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
24 (only retried tests) 6 0 0 0 18

🟢 2024-09-10 15:55:28 UTC Build successful.
🟢 2024-09-10 15:56:02 UTC ydbd size 8.4 GiB changed* by +31.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 7001202 merge: 6ac7f19 diff diff %
ydbd size 9 024 519 912 Bytes 9 024 551 872 Bytes +31.2 KiB +0.000%
ydbd stripped size 487 708 456 Bytes 487 709 672 Bytes +1.2 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 10, 2024

2024-09-10 09:32:15 UTC Pre-commit check linux-x86_64-release-clang14 for 6ac7f19 has started.
2024-09-10 09:32:19 UTC Artifacts will be uploaded here
2024-09-10 09:35:28 UTC ya make is running...
🟢 2024-09-10 10:15:31 UTC Build successful.

Copy link

github-actions bot commented Sep 10, 2024

2024-09-10 09:34:30 UTC Pre-commit check linux-x86_64-release-asan for 6ac7f19 has started.
2024-09-10 09:34:34 UTC Artifacts will be uploaded here
2024-09-10 09:37:47 UTC ya make is running...
🔴 2024-09-10 11:41:44 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11540 11458 0 20 29 33

🟢 2024-09-10 11:42:44 UTC Build successful.
🟢 2024-09-10 11:43:14 UTC ydbd size 5.6 GiB changed* by +34.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 7001202 merge: 6ac7f19 diff diff %
ydbd size 6 037 335 152 Bytes 6 037 370 344 Bytes +34.4 KiB +0.001%
ydbd stripped size 1 511 156 464 Bytes 1 511 161 328 Bytes +4.8 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation


TKikimrSettings GetKikimrSettings() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

нужно для всех тестов включить

@@ -51,6 +52,14 @@ NKikimr::TConclusionStatus TStandaloneSchemaUpdate::DoInitializeImpl(const TUpda
}
}

if (!AppData()->FeatureFlags.GetEnableSparsedColumns()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

проверку, лучше, в TColumnTableUpdate добавить

@aavdonkin aavdonkin requested review from a team as code owners September 11, 2024 12:41
Copy link

✅ Documentation build

Revision built successfully
Revision preview link

Copy link

github-actions bot commented Sep 11, 2024

2024-09-11 13:00:34 UTC Pre-commit check linux-x86_64-release-asan for 86eeabd has started.
2024-09-11 13:01:20 UTC Artifacts will be uploaded here
2024-09-11 13:05:14 UTC ya make is running...
2024-09-11 13:14:39 UTC Check cancelled

Copy link

github-actions bot commented Sep 11, 2024

2024-09-11 13:07:10 UTC Pre-commit check linux-x86_64-relwithdebinfo for 86eeabd has started.
2024-09-11 13:07:31 UTC Artifacts will be uploaded here
2024-09-11 13:10:44 UTC ya make is running...
2024-09-11 13:14:41 UTC Check cancelled

Copy link

github-actions bot commented Sep 11, 2024

2024-09-11 13:07:16 UTC Pre-commit check linux-x86_64-release-clang14 for 86eeabd has started.
2024-09-11 13:07:40 UTC Artifacts will be uploaded here
2024-09-11 13:10:51 UTC ya make is running...
2024-09-11 13:14:38 UTC Check cancelled

Copy link

github-actions bot commented Sep 11, 2024

2024-09-11 13:18:11 UTC Pre-commit check linux-x86_64-release-asan for 58fdd0d has started.
2024-09-11 13:18:50 UTC Artifacts will be uploaded here
2024-09-11 13:22:23 UTC ya make is running...
🔴 2024-09-11 15:36:35 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11807 11638 0 25 46 98

🟢 2024-09-11 15:37:36 UTC Build successful.
🟢 2024-09-11 15:38:10 UTC ydbd size 5.6 GiB changed* by +60.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 5a49f64 merge: 58fdd0d diff diff %
ydbd size 6 040 891 488 Bytes 6 040 953 032 Bytes +60.1 KiB +0.001%
ydbd stripped size 1 511 647 184 Bytes 1 511 657 424 Bytes +10.0 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 11, 2024

2024-09-11 13:18:52 UTC Pre-commit check linux-x86_64-relwithdebinfo for 58fdd0d has started.
2024-09-11 13:19:01 UTC Artifacts will be uploaded here
2024-09-11 13:22:04 UTC ya make is running...
🟡 2024-09-11 15:02:09 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
41097 35351 0 12 5707 27

2024-09-11 15:06:01 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-09-11 15:13:10 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
321 (only retried tests) 228 0 0 8 85

🟢 2024-09-11 15:13:17 UTC Build successful.
🟢 2024-09-11 15:13:59 UTC ydbd size 8.4 GiB changed* by +54.6 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 5a49f64 merge: 58fdd0d diff diff %
ydbd size 9 029 241 664 Bytes 9 029 297 528 Bytes +54.6 KiB +0.001%
ydbd stripped size 487 837 928 Bytes 487 839 208 Bytes +1.2 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 11, 2024

2024-09-11 13:24:38 UTC Pre-commit check linux-x86_64-release-clang14 for 58fdd0d has started.
2024-09-11 13:25:33 UTC Artifacts will be uploaded here
2024-09-11 13:29:22 UTC ya make is running...
🟢 2024-09-11 14:19:21 UTC Build successful.

@@ -133,6 +133,9 @@ Y_UNIT_TEST_SUITE(KqpOlapWrite) {

Y_UNIT_TEST(DefaultValues) {
auto settings = TKikimrSettings().SetWithSampleTables(false);
NKikimrConfig::TFeatureFlags featureFlags;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

перенеси, пожалуйста, инициацию флажка в конструктор TKikimrRunner

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Так я перенес, отсюда забыл убрать

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Лишний код тут убрал

Copy link

github-actions bot commented Sep 11, 2024

2024-09-11 16:52:49 UTC Pre-commit check linux-x86_64-relwithdebinfo for 690085c has started.
2024-09-11 16:53:14 UTC Artifacts will be uploaded here
2024-09-11 16:56:20 UTC ya make is running...
🟡 2024-09-11 18:38:32 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
41108 35383 0 10 5673 42

2024-09-11 18:42:05 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-09-11 18:49:23 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
295 (only retried tests) 202 0 0 7 86

🟢 2024-09-11 18:49:30 UTC Build successful.
🟢 2024-09-11 18:50:08 UTC ydbd size 8.4 GiB changed* by +54.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 3d518f6 merge: 690085c diff diff %
ydbd size 9 029 717 160 Bytes 9 029 773 008 Bytes +54.5 KiB +0.001%
ydbd stripped size 487 854 120 Bytes 487 855 464 Bytes +1.3 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 11, 2024

2024-09-11 16:53:46 UTC Pre-commit check linux-x86_64-release-clang14 for 690085c has started.
2024-09-11 16:53:56 UTC Artifacts will be uploaded here
2024-09-11 16:56:59 UTC ya make is running...
🟢 2024-09-11 17:46:05 UTC Build successful.

Copy link

github-actions bot commented Sep 11, 2024

2024-09-11 16:53:48 UTC Pre-commit check linux-x86_64-release-asan for 690085c has started.
2024-09-11 16:53:58 UTC Artifacts will be uploaded here
2024-09-11 16:57:02 UTC ya make is running...
🔴 2024-09-11 19:05:06 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11803 11651 0 24 32 96

🟢 2024-09-11 19:06:13 UTC Build successful.
🟢 2024-09-11 19:06:47 UTC ydbd size 5.6 GiB changed* by +50.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 5f125c0 merge: 690085c diff diff %
ydbd size 6 041 294 496 Bytes 6 041 345 928 Bytes +50.2 KiB +0.001%
ydbd stripped size 1 511 717 968 Bytes 1 511 722 576 Bytes +4.5 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@@ -97,6 +97,9 @@ struct TKikimrSettings: public TTestFeatureFlagsHolder<TKikimrSettings> {
exchangerSettings->SetStartDelayMs(10);
exchangerSettings->SetMaxDelayMs(10);
AppConfig.MutableColumnShardConfig()->SetDisabledOnSchemeShard(false);
NKikimrConfig::TFeatureFlags featureFlags;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

а зачем мы все фичи перезаписываем? один флаг нельзя поменять?

Copy link

github-actions bot commented Sep 12, 2024

2024-09-12 08:33:07 UTC Pre-commit check linux-x86_64-release-clang14 for 9ced25a has started.
2024-09-12 08:33:17 UTC Artifacts will be uploaded here
2024-09-12 08:36:16 UTC ya make is running...
🟢 2024-09-12 09:02:41 UTC Build successful.

Copy link

github-actions bot commented Sep 12, 2024

2024-09-12 08:33:14 UTC Pre-commit check linux-x86_64-relwithdebinfo for 9ced25a has started.
2024-09-12 08:33:24 UTC Artifacts will be uploaded here
2024-09-12 08:36:30 UTC ya make is running...
🟡 2024-09-12 09:58:14 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
41115 35419 0 1 5666 29

2024-09-12 10:01:56 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-09-12 10:12:11 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
267 (only retried tests) 175 0 0 7 85

🟢 2024-09-12 10:12:19 UTC Build successful.
🟢 2024-09-12 10:12:54 UTC ydbd size 8.4 GiB changed* by +54.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 42b25d4 merge: 9ced25a diff diff %
ydbd size 9 034 978 992 Bytes 9 035 034 848 Bytes +54.5 KiB +0.001%
ydbd stripped size 488 200 168 Bytes 488 201 512 Bytes +1.3 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 12, 2024

2024-09-12 08:33:48 UTC Pre-commit check linux-x86_64-release-asan for 9ced25a has started.
2024-09-12 08:33:58 UTC Artifacts will be uploaded here
2024-09-12 08:37:00 UTC ya make is running...
🔴 2024-09-12 10:39:57 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11817 11638 0 34 41 104

🟢 2024-09-12 10:41:01 UTC Build successful.
🟢 2024-09-12 10:41:39 UTC ydbd size 5.6 GiB changed* by +59.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 42b25d4 merge: 9ced25a diff diff %
ydbd size 6 044 822 040 Bytes 6 044 882 848 Bytes +59.4 KiB +0.001%
ydbd stripped size 1 512 777 904 Bytes 1 512 788 144 Bytes +10.0 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@aavdonkin aavdonkin merged commit ca964ba into ydb-platform:main Sep 12, 2024
10 of 12 checks passed
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 15, 2024
@aavdonkin aavdonkin linked an issue Sep 26, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature flag for sparsed columns
2 participants