Fix table stats aggregation for OLAP tables #8417

swalrus1 · 2024-08-28T16:02:54Z

Changelog entry

Fix aggregation of table stats for OLAP tables.

Changelog category

Bugfix

Additional information

Fixed:

Resource consumption metrics, Access time, Update time and PartCount did not update for column tables.
Data size and rows count were incorrect (too small) if there were multiple column table shards.

github-actions · 2024-08-28T16:30:15Z

⚪ 2024-08-28 16:30:15 UTC Pre-commit check linux-x86_64-release-asan for c6856bd has started.
⚪ 2024-08-28 16:32:54 UTC ya make is running...
⚫ 2024-08-28 16:35:32 UTC Check cancelled

github-actions · 2024-08-28T16:47:21Z

⚪ 2024-08-28 16:47:20 UTC Pre-commit check linux-x86_64-release-asan for 82a6232 has started.
⚪ 2024-08-28 16:50:16 UTC ya make is running...
⚫ 2024-08-28 17:36:27 UTC Check cancelled

github-actions · 2024-08-28T17:11:13Z

⚪ 2024-08-28 17:11:12 UTC Pre-commit check linux-x86_64-relwithdebinfo for 82a6232 has started.
⚪ 2024-08-28 17:14:18 UTC ya make is running...
⚫ 2024-08-28 17:36:25 UTC Check cancelled

github-actions · 2024-08-28T17:11:26Z

⚪ 2024-08-28 17:11:25 UTC Pre-commit check linux-x86_64-release-clang14 for 82a6232 has started.
⚪ 2024-08-28 17:14:12 UTC ya make is running...
⚫ 2024-08-28 17:36:24 UTC Check cancelled

github-actions · 2024-08-28T17:39:04Z

⚪ 2024-08-28 17:39:03 UTC Pre-commit check linux-x86_64-release-asan for 550474c has started.
⚪ 2024-08-28 17:41:47 UTC ya make is running...
⚫ 2024-08-28 17:47:08 UTC Check cancelled

github-actions · 2024-08-28T17:44:44Z

⚪ 2024-08-28 17:44:44 UTC Pre-commit check linux-x86_64-relwithdebinfo for 550474c has started.
⚫ 2024-08-28 17:47:03 UTC Check cancelled

github-actions · 2024-08-28T17:48:46Z

⚪ 2024-08-28 17:48:45 UTC Pre-commit check linux-x86_64-relwithdebinfo for f1e4a9b has started.
⚪ 2024-08-28 17:51:47 UTC ya make is running...
🟡 2024-08-28 18:57:47 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
39248	33871	0	4	5358	15

⚪ 2024-08-28 19:01:55 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-08-28 19:10:28 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
23 (only retried tests)	11	0	1	0	11

⚪ 2024-08-28 19:10:36 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-08-28 19:18:08 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
17 (only retried tests)	5	0	1	0	11

🟢 2024-08-28 19:18:15 UTC Build successful.
🟡 2024-08-28 19:18:51 UTC ydbd size 8.2 GiB changed* by +700.6 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `83be5ee`	merge: `f1e4a9b`	diff	diff %
ydbd size	8 808 868 152 Bytes	8 809 585 528 Bytes	+700.6 KiB	+0.008%
ydbd stripped size	480 364 712 Bytes	480 388 680 Bytes	+23.4 KiB	+0.005%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-28T17:51:59Z

⚪ 2024-08-28 17:51:59 UTC Pre-commit check linux-x86_64-release-clang14 for f1e4a9b has started.
⚪ 2024-08-28 17:55:29 UTC ya make is running...
🟢 2024-08-28 18:05:16 UTC Build successful.

github-actions · 2024-08-28T17:55:35Z

⚪ 2024-08-28 17:55:35 UTC Pre-commit check linux-x86_64-release-asan for f1e4a9b has started.
⚪ 2024-08-28 17:58:20 UTC ya make is running...
🔴 2024-08-28 19:29:57 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10087	10016	0	19	28	24

🟢 2024-08-28 19:30:51 UTC Build successful.
🟡 2024-08-28 19:31:20 UTC ydbd size 5.5 GiB changed* by +1.2 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `93d0e7f`	merge: `f1e4a9b`	diff	diff %
ydbd size	5 935 064 128 Bytes	5 936 288 304 Bytes	+1.2 MiB	+0.021%
ydbd stripped size	1 490 634 800 Bytes	1 490 931 536 Bytes	+289.8 KiB	+0.020%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-29T07:55:40Z

⚪ 2024-08-29 07:55:40 UTC Pre-commit check linux-x86_64-release-clang14 for 0eedcc6 has started.
⚪ 2024-08-29 07:58:25 UTC ya make is running...
🟢 2024-08-29 08:06:42 UTC Build successful.

github-actions · 2024-08-29T07:55:46Z

⚪ 2024-08-29 07:55:45 UTC Pre-commit check linux-x86_64-relwithdebinfo for 0eedcc6 has started.
⚪ 2024-08-29 07:58:37 UTC ya make is running...
🟡 2024-08-29 09:04:11 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
39262	33871	0	6	5366	19

⚪ 2024-08-29 09:07:48 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-08-29 09:15:27 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
32 (only retried tests)	20	0	0	0	12

🟢 2024-08-29 09:15:35 UTC Build successful.
🟢 2024-08-29 09:16:14 UTC ydbd size 8.2 GiB changed* by +81.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `e17a4b8`	merge: `0eedcc6`	diff	diff %
ydbd size	8 809 703 288 Bytes	8 809 786 576 Bytes	+81.3 KiB	+0.001%
ydbd stripped size	480 402 024 Bytes	480 402 664 Bytes	+640 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-29T07:55:48Z

⚪ 2024-08-29 07:55:48 UTC Pre-commit check linux-x86_64-release-asan for 0eedcc6 has started.
⚪ 2024-08-29 07:58:32 UTC ya make is running...
🔴 2024-08-29 09:24:11 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10084	10032	0	8	24	20

🟢 2024-08-29 09:25:04 UTC Build successful.
🟢 2024-08-29 09:25:35 UTC ydbd size 5.5 GiB changed* by +69.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `e17a4b8`	merge: `0eedcc6`	diff	diff %
ydbd size	5 936 332 096 Bytes	5 936 403 200 Bytes	+69.4 KiB	+0.001%
ydbd stripped size	1 490 960 848 Bytes	1 490 963 472 Bytes	+2.6 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

Fix table stats aggregation for OLAP tables

a428c00

github-actions bot added the bugfix label Aug 28, 2024

swalrus1 added 3 commits August 28, 2024 16:34

ашч

1ccaf05

fix

5698095

fix

73111d4

fix

b6c11a1

github-actions bot added bugfix and removed bugfix labels Aug 28, 2024

clean up

e4c7316

github-actions bot added bugfix and removed bugfix labels Aug 28, 2024

fix PartCount

e6715c3

github-actions bot added bugfix and removed bugfix labels Aug 29, 2024

ivanmorozov333 approved these changes Aug 29, 2024

View reviewed changes

ivanmorozov333 merged commit bf8c633 into ydb-platform:main Aug 29, 2024
11 of 13 checks passed

swalrus1 deleted the fix-table-stats-addition-for-olap-table branch August 29, 2024 14:21

niksaveliev mentioned this pull request Aug 29, 2024

Fix pqv1 max partitions #8485

Merged

siarheivesialou mentioned this pull request Aug 30, 2024

LOGBROKER 8891 fix not using optional fields in sqs json api properly #8523

Merged

stanislav-shchetinin pushed a commit to stanislav-shchetinin/ydb that referenced this pull request Aug 30, 2024

Fix table stats aggregation for OLAP tables (ydb-platform#8417)

102eea8

swalrus1 mentioned this pull request Sep 3, 2024

Update stable-24-3: add metrics to describe table #8688

Closed

swalrus1 added a commit to swalrus1/ydb that referenced this pull request Sep 3, 2024

Fix table stats aggregation for OLAP tables (ydb-platform#8417)

5516173

swalrus1 mentioned this pull request Sep 3, 2024

Update stable-24-3: add OLAP metrics to describe table #8689

Closed

swalrus1 added a commit to swalrus1/ydb that referenced this pull request Sep 3, 2024

Fix table stats aggregation for OLAP tables (ydb-platform#8417)

8b5f185

siarheivesialou mentioned this pull request Sep 4, 2024

fix memory leak in ymq proxy #8706

Merged

This was referenced Sep 4, 2024

YQ-3151 added issues for timeout and cancelled #8718

Merged

YQ kqprun added cancel after for script requests #8768

Merged

Fixed unit test ManyColumnShards #8777

Merged

niksaveliev mentioned this pull request Sep 9, 2024

Fix CLI max partitions default #8944

Merged

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 9, 2024

Fix table stats aggregation for OLAP tables (ydb-platform#8417)

03192cf

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 10, 2024

Fix table stats aggregation for OLAP tables (ydb-platform#8417)

68debfb

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 11, 2024

Fix table stats aggregation for OLAP tables (ydb-platform#8417)

d5f90e3

niksaveliev mentioned this pull request Sep 12, 2024

Fix reading without consumer from federation #9136

Merged

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 13, 2024

Fix table stats aggregation for OLAP tables (ydb-platform#8417)

ffdfc6d

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 14, 2024

Fix table stats aggregation for OLAP tables (ydb-platform#8417)

04254d1

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 15, 2024

Fix table stats aggregation for OLAP tables (ydb-platform#8417)

9e7109e

zverevgeny mentioned this pull request Sep 16, 2024

Integrate latest changes in CS into stable branch #9271

Closed

StekPerepolnen mentioned this pull request Sep 17, 2024

doc hc api images source #9376

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix table stats aggregation for OLAP tables #8417

Fix table stats aggregation for OLAP tables #8417

swalrus1 commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 29, 2024 •

edited

Loading

github-actions bot commented Aug 29, 2024 •

edited

Loading

github-actions bot commented Aug 29, 2024 •

edited

Loading

Fix table stats aggregation for OLAP tables #8417

Fix table stats aggregation for OLAP tables #8417

Conversation

swalrus1 commented Aug 28, 2024 • edited Loading

Changelog entry

Changelog category

Additional information

github-actions bot commented Aug 28, 2024 • edited Loading

github-actions bot commented Aug 28, 2024 • edited Loading

github-actions bot commented Aug 28, 2024 • edited Loading

github-actions bot commented Aug 28, 2024 • edited Loading

github-actions bot commented Aug 28, 2024 • edited Loading

github-actions bot commented Aug 28, 2024 • edited Loading

github-actions bot commented Aug 28, 2024 • edited Loading

github-actions bot commented Aug 28, 2024 • edited Loading

github-actions bot commented Aug 28, 2024 • edited Loading

github-actions bot commented Aug 29, 2024 • edited Loading

github-actions bot commented Aug 29, 2024 • edited Loading

github-actions bot commented Aug 29, 2024 • edited Loading

swalrus1 commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

github-actions bot commented Aug 29, 2024 •

edited

Loading

github-actions bot commented Aug 29, 2024 •

edited

Loading

github-actions bot commented Aug 29, 2024 •

edited

Loading