Dockerfile oss #11472

kobzonega · 2024-11-11T11:38:46Z

Changelog entry

Docker image

Base image

Base image is official ubuntu:focal with installed packages:

libcap2-bin (for setcap to binaries)
ca-certificates (for working with CA bundle)

Also base image included LICENSE, AUTHORS and README.md files from root of repository
and dynamic cpp libraries libiconv, liblibidn and libaio.

Base breakpad image

Extend base image with:

additional packages to collect and manage minidump format (binutils, gdb, strace, linux-tools-generic)
dynamic library libbreakpad_init.so from breakpad_init image (ydb/deploy/breakpad_init)
environment variable LD_PRELOAD for loading breakpad library on process start
environment variables BREAKPAD_MINIDUMPS_PATH and BREAKPAD_MINIDUMPS_SCRIPT to setup breakpad
binaries minidump_stackwalk and minidump-2-core to collect stacktrace and convert in coredump format
python script minidump_script.py as dumpCallback handler for breakpad

Image Types

Release

Image with minimal requirements to launch ydbd in container

ya package --docker ydb/deploy/docker/release/pkg.json

Used base image and included:

ydb cli binary
ydbd server strip'ed binary

Breakpad

Image with google breakpad assets to collect minidump

ya package --docker ydb/deploy/docker/breakpad/pkg.json

Used base image and included:

ydb cli binary
ydbd server strip'ed binary

Debug

Image with debug symbols and utils for dev purposes

ya package --docker ydb/deploy/docker/debug/pkg.json

Extend breakpad image with:

additional packages with debug utils (dnsutils, telnet, netcat-openbsd, iputils-ping, curl)
debug symbols for ydbd binary

Changelog category

Improvement

Additional information

...

github-actions · 2024-11-11T11:46:09Z

⚪ 2024-11-11 11:46:08 UTC Pre-commit check linux-x86_64-relwithdebinfo for 38d6a0b has started.
⚪ 2024-11-11 11:46:20 UTC Artifacts will be uploaded here
⚪ 2024-11-11 11:49:03 UTC ya make is running...
🟡 2024-11-11 12:15:58 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
3774	3661	0	1	36	76

⚪ 2024-11-11 12:16:33 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-11-11 12:21:17 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
76 (only retried tests)	2	0	0	0	74

🟢 2024-11-11 12:21:34 UTC Build successful.
🟢 2024-11-11 12:21:46 UTC ydbd size 2.8 GiB changed* by 0 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `8c8ed07`	merge: `38d6a0b`	diff	diff %
ydbd size	3 046 966 392 Bytes	3 046 966 392 Bytes	0 Bytes	0.000%
ydbd stripped size	482 241 624 Bytes	482 241 624 Bytes	0 Bytes	0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-11-11T12:01:01Z

⚪ 2024-11-11 12:01:00 UTC Pre-commit check linux-x86_64-release-asan for 38d6a0b has started.
⚪ 2024-11-11 12:01:11 UTC Artifacts will be uploaded here
⚪ 2024-11-11 12:03:52 UTC ya make is running...
🟡 2024-11-11 12:32:27 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
1397	1377	0	7	8	5

🟢 2024-11-11 12:32:38 UTC Build successful.
🟢 2024-11-11 12:32:55 UTC ydbd size 5.7 GiB changed* by -96 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `8c8ed07`	merge: `38d6a0b`	diff	diff %
ydbd size	6 165 970 720 Bytes	6 165 970 624 Bytes	-96 Bytes	-0.000%
ydbd stripped size	1 537 561 616 Bytes	1 537 561 552 Bytes	-64 Bytes	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

…file-oss

github-actions · 2024-11-13T17:54:48Z

⚪ 2024-11-13 17:54:48 UTC Pre-commit check linux-x86_64-release-asan for 6eecc78 has started.
⚪ 2024-11-13 17:54:52 UTC Artifacts will be uploaded here
⚪ 2024-11-13 17:57:21 UTC ya make is running...
🟡 2024-11-13 18:25:08 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
1400	1385	0	4	5	6

🟢 2024-11-13 18:25:20 UTC Build successful.
🟢 2024-11-13 18:25:40 UTC ydbd size 5.7 GiB changed* by -96 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `224fb1c`	merge: `6eecc78`	diff	diff %
ydbd size	6 171 783 352 Bytes	6 171 783 256 Bytes	-96 Bytes	-0.000%
ydbd stripped size	1 538 234 064 Bytes	1 538 234 000 Bytes	-64 Bytes	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-11-13T17:54:58Z

⚪ 2024-11-13 17:54:57 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6eecc78 has started.
⚪ 2024-11-13 17:55:08 UTC Artifacts will be uploaded here
⚪ 2024-11-13 17:57:42 UTC ya make is running...
🟢 2024-11-13 18:25:54 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
3775	3662	0	0	38	75

🟢 2024-11-13 18:26:17 UTC Build successful.
🟢 2024-11-13 18:26:28 UTC ydbd size 2.8 GiB changed* by 0 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `00bc3bd`	merge: `6eecc78`	diff	diff %
ydbd size	3 049 865 720 Bytes	3 049 865 720 Bytes	0 Bytes	0.000%
ydbd stripped size	482 447 552 Bytes	482 447 552 Bytes	0 Bytes	0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

iddqdex

А можно удаление tools/ydbd_slice/image/Dockerfile и tools/ydbd_slice/image/pkj.json сделать отдельным коммитом после того, как мы переключимся на новые пакеты?

github-actions · 2024-11-14T15:58:59Z

⚪ 2024-11-14 15:58:58 UTC Pre-commit check linux-x86_64-release-asan for e4f4205 has started.
⚪ 2024-11-14 15:59:09 UTC Artifacts will be uploaded here
⚫ 2024-11-14 15:59:53 UTC Check cancelled

github-actions · 2024-11-14T16:01:33Z

⚪ 2024-11-14 16:01:33 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6b21198 has started.
⚪ 2024-11-14 16:01:44 UTC Artifacts will be uploaded here
⚪ 2024-11-14 16:04:16 UTC ya make is running...
🟢 2024-11-14 16:30:41 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
3775	3662	0	0	36	77

🟢 2024-11-14 16:31:03 UTC Build successful.
🟢 2024-11-14 16:31:15 UTC ydbd size 2.8 GiB changed* by +6.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `f282430`	merge: `6b21198`	diff	diff %
ydbd size	3 052 992 072 Bytes	3 052 998 176 Bytes	+6.0 KiB	+0.000%
ydbd stripped size	483 022 688 Bytes	483 023 328 Bytes	+640 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-11-14T16:02:05Z

⚪ 2024-11-14 16:02:05 UTC Pre-commit check linux-x86_64-release-asan for 6b21198 has started.
⚪ 2024-11-14 16:02:17 UTC Artifacts will be uploaded here
⚪ 2024-11-14 16:04:54 UTC ya make is running...
🟡 2024-11-14 16:33:27 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
1400	1384	0	4	5	7

🟢 2024-11-14 16:33:40 UTC Build successful.
🟢 2024-11-14 16:33:57 UTC ydbd size 5.8 GiB changed* by +6.6 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `f282430`	merge: `6b21198`	diff	diff %
ydbd size	6 179 300 448 Bytes	6 179 307 192 Bytes	+6.6 KiB	+0.000%
ydbd stripped size	1 540 781 008 Bytes	1 540 781 776 Bytes	+768 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-11-14T19:18:04Z

⚪ 2024-11-14 19:18:03 UTC Pre-commit check linux-x86_64-relwithdebinfo for 7285086 has started.
⚪ 2024-11-14 19:18:40 UTC Artifacts will be uploaded here
⚪ 2024-11-14 19:21:42 UTC ya make is running...
🟢 2024-11-14 19:49:01 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
3775	3662	0	0	36	77

🟢 2024-11-14 19:49:28 UTC Build successful.
🟢 2024-11-14 19:49:40 UTC ydbd size 2.5 GiB changed* by 0 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `1c441a9`	merge: `7285086`	diff	diff %
ydbd size	2 681 690 664 Bytes	2 681 690 664 Bytes	0 Bytes	0.000%
ydbd stripped size	480 769 456 Bytes	480 769 456 Bytes	0 Bytes	0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-11-14T19:18:59Z

⚪ 2024-11-14 19:18:58 UTC Pre-commit check linux-x86_64-release-asan for 7285086 has started.
⚪ 2024-11-14 19:19:41 UTC Artifacts will be uploaded here
⚪ 2024-11-14 19:23:05 UTC ya make is running...
🟡 2024-11-14 19:50:17 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
1397	1384	0	3	5	5

🟢 2024-11-14 19:50:29 UTC Build successful.
🟢 2024-11-14 19:50:45 UTC ydbd size 4.9 GiB changed* by -96 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `1c441a9`	merge: `7285086`	diff	diff %
ydbd size	5 255 183 272 Bytes	5 255 183 176 Bytes	-96 Bytes	-0.000%
ydbd stripped size	1 350 165 328 Bytes	1 350 165 264 Bytes	-64 Bytes	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

kobzonega added 6 commits November 8, 2024 16:07

breakpad dockerfile

54adfd5

release dockerfile

8be35f1

debug dockerfile

3be9114

ydbd_slice uses new dockerfile

dc9c2e1

add coredumps volume for ydbd_slice kube templates

8116323

fixes

7e5b6ca

github-actions bot added the new-feature label Nov 11, 2024

kobzonega added 2 commits November 13, 2024 17:24

fixes

28f3166

Merge branch 'dockerfile-oss' of github.com:kobzonega/ydb into docker…

af3f877

…file-oss

github-actions bot added improvement and removed new-feature labels Nov 14, 2024

iddqdex requested changes Nov 14, 2024

View reviewed changes

github-actions bot added improvement and removed improvement labels Nov 14, 2024

revert changes

0787cae

kobzonega force-pushed the dockerfile-oss branch from 3c67d60 to 0787cae Compare November 14, 2024 15:59

release package folder

5f900f5

github-actions bot added improvement and removed improvement labels Nov 14, 2024

fomichev3000 approved these changes Nov 15, 2024

View reviewed changes

iddqdex approved these changes Nov 15, 2024

View reviewed changes

iddqdex merged commit 0a30799 into ydb-platform:main Nov 15, 2024
11 checks passed

kobzonega added a commit to kobzonega/ydb that referenced this pull request Nov 15, 2024

Dockerfile oss (ydb-platform#11472)

b2a7de9

kobzonega mentioned this pull request Nov 15, 2024

Dockerfile oss (cherry-pick to stable 24-3) #11633

Merged

GrigoriyPA mentioned this pull request Nov 25, 2024

YQ-3738 RD pass UV from filter to read actor #11940

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dockerfile oss #11472

Dockerfile oss #11472

kobzonega commented Nov 11, 2024 •

edited

Loading

github-actions bot commented Nov 11, 2024 •

edited

Loading

github-actions bot commented Nov 11, 2024 •

edited

Loading

github-actions bot commented Nov 13, 2024 •

edited

Loading

github-actions bot commented Nov 13, 2024 •

edited

Loading

iddqdex left a comment

github-actions bot commented Nov 14, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024 •

edited

Loading

Dockerfile oss #11472

Dockerfile oss #11472

Conversation

kobzonega commented Nov 11, 2024 • edited Loading

Changelog entry

Docker image

Base image

Base breakpad image

Image Types

Release

Breakpad

Debug

Changelog category

Additional information

github-actions bot commented Nov 11, 2024 • edited Loading

github-actions bot commented Nov 11, 2024 • edited Loading

github-actions bot commented Nov 13, 2024 • edited Loading

github-actions bot commented Nov 13, 2024 • edited Loading

iddqdex left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 14, 2024 • edited Loading

github-actions bot commented Nov 14, 2024 • edited Loading

github-actions bot commented Nov 14, 2024 • edited Loading

github-actions bot commented Nov 14, 2024 • edited Loading

github-actions bot commented Nov 14, 2024 • edited Loading

kobzonega commented Nov 11, 2024 •

edited

Loading

github-actions bot commented Nov 11, 2024 •

edited

Loading

github-actions bot commented Nov 11, 2024 •

edited

Loading

github-actions bot commented Nov 13, 2024 •

edited

Loading

github-actions bot commented Nov 13, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024 •

edited

Loading