Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dev): Use arm64 ClickHouse image on Apple arm64 machines #29081

Merged
merged 6 commits into from
Oct 12, 2021
Merged

feat(dev): Use arm64 ClickHouse image on Apple arm64 machines #29081

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
b8aa5dc
feat(dev): Use arm64 ClickHouse image on Apple arm64 machines
armenzg Oct 5, 2021
195ac37
Comment change
armenzg Oct 6, 2021
91b95d3
Address feedback
armenzg Oct 7, 2021
5a6a765
Update src/sentry/conf/server.py
armenzg Oct 12, 2021
ada1263
Remove logs bind mount
armenzg Oct 12, 2021
0b1fc03
Merge branch 'master' into armenzg/devservices/clickhouse-arm64
armenzg Oct 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion config/clickhouse/dist_config.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
<yandex>
<max_server_memory_usage_to_ram_ratio from_env="MAX_MEMORY_USAGE_RATIO" />
<max_server_memory_usage_to_ram_ratio>0.3</max_server_memory_usage_to_ram_ratio>


<remote_servers>
<cluster_one_sh>
Expand Down
2 changes: 1 addition & 1 deletion config/clickhouse/loc_config.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
<yandex>
<max_server_memory_usage_to_ram_ratio from_env="MAX_MEMORY_USAGE_RATIO" />
<max_server_memory_usage_to_ram_ratio>0.3</max_server_memory_usage_to_ram_ratio>
</yandex>
10 changes: 8 additions & 2 deletions src/sentry/conf/server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1802,11 +1802,17 @@ def build_cdc_postgres_init_db_volume(settings):
),
"clickhouse": lambda settings, options: (
{
"image": "yandex/clickhouse-server:20.3.9.70",
# altinity provides clickhouse support to other companies
# Official support: https://github.com/ClickHouse/ClickHouse/issues/22222
# This image is build with this script https://gist.github.com/filimonov/5f9732909ff66d5d0a65b8283382590d
"image": "yandex/clickhouse-server:20.3.9.70"
if not APPLE_ARM64
else "altinity/clickhouse-server:20.10.1.4844-testing-arm",
Comment on lines +1805 to +1810
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to move the comment to line 1809 to avoid confusion:

Suggested change
# altinity provides clickhouse support to other companies
# Official support: https://github.com/ClickHouse/ClickHouse/issues/22222
# This image is build with this script https://gist.github.com/filimonov/5f9732909ff66d5d0a65b8283382590d
"image": "yandex/clickhouse-server:20.3.9.70"
if not APPLE_ARM64
else "altinity/clickhouse-server:20.10.1.4844-testing-arm",
"image": "yandex/clickhouse-server:20.3.9.70"
if not APPLE_ARM64
# altinity provides clickhouse support to other companies
# Official support: https://github.com/ClickHouse/ClickHouse/issues/22222
# This image is build with this script https://gist.github.com/filimonov/5f9732909ff66d5d0a65b8283382590d
else "altinity/clickhouse-server:20.10.1.4844-testing-arm",

"pull": True,
"ports": {"9000/tcp": 9000, "9009/tcp": 9009, "8123/tcp": 8123},
"ulimits": [{"name": "nofile", "soft": 262144, "hard": 262144}],
"environment": {"MAX_MEMORY_USAGE_RATIO": "0.3"},
# The arm image does not properly load the MAX_MEMORY_USAGE_RATIO
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know why?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not certain yet. I tried it manually with -e, however, I believe the image does not accept the variable (i.e. MAX_MEMORY_USAGE_RATIO is not set as an ENV in the Docker image).

You can see in this screenshot that it is not defined in the image:
image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(i.e. MAX_MEMORY_USAGE_RATIO is not set as an ENV in the Docker image).

This is not relevant as you can pass anything via the -e option and it will be set inside at runtime. I'd say we should investigate this (not in this PR tho, as a follow up)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did try as well manually with -e. Added it to my backlog.

# from the environment in loc_config.xml, thus, hard-coding it there
"volumes": {
"clickhouse_dist"
if settings.SENTRY_DISTRIBUTED_CLICKHOUSE_TABLES
Expand Down