feat(dev): Use arm64 ClickHouse image on Apple arm64 machines #29081

armenzg · 2021-10-05T19:02:25Z

ClickHouse, currently, does not provide an arm64 Docker image for the clickhouse-server.

Until they do, we can use the experimental image that Altinity produces.
I believe this image is built by this script on an aarch64 host.

Fixes #28564

config/clickhouse/loc_config.xml

armenzg · 2021-10-06T14:21:35Z

On an Apple M1 machine this can be tested with sentry devservices up snuba clickhouse.

lynnagara · 2021-10-06T18:49:21Z

src/sentry/conf/server.py

+            # altinity provides clickhouse support to other companies
+            # Official support: https://github.com/ClickHouse/ClickHouse/issues/22222
+            # This image is build with this script https://gist.github.com/filimonov/5f9732909ff66d5d0a65b8283382590d
+            "image": "altinity/clickhouse-server:21.6.1.6734-testing-arm"


Is this the earliest ClickHouse version that works with ARM? That's a big jump from 20.3 to 21.6.

The version they have are 20.10.1.4844-testing-arm, 21.4.1.6307-testing-arm and 21.6.1.6734-testing-arm. I've downgraded to the 20.x series.

billyvg · 2021-10-06T14:34:56Z

src/sentry/conf/server.py

            "pull": True,
            "ports": {"9000/tcp": 9000, "9009/tcp": 9009, "8123/tcp": 8123},
            "ulimits": [{"name": "nofile", "soft": 262144, "hard": 262144}],
-            "environment": {"MAX_MEMORY_USAGE_RATIO": "0.3"},
+            # The arm image does not properly load the MAX_MEMORY_USAGE_RATIO


Do we know why?

I'm not certain yet. I tried it manually with -e, however, I believe the image does not accept the variable (i.e. MAX_MEMORY_USAGE_RATIO is not set as an ENV in the Docker image).

You can see in this screenshot that it is not defined in the image:

(i.e. MAX_MEMORY_USAGE_RATIO is not set as an ENV in the Docker image).

This is not relevant as you can pass anything via the -e option and it will be set inside at runtime. I'd say we should investigate this (not in this PR tho, as a follow up)

I did try as well manually with -e. Added it to my backlog.

armenzg · 2021-10-12T12:57:45Z

@BYK @lynnagara Billy is out this week. Would you mind reviewing this? Thanks

ClickHouse, currently, [does not provide an `arm64` Docker image for the clickhouse-server][issue]. Until they do, we can use the [experimental image that Altinity produces][image]. I believe this image is built by this [script][script] on an `aarch64` host. Fixes #28564 [issue]: ClickHouse/ClickHouse#22222 [image]: https://hub.docker.com/layers/altinity/clickhouse-server/21.6.1.6734-testing-arm/images/sha256-9a4516444fef9e0f11ee6b2de716d3b97b36bf05d9cc2d44c4596cfb0584dea6?context=explore [script]: https://gist.github.com/filimonov/5f9732909ff66d5d0a65b8283382590d

BYK

Mostly rejecting for the version mismatch.

BYK · 2021-10-12T13:58:35Z

src/sentry/conf/server.py

            "volumes": {
                "clickhouse_dist"
                if settings.SENTRY_DISTRIBUTED_CLICKHOUSE_TABLES
                else "clickhouse": {"bind": "/var/lib/clickhouse"},
+                "clickhouse_logs": {"bind": "/var/log/clickhouse-server"},


Why do we need this, especially as a bind mount?

This is because I needed to extract the logs. I guess I could have attached to the container. I don't need it. I will remove it.

src/sentry/conf/server.py

Co-authored-by: Burak Yigit Kaya <byk@sentry.io>

BYK · 2021-10-12T15:50:06Z

src/sentry/conf/server.py

+            # altinity provides clickhouse support to other companies
+            # Official support: https://github.com/ClickHouse/ClickHouse/issues/22222
+            # This image is build with this script https://gist.github.com/filimonov/5f9732909ff66d5d0a65b8283382590d
+            "image": "yandex/clickhouse-server:20.3.9.70"
+            if not APPLE_ARM64
+            else "altinity/clickhouse-server:20.10.1.4844-testing-arm",


You need to move the comment to line 1809 to avoid confusion:

Suggested change

# altinity provides clickhouse support to other companies

# Official support: https://github.com/ClickHouse/ClickHouse/issues/22222

# This image is build with this script https://gist.github.com/filimonov/5f9732909ff66d5d0a65b8283382590d

"image": "yandex/clickhouse-server:20.3.9.70"

if not APPLE_ARM64

else "altinity/clickhouse-server:20.10.1.4844-testing-arm",

"image": "yandex/clickhouse-server:20.3.9.70"

if not APPLE_ARM64

# altinity provides clickhouse support to other companies

# Official support: https://github.com/ClickHouse/ClickHouse/issues/22222

# This image is build with this script https://gist.github.com/filimonov/5f9732909ff66d5d0a65b8283382590d

else "altinity/clickhouse-server:20.10.1.4844-testing-arm",

In #29081, we changed the arm ClickHouse container at the last moment without realizing that the older version has this `libunwind: Unsupported .eh_frame_hdr version` [issue][issue]. We upgrade to a newer version that does not have profiling enabled. [issue]: ClickHouse/ClickHouse#15638

armenzg self-assigned this Oct 5, 2021

armenzg added the Component: Developer Environment This covers issues related to setting up a developer's environment label Oct 5, 2021

armenzg added this to the Add support for arm64 development services milestone Oct 5, 2021

armenzg linked an issue Oct 5, 2021 that may be closed by this pull request

No clickhouse Docker image for Mac Arm64 #28564

Closed

vercel bot deployed to Preview – storybook October 5, 2021 19:11 View deployment

vercel bot deployed to Preview – sentry October 5, 2021 19:11 View deployment

armenzg force-pushed the armenzg/devservices/clickhouse-arm64 branch from 715b1da to 4b0dd7c Compare October 5, 2021 19:31

vercel bot deployed to Preview – storybook October 5, 2021 19:31 View deployment

vercel bot deployed to Preview – sentry October 5, 2021 19:31 View deployment

vercel bot deployed to Preview – storybook October 6, 2021 13:43 View deployment

vercel bot deployed to Preview – sentry October 6, 2021 13:44 View deployment

armenzg changed the base branch from armenzg/dev/confluent-kafka-apple-m1 to master October 6, 2021 13:51

armenzg force-pushed the armenzg/devservices/clickhouse-arm64 branch from 1550afb to ef8b77e Compare October 6, 2021 13:59

vercel bot deployed to Preview – storybook October 6, 2021 13:59 View deployment

vercel bot deployed to Preview – sentry October 6, 2021 13:59 View deployment

armenzg force-pushed the armenzg/devservices/clickhouse-arm64 branch from ef8b77e to d796462 Compare October 6, 2021 14:00

vercel bot deployed to Preview – storybook October 6, 2021 14:02 View deployment

vercel bot deployed to Preview – sentry October 6, 2021 14:02 View deployment

vercel bot deployed to Preview – sentry October 6, 2021 14:14 View deployment

vercel bot deployed to Preview – storybook October 6, 2021 14:14 View deployment

vercel bot deployed to Preview – sentry October 6, 2021 14:17 View deployment

armenzg commented Oct 6, 2021

View reviewed changes

config/clickhouse/loc_config.xml Show resolved Hide resolved

vercel bot deployed to Preview – storybook October 6, 2021 14:17 View deployment

armenzg marked this pull request as ready for review October 6, 2021 14:20

armenzg requested review from billyvg and BYK October 6, 2021 14:21

lynnagara reviewed Oct 6, 2021

View reviewed changes

billyvg reviewed Oct 6, 2021

View reviewed changes

vercel bot deployed to Preview – sentry October 12, 2021 12:56 View deployment

vercel bot deployed to Preview – storybook October 12, 2021 12:56 View deployment

armenzg enabled auto-merge (squash) October 12, 2021 13:00

armenzg added 3 commits October 12, 2021 09:21

Comment change

195ac37

Address feedback

91b95d3

armenzg force-pushed the armenzg/devservices/clickhouse-arm64 branch from 2fd876b to 91b95d3 Compare October 12, 2021 13:22

vercel bot deployed to Preview – storybook October 12, 2021 13:22 View deployment

vercel bot deployed to Preview – sentry October 12, 2021 13:22 View deployment

BYK requested changes Oct 12, 2021

View reviewed changes

Update src/sentry/conf/server.py

5a6a765

Co-authored-by: Burak Yigit Kaya <byk@sentry.io>

vercel bot deployed to Preview – sentry October 12, 2021 15:26 View deployment

vercel bot deployed to Preview – storybook October 12, 2021 15:26 View deployment

Remove logs bind mount

ada1263

vercel bot deployed to Preview – sentry October 12, 2021 15:31 View deployment

vercel bot deployed to Preview – storybook October 12, 2021 15:31 View deployment

armenzg requested review from BYK and fpacifici October 12, 2021 15:31

Merge branch 'master' into armenzg/devservices/clickhouse-arm64

0b1fc03

vercel bot deployed to Preview – sentry October 12, 2021 15:34 View deployment

vercel bot deployed to Preview – storybook October 12, 2021 15:34 View deployment

BYK approved these changes Oct 12, 2021

View reviewed changes

armenzg merged commit f7238b6 into master Oct 12, 2021

armenzg deleted the armenzg/devservices/clickhouse-arm64 branch October 12, 2021 16:52

armenzg mentioned this pull request Oct 13, 2021

fix(dev): ClickHouse dev container on arm64 has errors #29284

Merged

github-actions bot locked and limited conversation to collaborators Oct 28, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(dev): Use arm64 ClickHouse image on Apple arm64 machines #29081

feat(dev): Use arm64 ClickHouse image on Apple arm64 machines #29081

armenzg commented Oct 5, 2021

armenzg commented Oct 6, 2021

lynnagara Oct 6, 2021

armenzg Oct 7, 2021

billyvg Oct 6, 2021

armenzg Oct 7, 2021

BYK Oct 12, 2021

armenzg Oct 12, 2021

armenzg commented Oct 12, 2021

BYK left a comment

BYK Oct 12, 2021

armenzg Oct 12, 2021

BYK Oct 12, 2021

feat(dev): Use arm64 ClickHouse image on Apple arm64 machines #29081

feat(dev): Use arm64 ClickHouse image on Apple arm64 machines #29081

Conversation

armenzg commented Oct 5, 2021

armenzg commented Oct 6, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

armenzg commented Oct 12, 2021

BYK left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment