Installing error when using python:alpine as the base image

[rabbit19981023]

Hello,

I'm trying to bundle my app into a docker image.

When using python:alpine as the base image, it causes errors like this (while python:slim doesn't cause this error):

#0 22.01   Downloading fugashi-1.2.1.tar.gz (337 kB)
#0 22.08      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 338.0/338.0 kB 4.7 MB/s eta 0:00:00
#0 22.11   Preparing metadata (setup.py): started
#0 22.86   Preparing metadata (setup.py): finished with status 'error'
#0 22.87   error: subprocess-exited-with-error
#0 22.87
#0 22.87   × python setup.py egg_info did not run successfully.
#0 22.87   │ exit code: 1
#0 22.87   ╰─> [9 lines of output]
#0 22.87       Traceback (most recent call last):
#0 22.87         File "<string>", line 2, in <module>
#0 22.87         File "<pip-setuptools-caller>", line 34, in <module>
#0 22.87         File "/tmp/pip-install-wxx7nb2x/fugashi_e3388a1cdbb74184951bb5024087448f/setup.py", line 15, in <module>
#0 22.87           output, data_files = check_libmecab()
#0 22.87                                ^^^^^^^^^^^^^^^^
#0 22.87         File "/tmp/pip-install-wxx7nb2x/fugashi_e3388a1cdbb74184951bb5024087448f/fugashi_util.py", line 58, in check_libmecab
#0 22.87           raise RuntimeError("Could not configure working env. Have you installed MeCab?")
#0 22.87       RuntimeError: Could not configure working env. Have you installed MeCab?
#0 22.87       [end of output]
#0 22.87
#0 22.87   note: This error originates from a subprocess, and is likely not a problem with pip.
#0 22.87 error: metadata-generation-failed
#0 22.87
#0 22.87 × Encountered error while generating package metadata.
#0 22.87 ╰─> See above for output.
#0 22.87
#0 22.87 note: This is an issue with the package mentioned above, not pip.
#0 22.87 hint: See above for details.

I'm not sure if it's just that mecab doesn't support the platform, or there's something missing in check_libmecab().

[polm]

This happens if you build from source and don't have MeCab installed previously. fugashi will not install MeCab for you.

It seems likely you're using something that doesn't have a wheel - what version of Python and what architecture are you using?

[rabbit19981023]

This is the Dockerfile that causes this problem:

FROM python:alpine

RUN pip install fugashi

I've checked the python/pip version and the architecture, I found that they uses the same version & architecture:

"PYTHON_VERSION=3.11.4",
"PYTHON_PIP_VERSION=23.1.2",
"PYTHON_SETUPTOOLS_VERSION=65.5.1",

"Architecture": "amd64",
"Os": "linux",

docker inspect python:alpine:

[
    {
        "Id": "sha256:35f8462d36fe25f1117df6aa781567fe73dcd0796c859252c0894b1e308e27ce",
        "RepoTags": [
            "python:alpine"
        ],
        "RepoDigests": [
            "python@sha256:995c7fcdf9a10e0e1a4555861dac63436b456822a167f07b6599d4f105de6fa0"
        ],
        "Parent": "",
        "Comment": "buildkit.dockerfile.v0",
        "Created": "2023-06-07T09:49:17Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "LANG=C.UTF-8",
                "GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D",
                "PYTHON_VERSION=3.11.4",
                "PYTHON_PIP_VERSION=23.1.2",
                "PYTHON_SETUPTOOLS_VERSION=65.5.1",
                "PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/0d8570dc44796f4369b652222cf176b3db6ac70e/public/get-pip.py",
                "PYTHON_GET_PIP_SHA256=96461deced5c2a487ddc65207ec5a9cffeca0d34e7af7ea1afc470ff0d746207"
            ],
            "Cmd": [
                "python3"
            ],
            "ArgsEscaped": true,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 57236257,
        "VirtualSize": 57236257,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/14d1964323fa430bf9393e101ed13c931787ab61a7206dcfec4c2158002c9dbe/diff:/var/lib/docker/overlay2/a311e37f352cb400d32b5ebe7d065cdcf8a6f82c898ab531ff1a68e6e16df3c4/diff:/var/lib/docker/overlay2/9f6fb011edc7bfbc9c08d1ae34dff39c0e979d5a20efdaf16a10757b49e92ff3/diff:/var/lib/docker/overlay2/7caa74b1829fe320818470924e8b03eb545df1ee04dd78778166d31ed097ebc2/diff",
                "MergedDir": "/var/lib/docker/overlay2/3bfc1b784a5ab1852287f858da8d07e35870e2b14ab7d0bee88cc069d96160ac/merged",
                "UpperDir": "/var/lib/docker/overlay2/3bfc1b784a5ab1852287f858da8d07e35870e2b14ab7d0bee88cc069d96160ac/diff",
                "WorkDir": "/var/lib/docker/overlay2/3bfc1b784a5ab1852287f858da8d07e35870e2b14ab7d0bee88cc069d96160ac/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:bb01bd7e32b58b6694c8c3622c230171f1cec24001a82068a8d30d338f420d6c",
                "sha256:208977ac81d79a33082babcb62e6971018ec28df3b7a2d7c8d1c4034251cfa1c",
                "sha256:3b5ed29797e6a75c5005fb96eeefb394b19e43e6e93d8713e489f7e1c65ead8c",
                "sha256:a3941c1792e99e5aa3a54a73ebb8b90ead3fb4cb9640dfe845b28c44bdb27cee",
                "sha256:31215aa3e6aec2932309c297a4bbf1154854a7a16314145a1982a59f68d5ed75"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

docker inspect python:slim:

[
    {
        "Id": "sha256:17e756a5d8f29e5b448bca7c0f4be9fa690cf52364b796833bb9454fe4bd99dc",
        "RepoTags": [
            "python:slim"
        ],
        "RepoDigests": [
            "python@sha256:7ae02e72226c12a65361f7fcf4428ab53b402abcbfc377720b61314cc3bed266"
        ],
        "Parent": "",
        "Comment": "buildkit.dockerfile.v0",
        "Created": "2023-06-07T09:49:17Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "LANG=C.UTF-8",
                "GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D",
                "PYTHON_VERSION=3.11.4",
                "PYTHON_PIP_VERSION=23.1.2",
                "PYTHON_SETUPTOOLS_VERSION=65.5.1",
                "PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/0d8570dc44796f4369b652222cf176b3db6ac70e/public/get-pip.py",
                "PYTHON_GET_PIP_SHA256=96461deced5c2a487ddc65207ec5a9cffeca0d34e7af7ea1afc470ff0d746207"
            ],
            "Cmd": [
                "python3"
            ],
            "ArgsEscaped": true,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 127782111,
        "VirtualSize": 127782111,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/6980c424aaa55d239c4416de0e204e32de80895a97a953d619ed93ab2fbffd81/diff:/var/lib/docker/overlay2/c67213d164717c659ec73107527f717d0c84a3ff2e9c8fb7b371400bd19c751c/diff:/var/lib/docker/overlay2/b00310ddaf9661729e2a6c2f10054bed4c4e9c29cb566ff55c728e4751469899/diff:/var/lib/docker/overlay2/b889c1cb27386b8d5249f21fd7c9ad774c4f517a597add929dd77fa217647570/diff",
                "MergedDir": "/var/lib/docker/overlay2/2fd8799878862e14a892b3e3b45130e630f198a8a021c7839838408629ef2c6f/merged",
                "UpperDir": "/var/lib/docker/overlay2/2fd8799878862e14a892b3e3b45130e630f198a8a021c7839838408629ef2c6f/diff",
                "WorkDir": "/var/lib/docker/overlay2/2fd8799878862e14a892b3e3b45130e630f198a8a021c7839838408629ef2c6f/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:0cc1f01656262cc1319655e8570146e4aa190c3fb8c7e81c353760c44a96c13b",
                "sha256:ee3d5254900ba656373f0ea714661d573bf93cbecfc3e494c9175edc67b5d94a",
                "sha256:540a61bb56db8161aff23bf229b336c35ffb73a7d78556806ec95d3f4ebf5a38",
                "sha256:05c5e11e5347f7cbbf374c2d2f0430073092a294fbb498a266c789c09cf9716f",
                "sha256:5d99a0cd99672ee66f09d7b28933af64b9d6d7e45d52d0a69f5369413218e74c"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

[polm]

Hm, those images both seem to have the same architecture, so not sure what's up. Can you check the verbose output of pip for both of them and see why it decides to install from source instead of from a wheel?

[rabbit19981023]

Finally I found the reason why this happens: Alpine Linux uses musl, while manylinux wheels are based on glibc. (See pypa/pip#9735)

And I also found the working tags for Alpine are *-py3-none-any.whl or *-cp311-cp311-musllinux_1_1_x86_64.whl.

[polm]

Thank you for clarifying, that is something that wouldn't have occurred to me.

Since nobody has ever brought this up before, I don't consider it a priority at the moment, though I understand Alpine is widely used. If someone would like to make a PR to add musl packages to wheel builds I'd be happy to consider it.

In the meantime, you can install MeCab before fugashi, and then building fugashi from source should be fast and work without issue.

[rabbit19981023]

I would just use python:slim, it's more straightforward and also suites my needs (have a small-size container).

And maybe just need to mention this issue in README would help most of the users.

[polm]

Thank you for the clarification. I updated the README with a list of platforms for which wheels are not available.