Python 3.9

[stchris]

This includes the following changes:

• changes the base image from ubuntu:20.04 to python:3.9, which is Debian based (Python 3.9 release notes)
• runs tests in dev mode and enables tracemalloc to allow us to see deperecation warnings and resource misusage. This slows down test runs a little bit (maybe 10s on top of the 150s on my machine), but allows us to catch some warnings much earlier.
• updates a deprecated way to get the current version

Note

Up for discussion: should we have a feature flag which allows us to turn on dev mode for the application? This way we could get deprecation and resource usage warnings from the real application in the staging env.

[tillprochaska]

One thing related to transliteration that would need to be fixed, otherwise this looks good to me.

[tillprochaska]

Not sure if this is in scope for this PR, so feel free to ignore this: I ran apt list --installed in a stock python:3.9 container which included the following:

• ca-certificates
• libpq-dev
• curl
• git
• libxml2-dev
• libxslt1-dev

I think it would be nice to remove stuff from the list that isn’t strictly required, just to make maintenance easier and prevent confusion around why certain packages are installed in the future.

I removed the packages listed above from the Dockerfile and built the image without problems, so maybe we can just drop them?

[stchris]

Thanks, I didn't bother to check! Will remove those!

[tillprochaska]

I also noticed that you removed python3-icu. While this is an optional dependency and Aleph will run without it, it is recommended for improved transliteration in some alphabets. This affects transliterations displayed in the UI, but also xref etc.

Didn’t want to commit directly to your PR, but here’s a test case that you can add to test_entities_api.py that succeeds if PyICU is installed and fail otherwise.

    def test_view_transliterate(self):
        role, headers = self.login(is_admin=True)

        data = {
            "id": "1",
            "schema": "Person",
            "properties": {
                "name": ["İlham Əliyev"],
            },
        }
        entity = self.create_entity(data, self.col)
        index_entity(entity)

        res = self.client.get(f"/api/2/entities/{entity.id}", headers=headers)
        assert res.json["properties"]["name"][0] == "İlham Əliyev"
        assert res.json["latinized"]["İlham Əliyev"] == "Ilham Aliyev"

You can install PyICU as a Debian package python3-icu or via pip as pyicu (but I think that will build it from source).