Use fixed versions of CPAN distributions for rebuilding Docker images

[bschmalhofer]

When the check for changes in the base images is implemented then there will be fairly frequent rebuilds of Docker images.
These rebuilds include the installation of CPAN distributions into the Docker image. Currently most of the CPAN dependencies are installed with their most current versions on CPAN. This is fine for infrequent builds as there the builds are tested before they are tagged and built automatically on Docker Hub. For rebuilding already released versions we want a more stable setup. Specifically we want that the same versions of CPAN distrubutions are installed into the rebuilt image. This can be achieved with cpanfile.snapshot support in Carton.

See also #210.

Originally posted by @bschmalhofer in #3448 (comment)

[bschmalhofer]

The inital approach is that local build update cpanfile,snapshot and that the automated builds use it.

TODO:

• enhance bin/docker/build_docker_images.sh to copy cpanfile.snapshot from the image to the git sandbox
• add cpanfile.snapshot , or maybe cpanfile.docker.snapshot, to the git repository
• tell the automated builds on hub.docker.com to use carton install --deployment
• decide whether the rebuilds are done on Github or on Docker Hub

[bschmalhofer]

The automated builds initially did not accept the heredoc syntax in the Dockerfile otobo.web.dockerfile . Declaring the Dockerfile frontend in the Dockerfile did the trick.

$ more otobo.web.dockerfile -n 5
# syntax=docker/dockerfile:1.9

# This is the build file for the OTOBO web docker image.
# The services OTOBO web and OTOBO daemon use the same image.
# There is also an extra build target otobo-web-kerberos that adds support for Kerberos.

[bschmalhofer]

There still is a problem. When changes from one branch are merged the next higher version branch, e.b. rel-10_1 merged into rel-11_0, then cpanfile.docker.snapshot would also be merged. Excluding files from git merge is not trivially possible.

[bschmalhofer]

I inspected the Github action https://github.com/RotherOSS/otobo/blob/rel-11_0/.github/workflows/cache_local_lib.yml . The action does not consider cpanfile.docker.snapshot when populating the cache. But this is fine. The cache is only used for CodePolicy checks, it does not matter whether the fixed or the most current CPAN module versions are used.

[bschmalhofer]

Another problem are Carton and its dependencies. These are currently installed in otobo.web.dockerfile with cpanm . This means that the most current versions are used. A simple improvement would be to install Carton only for local builds. For devel and release build we can use cpm. cpm handles _cpanfile.snapshot, at least https://github.com/miyagawa/Carmel?tab=readme-ov-file#cpm says so.

[bschmalhofer]

Well, that did not work out. cpm uses Carton::Snapshot for supporting cpanfile.snapshot . This means that carton must be installed. So nothing is gained by using cpm in deployments.

The next idea is actually a somewhat secret option. carton provides the command fatpack , https://metacpan.org/dist/Carton/view/lib/Carton/Doc/Fatpack.pod . This can be used to provide a fatpacked version of carton. The fatpacked version can then be used for both creating cpanfile.snapshot and using cpanfile.snapshot in deployment. In our use case deployment means the generation of a Docker image.

[bschmalhofer]

Switched to fatpacked carton script. The unitest show no new failures, just the 10 already known failures. Looks good.

See perl-carton/carton#237 and miyagawa/cpanminus#577 for the needed workaround for carton fatpack.