Fully reproducible repo2docker builds

[weiji14]

Ensure that people can reproduce the cryocloud software environment locally and in the future.

Some actionable steps:

• Tag versions of the docker image that has been deployed to the CryoCloud JupyterHub using YYYY.MM.DD syntax (Workflow to retag docker images with CalVer string #32)
• Use lockfiles and or version pinning for dependencies installed via conda/pip/apt (Using unified conda-lock file in postBuild for reproducible conda environment #14)
• Ensure cross platform compatibility on Linux/macOS/Windows

Relevant discussions previously:

@weiji14 - absolutely!! My take on this issue is that we should take a "two-level" approach:

• Once we settle on a complete list of packages for the base environment.yml file, then we run a script @yuvipanda wrote that will write in the current conda-forge package versions in it. But that's not a conda lock file, b/c it remains platform independent as it won't have build hashes in it. This makes it easy for users to replicate the environment locally across OSes, which I think is a very important feature (not everything can be done locally, but it's very valuable for users to at least have a reproducible offline workflow that matches the cloud service as much as possible).
• Then, we generate a conda-lock.yml file to pin the docker images with exact builds. We tag the image with a version number at that point.

Regarding versioning, my suggestion would be to use an 2022.X.Y scheme, where the "major" number X changes when new packages are added, whereas the "minor" Y changes when any package version or even build is updated. This will let people say "This project needs the cryohub environment 2022.3.4" with that being a pretty unambiguous reference.

As for the apt packages, I think that's a bit less of a concern, since they change more slowly and we can ensure that a given YYYY series never changes the underlying Ubuntu base image, so only minor version fixes will go in. I don't know of the equivalent of a conda-lock for apt packages, but that's probably just my ignorance.

Originally posted by @fperez in CryoInTheCloud/CryoCloudWebsite#1 (comment)

Ideally, we would use this conda-linux-64.lock to build the docker image for full reproducibility, but there are some PyPI-only packages right now that are not on conda-forge.

@yuvipanda, have you thought about creating conda-forge packages for https://github.com/yuvipanda/jupyter-desktop-server and https://github.com/yuvipanda/jupyter-syncthing-proxy?

Originally posted by @weiji14 in #9 (comment)

References:

• https://repo2docker.readthedocs.io/en/latest/specification.html
• https://www.anaconda.com/blog/8-levels-of-reproducibility

[fperez]

Awesome - many thanks for opening this here so we track progress!

@tsnow03 we should mention this on Friday so the community users are aware of the longer-term view regarding these points, even if not all the pieces will be in place by then.

[yuvipanda]

Also, I think right now repo2docker doesn't actually respect the lockfile at all, so I don't think it's used. The 'right' way to fix this is to add lockfile support to repo2docker. Until then, I suggest:

1. renaming the environment.yml to something else, so it doesn't trigger the repo2docker behavior
2. Adding a postBuild bash file that installs the environment from the lockfile

[weiji14]

Also, I think right now repo2docker doesn't actually respect the lockfile at all, so I don't think it's used. The 'right' way to fix this is to add lockfile support to repo2docker.

Yep, that is being tracked by jupyterhub/repo2docker#1157.

Until then, I suggest:

1. renaming the environment.yml to something else, so it doesn't trigger the repo2docker behavior

2. Adding a postBuild bash file that installs the environment from the lockfile

Ok, then we will need to use the unified conda-lock.yml file rather than the current explicit lockfile conda-linux-64.lock which doesn't handle pip-only dependencies. Let me see if I can get to working using postBuild.

[weiji14]

@yuvipanda, have you thought about creating conda-forge packages for https://github.com/yuvipanda/jupyter-desktop-server and https://github.com/yuvipanda/jupyter-syncthing-proxy?

Turns out we really do need these two packages to be on conda-forge, otherwise conda-lock won't work, see #14 (comment). So I've started Pull Requests at conda-forge/staged-recipes#21368 and conda-forge/staged-recipes#21369 to add those to conda-forge 🙂