Switch to poetry and make reuse (pretty much) reproducible
#509
Conversation
This commit adds everything needed to switch dependency management over to poetry. This has the potential to streamline the development/publishing workflows quite a bit. Here is poetry's documentation: https://python-poetry.org/docs/
reuse (pretty much) reproducible
This adds compatibility with Nix, the purely functional package manager. This gives us a declarative development environment with `reuse` in editable mode and a fully reproducible build of `reuse` and its entire dependency tree across multiple architectures. As a cherry on top, it can also build a fully reproducible docker image. If you are curious about Nix, the package manager, check out the manual: https://nixos.org/manual/nix/stable/ For more information about Flakes, this blog post is helpful: https://www.tweag.io/blog/2020-05-25-flakes/
|
This is awesome! I need to find some time to review this. |
|
@carmenbianca did you already find time to review this or is this good to go for you? It's a bit of a blocker for further steps :) |
There was a problem hiding this comment.
Finally got the chance to review this. I can't speak to the Nix scrips—the syntax is completely foreign to me, and all my exposure to Nix is a single super interesting talk.
The poetry config makes perfect sense to me.
Some notes:
- Basic usage should probably be documented.
CONTRIBUTING.mdis probably a good destination for this. As a matter of fact, there's some stuff in the README that should probably be moved to that destination, but we can delay that. - There is currently no feature parity between the poetry config and the setuptools config.
- Translations aren't included in the built package.
- The long description is not included in the built package.
- The license files are not included in the built package.
However, none of these things are blockers if we want to maintain this as a parallel project until we deprecate the setuptools methods. If we do deprecate the setuptools methods, I think it'd be nice to get rid of the custom Makefile as well. It's a bit clunky and awkward here and there.
LGTM, thanks a lot @linozen, and sorry for the long wait!
edit: When this is merged, it's a good idea to create an issue for feature paritry.
|
@linozen LGTM! Is the plan to completely switch to this as a build system at some point or does it mean we have to maintain two separate setups forever? Sorry if that is a silly question, but I'm not familiar enough with this new part yet. |
Thanks for approving. So, I think it would be beneficial to commit fully to The Nix* stuff is mainly there for me so I can develop effectively. Furthermore, it also allows other Nix users to spin up a development environment in no time. More generally, IMHO efforts like Nix or Guix that focus on reproducibility, declarativeness and reliability to be the future of packaging any application. For now, I'd just be very happy if the two files ending in |
|
This should close #289. More generally, this is a big step towards better reproducibility of both the development environment and the resulting binaries. This is the case as poetry captures the entire dependency tree of
reuseinpoetry.lock.In addition, it also adds some files that ensure full reproducibility (basically by pinning everything, incl. compilers etc.) via Nix, a purely functional package manager. It's easy to install on any operating system. For more information about Nix flakes (a newish feature and soon-to-standard of the Nix ecosystem), this blog post might be helpful.
For more information about why reproducibility matters, see the docs of reproducible-builds.org
For now, this entire PR (if pushed to master) can co-exist happily with the build/dev/publish infrastructure we have in place currently. I can commit to keeping this branch up-to-date until we have everything in place for the merge - but as I said, there will be no conflict. Feel free to ping me if you have questions/reservations re: this PR, the switch to poetry or reproducible builds (with Nix) more generally.