[bug] License Zipper is skipping packages

[fschoenm]

There seem to be situations where the license zipper (and maybe other deployers) cannot collect files from all dependencies. This seems to happen if Conan decides that it can skip certain dependencies (e.g. header-only libs that have already been compiled into a static lib).

However, of course the license of skipped packages might still mandate me to bring it along, which is entirely unrelated to which packages have to be available for the build process.

Is there even a way to handle this situation correctly (or maybe I'm operating it wrong)? Otherwise Conan might be inadequate to correctly handle license obligations.

Steps to reproduce

I tried as example with the fast-dds package that depends on asio:

0. Install conan-extensions with conan config install.
1. Make sure all packages exist in your package cache:

$ conan install --requires fast-dds/2.10.1 -b missing
[...]

2. Call conan install again with the license deployer:

$ conan install --requires fast-dds/2.10.1 --deployer=licenses
[...]
======== Computing dependency graph ========
Graph root
    cli
Requirements
    asio/1.28.0#83b60467ba4e3487d61a29a722166a0b - Cache
    fast-cdr/1.0.27#8dbda822b54bd311d8765cd3ea9b3381 - Cache
    fast-dds/2.10.1#41c9359faf3149a48652b3d049a2883a - Cache
    foonathan-memory/0.7.3#e15d6872869ad2a42dc4e8b95e624772 - Cache
    tinyxml2/9.0.0#f53e08b723411bc90730f934c6c2f511 - Cache
Build requirements
    cmake/3.27.0#1916e6ab33353145b93244d065a2c868 - Cache
Resolved version ranges
    cmake/[>=3.16.3 <4]: cmake/3.27.0

======== Computing necessary packages ========
Requirements
    asio/1.28.0#83b60467ba4e3487d61a29a722166a0b:da39a3ee5e6b4b0d3255bfef95601890afd80709#a99d859c17cb5264f9ecec84095602ae - Skip
    fast-cdr/1.0.27#8dbda822b54bd311d8765cd3ea9b3381:d841a431cd09fff8a601e5cb5d88ba4c9a0ac886#118448c3308155aa9521fbbab814f2f7 - Cache
    fast-dds/2.10.1#41c9359faf3149a48652b3d049a2883a:e10bb7c35eee6d3b2521d8cc27ed8664095ea09e#a3be3e7f2ecdbcbe29dc57ce732cb7bc - Cache
    foonathan-memory/0.7.3#e15d6872869ad2a42dc4e8b95e624772:a7502e7e8a977fc7b55a68c566cea99e85523872#587a800f6de0077c6e37fce471c27920 - Cache
    tinyxml2/9.0.0#f53e08b723411bc90730f934c6c2f511:d841a431cd09fff8a601e5cb5d88ba4c9a0ac886#1f0cab4b10005ca32472a4f43e15fa19 - Cache
Build requirements
    cmake/3.27.0#1916e6ab33353145b93244d065a2c868:63fead0844576fc02943e16909f08fcdddd6f44b#ca1493fa25e1944ebc4de1c49ac0fad5 - Skip

======== Installing packages ========
fast-cdr/1.0.27: Already installed! (1 of 4)
foonathan-memory/0.7.3: Already installed! (2 of 4)
foonathan-memory/0.7.3: Appending PATH env var with : /home/fschoenm/devel/conan-license-test/conan/p/b/foona04b81a8e3ccb8/p/bin
tinyxml2/9.0.0: Already installed! (3 of 4)
fast-dds/2.10.1: Already installed! (4 of 4)
WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X:
WARN: deprecated:     'cpp_info.names' used in: fast-dds/2.10.1, fast-cdr/1.0.27, foonathan-memory/0.7.3
WARN: deprecated:     'cpp_info.build_modules' used in: fast-dds/2.10.1, fast-cdr/1.0.27, foonathan-memory/0.7.3
WARN: deprecated:     'env_info' used in: foonathan-memory/0.7.3

======== Finalizing install (deploy, generators) ========
deployer(licenses): /home/fschoenm/devel/conan-license-test/conan/p/b/fast-8bd3ddd4b92b9/p/licenses
deployer(licenses): /home/fschoenm/devel/conan-license-test/licenses/fast-dds/2.10.1
deployer(licenses): /home/fschoenm/devel/conan-license-test/conan/p/b/tinyx15bb29e242aad/p/licenses
deployer(licenses): /home/fschoenm/devel/conan-license-test/licenses/tinyxml2/9.0.0
deployer(licenses): /home/fschoenm/devel/conan-license-test/conan/p/b/fast-480486eeb23cb/p/licenses
deployer(licenses): /home/fschoenm/devel/conan-license-test/licenses/fast-cdr/1.0.27
deployer(licenses): /home/fschoenm/devel/conan-license-test/conan/p/b/foona04b81a8e3ccb8/p/licenses
deployer(licenses): /home/fschoenm/devel/conan-license-test/licenses/foonathan-memory/0.7.3
deployer(licenses): ['fast-dds/2.10.1/LICENSE', 'tinyxml2/9.0.0/LICENSE.txt', 'fast-cdr/1.0.27/LICENSE', 'foonathan-memory/0.7.3/LICENSE']

[...]

3. Check the licenses.zip:

$ unzip -l licenses.zip
Archive:  licenses.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
    11358  04-04-2023 14:53   fast-dds/2.10.1/LICENSE
      808  06-07-2021 02:10   tinyxml2/9.0.0/LICENSE.txt
    11358  03-22-2023 07:29   fast-cdr/1.0.27/LICENSE
      902  01-11-2023 12:44   foonathan-memory/0.7.3/LICENSE
---------                     -------
    24426                     4 files

As you can see, the asio license is missing.

[AbrilRBS]

Hi @fschoenm thanks for your report.

You're not missing anything, it's true that right now the deployer is not processing the packages that have been skipped. The good news is that this is something that can be addressed in the client to get working, we'll just need a few moments to figure out the best way to approach it, so thanks for your patience :)

[fschoenm]

Thanks for the answer @RubenRBS.

Is this something you're going to tackle in the near future, or is it not worth the wait? I want to decide if we can rely on Conan to handle OSS obligations in our internal projects.

[AbrilRBS]

Yes! We're proposing a new conf that will disable package skipping, so that every dependency in the graph is fetched. We'll ping you once it's ready for use, but it shouldn't take too long.

Thinking ahead, at some point the proper solution might be to have licenses be more of an integral part of the Conan client itself, modeling it in a more native way, but that's some ways off :)

[prince-chrismc]

This is coming in 2.0.10 soon hopefully 🤞

[AbrilRBS]

Available in 2.0.10 now, let us know if you find any issues with it :)