conan install failing with stacktrace

[klausholstjacobsen]

What is your question?

conan 2.0.16

I have a lockfile created with --lockfile-packages. Inhere I have a dependency on libcurl:

        [
            "libcurl/8.4.0#c0e7f02841de4c02cf4f52a54328ebaf%1705889426.739",
            {
                "e2b8ce0092c007a3b9528a956da3a687ad4bfb5a": "e87a5c3e1453c022b01e87f5bb01b540"
            }
        ],

The package has been build and is in my remote, but not in my local cache.
Then I call install to get the dependencies installed.

conan install --lockfile-packages --lockfile=Build/Locks/x86_64-pc-linux-gnu.lock --profile:build Build/Profiles/x86_64-pc-linux-gnu --profile:host Build/Profiles/x86_64-pc-linux-gnu -s:h build_type=Release -s:h &:build_type=Release --update conanfile.py

When calculating what to download from my remote I see this in my graph root:

boost/1.72.0#31df8f6cf801bd2ad49f3f37b161d770 - Cache (conan-virtual)
libcurl/8.4.0#c0e7f02841de4c02cf4f52a54328ebaf - Cache (conan-virtual)
openssl/1.1.1q#cec037cbac52f0c1be52ae8577bd84d0 - Cache (conan-virtual)

which looks fine. The rrev matches!
Later in the output I see:

boost/1.72.0#31df8f6cf801bd2ad49f3f37b161d770:2d0b05800c60a849f908be56adf0271307ce8180#e97361f05636c84310f39576d0ed1a82 - Cache
libcurl/8.4.0#c0e7f02841de4c02cf4f52a54328ebaf:e2b8ce0092c007a3b9528a956da3a687ad4bfb5a#e87a5c3e1453c022b01e87f5bb01b540 - None
openssl/1.1.1q#cec037cbac52f0c1be52ae8577bd84d0:c6d8d2e89cc76cb0a85f21a53647da7b117beff6#514468b2b707d4e7dd632a4037323bed - Cache

The rrev, package id and prev matches what I have built and uploaded. But it has "None" at the end of the line?
It then finishes with a trace:

ERROR: Traceback (most recent call last):
File "/lib/python3.10/site-packages/conan/cli/cli.py", line 270, in main
cli.run(args)
File "/lib/python3.10/site-packages/conan/cli/cli.py", line 180, in run
command.run(self._conan_api, args[0][1:])
File "/lib/python3.10/site-packages/conan/cli/command.py", line 126, in run
info = self._method(conan_api, parser, *args)
File "/lib/python3.10/site-packages/conan/cli/commands/install.py", line 73, in install
conan_api.install.install_binaries(deps_graph=deps_graph, remotes=remotes)
File "/lib/python3.10/site-packages/conan/api/subapi/install.py", line 21, in install_binaries
installer.install(deps_graph, remotes)
File "/lib/python3.10/site-packages/conans/client/installer.py", line 258, in install
self._handle_package(package, install_reference, handled_count, package_count)
File "/lib/python3.10/site-packages/conans/client/installer.py", line 304, in _handle_package
assert package.binary in (BINARY_CACHE, BINARY_BUILD, BINARY_DOWNLOAD, BINARY_UPDATE)
AssertionError

Any clue about what's going on?

Regards

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @klausholstjacobsen

The --lockfile-packages is an "undocumented" feature, and not available for production. It will likely be removed soon.
It appears only in the embedded conan xxx -h help, but it is not documented in the relevant usage sections of lockfiles in the docs:

• https://docs.conan.io/2/tutorial/consuming_packages/intro_to_versioning.html
• https://docs.conan.io/2/tutorial/versioning/lockfiles.html
  And there is no intention to document it at the moment.

It seems that we should try to hide it from the argparse that generates the -h or at least put some clarification on code for that automatically generated help.

The reason is that locking packages shouldn't be really necessary ever. Locking the recipe-revisions, that identify all the "source" origins, including the versions and the hashes of the input source code should be more than enough to achieve full reproducibility. Note that:

• locking the package-ids is unnecessary, they are a function of the source + profile inputs.
• Having more than 1 package-revision is considered in Conan 2.0 a model or process error. No package-id should have more than 1 package-revision, and while Conan will resolve to latest package-revision, it will not take it into account for example to compute new package-ids of the consumers.

I will do a PR to hide the argument from the CLI help

[klausholstjacobsen]

@memsharded your comment makes perfect sense. We just liked the added confidence in point-pointing dependencies exactly down to the prev. This way If I, in 6 months time, checkout a git commit from today I can be absolutely certain what dependencies gets pulled and linked.

Can we somehow enforce single prev's in our remote? I know this indicates flaws in our process, but we all know that developers can be a difficult herd to manage. :-)

Regards
Klaus

[memsharded]

Can we somehow enforce single prev's in our remote? I know this indicates flaws in our process, but we all know that developers can be a difficult herd to manage. :-)

From https://docs.conan.io/2/knowledge/guidelines.html

Developers should not be able to upload to “development” and “production” repositories in the server. Only CI builds have write permissions in the server. Developers should only have read permissions and at most to some “playground” repositories used to work and share things with colleagues, but which packages are never used, moved or copied to the development or production repositories.

This is not something Conan specific, but a general best practice in devops.

It is not possible to protect in the server side against multiple package-revisions. The package-revisions are still a potentially useful feature against system failures, that allows to recover and move forward under some exceptional circumstances, this is why they haven't been fully removed from Conan and server.

If you want to protect against this, there would be several approaches:

• Make sure that only --build=missing is used, not other --build policies that could re-build already existing binaries
• Use the conan list pkg/version:*#* --format=json to retrieve package revisions and check the json to validate only 1 package revision there
  • You can automate that with a Conan custom command, and integrate the check easily in CI, for example

[memsharded]

This was automatically closed by #15499, please feel free to re-open or submit a new ticket to continue the conversation if you have any further question. Thanks!