[question] How to see if package is taken from Conan download cache or downloaded from Artifactory?

[MTomBosch]

In order to better understand the download traffic from our Artifactory instance towards our CI/CD build machines that are supposed to use the Conan download cache I would need to know if there is a way to configure Conan to log to console (or log file) when a package is not downloaded from Artifactory but taken from the download cache.

I was going through the documentation and even playing around with some cli args but could not find any way to achieve this.

Do you have any ideas how to get to know this information without attaching some network sniffing tools to the CI/CD machines?

• I've read the CONTRIBUTING guide.

[lasote]

You can use CONAN_TRACE_FILE environment variable. Traces will be recorded to a file like: https://docs.conan.io/en/latest/mastering/logging.html#logging-and-debugging
This is only Conan 1.X. We don't have something similar in develop2 yet.

[MTomBosch]

OK. Thank you. Will try that out. Is there a special term that I can search for to know whether data was downloaded to the package cache or the download cache?

[TheMarTom]

Coming back to this topic. When I run an install then I can see some hints on the console whether the package is in the cache or will be/has been downloaded. See below example. For the package astree-profiles it once says "Donloaded" and then "Download". What does this mean?

Build requirements
astree/22.10.1@ci/tools from '...-conan-all-virtual' - Cache
astree-profiles/0.1.0@ci/stable from '...-conan-all-virtual' - Downloaded
Build requirements packages
astree/22.10.1@vrte-ci/tools:cb054d0b3e1ca595dc66bc2339d40f1f8f04ab31 - Cache
astree-profiles/0.1.0@vrte-ci/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Download

Thank you

[AbrilRBS]

Hi @TheMarTom thanks for your question (Note that usually it's easier to get answers for your questions by opening new issues, it's pretty lucky I saw this one!)

The first Download refers to the recipe itself being downloaded to compute the dependency graph (In opposition to it being already in Cache or being an Editable)
The second Download refers to the package itself, where it signifies that the package that matches your requested configuration (os, compiler, arch, build type etc) is being downloaded from your server, where it has been pre-built, instead of being already in the Cache, going to be "Build" locally, Missing when the binary is not found and you didn't tell Conan to build it, and Skip when Conan knows it won't need to actually build it :)

Hope this helps a little, feel free to open new issues if you have any more questions!

[TheMarTom]

@RubenRBS Thanks a lot for the explanation. Is the output somehow different when a file is copied from the download cache to the package cache?

[MTomBosch]

I made a detailed analysis with 1.60.1. And here are the results:

It is possible to figure out from the console log/conan trace file whether a package (recipe and package files) was taken
from the download cache instead of Artifactory. However for some scenarios the absence of certain entries in the console log/
trace file must be checked before a decision can be taken.

In short for checking if a package was taken from the download cache and then copied over to the package cache since the
package cache is not having the most current recipe/package files you have check if:

• The recipe or package is marked as Download/Downloaded when looking at the console log or DOWNLOADED_RECIPE/DOWNLOADED_PACKAGE when looking at the Conan trace file

  and

• there is no "DEBUG :file_downloader.py"/"ACTION DOWNLOAD" visible in the console log/trace file for the related files
  of the particular package in question.