[question] Meaning of "skip: True" in "conan graph" output

[IvanRibakov]

What is your question?

I'm using Conan v2.0.17 graph command to load dependency information into external SCA tool. I've been using the -c tools.build:download_source=True argument to force downloading of all sources, which I then pass to SCA tool to do it's job.

However, after (I presume) a recent update of one of the direct dependencies above process broke down because one of the transitive dependency's sources are no longer being downloaded.

Here's the command(s) I've been running and the outputs I'm getting (pay attention to the jinja2cpp (direct dep) and rapidjson (transitive)):

$ conan graph info -pr:b src/conan-el9.profile -pr:h src/conan-el9.profile src -c tools.build:download_source=True

======== Computing dependency graph ========
Graph root
    conanfile.py (zroute/None): /data/work/git/myapp/src/conanfile.py
Requirements
    boost/1.83.0#658a6b1d7aa62a019cbba3f4ae363cb7 - Cache
    c-ares/1.22.1#d030cc7e14b2ebd4d351446ae158488a - Cache
    cctz/2.3#567ed7e4ca84e5380d816228c996e41d - Cache
    cpr/1.9.3#9f5c70878c93ce833074e9d31ff0dd96 - Cache
    expected-lite/0.6.3#262026767cf9d5ab1fdccb25c9bdbac2 - Cache
    fmt/10.2.0#43aafa5364c2359fc45be77d2587ab15 - Cache
    jinja2cpp/1.2.1#643e50d77dfadb2102ae27b9bc3ef60c - Cache
    json-schema-validator/2.1.0#105bcac54ce761c10bce5a1779359e51 - Cache
    libcurl/8.5.0#4fc73f2d68bae7c25d11a8b21d52233d - Cache
    nlohmann_json/3.11.2#1ded6ae5d200a68ac17c51d528b945e2 - Cache
    oatpp/1.3.0#845d8538ed10c5a2d742a87056150e95 - Cache
    openssl/3.2.0#d81e0a8c08372368fe7cb757280e2196 - Cache
    optional-lite/3.5.0#c5c705364f2036db5a3271c9e87e9ab9 - Cache
    rapidjson/cci.20220822#1b9d8c2256876a154172dc5cfbe447c6 - Cache
    string-view-lite/1.7.0#b58c5ddb877597aea1cdf09a9e4f2656 - Cache
    tree-sitter/0.17.3#a6ad43e0c312244858a8af2e9766a5c3 - Cache
    variant-lite/2.0.0#780ff2755d9b871f363162878f2f2e6d - Cache
    xerces-c/3.2.3#5ccde60faff0750c2830568c5526bfe9 - Cache
    zlib/1.2.13#97d5730b529b4224045fe7090592d4c1 - Cache
Test requirements
    catch2/2.13.10#d8541ec3a7e1236ecc46a5952896969e - Cache
    trompeloeil/43#1e21eef8dbb3a6710bcbdad6409c0f05 - Cache
Build requirements
    autoconf/2.71#53be95d228b2dcb30dc199cb84262d8f - Cache
    automake/1.16.5#058bda3e21c36c9aa8425daf3c1faf50 - Cache
    b2/4.10.1#1b290403d8648c79f468f5a6496f829a - Cache
    gnu-config/cci.20210814#dc430d754f465e8c74463019672fb97b - Cache
    libtool/2.4.7#08316dad5c72c541ed21e039e4cf217b - Cache
    m4/1.4.19#b38ced39a01e31fef5435bc634461fd2 - Cache
    meson/1.2.2#aace9dcc1db58fa42ecb5292f724092d - Cache
    ninja/1.11.1#77587f8c8318662ac8e5a7867eb4be21 - Cache
    pkgconf/2.1.0#27f44583701117b571307cf5b5fe5605 - Cache
Resolved version ranges
    libcurl/[>=7.78.0 <9]: libcurl/8.5.0
    openssl/[>=1.1 <4]: openssl/3.2.0
Overrides
    nlohmann_json/3.11.2#1ded6ae5d200a68ac17c51d528b945e2: ['nlohmann_json/3.11.2#1ded6ae5d200a68ac17c51d528b945e2', None]
    nlohmann_json/3.10.5: ['nlohmann_json/3.11.2']

======== Computing necessary packages ========
...
jinja2cpp/1.2.1: Checking 7 compatible configurations
jinja2cpp/1.2.1: Compatible configurations not found in cache, checking servers
jinja2cpp/1.2.1: '55b54255c7f44b0216d6a848fd5a9438d0e636a0': compiler.cppstd=14
jinja2cpp/1.2.1: 'c4f9d70bf105fea2232707734396836202680a33': compiler.cppstd=gnu14
jinja2cpp/1.2.1: '62e3da0592c8a5c2bee3b1b6d5d573dc994d4cbe': compiler.cppstd=17
jinja2cpp/1.2.1: 'f5b1570789848ea796321bc2a4fdc9d09de5cf52': compiler.cppstd=20
jinja2cpp/1.2.1: '5b568eff1a348163f177098fca54614369881f75': compiler.cppstd=gnu20
jinja2cpp/1.2.1: '2518dc757376d921dca1114c1f811cfc5c5b5cd1': compiler.cppstd=23
jinja2cpp/1.2.1: '6fd06c0eb00e0ea8b3fa9275582d7468114e7a86': compiler.cppstd=gnu23
...
Requirements
    boost/1.83.0#658a6b1d7aa62a019cbba3f4ae363cb7:e000b98e46184bbda2ac5c1ec704c5ee13f39906 - Missing
    c-ares/1.22.1#d030cc7e14b2ebd4d351446ae158488a:e3c52a2f5bd5dc2a32df2d25f9f70a04cfd7bea3#adc9883e02a821bc501f765c0ba17388 - Download (conancenter)
    cctz/2.3#567ed7e4ca84e5380d816228c996e41d:53939a69492e42ec7a5c29c2c5dee0ee9a58fe05#79c81c823fbd2b9325b0ccfd475eb0c3 - Download (conancenter)
    cpr/1.9.3#9f5c70878c93ce833074e9d31ff0dd96:f589dccdc84b6d257a402da892773367d2bf99f3 - Missing
    expected-lite/0.6.3#262026767cf9d5ab1fdccb25c9bdbac2:da39a3ee5e6b4b0d3255bfef95601890afd80709#beb7503b67488b722e231f67855dbbf6 - Download (conancenter)
    fmt/10.2.0#43aafa5364c2359fc45be77d2587ab15:ab63c9a7d02c6c3c8452b305f3a53c2446c7322d#d02ceed90a12df85e5eed5dae4019904 - Download (conancenter)
    jinja2cpp/1.2.1#643e50d77dfadb2102ae27b9bc3ef60c:0a20ff0a0fe284b5f2877e666f334f3611bfe68e - Missing
    json-schema-validator/2.1.0#105bcac54ce761c10bce5a1779359e51:9cba398f441678f096be93941431e0b6c4a5f0a4 - Missing
    libcurl/8.5.0#4fc73f2d68bae7c25d11a8b21d52233d:ff3b6bf68ed1ab590223bd337df09d9022c1b694 - Missing
    nlohmann_json/3.11.2#1ded6ae5d200a68ac17c51d528b945e2:da39a3ee5e6b4b0d3255bfef95601890afd80709#2d1a5b1f5d673e1dab536bed20ce000b - Download (conancenter)
    oatpp/1.3.0#845d8538ed10c5a2d742a87056150e95:13be611585c95453f1cbbd053cea04b3e64470ca#ef61ae99aaa4072f45e5ff01f67389c3 - Download (conancenter)
    openssl/3.2.0#d81e0a8c08372368fe7cb757280e2196:517a2f66278c9fbeb4bb0b64fc531fe112929bc9 - Missing
    optional-lite/3.5.0#c5c705364f2036db5a3271c9e87e9ab9:da39a3ee5e6b4b0d3255bfef95601890afd80709#caf9c39526bde4f9b7a724eed9125966 - Download (conancenter)
    string-view-lite/1.7.0#b58c5ddb877597aea1cdf09a9e4f2656:da39a3ee5e6b4b0d3255bfef95601890afd80709#fbd15e2b6dc28e7d39cca01a8a299135 - Download (conancenter)
    tree-sitter/0.17.3#a6ad43e0c312244858a8af2e9766a5c3:b647c43bfefae3f830561ca202b6cfd935b56205#5d3aa51cf09f937941a5549f3b2382ae - Download (conancenter)
    variant-lite/2.0.0#780ff2755d9b871f363162878f2f2e6d:da39a3ee5e6b4b0d3255bfef95601890afd80709#92a9db7116ed47cbcc5951285ddcd4ab - Download (conancenter)
    xerces-c/3.2.3#5ccde60faff0750c2830568c5526bfe9:ca6cd8782efa45a0b0eb714f69f327b26c26a578#ab42fcb4db53f38f9528180f63104969 - Download (conancenter)
    zlib/1.2.13#97d5730b529b4224045fe7090592d4c1:b647c43bfefae3f830561ca202b6cfd935b56205#193bca86926abaf79971c774228e6249 - Download (conancenter)
Test requirements
    catch2/2.13.10#d8541ec3a7e1236ecc46a5952896969e:da39a3ee5e6b4b0d3255bfef95601890afd80709#d747b95b8e7ed466e354429ffd7f048f - Download (conancenter)
    trompeloeil/43#1e21eef8dbb3a6710bcbdad6409c0f05:da39a3ee5e6b4b0d3255bfef95601890afd80709#389432b631b3ca92bc47c1d670fbab3f - Download (conancenter)
Build requirements
Skipped binaries
    rapidjson/cci.20220822, autoconf/2.71, automake/1.16.5, b2/4.10.1, gnu-config/cci.20210814, libtool/2.4.7, m4/1.4.19, meson/1.2.2, ninja/1.11.1, pkgconf/2.1.0

And now the actual graph information:

======== Basic graph information ========
zroute/None:
  ref: myapp/None
  id: 0
  recipe: Consumer
  package_id: 9af1aba8641972472157f339a7c4190fa190cb5f
  prev: None
  rrev: None
  rrev_timestamp: None
  prev_timestamp: None
  remote: None
  binary_remote: None
  build_id: None
  binary: None
  invalid_build: False
  info_invalid: None
  name: zroute
  user: None
  channel: None
  url: https://github.com/myapp
  license: Copyright (c) 2018-2025 Myapp
  author: None
  description: Myapp
  homepage: None
  build_policy: None
  upload_policy: None
  revision_mode: hash
  provides: None
  deprecated: None
  win_bash: None
  win_bash_run: None
  default_options:
    shared: False
    fPIC: True
    libcurl/*:with_c_ares: True
    boost/*:pch: False
    boost/*:without_filesystem: False
    boost/*:without_atomic: False
    boost/*:without_system: False
    boost/*:without_chrono: True
    boost/*:without_container: True
    boost/*:without_context: True
    boost/*:without_contract: True
    boost/*:without_coroutine: True
    boost/*:without_exception: True
    boost/*:without_fiber: True
    boost/*:without_graph: True
    boost/*:without_graph_parallel: True
    boost/*:without_iostreams: True
    boost/*:without_json: True
    boost/*:without_locale: True
    boost/*:without_log: True
    boost/*:without_math: True
    boost/*:without_mpi: True
    boost/*:without_nowide: True
    boost/*:without_program_options: True
    boost/*:without_python: True
    boost/*:without_random: True
    boost/*:without_regex: True
    boost/*:without_serialization: True
    boost/*:without_stacktrace: True
    boost/*:without_test: True
    boost/*:without_thread: True
    boost/*:without_timer: True
    boost/*:without_type_erasure: True
    boost/*:without_wave: True
  options_description: None
  version: None
  topics: None
  package_type: static-library
  settings:
    os: Linux
    arch: x86_64
    compiler: gcc
    compiler.cppstd: gnu17
    compiler.libcxx: libstdc++11
    compiler.version: 11
    build_type: Release
  options:
    fPIC: True
    shared: False
  options_definitions:
    shared: ['True', 'False']
    fPIC: ['True', 'False']
  generators: []
  python_requires: None
  system_requires:
  recipe_folder: /data/work/git/myapp/src
  source_folder: None
  build_folder: None
  generators_folder: None
  package_folder: None
  cpp_info:
    root:
      includedirs: ['include']
      srcdirs: None
      libdirs: ['lib']
      resdirs: None
      bindirs: ['bin']
      builddirs: None
      frameworkdirs: None
      system_libs: None
      frameworks: None
      libs: None
      defines: None
      cflags: None
      cxxflags: None
      sharedlinkflags: None
      exelinkflags: None
      objects: None
      sysroot: None
      requires: None
      properties: None
  conf_info:
  label: conanfile.py (myapp/None)
  dependencies:
    ...
    30:
      ref: jinja2cpp/1.2.1
      run: False
      libs: True
      skip: False
      test: False
      force: False
      direct: True
      build: False
      transitive_headers: None
      transitive_libs: None
      headers: True
      package_id_mode: minor_mode
      visible: True
    ...
    33:
      ref: rapidjson/cci.20220822
      run: False
      libs: False
      skip: True
      test: False
      force: False
      direct: False
      build: False
      transitive_headers: None
      transitive_libs: None
      headers: False
      package_id_mode: None
      visible: True
    ...
  context: host
  test: False
jinja2cpp/1.2.1#643e50d77dfadb2102ae27b9bc3ef60c:
  ref: jinja2cpp/1.2.1#643e50d77dfadb2102ae27b9bc3ef60c
  id: 30
  recipe: Cache
  package_id: 0a20ff0a0fe284b5f2877e666f334f3611bfe68e
  prev: None
  rrev: 643e50d77dfadb2102ae27b9bc3ef60c
  rrev_timestamp: 1704192585.914
  prev_timestamp: None
  remote: None
  binary_remote: None
  build_id: None
  binary: Missing
  invalid_build: False
  info_invalid: None
  name: jinja2cpp
  user: None
  channel: None
  url: https://github.com/conan-io/conan-center-index
  license: MIT
  author: None
  description: Jinja2 C++ (and for C++) almost full-conformance template engine implementation
  homepage: https://jinja2cpp.dev/
  build_policy: None
  upload_policy: None
  revision_mode: hash
  provides: None
  deprecated: None
  win_bash: None
  win_bash_run: None
  default_options:
    shared: False
    fPIC: True
  options_description: None
  version: 1.2.1
  topics: ['cpp14', 'cpp17', 'jinja2', 'string templates', 'templates engine']
  package_type: static-library
  settings:
    os: Linux
    arch: x86_64
    compiler: gcc
    compiler.cppstd: gnu17
    compiler.libcxx: libstdc++11
    compiler.version: 11
    build_type: Release
  options:
    fPIC: True
    shared: False
  options_definitions:
    shared: ['True', 'False']
    fPIC: ['True', 'False']
  generators: []
  python_requires: None
  system_requires:
  recipe_folder: /home/devenv/.conan2/p/jinjacf87cca18d2bc/e
  source_folder: /home/devenv/.conan2/p/jinjacf87cca18d2bc/s/src
  build_folder: None
  generators_folder: None
  package_folder: None
  cpp_info:
    root:
      includedirs: ['include']
      srcdirs: None
      libdirs: ['lib']
      resdirs: None
      bindirs: ['bin']
      builddirs: None
      frameworkdirs: None
      system_libs: None
      frameworks: None
      libs: None
      defines: None
      cflags: None
      cxxflags: None
      sharedlinkflags: None
      exelinkflags: None
      objects: None
      sysroot: None
      requires: None
      properties: None
  conf_info:
  label: jinja2cpp/1.2.1
  dependencies:
    ...
    33:
      ref: rapidjson/cci.20220822
      run: False
      libs: False
      skip: False
      test: False
      force: False
      direct: True
      build: False
      transitive_headers: None
      transitive_libs: None
      headers: True
      package_id_mode: minor_mode
      visible: True
    ...
  context: host
  test: False
rapidjson/cci.20220822#1b9d8c2256876a154172dc5cfbe447c6:
  ref: rapidjson/cci.20220822#1b9d8c2256876a154172dc5cfbe447c6
  id: 33
  recipe: Cache
  package_id: da39a3ee5e6b4b0d3255bfef95601890afd80709
  prev: a99fd0306f219909b3c1b1059abc47e9
  rrev: 1b9d8c2256876a154172dc5cfbe447c6
  rrev_timestamp: 1705070244.747
  prev_timestamp: 1705070466.44
  remote: None
  binary_remote: conancenter
  build_id: None
  binary: Skip
  invalid_build: False
  info_invalid: None
  name: rapidjson
  user: None
  channel: None
  url: https://github.com/conan-io/conan-center-index
  license: MIT
  author: None
  description: A fast JSON parser/generator for C++ with both SAX/DOM style API
  homepage: http://rapidjson.org
  build_policy: None
  upload_policy: None
  revision_mode: hash
  provides: None
  deprecated: None
  win_bash: None
  win_bash_run: None
  default_options: None
  options_description: None
  version: cci.20220822
  topics: ['rapidjson', 'json', 'parser', 'generator']
  package_type: header-library
  settings:
    os: Linux
    arch: x86_64
    compiler: gcc
    compiler.cppstd: gnu17
    compiler.libcxx: libstdc++11
    compiler.version: 11
    build_type: Release
  options:
  options_definitions:
  generators: []
  python_requires: None
  system_requires:
  recipe_folder: /home/devenv/.conan2/p/rapid67f21ae5216f9/e
  source_folder: None
  build_folder: None
  generators_folder: None
  package_folder: None
  cpp_info:
    root:
      includedirs: ['include']
      srcdirs: None
      libdirs: ['lib']
      resdirs: None
      bindirs: ['bin']
      builddirs: None
      frameworkdirs: None
      system_libs: None
      frameworks: None
      libs: None
      defines: None
      cflags: None
      cxxflags: None
      sharedlinkflags: None
      exelinkflags: None
      objects: None
      sysroot: None
      requires: None
      properties: None
  conf_info:
  label: rapidjson/cci.20220822
  dependencies:
  context: host
  test: False

The only thing that stands out to me is the skip: True in the global list of dependencies and source_folder: None. Can someone please explain to me what is actually happening here? Is rapidjson actually a transitive dependency or not? Why it's sources are not being downloaded unlike those of the rest of the dependencies?

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @IvanRibakov

Thanks for your question.

When Conan knows that some package is actually not necessary for the consumption, it is able to skip the download of the binaries, and the evaluation of the package_info(), as these will not be providing any information to consumers. This can happen in different scenarios, like unneeded tool_requires (because the binaries already exist, so not necessary tools to build from source), or when libraries are hidden behind other libraries, like a static library being linked and fully embedded as an implementation detail by a shared library, or a header-only library being an implementation detail of a static library.

You can see in Skipped binaries output how in this case most of the tool_requires are skipped, and also the rapidjson library.

The tools.graph:skip_binaries configuration can disable this, and force the retrieval of the binary, even if it wouldn't be necessary, but of course it will be slower, as it might need to download and unzip more package binaries.

[IvanRibakov]

I just found something very strange: adding --build=\"*\"works the first time (I get the source path), but doesn't work on subsequent runs. When I say first run I mean first run after purging Conan cache.

[memsharded]

That might be something to check indeed, I'll try to have a look

[IvanRibakov]

@memsharded thanks for the earlier explanation. I suspected that it was something along those lines. Unfortunately, it still leaves me wondering about my specific case: jinja2cpp and rapidjson. Why is rapidjson sometimes marked for skipping when it appears in the recipe as dependency and has some occurrences in the code?

Could it have something to do with rapidjson being header-only library? Or with recipe missing transitive_headers=True?

[IvanRibakov]

Is there any way to "interrogate" Conan about the reasons it has marked dependency as "skipped"?

[memsharded]

Yes, if the jinja public headers do #includes to rapidjson headers, then they should be marked as transitive_headers=True. But that would have been typically caught by CI, maybe it is just for some specific jinja headers that are not covered in the test_package?

Is there any way to "interrogate" Conan about the reasons it has marked dependency as "skipped"?

Not directly, just analyzing the graph info result, with the requires traits, but that is not an easy task.

[AbrilRBS]

Hi, sorry that we missed this issue - in Conan 2.0.10, we introduced the tools.graph:skip_binaries conf in #14466 that allows users to opt-out of binnary skipping in the Conan graph, which will force everything to be downloaded/compiled, as to avoid issues like the one you faced. Let us know if this works/feel free to open a new issue if needed, closing as solved for now :)