[feature] conan list merge command

[DoDoENT]

What is your suggestion?

This is akin to conan lock merge.

This is useful when you create a package and immediately build multiple binaries for it, with different configurations (e.g. debug, release, various options) and you want to upload all binaries in a single conan upload call so that you benefit from Conan's parallel upload feature.

The flow would be something like this (avoiding multiple conan create calls because each call creates a different revision for some reason).

conan export . --name pkg --version 1.0

conan install --requires pkg/1.0 -s build_type=Debug --format=json > graph-debug.json
conan install --requires pkg/1.0 -s build_type=Release --format=json > graph-release.json

conan list --graph=graph-debug.json --graph-binaries=build --format=json > built-debug.json
conan list --graph=graph-release.json --graph-binaries=build --format=json > built-release.json

conan list merge --list=built-debug.json --list=built-release.json --list-out=built.json

conan upload -r myremote -c --list=built.json

Of course, this can be implemented as a custom command (but not as conan list merge because list command is already reserved). However, I think that having this command by default would be a good value.

[AbrilRBS]

Hi @DoDoENT thanks a lot for your request. While this is something that we think has some value being added, we need to think the UX/implementation of this one, as internally it's not straightforward to create subcommands for conan list.

In the meantime and for anyone else finding this problem, if you are using jq (A really useful json manipulation tool), this is a built-in functionality of it: (A bit advanced with a peculiar syntax and not as easy as what conan will provide, but can help for now)

$ conan list "zlib/*:*#*" -p "build_type=Debug" --format=json > graph-debug.json

$ conan list "zlib/*:*#*" -p "build_type=Release" --format=json > graph-release.json

$ jq -s '.[0] * .[1]' graph-release.json graph-debug.json 
{
  "Local Cache": {
    "zlib/1.2.13": {
      "revisions": {
        "e377bee636333ae348d51ca90874e353": {
          "timestamp": 1682597484.674,
          "packages": {
            "76f7d863f21b130b4e6527af3b1d430f7f8edbea": {
              "revisions": {
                "75c9bf351695f8ce2f6b17458bd507ed": {
                  "timestamp": 1686666077.41796
                }
              },
              "info": {
                "settings": {
                  "arch": "armv8",
                  "build_type": "Release",
                  "compiler": "apple-clang",
                  "compiler.version": "14",
                  "os": "Macos"
                },
                "options": {
                  "fPIC": "True",
                  "shared": "False"
                }
              }
            },
            "8359538fee5b146800be789e3d3f504010b3bc01": {
              "revisions": {
                "dba9c2b0bd86a13f5770b91195e73888": {
                  "timestamp": 1686672985.748242
                }
              },
              "info": {
                "settings": {
                  "arch": "armv8",
                  "build_type": "Debug",
                  "compiler": "apple-clang",
                  "compiler.version": "14",
                  "os": "Macos"
                },
                "options": {
                  "fPIC": "True",
                  "shared": "False"
                }
              }
            }
          }
        }
      }
    }
  }
}

[DoDoENT]

Great idea for jq! Never thought of it, even though I used it before for other stuff.

In the meantime, I've already created a custom command merge-lists that does the merging of MultiPackageList objects.

UX-wise, since subcommands for list are complicated, maybe

conan list merge --list=built-debug.json --list=built-release.json --list-out=built.json

could be something like

conan list --merge=built-debug.json --merge=built-release.json --format=json > built.json

I don't know if it's too confusing or not. It does look a bit unintuitive.

[memsharded]

We are going with the conan pkglist merge command for Conan 2.3