Multi-crate workspace projects support

[MOZGIII]

I'm trying to use cargo-wix in a multi-crate workspace, and it doesn't work.

> cargo wix --nocapture
Error[4] (Manifest): No 'name' field found in the package's manifest (Cargo.toml)

The error is on point, since the Cargo.toml manifest only contains the workspace definition, and doesn't have a particular crate name.

[MOZGIII]

I ended up with cargo wix -n myapp --install-version 0.0.1 to get it working. It's great this crate is built with that kind of flexiubility in mind, kudos for that! I'd still love to see proper suppor for multi-workspace configs.

My layout is the following:

/core
  /core/Cargo.toml
  /core/src/lib.rs
/binary1
  /binary1/Cargo.toml
  /binary1/src/main.rs
/binary2
  /binary2/Cargo.toml
  /binary2/src/main.rs
/wix
  /win/main.wxs

My end goal is to build a single installer containing both binary1 and binary2. I think I can acheive it now if I pass the name and install-version flags and customize the .wsx.
I'd like to have my install-version value kept somewhere in the repo. This is the main problem for me so far.
Further improvement for this feature can be the better init process for multi-crate projects, but for me it only took about a minute to figure out how to set it up after the wix init withg a single crate, so it might not worth the effort.

[MOZGIII]

Looks like #20 may help if it's implemented in a way that doesn't depend on Cargo.toml having a package field (which root Cargo.tomls in multi-create workspaces can't have).

[volks73]

Supporting multi-crate workspace project has never occurred to me! Thanks for the feature request and making me aware.

I ended up with cargo wix -n myapp --install-version 0.0.1 to get it working. It's great this crate is built with that kind of flexiubility in mind, kudos for that!

It makes me happy to hear that the focus on flexibility for this sub-command to capture as many use-cases as possible has worked out, even for a use case I completely missed.

My end goal is to build a single installer containing both binary1 and binary2. I think I can achieve it now if I pass the name and install-version flags and customize the .wsx.

This should be achievable with editing the .wxs file. I would actually recommend editing the WXS file instead of using the -n,--name option because the display names, executable names, product names, etc. may be different depending on the invocation of the cargo wix init or cargo wix print subcommands used to create the initial WXS file. Editing the WXS file will also avoid having to remember to use the -n,--name option every time (I acknowledge #20 would resolve this as well. Looks like I need to get going on that feature).

I'd like to have my install-version value kept somewhere in the repo. This is the main problem for me so far.

Hmm, without #20, this does look to be an issue. Maybe if you are using cargo and this subcommand within a larger build script (like a .bat file), you could keep the install version (which is really the workspace version) there?

I'd still love to see proper support for multi-workspace configs.

Further improvement for this feature can be the better init process for multi-crate projects, but for me it only took about a minute to figure out how to set it up after the wix init withg a single crate, so it might not worth the effort.

I would like to add better support for workspace projects to this subcommand. My initial concern is having to loop through all members in the workspace, determine which members are binaries as opposed to libraries, and then add each binary to the components and feature list for the installer within the WXS file, which would requiring editing the built-in template. This is all relatively straight-forward programming and would be super slick, but how often would this feature be needed? As you mention, it only took a minute or two to modify the WXS file to your specific needs. What is the majority work-flow, desired installer for workspace-based projects?

This is my initial thought on the installation layout for a workspace-based project:

C:\Program Files\MyWorkspaceProject\
  bin
    Binary1.exe
    Binary2.exe
  License.rtf

Looks like #20 may help if it's implemented in a way that doesn't depend on Cargo.toml having a package field (which root Cargo.tomls in multi-create workspaces can't have).

My initial thought on this: #20 would not depend on the existence of the package section. In fact, it would override any value in the package section.

Interestingly, I just found this information in the Cargo book:

In workspace manifests, if the package table is present, the workspace root crate will be treated as a normal package, as well as a workspace. If the package table is not present in a workspace manifest, it is called a virtual manifest.

This seems to indicate that the root Cargo.toml that defines the members of the workspace can also have a package section. If the workspace manifest can have a package section, then all of the fields for a package should be available, including the version (for install version).

I created a temporary workspace project to play with this a little. The following project layout was used:

Workspace
  core
    src
      lib.rs
    Cargo.toml
  binary1
    src
      main.rs
    Cargo.toml
  binary2
    src
      main.rs
    Cargo.toml
  Cargo.toml

The contents of the workspace Cargo.toml are as follows:

[workspace]
members = [
  "binary1",
  "binary2",
  "core"
]

[package]
name = "my_workspace_project"
authors = ["Workspace Author <author@example.com>"]
version = "0.0.1"

[lib]
path = "core/src/lib.rs"

A [lib] or [[bin]] section is needed for the package section. Since the project has multiple binaries, I used the core library and the [lib] section. This required a path field; otherwise, I could not build the project/workspace with cargo. This is more-or-less a placeholder and to workaround the requirement for Cargo.

Then I did a cargo wix init command from the root project directory, which would use the workspace Cargo.toml file that contained both the workspace and package sections. The expected warning messages appeared, but a wix\main.wxs was created. Next, I tried the cargo wix command and received the following error:

The 'light' application failed with exit code = 103. Consider using the '--nocapture' flag to obtain more information.

The linker (light.exe) could not find the target\release\my_workspace_project.exe file, which is expected because that does not exist. I modified the wix\main.wxs file to include the two binaries instead of the my_worspace_project.exe file and tried the cargo wix default command again. It worked! A my_workspace_project-0.0.1-x86_64.msi was created in the target\wix folder.

I ran the installer. The following installation layout was created:

C:\Program Files\my_workspace_project\bin
  binary1.exe
  binary2.exe

and the PATH system environment variable contained the C:\Program Files\my_workspace_project\bin folder. The application appeared in the Program and Features control panel with the correct version number from the workspace Cargo.toml file. Uninstall appeared to work correctly as well.

In summary, I think you can add a package section to your workspace Cargo.toml file and make some small modifications to the generated wix\main.wxs to make it all work and maintain a workspace/install version without needing the -n,--name or -i,--install-version options, and no changes necessary to the cargo wix subcommand. I still think support for workspace projects can be improved.

[MOZGIII]

Nice! Here's my project, as an additional data for this issue: https://github.com/MOZGIII/rebootinto
And here's a sample build: https://ci.appveyor.com/project/MOZGIII/rebootinto/builds/24093596

I see the point about the virtual and non-virtual workspaces, though I'd like to have proper support for virtual manifests too - mainly because I don't like to have a package at the top level, because it doesn't have a purpose other that holding the installer data.

That said, I'll try adding the package to the toplevel Cargo.lock - maybe it will just work fine for me. I want core and installer to have separate versions though - as core is updated the least, while the binaries are changing often, and the installer should be updated for every binary update.

I would also consider building multiple installers - one per each workspace project. In fact, I think it makes even more sense for my project than packaging everything in all-in-one installer...

[volks73]

Thank you for sharing your project. It will be useful as a template to implement better support for workspace projects.

I see the point about the virtual and non-virtual workspaces, though I'd like to have proper support for virtual manifests too - mainly because I don't like to have a package at the top level, because it doesn't have a purpose other that holding the installer data.

I agree with not liking the package section just be a placeholder. It is just a "hack", or workaround, to get this subcommand to work really. Ultimately, #20 would eliminate the need for the placeholder package section.

Interestingly, as an experiment, I changed the Cargo.toml from my test workspace project to the following:

[workspace]
members = [
  "binary1",
  "binary2",
  "core"
]

name = "my_workspace_project"
authors = ["Workspace Author <author@example.com>"]
version = "0.0.1"

I was able to build the workspace, but I received the following warnings:

warning: Cargo.toml: unused manifest key: workspace.authors
warning: Cargo.toml: unused manifest key: workspace.name
warning: Cargo.toml: unused manifest key: workspace.version

So, we can add other keys to the workspace section of the Cargo.toml. It is currently hard-coded to look for the package section in this subcommand, but I wanted to make a note this was possible as I begin to think about adding better support for workspace-based projects.

I would also consider building multiple installers - one per each workspace project. In fact, I think it makes even more sense for my project than packaging everything in all-in-one installer...

This all depends on how you envision your distribution of your project. A workspace is a grouping of libraries and binaries that have some build dependence on each other. It is supposed to make organizing and developing multiple, related project easier. I foresee two scenarios for installers from workspace projects:

1. Are you distributing a "suite" of applications, where each application is a binary from the workspace? If yes, then a single installer with each binary being a feature/component to install makes sense. A possible installation layout would be:

   C:\Program Files\MySuite\
     bin
       Binary1.exe
       Binary2.exe
     lib
       Library1.dll
       Library2.dll
       Library3.dll
     doc
       Suite-Manual.html 
     License.rtf
   

   and a single MySuite entry would appear in the Add/Remove Programs control panel.

2. If each binary is actually a separate application with no meaningful relationship or grouping between them for the end-user, then separate installers for each binary makes sense. In this scenario, the workspace is only meant for helping the developer organize and manage the codebase and has no mirror, or meaning, for the end-user to organize and manage the end product. A possible installation layout would be:

   C:\Program Files\MyOrganization\
      Binary1
        bin
          Binary1.exe
        lib
          Library1.dll
          Library3.dll
        doc
          Binary1-Manual.html
       License.rtf
      Binary2
        bin
          Binary2.exe
        lib
          Library2.dll
          Library3.dll 
        doc
          Binary2-Manual.html
        License.rtf
   

   and Binary1 and Binary2 would appear as separate entries in the Add/Remove Programs control panel.

There are, of course, a plethora of alternatives. For example, you could create individual installers for each binary, and then create an installer of installers. This is supported with WiX but not currently supported, nor do I expect it in the future, for this subcommand.

I will have to think about this some more for adding workspace project support. Some questions to think about:

1. Is it possible to accomplish these two scenarios with a single WXS template?
2. How would the developer indicate which scenario/layout to use?
3. Are there other, possibly more common, scenarios that should be supported?
4. How much of this needs to be incorporated into the subcommand versus left to post-initialization editing of the main.wxs file?

[Songtronix]

This missing feature currently really complicates my CI setup :/
Any progress being made?

[volks73]

I still do not have a clear picture of how to implement workspace support given the above discussion, questions, and variety of project layouts. After reviewing some discussion on workspaces for the cargo-deb project, I am not even sure any new enhancement/feature is even needed. cargo-deb will not be implementing one deb for multiple packages (projects within workspaces):

...
I probably won't be adding support for making one deb from multiple packages in one go, since the deb needs package metadata, and combining metadata from multiple packages complicates things too much.

However, they did add the -p option, which allows the user to select a package for creating a single deb. Thus, if you wanted to create separate debs for each package in the workspace, it would require multiple cargo deb -p <PACKAGE> calls. Adding a similar option/flag to the CLI might be the quickest and most broadly applicable to workspace-based projects.

I would like to callout and mention @notriddle's comment in the cargo-deb discussion on workspaces, where they are using both cargo-deb and cargo-wix for the same workspace project. I would be curious to hear more about their project organization and how that all works.

Would it be possible for you (@Songtronix) to elaborate a little more on your project layout, your expectation for support for workspaces, and why cargo-wix currently complicates your CI setup? I believe a combination of modifying the WXS file and the "recently" implemented and released #20 feature in v0.2 maybe enough to un-complicate your CI setup and generally support workspaces.

[Songtronix]

Well I could in theory switch from (Github actions):

- name: Build
  run: cargo build --bin airshipper --release     
- name: MSI Installer
  run: cargo wix --no-build --nocapture -n client --install-version ${{ steps.vars.outputs.version }}
- name: proper naming
  run: |
    ren target/wix/client-${{ steps.vars.outputs.version }}-x86_64.msi airshipper-${{ steps.vars.outputs.version }}-x86_64.msi
# Upload artifact
- name: Upload artifact
  uses: actions/upload-artifact@v1
  with:
    name: airshipper-windows-installer
    path: target/wix/airshipper-${{ steps.vars.outputs.version }}-x86_64.msi

To (with the -p option):

- name: MSI Installer
  run: cargo wix --nocapture -p client
# Upload artifact
- name: Upload artifact
  uses: actions/upload-artifact@v1
  with:
    name: airshipper-windows-installer
    path: target/wix/airshipper-${{ steps.vars.outputs.version }}-x86_64.msi

As you see, to avoid that the server gets compiled, I do a cargo build first then invoke cargo wix to generate the msi. However the msi will be named client instead of airshipper which would have been the case without the workspace.Therefore I rename it and finally upload it. Furthermore I need to pass the install version too which would be redudant if parsed from Cargo.toml.

[volks73]

Hmm, I looked at your airshipper project. Is it possible to move the wix folder from the root project directory to the client directory and then have the CI execute cargo wix from within the client directory or passing the path to the main.wxs file within the client package directory to cargo wix? I think this would at least eliminate the renaming step.

I am missing why you need to pass the install version. Is the client a different version than the workspace and server package?

[Songtronix]

I have to pass the version otherwise the command does not work at all:

Error[4] (Manifest): No 'version' field found in the package's manifest (Cargo.toml)

I've tested moving the wix folder into client and it looks like this works perfectly and removes the renaming bit and explicitly stating the version, thanks!

[volks73]

I've tested moving the wix folder into client and it looks like this works perfectly and removes the renaming bit and explicitly stating the version, thanks!

@Songtronix Great! I am glad this worked for you. So, I am going to assume that moving the wix folder into the client folder means that for your workspace layout, you do not actually need the -p option, or something similar.

This would indicate to me that there is already some workspace-related support in cargo-wix if each package within the workspace needs a separate installer. For example, if you wanted to provide an installer for your server package, you would create a separate wix sub-folder within the server sub-folder and have a different main.wxs file for it. The destination of the binaries for the client and server would be defined in their respective WXS files, which could be the same destination.

Really only workspace support is needed if you want a single installer for all of the packages in a workspace, but I think this already works as well because if you did not need to exclude the server package from the installer, I think everything would have worked. The [package.metadata.wix] section in the workspace/root Cargo.toml file could be used to adjust names, versions, etc. for the single installer. I need to test this and think about it some more. Creating some tests would be helpful.

[Songtronix]

One annoyance is that the target directory will be in client now which effectively removes my caching completely and now I need to symlink or such which is the same level of annoyance as renaming the msi file.
I could cache this directory specifically however this will consume up my available storage too fast.

EDIT: I now know what the issue was for why I was going the way I did all along:
If I cd into client and run cargo-wix it will invoke cargo and cargo will create the target directory at root of the workspace not in client. So I have to copy the entire target directory over for cargo-wix to be able to build the msi installer.

I'm really confused as to why it worked fine when I've tested it...

[volks73]

Would a combination of the cargo wix -o <output> <INPUT> option and positional argument eliminate this annoyance? For example, from the workspace root folder:

cargo wix -o target\wix\ client\wix\main.wxs

This would use the main.wxs file now located into the client package but the output would be same destination as before?

v0.2 --help documentation for the -o, --output:

Sets the destination file name and path for the created installer, or the destination folder for the installer with the default file name. If the path is to an existing folder or has a trailing slash (forward and backward), then the default installer file name will be used and the installer will be available in the folder after creation. Otherwise, this value overwrites the default file name and path for the installer. The default is to create an installer with the <product-name>-<version>-<arch>.msi file name in the target\wix folder.

Please note the requirement for a trailing forward/backward slash to output to a folder and keep the default naming, which includes the version number from the client's Cargo.toml.

Given this, a -p-like option is probably good to add/implement. The -p short name is also unused for the default/create subcommand. So, a -p, --package <PACKAGE> option can be added. I think I will move this to a separate issue/enhancement because it does not necessarily support the single installer for all packages, which maybe should be the default if the cargo wix subcommand is executed from the workspace root with the [workspaces] section present in the Cargo.toml file.

[Songtronix]

I've moved the wix folder but your suggestion does not work:

[volks73]

Ah, cargo-wix when called from the workspace root is looking at the virtual manifest, Cargo.toml, and not the package manifest, client\Cargo.toml. The virtual manifest does not have the name field. You could possibly add a package.metadata.wix section to the virtual manifest with a name field.

This data point indicates that the eventual -p option would need to override looking for the virtual manifest and instead look for the package manifest if/when executing from the workspace directory instead of the child package directory.

[volks73]

Workspace support might be supported with the new implementation that resolves #70 as of 4cf7cc0.

A path to the client's Cargo.toml file could be used and the installer should be created relative to the client's manifest as opposed to the virtual manifest. This should eliminate all of the workarounds (renaming) in combination with the defining a [package.metadata.wix] section for the client's Cargo.toml file.

I would be curious to know if the resolution of #70 does work for you (@Songtronix).

[Songtronix]

Running cargo wix client/Cargo.toml --nocapture with wix folder moved into client works as expected.
Very nice job!

[volks73]

@Songtronix, that is good hear! Given this result, do you believe a -p-like option as specified in the #94 enhancement is needed?

[Songtronix]

Hmm in my usecase it won't be needed. Furthermore can't think of any other usecase which couldn't be satisified with cargo wix <cargo.toml path> but with the -p option. Maybe it will make it easier to use if you're already familiar with cargo-deb.

However I've not tested one thing: Does in theory cargo wix client/Cargo.toml --nocapture only compile the client crate or will it compile the entire workspace?

[volks73]

Does in theory cargo wix client/Cargo.toml --nocapture only compile the client crate or will it compile the entire workspace?

It should only build the client package, not the entire workspace, but I also admit I have not fully tested this.

[roblabla]

Workspaces can now have metadata: rust-lang/cargo#8309. This is already supported by cargo-metadata too!

This is a really new feature, so we might want to wait for a stable cargo version to support it. Once it is supported, it removes the only real roadblock I had towards supporting this feature!

[CaoKha]

any update on this?
After doing this:

cargo wix init -p e2ee-cli
cargo wix -p e2ee-cli 

I still got error:

error LGHT0103 : The system cannot find the file 'wix\License.rtf'.