Windows deploy broken due to Cargo bug

[SonicZentropy]

Windows currently fails to deploy. The bug is related to this Cargo meta-issue. Cargo is still trying to determine how to fix them properly in general.

I forked Cargo, fixed our instance of the bug by canonicalizing the malformed path, and patched Shuttle with it here which I've tested working on Windows/WSL(Ubuntu) and the normal CI tests. The change may be small and innocuous enough that the Cargo folks would accept a PR?

[chesedo]

I wonder how cargo gets the absolute path without canonizalizing? Maybe following the same approach in shuttle will fix it for us 🤷‍♂️

[chrisheib]

Just following the discord bot example and running into the same problem:

discordbot on  master [?] is 📦 v0.1.0 via 🦀 v1.65.0-nightly 
❯ cargo shuttle run
    Building \\?\C:\Users\aaa\Downloads\discordbot
   Compiling hyper v0.14.20
   Compiling tokio-stream v0.1.9
   Compiling mime_guess v2.0.4
   Compiling log v0.4.17
   Compiling indexmap v1.9.1
   Compiling pear v0.2.3
   Compiling url v2.3.1
   Compiling time v0.3.14
   Compiling futures-executor v0.3.24
   Compiling uncased v0.9.7
   Compiling devise_codegen v0.3.1
   Compiling tokio-native-tls v0.3.0
   Compiling serde_json v1.0.85
   Compiling toml v0.5.9
   Compiling ubyte v0.10.3
error: couldn't read \\?\C:\Users\aaa\Downloads\discordbot\target\debug\build\mime_guess-46fc927e4a1f756a\out/mime_types_generated.rs: Die Syntax für den Dateinamen, Verzeichnisnamen oder die Datenträgerbezeichnung ist falsch. (os error 123)
 --> C:\Users\aaa\.cargo\registry\src\github.com-1ecc6299db9ec823\mime_guess-2.0.4\src\impl_bin_search.rs:4:1
  |
4 | include!(concat!(env!("OUT_DIR"), "/mime_types_generated.rs"));
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: this error originates in the macro `include` (in Nightly builds, run with -Z macro-backtrace for more info)

error: could not compile `mime_guess` due to previous error
warning: build failed, waiting for other jobs to finish...
Error: 1 job failed

Unfortunately I can't continue with trying this out until I get a solution for this :(

[urtokk]

@chrisheib I had the same problem and worked around it by using github actions. If that's a possiblity for you

[oddgrd]

Hi @chrisheib ! As urtokk said above, until we find a way to work around this bug in cargo, Windows users can deploy remotely using a GitHub Action.

When deploying certain projects (projects that rely on getting secrets in the shuttle_service::main function), the project will have to be deployed twice (only for the first two deploys). If you're deploying via GitHub actions, the easiest thing to do is to set the workflow to run on workflow_dispatch. An example of this is linked below. This way you can manually run it after setting the required secrets in the repository, and then run it again if needed. This is a known bug that we are working on resolving.

Here is an example repo where I deployed the serenity-postgres example from the Shuttle examples: https://github.com/oddgrd/postgres-test-deploy

[xobs]

This appears to be due to abonander/mime_guess#78 which is fixed in abonander/mime_guess#79

You might try adding something like this to your Cargo.toml file while waiting for the PR to get merged (note: I haven't tested this, so it may need modification):

[patch.crates-io.mime_guess]
git="https://github.com/NathanSWard/mime_guess.git"
branch="platform-path-separators"

[oddgrd]

Thanks for this @xobs ! I tested this out on Windows (briefly), and it did indeed fix cargo shuttle run for me. Unfortunately for this to work with deployment we need to wait for the fix to be merged in mime_guess and incorporated in serenity. Shuttle doesn't support git dependencies yet.

Hopefully you can play around with cargo shuttle run in the meantime if you are on Windows with this fix, and deploy through CI.

[tguichaoua]

@chrisheib, I had the same problem and solved it by running cargo build before cargo shuttle run.

[christos-h]

So if you don't want to use GH actions - I have a quick and easy workaround using Docker. Haven't tested it on Windows yet but I don't see why it wouldn't work.

First you need to create a Dockerfile:

FROM rust:latest
RUN rustup default nightly
RUN cargo install cargo-shuttle

and then build an image (while in the same directory as the Dockerfile):

docker build -t cargo-shuttle .

This will create the image called cargo-shuttle - but this will take a while the first time you do it (5 minutes is normal).

The last step is this docker command which mounts your shuttle credentials and your project directory to the container:

docker run -v <path-to-shuttle-config>:/root/.config/shuttle -v <path-to-project>:/root/<project-name> -w /root/<project-name>/ cargo-shuttle cargo shuttle deploy --no-test --allow-dirty

The <path-to-shuttle-config> should be C:\Users\<username>\AppData\Roaming\shuttle as per: https://docs.rs/dirs/4.0.0/dirs/fn.config_dir.html but I don't have windows so I can't test this.

Modification may be necessary as I don't have a windows machine - but hope this works!

[oddgrd]

I can confirm that Christos' solution works for deploying on Windows, I tested it by deploying this project on Windows. Thanks Christos!

Note: I first tried to do this with Git Bash, which didn't work due to pathing issues. Doing it in PowerShell went without a hitch, though.

[RagibHasin]

Using dunce crate in a local cargo-shuttle build seems to solve it for me.

[oddgrd]

Great catch @RagibHasin! We did discover this fix too yesterday: #721, but we have had this bug for months. 😅

This fix will be included in the next release which is scheduled for monday, so I'll close this issue (at long last!).

[RagibHasin]

I was about to make a PR on the main branch 😅. Is all future development happening on the shuttle-next branch?

[oddgrd]

Haha, thanks anyway! 😂 No, shuttle-next is a development branch for a major refactor we've been working on for a while now. We will return to developing on main after this release. 👍