Windows: don't shell out to cmd.exe

[laszlocsomor]

Description of the problem / feature request / question:

cmd.exe doesn't support UNC paths so we can't shell out to tools through it and use long paths.

Example where this occurs: WindowsFileSystem.createDirectoryJunction

Related issue: #2181

Environment info

• Operating System: Windows 10

[laszlocsomor]

This issue is actually blocking #2181 .

[laszlocsomor]

P2, because there is a workaround: use --output_user_root=<something short>.

You can even create a virtual drive:

subst d: c:\some\path\where\bazel\can\put\stuff

then just use --output_user_root=/d/.

[laszlocsomor]

Re: #2190 (comment) and WindowsFileSystem.createDirectoryJunction, that's actually a bad example, because mklink is a built-in command of cmd.exe, so we'll have to implement the junction creation in JNI anyway.

The CommandBuilder however suffers from this limitation.

[abergmeier-dsfishlabs]

Isn't this rather a bug?

[petemounce]

Generally don't use cmd, because MS announced that they will (finally!!) be deprecating it (over some time): http://www.osnews.com/story/29502/Microsoft_replaces_cmd_with_PowerShell_in_new_Windows_10_build

[laszlocsomor]

@petemounce : AIUI they aren't deprecating cmd, just making it non-default. So cmd will be around... *in ghastly voice* foreveeeeeeer.

[petemounce]

Sure. Still don't use it. :p

[dslomov]

Powershell is very very slow to start (as in, >10x slower than cmd.exe as @philwo measured)
So we avoid using powershell as a shell.
See https://docs.google.com/document/d/1z6Xv95CJYNYNYylcRklA6xBeesNLc54dqXfri0z0e14/ for some plans

[davido]

Another reason to not shell out to cmd.exe is 8191 command line limit: #3579.

[laszlocsomor]

This is the only non-test place I found where Bazel shells out to cmd.exe:

bazel/src/main/java/com/google/devtools/build/lib/util/CommandBuilder.java

Line 163 in b874254

return new String[] { "CMD.EXE", "/S", "/E:ON", "/V:ON", "/D", "/C",

I have a hunch (that I need to verify) that you can CreateProcess with other executables than .exe files, so automatically using cmd may be unnecessary:

bazel/src/main/java/com/google/devtools/build/lib/util/CommandBuilder.java

Lines 146 to 147 in b874254

	// Automatically enable CMD.EXE use if we are executing something else besides "*.exe" file.
	// When use CMD.EXE to invoke a bat/cmd file, the file path must have '\' instead of '/'

[laszlocsomor]

PR #5007 was rolled back in 8358148.

[laszlocsomor]

Apparently the CommandBuilder.java can still shell out to cmd.exe, so this bug is not fixed.

[laszlocsomor]

Other than CommandBuilder.java, the Android tools' junction helper (that I wrote) also uses cmd:

bazel/tools/android/junction.py

Line 89 in acb1bd1

"cmd.exe /C mklink /J \"%s\" \"\\\\?\\%s\"" % (name, target),

These are the only two locations I'm aware of. (And some more in Bazel's own tests.)

[laszlocsomor]

I'm not sure if this bug is really that big of a deal:

• For the Android tools: mklink supports long paths, even when invoked from cmd. The limiting factor for the path length of the junction target is the 8K command line limit.
• Bazel shells out to cmd.exe only when:
  • the action sets /bin/sh or /bin/bash as its executable (I could not find any native rules doing so; I can't rule out Starlark rules doing it via the deprecated ctx.action()), or
  • when the argv0 does not end with ".exe", which should only happen on Windows if the extension is ".bat" or ".cmd" (very rare) which are fine to run under cmd.exe

[laszlocsomor]

Because of #2190 (comment), I'm marking it as P4.

[meisterT]

Closing for the reasons mentioned above.