[MonoAOTCompiler] more properties & custom WorkingDirectory

[jonathanpeppers]

Fixes: #56163

PR #58523 fixed something on Windows, but it didn't actually address
our issues on Android.

A directory name like foo Ümläüts fails:

• mkdir 'foo Ümläüts' ; cd 'foo Ümläüts'
• dotnet new android
• dotnet build -c Release -p:RunAOTCompilation=true (adding
  -p:EnableLLVM=true complicates further)

The error:

Precompiling failed for C:\src\foo Ümläüts\obj\Release\android-arm64\linked\System.Private.CoreLib.dll: Error: Loaded assembly 'C:\src\foo ├£ml├ñ├╝ts\obj\Release\android-arm64\linked\System.Private.CoreLib.dll' doesn't match original file name 'C:\foo ▄mlΣⁿts\obj\Release\android-arm64\linked\System.Private.CoreLib.dll'. Set MONO_PATH to the assembly's location.

Reviewing the existing AOT implementation in Xamarin.Android, I found
out why Xamarin.Android works:

[AOT] response file obj\Release\120\aot\arm64-v8a\App36.dll\response.txt: --llvm "--aot=temp-path=obj\Release\120\aot\arm64-v8a\App36.dll,llvm-path=C:\Program Files\Microsoft Visual Studio\2022\Preview\MSBuild\Xamarin\Android,outfile=obj\Release\120\aot\arm64-v8a\libaot-App36.dll.so,msym-dir=obj\Release\120\aot\arm64-v8a,asmwriter,mtriple=aarch64-linux-android,tool-prefix=C:\Program Files (x86)\Android\android-sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android-,ld-name=ld.EXE,ld-flags=\"-LC:\Program Files (x86)\Android\android-sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\lib\gcc\aarch64-linux-android\4.9.x\";\"-LC:\Program Files (x86)\Android\android-sdk\ndk-bundle\platforms\android-21\arch-arm64\usr\lib\";\"C:\Program Files (x86)\Android\android-sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\lib\gcc\aarch64-linux-android\4.9.x\libgcc.a\";\"C:\Program Files (x86)\Android\android-sdk\ndk-bundle\platforms\android-21\arch-arm64\usr\lib\libc.so\";\"C:\Program Files (x86)\Android\android-sdk\ndk-bundle\platforms\android-21\arch-arm64\usr\lib\libm.so\"" C:\Users\jopepper\source\repos\App36\App36\obj\Release\120\android\assets\shrunk\App36.dll

1. Xamarin.Android passes relative paths. The foo Ümläüts
   directory name doesn't even come into play for some arguments.

2. With LLVM, ld-flags contains a ;.

The existing code splits on ; and joins on ,:

runtime/src/tasks/AotCompilerTask/MonoAOTCompiler.cs

Lines 505 to 509 in 25c2073

	var a = assemblyItem.GetMetadata("AotArguments");
	if (a != null)
	{
	aotArgs.AddRange(a.Split(new char[]{ ';' }, StringSplitOptions.RemoveEmptyEntries));
	}

So we lose any ; delimiters for the ld-flags value, they get
replaced by ,.

I think the solution here is:

1. Add several missing properties to <MonoAOTCompiler/> so we don't
   have to rely on the %(AotArguments) item metadata. No splitting
   on ; would be required, ld-flags can be passed in and used as-is.

2. Add a new WorkingDirectory property. When this is set, assume
   paths passed in might be relative -- and don't transform paths by
   calling Path.GetFullPath().

Lastly, I fixed a place where the UTF8 encoding wasn't passed when
MSBuild logging the response file.

These changes I tried to make in a way where this shouldn't break
other .NET workloads like wasm. If existing MSBuild targets call this
task (not using the new properties), the behavior should remain
unchanged.

I tested these changes by commiting a modified MonoAOTCompiler.dll:

dotnet/android#6562

I'm able to enable several AOT tests related to #56163.

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

[steveisok]

/azp run runtime-manual

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[steveisok]

The runtime-manual failures are unrelated. The AOT legs for wasm and iOS were successful.

@radical and @akoeplinger please review.

[radical]

I wonder if it might be better to have a ITaskItem? CommonAotArguments with metadata which act as the key-value pairs to pass to --aot.

[jonathanpeppers]

LdFlags will contain a ; on Android. So it seems like using item metadata would always have an issue?

Our usage is now:

<MonoAOTCompiler
    Triple="$(_Triple)"
    ToolPrefix="$(_ToolPrefix)"
    MsymPath="$(_MsymPath)"
    Assemblies="@(_MonoAOTAssemblies)"
    CompilerBinaryPath="$(_MonoAOTCompilerPath)"
    DisableParallelAot="$(_DisableParallelAot)"
    IntermediateOutputPath="$(IntermediateOutputPath)"
    LibraryFormat="So"
    Mode="$(AndroidAotMode)"
    OutputDir="$(IntermediateOutputPath)aot\"
    OutputType="Library"
    UseAotDataFile="false"
    UseLLVM="$(EnableLLVM)"
    LLVMPath="$(_LLVMPath)"
    LdName="$(_LdName)"
    LdFlags="$(_LdFlags)"
    WorkingDirectory="$(MSBuildProjectDirectory)"
    AotArguments="$(AndroidAotAdditionalArguments)">
  <Output TaskParameter="CompiledAssemblies" ItemName="_AotCompiledAssemblies" />
  <Output TaskParameter="FileWrites"         ItemName="FileWrites" />
</MonoAOTCompiler>

But maybe $(AndroidAotAdditionalArguments) could be passed in a different way? What would the XML look like for ITaskItem? CommonAotArguments?

[radical]

LdFlags will contain a ; on Android. So it seems like using item metadata would always have an issue?

We won't split the metadata on CommonAotArguments.
As for the metadata names, we could either use the real argument name, like <ld-flags>foo; bar</ld-flags>, or we could have <LdFlags>foo; bar</LdFlags> and convert to the real name in the task.

  <CommonAotArguments Include="common">
    <ld-flags>foo;bar</ld-flags>
    <ld-name>xyz</ld-name>
  </CommonAotArguments>

or

  <CommonAotArguments Include="common">
    <LdFlags>foo;bar</LdFlags>
    <LdName>xyz</LdName>
  </CommonAotArguments>

And you could have a <PassVerbatim>..</PassVerbatim> metadata (or better name) for just any additional string to be passed to --aot.

Feel free to push back, I'm just thinking aloud.

[jonathanpeppers]

The weird part about using @(CommonAotArguments), is that you put Include="" with any random word inside. You can't leave out Include, or that would be an empty item group. I also like the idea of having C# properties, as it checks for typos -- you'll get an MSBuild error if you put LdlFlags instead of LdFlags.

Adding something like @(CommonAotArguments) might be ok if there is some unknown set. At least for Android, these changes now include every parameter we use. Are there many more not represented here?

[jonathanpeppers]

@steveisok how would we get this on the mauipre branch? Is there a magic comment to backport it?

[steveisok]

/backport to release/6.0-maui

[github-actions]

Started backporting to release/6.0-maui: https://github.com/dotnet/runtime/actions/runs/1583847353