Linker Problem for MAUI on IOS

[hssoab326]

Describe the bug
When i was building maui ios app on Azure DevOps, it fails to build with the following error.
It can be built when <MtouchLink>None</MtouchLink> is added to the csproj.
However, App size and build time will be doubled, it is not desireable.

2022-08-02T02:15:41.3706750Z Tool xcrun execution finished (exit code = 1).
2022-08-02T02:15:41.3725910Z
2022-08-02T02:15:41.3727470Z Undefined symbols for architecture arm64:
2022-08-02T02:15:41.3728050Z "OBJC_CLASS$_MLPredictionOptions", referenced from:
2022-08-02T02:15:41.3729980Z objc-class-ref in onnxruntime(model.o)
2022-08-02T02:15:41.3730730Z "OBJC_CLASS$_MLModelConfiguration", referenced from:
2022-08-02T02:15:41.3732100Z objc-class-ref in onnxruntime(model.o)
2022-08-02T02:15:41.3732860Z "OBJC_CLASS$_MLModel", referenced from:
2022-08-02T02:15:41.3733630Z objc-class-ref in onnxruntime(model.o)
2022-08-02T02:15:41.3734990Z "OBJC_CLASS$_MLFeatureValue", referenced from:
2022-08-02T02:15:41.3735630Z objc-class-ref in onnxruntime(model.o)
2022-08-02T02:15:41.3736260Z "OBJC_CLASS$_MLMultiArray", referenced from:
2022-08-02T02:15:41.3736960Z objc-class-ref in onnxruntime(model.o)
2022-08-02T02:15:41.3738070Z ld: symbol(s) not found for architecture arm64
2022-08-02T02:15:41.3739440Z clang: error: linker command failed with exit code 1 (use -v to see invocation)
2022-08-02T02:15:41.3743170Z
2022-08-02T02:15:41.3745190Z /Users/runner/hostedtoolcache/dotnet/packs/Microsoft.iOS.Sdk/15.4.402/targets/Xamarin.Shared.Sdk.targets(1214,3): error : clang++ exited with code 1

Urgency
None

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS - 12 (Azure DevOps)
• ONNX Runtime installed from (source or binary): Nuget
• ONNX Runtime version: 1.12.0
• Python version: -
• Visual Studio version (if applicable): -
• GCC/Compiler version (if compiling from source): -
• CUDA/cuDNN version: -
• GPU model and memory: -

To Reproduce
Create maui project with onnxruntime.
Build for iphone.

Expected behavior
it builds.

Screenshots
The build task.

[skottmckay]

@mikeparker104 any ideas for this one? It appears that some CoreML pieces are removed or not included. Not sure if there's an mtouch setting we can tweak.

https://docs.microsoft.com/en-us/xamarin/ios/deploy-test/linker?tabs=macos

FWIW we mark the C# ORT library as LinkerSafe but I wouldn't expect that to influence Objective-C level CoreML symbols being removed.

[skottmckay]

@hssoab326 do you get errors if building locally with

• the dotnet CLI
• Visual Studio 2022 Preview

Are you using the new single project structure (one csproj builds everything) or the older Xamarin setup with platform specific csproj files?

How/where is Microsoft.ML.OnnxRuntime referenced in the project file/s?

[hssoab326]

@hssoab326 do you get errors if building locally with

• the dotnet CLI
• Visual Studio 2022 Preview

The same errors occur when building with VS2022 preview or with the dotnet CLI.
It also happens when building for iphone simulator as long as MTouchLink is set to SdkOnly.
Setting MTouchLink to full can build without problem. But it breaks MAUI.

Are you using the new single project structure (one csproj builds everything) or the older Xamarin setup with platform specific csproj files?

I am using the new single project structure.

How/where is Microsoft.ML.OnnxRuntime referenced in the project file/s?

Like this.

In the ML csproj

[skottmckay]

Can you please try adding a reference to Microsoft.ML.OnnxRuntime in your MAUI App csproj.

[skottmckay]

If that doesn't work, could you possibly try building the example app locally? We've tested that and were able to use 'dotnet build' and 'dotnet publish' successfully with both SdkOnly and Full. This would help rule out the issue being due to your project structure.

https://github.com/microsoft/onnxruntime-inference-examples/tree/main/mobile/examples/Maui/MauiVisionSample

[hssoab326]

Can you please try adding a reference to Microsoft.ML.OnnxRuntime in your MAUI App csproj.

Same Error.
Heres some azure dev ops agent info that might help.

Operating System

• macOS
• 12.5
• 21G72

Virtual Environment

• Environment: macos-12
• Version: 20220801.1
• Included Software: https://github.com/actions/virtual-environments/blob/macOS-12/20220801.1/images/macos/macos-12-Readme.md
• Image Release: https://github.com/actions/virtual-environments/releases/tag/macOS-12%2F20220801.1

If that doesn't work, could you possibly try building the example app locally? We've tested that and were able to use 'dotnet build' and 'dotnet publish' successfully with both SdkOnly and Full. This would help rule out the issue being due to your project structure.

https://github.com/microsoft/onnxruntime-inference-examples/tree/main/mobile/examples/Maui/MauiVisionSample

Same behaviour when building the example app locally. It fails when MtouchLink is set to SdkOnly.
dotnet cli also fails to build locally.

OS: MacOS Monterey 12.5
Xcode Version: 13.4.1

[skottmckay]

Can you please provide the output from dotnet --info as well as the full command line/s and output from your attempt to build the sample app so we can make sure we're testing exactly the same way?

Can you also run a build of the sample app to produce a binlog and share the log here? Use the -bl parameter. By default it will output to a file called msbuild.binlog in the current directory.

[skottmckay]

We were able to reproduce the issue in a CI. The sample app builds without a problem (so local builds and testing are fine). It's the 'publish' phase that seems to be the issue with some coreml symbols used by the ORT library going missing. We'll contact the MAUI folks to investigate given that's not something (AFAIK) we have any control over the behaviour of.

[skottmckay]

Short term could you try adding this workaround to your csproj to manually insert CoreML into the frameworks the app links against? Adjust the 'Condition' if needed.

<!-- Manually add CoreML to the frameworks we link against. -->
<Target Name="AddCoreML" Condition="'$(TargetFramework)' == 'net6.0-ios'" AfterTargets="_LoadLinkerOutput" BeforeTargets="_ComputeLinkNativeExecutableInputs">
    <ItemGroup>
        <_LinkerFrameworks Include="CoreML" />
    </ItemGroup>
</Target>

[hssoab326]

It successfully built with the workaround. Thanks.

[skottmckay]

Awesome. Glad it works. I'll leave this issue open until we figure out a long term fix with the MAUI folks.

[rolfbjarne]

The problem is that we (MAUI on iOS) don't know that the Microsoft.ML.OnxxRuntime NuGet uses the CoreML framework, so we don't link with it.

As a sidenote, one reason this happens is because the native onxxruntime is shipped as a static library, and not as a framework. This means that a potential solution would be to ship the onxxruntime as a framework instead.

Another potential solution would be to include the fact that the onxxruntime static library needs to link with CoreML by enabling auto linking when building the static library (see https://www.smileykeith.com/2022/02/23/lc-linker-option/ for more information about auto linking). Seems like it builds down to passing -fmodules to clang when building the static library (note that the other options mentioned in that article, such as passing --linker-option=... to clang, don't work because of clang limitations, as I found out here).

[DRAirey1]

I'm running into this with a new Maui application. I use multiple projects for the View Models and Views (forms). Nothing fancy, just trying to port a working XAMARIN project to the new Maui architecture.

[skottmckay]

I'm running into this with a new Maui application. I use multiple projects for the View Models and Views (forms). Nothing fancy, just trying to port a working XAMARIN project to the new Maui architecture.

@DRAirey1 We should have a fix in the next release. In the meantime, please try the workaround to manually add CoreML to the _LinkerFrameworks property.

[DRAirey1]

Didn't fix anything.

5>Tool xcrun execution finished (exit code = 1).
5>ld: framework not found System
5>clang: error: linker command failed with exit code 1 (use -v to see invocation)
5>C:\Program Files\dotnet\packs\Microsoft.iOS.Sdk\15.4.446-ci.-release-6-0-4xx.446\targets\Xamarin.Shared.Sdk.targets(1219,3): error : clang++ exited with code 1
5>C:\Program Files\dotnet\packs\Microsoft.iOS.Sdk\15.4.446-ci.-release-6-0-4xx.446\targets\Xamarin.Shared.Sdk.targets(1219,3): error :
5>Done building project "ThetaRex.OpenBook.Demo.csproj" -- FAILED.
5>ThetaRex.OpenBook.Demo -> C:\source\Repos\openbook-demo\ThetaRex.OpenBook.Demo\bin\Release\net6.0-windows10.0.19041.0\win10-x64\ThetaRex.OpenBook.Demo.dll

Everything is up to date in NuGet. I'll be damned if I can figure out what framework it can't find, but this is infuriating.

[skottmckay]

That seems like a different issue. The linker problem in the original issue was because CoreML was not included and the ORT library required it. The linker not being able to find the System framework seems far more fundamental. If you were to temporarily remove the usage of ORT do you still get the same error?

[DRAirey1]

Never mind. It appears to be MSAL that's causing the issue.

[rolfbjarne]

ld: framework not found System

That's likely a variation of AzureAD/microsoft-authentication-extensions-for-dotnet#180

[skottmckay]

Will be fixed in the 1.13 release so closing this issue.