GC crash in .NET 9 UWP and WinUI 3 apps published with Native AOT on win-x86

[andrew-kulikov]

Description

I am migrating an UWP applicaiton to .NET 9 and found a problem with win-x86 Native AOT publish. The app works when it's either unpackaged or published using win-x64 runtime. But when it's published using win-x86 runtime, gc crashes on random instructions when trying to collect memory. Example stack trace from crash dump:

The thread tried to read from or write to a virtual address for which it does not have the appropriate access.

UwpDotnet.exe!WKS::gc_heap::mark_phase(int condemned_gen_number=5150738)	C++
UwpDotnet.exe!WKS::gc_heap::gc1()	C++
UwpDotnet.exe!WKS::gc_heap::garbage_collect(int n)	C++
UwpDotnet.exe!WKS::GCHeap::GarbageCollectGeneration(unsigned int gen=0, gc_reason reason=reason_alloc_soh)	C++
UwpDotnet.exe!WKS::gc_heap::trigger_gc_for_alloc(int gen_number=0, gc_reason gr=reason_alloc_soh, WKS::GCDebugSpinLock * msl=0x0082f010, bool loh_p=false, WKS::msl_take_state take_state=mt_try_budget)	C++
[Inline Frame] UwpDotnet.exe!WKS::gc_heap::try_allocate_more_space(alloc_context *)	C++
UwpDotnet.exe!WKS::gc_heap::allocate_more_space(alloc_context * acontext=0x01640358, unsigned int size=4012, unsigned int flags=0, int alloc_generation_number=0)	C++
[Inline Frame] UwpDotnet.exe!WKS::gc_heap::allocate(unsigned int)	C++
UwpDotnet.exe!WKS::GCHeap::Alloc(gc_alloc_context * context=0x0082f010, unsigned int size=4012, unsigned int flags=0)	C++
UwpDotnet.exe!GcAllocInternal(MethodTable * pEEType=0x008f319c, unsigned int uFlags=0, unsigned int numElements=1000, Thread * pThread=0x01640358)	C++
UwpDotnet.exe!RhpGcAlloc(MethodTable * pEEType=0x008f319c, unsigned int uFlags=0, unsigned int numElements=1000, PInvokeTransitionFrame * pTransitionFrame=0x0782f8dc)	C++
UwpDotnet.exe!@RhpNewArray@8()	Unknown
>	UwpDotnet.exe!UwpDotnet_UwpDotnet_Program__RunLoad()	Unknown
UwpDotnet.exe!S_P_CoreLib_System_Threading_ThreadStart__InvokeOpenStaticThunk()	Unknown
UwpDotnet.exe!S_P_CoreLib_System_Threading_Thread__StartThread()	Unknown
UwpDotnet.exe!S_P_CoreLib_System_Threading_Thread__ThreadEntryPoint()	Unknown
ntdll.dll!77cc809e()	Unknown
[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll]	
ntdll.dll!77cc806e()	Unknown

The same problem is reproduced for both WinUI 3 and UWP apps, but not reproduces for console application.

Reproduction Steps

It's reproducible on both UWP and WinUI 3, but for simplicity specifying WinUI 3 here.

1. Create WinUI 3 app on net9.0-windows10.0.26100.0

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>WinExe</OutputType>
    <TargetFramework>net9.0-windows10.0.26100.0</TargetFramework>
    <TargetPlatformMinVersion>10.0.17763.0</TargetPlatformMinVersion>
    <RootNamespace>WinUIAppTest</RootNamespace>
    <ApplicationManifest>app.manifest</ApplicationManifest>
    <Platforms>x86;x64;arm64</Platforms>
    <RuntimeIdentifiers>win-x86;win-x64;win-arm64</RuntimeIdentifiers>
    <PublishProfile>win-$(Platform).pubxml</PublishProfile>
    <UseWinUI>true</UseWinUI>
    <EnableMsixTooling>true</EnableMsixTooling>
    <Nullable>enable</Nullable>
    <AppendRuntimeIdentifierToOutputPath>false</AppendRuntimeIdentifierToOutputPath>
    <AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
    <PackageCertificateKeyFile>../TemporaryKey.pfx</PackageCertificateKeyFile>
  </PropertyGroup>

  <ItemGroup>
    <Content Include="Assets\SplashScreen.scale-200.png" />
    <Content Include="Assets\LockScreenLogo.scale-200.png" />
    <Content Include="Assets\Square150x150Logo.scale-200.png" />
    <Content Include="Assets\Square44x44Logo.scale-200.png" />
    <Content Include="Assets\Square44x44Logo.targetsize-24_altform-unplated.png" />
    <Content Include="Assets\StoreLogo.png" />
    <Content Include="Assets\Wide310x150Logo.scale-200.png" />
  </ItemGroup>

  <ItemGroup>
    <Manifest Include="$(ApplicationManifest)" />
  </ItemGroup>

  <ItemGroup Condition="'$(DisableMsixProjectCapabilityAddedByProject)'!='true' and '$(EnableMsixTooling)'=='true'">
    <ProjectCapability Include="Msix" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.Windows.SDK.BuildTools" Version="10.0.26100.1742" />
    <PackageReference Include="Microsoft.WindowsAppSDK" Version="1.6.240923002" />
  </ItemGroup>

  <PropertyGroup Condition="'$(DisableHasPackageAndPublishMenuAddedByProject)'!='true' and '$(EnableMsixTooling)'=='true'">
    <HasPackageAndPublishMenu>true</HasPackageAndPublishMenu>
  </PropertyGroup>
</Project>

2. Configure NativeAOT publish in win-x86.pubxml

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <PublishProtocol>FileSystem</PublishProtocol>
    <Platform>x86</Platform>
    <RuntimeIdentifier>win-x86</RuntimeIdentifier>
    <PublishAot>true</PublishAot>
    <SelfContained>true</SelfContained>
  </PropertyGroup>
</Project>

3. Add some GC-intensive worker in App.xaml.cs

using System;
using System.Collections.Concurrent;
using System.Threading;
using Microsoft.UI.Xaml;

namespace WinUIAppTest;

public partial class App : Application
{
    private Window? m_window;

    public App()
    {
        InitializeComponent();

        var t = new Thread(RunLoad);
        t.Start();
    }

    protected override void OnLaunched(LaunchActivatedEventArgs args)
    {
        m_window = new MainWindow();
        m_window.Activate();
    }

    public static void RunLoad()
    {
        var queue = new ConcurrentQueue<object>();
        while (true)
        {
            var obj = new
            {
                Property1 = new int[1000]
            };
            if (queue.Count > 100000)
            {
                queue.TryDequeue(out _);
            }

            queue.Enqueue(obj);
            if (Random.Shared.Next() % 1000 == 0)
            {
                Thread.Sleep(10);
            }
        }
    }
}

4. Publish app

dotnet publish /p:Platform=x86 /p:GenerateAppxPackageOnBuild=true

5. Install msix package and run app

Expected behavior

App launches and runs.

Actual behavior

App crashes with stacktrace specified in descripiton.

Regression?

No, since x86 is available in .NET 9 only.

Known Workarounds

No response

Configuration

Dotnet info:

.NET SDK:
 Version:           9.0.100
 Commit:            59db016f11
 Workload version:  9.0.100-manifests.4a280210
 MSBuild version:   17.12.7+5b8665660

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19045
 OS Platform: Windows
 RID:         win-x64
 Base Path:   C:\Program Files\dotnet\sdk\9.0.100\

.NET workloads installed:
 [android]
   Installation Source: SDK 9.0.100, VS 17.12.35521.163
   Manifest Version:    35.0.7/9.0.100
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.android\35.0.7\WorkloadManifest.json
   Install Type:              Msi

 [aspire]
   Installation Source: SDK 9.0.100
   Manifest Version:    8.2.2/8.0.100
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\8.2.2\WorkloadManifest.json
   Install Type:              Msi

 [ios]
   Installation Source: SDK 9.0.100, VS 17.12.35521.163
   Manifest Version:    18.1.9163/9.0.100
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.ios\18.1.9163\WorkloadManifest.json
   Install Type:              Msi

 [maccatalyst]
   Installation Source: SDK 9.0.100, VS 17.12.35521.163
   Manifest Version:    18.1.9163/9.0.100
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.maccatalyst\18.1.9163\WorkloadManifest.json
   Install Type:              Msi

 [macos]
   Installation Source: SDK 9.0.100
   Manifest Version:    15.1.9163/9.0.100
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.macos\15.1.9163\WorkloadManifest.json
   Install Type:              Msi

 [maui-windows]
   Installation Source: SDK 9.0.100, VS 17.12.35521.163
   Manifest Version:    9.0.0/9.0.100
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.maui\9.0.0\WorkloadManifest.json
   Install Type:              Msi

Configured to use loose manifests when installing new manifests.

Host:
  Version:      9.0.0
  Architecture: x64
  Commit:       9d5a6a9aa4

.NET SDKs installed:
  7.0.410 [C:\Program Files\dotnet\sdk]
  9.0.100 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Windows SDKs installed;

• 10.0.26100.0
• 10.0.22621.0
• 10.0.18362.0
• 10.0.17763.0
• 10.0.16299.0

Other information

Repo with reproduction examples: https://github.com/andrew-kulikov/native-aot-test

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

[Sergio0694]

👀 cc. @manodasanW

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

Thank you for the good repro! I have a PR out with a fix.