Add VMR leg that builds test projects #44843

ViktorHofer · 2024-11-13T13:08:30Z

Running as part of the PR to get some initial results.

Fixes dotnet/source-build#4168 Running as part of the PR to get some initial results.

ViktorHofer · 2024-11-13T15:48:28Z

src\winforms\src\System.Windows.Forms.Design\tests\UnitTests\System\Windows\Forms\Design\ToolStripContentPanelDesignerTests.cs(43,67): error CS8620: (NETCORE_ENGINEERING_TELEMETRY=Build) Argument of type 'string[]' cannot be used for parameter 'span' of type 'Span<string?>' in 'bool MemoryExtensions.Contains<string?>(Span<string?> span, string? value)' due to differences in the nullability of reference types.

@am11 did we see this one when we updated the VMR to the .NET 10 SDK? Can't remember if we talked about it.

am11 · 2024-11-13T17:21:49Z

Yup, it was #43015 (comment). It will probably need a fix like dotnet/winforms@main...am11:winforms:patch-1. I'll try building it on windows soon.

ViktorHofer · 2024-11-19T12:39:25Z

    D:\a\_work\1\vmr\src\efcore\test\EFCore.Specification.Tests\CustomConvertersTestBase.cs(1187,49): error CS0023: Operator '.' cannot be applied to operand of type 'void' [D:\a\_work\1\vmr\src\efcore\test\EFCore.Specification.Tests\EFCore.Specification.Tests.csproj]
    D:\a\_work\1\vmr\src\efcore\test\EFCore.Specification.Tests\CustomConvertersTestBase.cs(1188,49): error CS0023: Operator '.' cannot be applied to operand of type 'void' [D:\a\_work\1\vmr\src\efcore\test\EFCore.Specification.Tests\EFCore.Specification.Tests.csproj]
    D:\a\_work\1\vmr\src\efcore\test\EFCore.Specification.Tests\Query\PrimitiveCollectionsQueryTestBase.cs(430,67): error CS8620: Argument of type 'string[]' cannot be used for parameter 'span' of type 'Span<string?>' in 'bool MemoryExtensions.Contains<string?>(Span<string?> span, string? value)' due to differences in the nullability of reference types. [D:\a\_work\1\vmr\src\efcore\test\EFCore.Specification.Tests\EFCore.Specification.Tests.csproj]
    D:\a\_work\1\vmr\src\efcore\test\EFCore.Specification.Tests\Query\PrimitiveCollectionsQueryTestBase.cs(433,68): error CS8620: Argument of type 'string[]' cannot be used for parameter 'span' of type 'Span<string?>' in 'bool MemoryExtensions.Contains<string?>(Span<string?> span, string? value)' due to differences in the nullability of reference types. [D:\a\_work\1\vmr\src\efcore\test\EFCore.Specification.Tests\EFCore.Specification.Tests.csproj]
    D:\a\_work\1\vmr\src\efcore\test\EFCore.Specification.Tests\Query\PrimitiveCollectionsQueryTestBase.cs(577,67): error CS8620: Argument of type 'string[]' cannot be used for parameter 'span' of type 'Span<string?>' in 'bool MemoryExtensions.Contains<string?>(Span<string?> span, string? value)' due to differences in the nullability of reference types. [D:\a\_work\1\vmr\src\efcore\test\EFCore.Specification.Tests\EFCore.Specification.Tests.csproj]

@am11 would you want to help me with driving this one in? I currently have some other obligations and won't be able to get back to this PR anytime soon. See the above efcore test failure. They need the same first class Span reaction update.

There's also a winforms test failure in the PR that I don't really understand yet.

am11 · 2024-11-19T13:36:51Z

I think #44922 is bringing winforms change. I'll check efcore.

ViktorHofer · 2024-11-20T20:29:06Z

The efcore issue is weird:

2024-11-20T20:05:12.6651643Z     FSC : error FS2014: A problem occurred writing the binary 'D:\a\_work\1\vmr\src\efcore\artifacts\obj\EFCore.FSharp.FunctionalTests\Release\net9.0\refint\Microsoft.EntityFrameworkCore.FSharp.FunctionalTests.dll': A call to StrongNameSignatureSize failed (Invalid Public Key blob) [D:\a\_work\1\vmr\src\efcore\test\EFCore.FSharp.FunctionalTests\EFCore.FSharp.FunctionalTests.fsproj]

The winforms failure is also still happening.

am11 · 2024-11-20T20:38:27Z

Looks like a known bug dotnet/fsharp#17451. A solution (workaround?) is here dotnet/fsharp#11546 (comment).

Since it is a test assembly perhaps we should just disable strong name signing?

ViktorHofer · 2024-11-21T10:14:57Z

It would be interesting to understand why the fsharp test build behaves differently with the VMR.

NikolaMilosavljevic · 2024-12-03T22:40:08Z

efcore issue is caused by /p:FullAssemblySigningSupported=false which is set in VMR build. This causes PublicSign to not be set to false here: https://github.com/dotnet/arcade/blob/2fd9b394c6ef3ad22639d83b8da649c9350f88f6/src/Microsoft.DotNet.Arcade.Sdk/tools/StrongName.targets#L50

I'll try to set PublicSign to false explicitly in this test project.

ViktorHofer · 2024-12-04T13:19:04Z

cc @mmitche as you approved the Arcade PR: dotnet/arcade#12940 and might have context here.

mmitche · 2024-12-04T18:18:23Z

I believe this should be conditioned on source-only or non-source only builds, rather than a blanket setting across all VMR builds. That said, I'm slightly uncomfortable with where the setting is put. I think it should actually be in StrongName.targets or generally within the arcade infra, rather than the source build control setup.

mmitche · 2024-12-04T18:18:34Z

I'll file an issue and open a PR to fix this.

mmitche · 2024-12-04T18:21:56Z

dotnet/arcade#15301

eng/pipelines/templates/stages/vmr-build.yml

NikolaMilosavljevic · 2025-01-31T18:25:09Z

Tests build is failing in winforms with:

The command "
      call "D:\a\_work\1\vmr\src\winforms\eng\init-vs-env.cmd" 
      
       cmake -G "Visual Studio 17 2022" -A Win32 -B "D:\a\_work\1\vmr\src\winforms\artifacts\obj\NativeTests\x86\Release" -S "D:\a\_work\1\vmr\src\winforms\src\System.Windows.Forms\tests\InteropTests\NativeTests" -DCMAKE_INSTALL_PREFIX=D:\a\_work\1\vmr\src\winforms\artifacts\bin\NativeTests\x86\Release --log-level=WARNING
      " exited with code 1.

I have not seen this locally and it is likely due to host configuration issues.

I'm going to opt-out of building tests in winforms and create a tracking issue in dotnet/source-build

NikolaMilosavljevic · 2025-02-03T16:40:23Z

Hmm, this PR is now hitting an odd error that I don't think is related.

It's failure to restore vstest package during SDK repo build:
D:\a\_work\1\vmr\src\sdk\test\Microsoft.NET.Sdk.BlazorWebAssembly.Tests\Microsoft.NET.Sdk.BlazorWebAssembly.Tests.csproj error NU1603: Warning As Error: Microsoft.NET.Test.Sdk 17.14.0-ci depends on Microsoft.TestPlatform.TestHost (>= 17.14.0-ci) but Microsoft.TestPlatform.TestHost 17.14.0-ci was not found. Microsoft.TestPlatform.TestHost 17.14.0-preview-25071-04 was resolved instead. [D:\a\_work\1\vmr\src\sdk\sdk.sln]

I have not seen this in local build, but I have not refreshed the local sha in a week. I'm updating local sha to repro this.

There were some recent changes in versioning i.e. the use of official or dev labels. @ViktorHofer @mmitche does this sound familiar?

I'm not sure which repo builds Microsoft.TestPlatform.TestHost - I don't see this produced in local build and darc tool is not working for me atm. Perhaps a new dependency that should be added in vstest or sdk.

mmitche · 2025-02-03T18:05:56Z

Hmm, this PR is now hitting an odd error that I don't think is related.

It's failure to restore vstest package during SDK repo build: D:\a\_work\1\vmr\src\sdk\test\Microsoft.NET.Sdk.BlazorWebAssembly.Tests\Microsoft.NET.Sdk.BlazorWebAssembly.Tests.csproj error NU1603: Warning As Error: Microsoft.NET.Test.Sdk 17.14.0-ci depends on Microsoft.TestPlatform.TestHost (>= 17.14.0-ci) but Microsoft.TestPlatform.TestHost 17.14.0-ci was not found. Microsoft.TestPlatform.TestHost 17.14.0-preview-25071-04 was resolved instead. [D:\a\_work\1\vmr\src\sdk\sdk.sln]

I have not seen this in local build, but I have not refreshed the local sha in a week. I'm updating local sha to repro this.

There were some recent changes in versioning i.e. the use of official or dev labels. @ViktorHofer @mmitche does this sound familiar?

I'm not sure which repo builds Microsoft.TestPlatform.TestHost - I don't see this produced in local build and darc tool is not working for me atm. Perhaps a new dependency that should be added in vstest or sdk.

That looks like a new package dependency that is not being built. It's built out of vstest normally: https://github.com/microsoft/vstest/blob/main/src/package/Microsoft.TestPlatform.TestHost/Microsoft.TestPlatform.TestHost.csproj#L7-L8

mmitche · 2025-02-03T18:13:15Z

@NikolaMilosavljevic #46480

mmitche · 2025-02-03T20:12:02Z

@NikolaMilosavljevic #46480

@NikolaMilosavljevic vstest just doesn't build on non-Windows. I'm updating the conditional to pack on Windows too, but we should disable building the test on non-Windows.

NikolaMilosavljevic · 2025-02-03T20:19:55Z

@NikolaMilosavljevic #46480

@NikolaMilosavljevic vstest just doesn't build on non-Windows. I'm updating the conditional to pack on Windows too, but we should disable building the test on non-Windows.

vstest repo gets built as part of source-build and Linux unified build. I presume we should not build vstest test projects on non-Windows, right?

mmitche · 2025-02-03T20:30:46Z

@NikolaMilosavljevic #46480

@NikolaMilosavljevic vstest just doesn't build on non-Windows. I'm updating the conditional to pack on Windows too, but we should disable building the test on non-Windows.

vstest repo gets built as part of source-build and Linux unified build. I presume we should not build vstest test projects on non-Windows, right?

This looks different. it looks like an SDK test project, that is referencing a vstest package that is not produced on Linux.

NikolaMilosavljevic · 2025-02-03T20:47:21Z

@NikolaMilosavljevic #46480

@NikolaMilosavljevic vstest just doesn't build on non-Windows. I'm updating the conditional to pack on Windows too, but we should disable building the test on non-Windows.

vstest repo gets built as part of source-build and Linux unified build. I presume we should not build vstest test projects on non-Windows, right?

This looks different. it looks like an SDK test project, that is referencing a vstest package that is not produced on Linux.

I see - there are 60+ sdk tests that need this vstest package. Perhaps it's best to not allow sdk repo to build tests on non-Windows.

I'll create a tracking issue in source-build.

NikolaMilosavljevic · 2025-02-05T15:59:41Z

I've merged the latest from main to pick up #46480 - this should resolve the last build issue.

NikolaMilosavljevic · 2025-02-05T20:07:44Z

First successful build of the '_BuildTests' leg - it took 106 minutes. @ViktorHofer

We are skipping tests in several repos, i.e. aspnetcore, so that will add some time, but hopefully keep it below 2 hours limit.

There is a failure in repo's test leg, which is unrelated.

eng/pipelines/templates/stages/vmr-build.yml

ViktorHofer · 2025-02-05T21:12:03Z

@NikolaMilosavljevic changes LGTM

NikolaMilosavljevic · 2025-02-06T15:54:44Z

macOS test legs are timing out. This is unrelated as the PR is not affecting the repo build.

@mmitche @MichaelSimons @marcpopMSFT can someone merge this? I don't have permissions.

Add VMR leg that builds test projects

b1fdc5e

Fixes dotnet/source-build#4168 Running as part of the PR to get some initial results.

ViktorHofer requested review from a team as code owners November 13, 2024 13:08

dotnet-issue-labeler bot added Area-Infrastructure untriaged Request triage from a team member labels Nov 13, 2024

am11 mentioned this pull request Nov 13, 2024

Fix nullability error CS8620 in tests dotnet/winforms#12475

Merged

Merge branch 'main' into VMRDotNetBuildTestsLeg

a9708d9

am11 mentioned this pull request Nov 19, 2024

Fix build with latest roslyn dotnet/efcore#35149

Merged

Merge branch 'main' into VMRDotNetBuildTestsLeg

efaa5f3

Merge branch 'main' into VMRDotNetBuildTestsLeg

9de5c8c

Merge branch 'main' into VMRDotNetBuildTestsLeg

3341ca8

NikolaMilosavljevic added 4 commits December 10, 2024 09:01

Merge branch 'main' into VMRDotNetBuildTestsLeg

16955cb

Merge branch 'main' into VMRDotNetBuildTestsLeg

842a50a

Merge branch 'main' into VMRDotNetBuildTestsLeg

5c93ec6

Merge branch 'main' into VMRDotNetBuildTestsLeg

60a2506

ViktorHofer commented Jan 30, 2025

View reviewed changes

eng/pipelines/templates/stages/vmr-build.yml Outdated Show resolved Hide resolved

NikolaMilosavljevic added 2 commits January 30, 2025 11:31

Merge branch 'main' into VMRDotNetBuildTestsLeg

8517d15

Move the job to PR section

e99b723

build-analysis bot mentioned this pull request Jan 30, 2025

dotnet-watch tests failing #40006

Open

NikolaMilosavljevic mentioned this pull request Jan 31, 2025

WinForms tests are not building in VMR dotnet/source-build#4875

Open

NikolaMilosavljevic added 3 commits January 31, 2025 10:33

Opt-out of winforms tests build

2a71f65

Merge branch 'main' into VMRDotNetBuildTestsLeg

5ae460f

Merge branch 'main' into VMRDotNetBuildTestsLeg

8a09027

NikolaMilosavljevic mentioned this pull request Feb 3, 2025

Microsoft.TestPlatform.TestHost does not build on non-Windows dotnet/source-build#4882

Closed

Do not build sdk tests on non-Windows platforms

9f8dde9

build-analysis bot mentioned this pull request Feb 4, 2025

Template engine tests hitting known parallelism issue in GetVisualStudioInstances #44878

Open

Merge branch 'main' into VMRDotNetBuildTestsLeg

5a41076

ViktorHofer commented Feb 5, 2025

View reviewed changes

eng/pipelines/templates/stages/vmr-build.yml Outdated Show resolved Hide resolved

Update eng/pipelines/templates/stages/vmr-build.yml

6263863

ViktorHofer marked this pull request as ready for review February 5, 2025 21:11

NikolaMilosavljevic approved these changes Feb 5, 2025

View reviewed changes

MichaelSimons merged commit b513b97 into main Feb 6, 2025
34 of 38 checks passed

MichaelSimons deleted the VMRDotNetBuildTestsLeg branch February 6, 2025 16:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add VMR leg that builds test projects #44843

Add VMR leg that builds test projects #44843

ViktorHofer commented Nov 13, 2024

ViktorHofer commented Nov 13, 2024 •

edited

Loading

am11 commented Nov 13, 2024

ViktorHofer commented Nov 19, 2024

am11 commented Nov 19, 2024

ViktorHofer commented Nov 20, 2024

am11 commented Nov 20, 2024

ViktorHofer commented Nov 21, 2024

NikolaMilosavljevic commented Dec 3, 2024

ViktorHofer commented Dec 4, 2024 •

edited

Loading

mmitche commented Dec 4, 2024

mmitche commented Dec 4, 2024

mmitche commented Dec 4, 2024

NikolaMilosavljevic commented Jan 31, 2025

NikolaMilosavljevic commented Feb 3, 2025

mmitche commented Feb 3, 2025

mmitche commented Feb 3, 2025

mmitche commented Feb 3, 2025

NikolaMilosavljevic commented Feb 3, 2025

mmitche commented Feb 3, 2025

NikolaMilosavljevic commented Feb 3, 2025 •

edited

Loading

NikolaMilosavljevic commented Feb 5, 2025

NikolaMilosavljevic commented Feb 5, 2025

ViktorHofer commented Feb 5, 2025

NikolaMilosavljevic commented Feb 6, 2025

Add VMR leg that builds test projects #44843

Add VMR leg that builds test projects #44843

Conversation

ViktorHofer commented Nov 13, 2024

ViktorHofer commented Nov 13, 2024 • edited Loading

am11 commented Nov 13, 2024

ViktorHofer commented Nov 19, 2024

am11 commented Nov 19, 2024

ViktorHofer commented Nov 20, 2024

am11 commented Nov 20, 2024

ViktorHofer commented Nov 21, 2024

NikolaMilosavljevic commented Dec 3, 2024

ViktorHofer commented Dec 4, 2024 • edited Loading

mmitche commented Dec 4, 2024

mmitche commented Dec 4, 2024

mmitche commented Dec 4, 2024

NikolaMilosavljevic commented Jan 31, 2025

NikolaMilosavljevic commented Feb 3, 2025

mmitche commented Feb 3, 2025

mmitche commented Feb 3, 2025

mmitche commented Feb 3, 2025

NikolaMilosavljevic commented Feb 3, 2025

mmitche commented Feb 3, 2025

NikolaMilosavljevic commented Feb 3, 2025 • edited Loading

NikolaMilosavljevic commented Feb 5, 2025

NikolaMilosavljevic commented Feb 5, 2025

ViktorHofer commented Feb 5, 2025

NikolaMilosavljevic commented Feb 6, 2025

ViktorHofer commented Nov 13, 2024 •

edited

Loading

ViktorHofer commented Dec 4, 2024 •

edited

Loading

NikolaMilosavljevic commented Feb 3, 2025 •

edited

Loading