Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GivenDotnetTest Failures #46923

Open
nagilson opened this issue Feb 18, 2025 · 0 comments
Open

GivenDotnetTest Failures #46923

nagilson opened this issue Feb 18, 2025 · 0 comments

Comments

@nagilson
Copy link
Member

nagilson commented Feb 18, 2025

In the past week, we've seen a number of failures from sdk\test\dotnet-test.Tests\GivenDotnetTestBuildsAndRunsTests.cs -- it is failing in main almost 80% of the time in one test or another. This is a relatively new issue as they're new.

For example, we saw RunMultipleTestProjectsWithDifferentFailures_ShouldReturnOneAsExitCode fail with the following:

   Assert.Matches() Failure: Pattern not found in value
Regex: ".+Debug\\\\net10.0\\\\TestProject\\.dll\\s+\\(net1"···
Value: "  Determining projects to restore...\r\n  Restored"···

And another example is ``RunMultipleTestProjectsWithDifferentFailures_ShouldReturnExitCodeGenericFailure`, which failed here:
#46916

net 00:01:14.37]     Microsoft.DotNet.Cli.Test.Tests.GivenDotnetTestBuildsAndRunsTests.RunMultipleTestProjectsWithDifferentFailures_ShouldReturnExitCodeGenericFailure(configuration: "Release") [FAIL]
[xUnit.net 00:01:14.37]       Assert.Matches() Failure: Pattern not found in value
[xUnit.net 00:01:14.37]       Regex: ".+Release\\\\net10.0\\\\OtherTestProject\\.dll\\s+"···
[xUnit.net 00:01:14.37]       Value: "Running tests from C:\\h\\w\\B0DB094E\\t\\dotnetSd"···
[xUnit.net 00:01:14.37]       Stack Trace:

cc @mariam-abdulla when you have an opportunity, please take a look at these tests. I will be disabling them for now because they are blocking PRs. Thanks for your help!

@dotnet-issue-labeler dotnet-issue-labeler bot added Area-dotnet test untriaged Request triage from a team member labels Feb 18, 2025
@nagilson nagilson added Infrastructure and removed untriaged Request triage from a team member labels Feb 18, 2025
nagilson added a commit that referenced this issue Feb 18, 2025
Please see #46923 for more information.
nagilson added a commit to nagilson/sdk that referenced this issue Feb 21, 2025
In this PR a few days ago I disabled some similar tests:
dotnet#46924

It turns out that even more of them are failing about 80%+ of the time
after some recent changes to these tests.

See here: dotnet#47003 (review)
here: dotnet#47004 (review)
here: https://github.com/dotnet/sdk/pull/47004/checks?check_run_id=37586911841
here: dotnet#47014
etc.

I've already made an issue to track this: dotnet#46923
So I'll keep using that one. The maintainer has been notified.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants