[ci] Remove separate SmokeTests jobs. #7872
Conversation
17f5d46 to
0d62b6c
Compare
0d62b6c to
d265580
Compare
| # MSBuild Smoke Tests - Win | ||
| - template: yaml-templates/run-msbuild-tests.yaml | ||
| parameters: | ||
| testOS: Windows | ||
| jobName: win_smoke_msbuild_tests | ||
| jobDisplayName: Windows > Tests > MSBuild | ||
| agentCount: 2 | ||
| testFilter: cat = SmokeTests | ||
| installApkDiff: true |
There was a problem hiding this comment.
I think this step built the repo on Windows. Would we lose anything there?
Could we potentially break the build on Windows and not know until someone tries it?
There was a problem hiding this comment.
I believe you are referring to the Windows build + run some in-tree tests on it, which is this stage:
It will still run here.
The tests that were previously built by the job(s) you referenced are simply moved to the MSBuild stage Windows test.
There was a problem hiding this comment.
The windows build stage (and testing of in-tree build artifacts rather than nugets) is part of https://github.com/xamarin/xamarin-android/blob/452e742e214cad42f2f30ea74097e422e49c04e3/build-tools/automation/yaml-templates/build-windows.yaml and will still be covered. This job was only running SmokeTests from Xamarin.Android.Build.Tests
|
|
||
| - template: run-xaprepare.yaml | ||
| parameters: | ||
| arguments: --s=DetermineApplicableTests |
There was a problem hiding this comment.
So we could probably delete --s=DetermineApplicableTests in xaprepare, but maybe we don't have to do that here.
* main: [AOT] provide libc and libm stubs so NDK is not required (dotnet#7475) [MSBuildDeviceIntegration] Handle Debugger errors (dotnet#7864) [ci] Remove separate SmokeTests jobs. (dotnet#7872) Bump to dotnet/installer@cddf8e6 8.0.100-preview.3.23163.4 (dotnet#7885) [Actions] Do not double quote the commit message (dotnet#7887) [Actions] Ensure that the commit message is valid json. (dotnet#7884) [github] Use ".NET Android", not "Android for .NET" (dotnet#7882) Bump `$(AndroidNet7Version)` (dotnet#7883) [ci] Install fewer Android SDK platforms on test agents. (dotnet#7874)
* main: [AOT] provide libc and libm stubs so NDK is not required (dotnet#7475) [MSBuildDeviceIntegration] Handle Debugger errors (dotnet#7864) [ci] Remove separate SmokeTests jobs. (dotnet#7872) Bump to dotnet/installer@cddf8e6 8.0.100-preview.3.23163.4 (dotnet#7885) [Actions] Do not double quote the commit message (dotnet#7887) [Actions] Ensure that the commit message is valid json. (dotnet#7884) [github] Use ".NET Android", not "Android for .NET" (dotnet#7882) Bump `$(AndroidNet7Version)` (dotnet#7883) [ci] Install fewer Android SDK platforms on test agents. (dotnet#7874)
* main: [AOT] provide libc and libm stubs so NDK is not required (dotnet#7475) [MSBuildDeviceIntegration] Handle Debugger errors (dotnet#7864) [ci] Remove separate SmokeTests jobs. (dotnet#7872) Bump to dotnet/installer@cddf8e6 8.0.100-preview.3.23163.4 (dotnet#7885) [Actions] Do not double quote the commit message (dotnet#7887) [Actions] Ensure that the commit message is valid json. (dotnet#7884)
Context: #7850 (comment)
With the re-parallelization of our test suites (#7804, #7850), combined with the fact that most PR's end up running the full test suite anyways, the separate "Smoke Tests" jobs aren't really saving us much, if any, CI time. We feel we would be better off using those additional agents to run the full test suite faster.
This PR removes the separate "Smoke Tests" jobs and runs all tests in the same jobs.
Additionally, as the
[Category ("SmokeTests")]NUnit attribute is now only used by the Windows build stage to run in-tree tests, it has been cut back to the absolute bare minimum, saving ~45 minutes. Note that all the tests will continue to be run against the Windows installer, this only runs fewer tests for the Windows in-tree tests.