-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add test leg to run runtime tests on Android arm64 #44560
Conversation
Tagging subscribers to this area: @directhex Issue meta data
|
3044595
to
19cde2c
Compare
d550f4c
to
f587498
Compare
…hen building Android apps
8a74d1e
to
3f05be1
Compare
@safern - this is a new lane too, what needs to be done to add it to the staging pipeline first ? |
@SamMonoRT So, my understanding is basically that to add to staging, you add the lanes to runtime-staging.yml instead of runtime.yml. You can use @safern's original PR moving the android emulator legs as an example: https://github.com/dotnet/runtime/pull/45178/files |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, thank you! Please make sure to revert the instrumentation changes to runtime.yml before merging in :-).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add it to runtime-staging
pipeline first to make sure it is stable before we move it to the main pipeline.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's put it in staging first as @SamMonoRT suggested. Other than that, looks good.
According to the log here. On some Android devices, it took ~2.5min to install an Android app (size: 11.4 Mb). This work item contains one test wrapper, which includes 82 tests. That means this work item does 82 times app installation, which is more than 3 hours. Some devices are faster than the others. It seems to me that the devices were picked randomly for each work item. So the best solution is to avoid unnecessary installations. For each test wrapper, all tests use the same Android app. @akoeplinger suggested to install/uninstall once in the initialize/cleanup methods for each test wrapper. That would require this functionality(dotnet/xharness#397) in place. So for this PR, I will probably have to disable quite a few tests. And enable them when that functionality is ready to use. |
<!-- Set Test Wrapper running host OS --> | ||
<PropertyGroup> | ||
<TestWrapperTargetsWindows>false</TestWrapperTargetsWindows> | ||
<TestWrapperTargetsWindows Condition=" ('$(TargetsWindows)' != '' And '$(TargetsWindows)' ) OR ('$(TargetOS)' == 'Android' And '$(TargetArchitecture)' == 'arm64' )">true</TestWrapperTargetsWindows> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we use ".cmd" scripts on arm64 android?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the trick is that, while traditional CoreCLR builds deal with a pair of platforms (hosting / targeting), for Mono device testing there's a triplet of platforms - host, test driver, target. Arm64 Android tablets use Windows test drivers so at that level yes, they use ".cmd" scripts.
This work contributes to the project of bringing up test coverage for Mono (#33381)
488 runtime tests ran and passed on this newly added CI lane. 1856 runtime tests were disabled for this lane, due to long running work items and oversized Android apps. Created two issues to track those two issues separately. (#45965 and #45568)