Switch to full NGEN #6764
Switch to full NGEN #6764
Conversation
I don't see IBC data generated for MSBuild.exe here: internal link Do you happen to know how the set of assemblies to profile with OptProf is defined? |
|
I don't, unfortunately. This is promising but blank: msbuild/eng/config/OptProf.json Lines 35 to 36 in c88325c |
|
I've pushed a shot in the dark commit. Will have results tomorrow. |
| @@ -10,10 +10,12 @@ vs.dependencies | |||
| vs.dependency id=Microsoft.VisualStudio.PackageGroup.NuGet | |||
|
|
|||
| vs.relatedProcessFiles | |||
| vs.relatedProcessFile Path="[InstallDir]\MSBuild\Current\Bin\MSBuild.exe" | |||
There was a problem hiding this comment.
As I recall relatedProcessFile means "use the RestartManager API to see if this file is in use and if so complain to the user rather than failing to overwrite it during setup/upgrade."
There was a problem hiding this comment.
Yup, it didn't work.
@genlu, may I ask for your expert advice here? Does setting the ApplyNgenOptimization prop enable OptProf end-to-end or is there some other place where we indicate for which assemblies optimization data should be created?
This is our OptProf training pipeline: internal link
There was a problem hiding this comment.
ApplyNgenOptimization doesn't do it, we need to enable generating optimization data for the assembly first.
##[error].packages\microsoft.dotnet.arcade.sdk\6.0.0-beta.21379.2\tools\OptimizationData.targets(111,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) No optimization data found for assemblies: C:\a\1\s\artifacts\obj\MSBuild\Release\net472\MSBuild.exe
Let's get this in then and work on MSBuild.exe separately.
There was a problem hiding this comment.
Sorry I missed your question, notification from msbuild repo is being filtered out from my inbox. Please let me know if there's anything else I could help :)
There was a problem hiding this comment.
@genlu yes, the question is still valid. How does the OptProf infrastructure determine for which assemblies it should create optimization data? We noticed that it's not being done for MSBuild.exe and we would like to fix it. Thank you!
There was a problem hiding this comment.
This is specified in our optprof runsetting file, which points to the config files we update with each of our insertion (in the directory shown in the screenshot below)
The content of those config files are auto-generated based on this file during our build.
| @@ -20,7 +20,7 @@ | |||
| <IsPackable>true</IsPackable> | |||
| <PackageDescription>This package contains the $(MSBuildProjectName) assembly which is used to create, edit, and evaluate MSBuild projects.</PackageDescription> | |||
| <IncludeSatelliteOutputInPack>false</IncludeSatelliteOutputInPack> | |||
| <ApplyNgenOptimization Condition="'$(TargetFramework)' == '$(FullFrameworkTFM)'">partial</ApplyNgenOptimization> | |||
| <ApplyNgenOptimization Condition="'$(TargetFramework)' == '$(FullFrameworkTFM)'">full</ApplyNgenOptimization> | |||
There was a problem hiding this comment.
Why are all these only on full?
I was momentarily confused as to why you'd have to change it for four projects individually rather than just one change for them all, but it doesn't really make sense to ngen anything from other assemblies, so 👍
There was a problem hiding this comment.
Why are all these only on full?
ngen is only a thing on .NET Framework; in .NET Core it was replaced with "crossgen" (and now crossgen2) which has a different set of rules.
There was a problem hiding this comment.
I think you've told me that before...I'll try to remember this time.
|
For the record, I'm fine with taking this (without the MSBuild.exe changes) and then separately figuring out MSBuild.exe training data. |
|
We got buy-off for this part (that is, without MSBuild.exe changes) from the VS perf team, right? So this is good to merge when possible? |
|
Yes, please merge. |
Context
We are currently pre-compiling only code that gets executed as part of our IBC training scenarios. This results in smaller native images but the coverage is not perfect and it's easy to miss a code path and cause JITting at run-time.
Changes Made
With Visual Studio switching to 64-bit, address space is no longer a concern and the positive impact of pre-compiling everything outweighs the cost of larger image sizes.
Testing
Experimental VS insertion showing improvements in # of methods JITted and wall-clock time.