[release/7.0] Fix non-determinism in Regex source generator #78149
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The source generator enumerates a Hashtable to write out its contents. When the keys of the Hashtable are strings, string hash code randomization may result in the order of that enumeration being different in different processes, leading to non-deterministic ordering of values written out and thus non-deterministic source generator output.
|
Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions Issue DetailsBackport of #78103 to release/7.0 /cc @stephentoub Customer ImpactTestingRiskIMPORTANT: Is this backport for a servicing release? If so and this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.
|
joperezr
approved these changes
Nov 10, 2022
|
Signed off by area owners and approved by Tactics. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of #78103 to release/7.0
/cc @stephentoub
Customer Impact
Non-deterministic build output, such that compiling the same inputs multiple times may result in different bits in the resulting binaries. The source generator has a Hashtable built up by the Regex implementation, and that Hashtable is then enumerated to write its contents into the C# source being generated. That Hashtable has string keys. In .NET Core, string hash codes are randomized, such that a string's hash code potentially changes from process to process. And Hashtable's enumeration order is influenced by the hash codes of its keys, as it enumerates its internal buckets and the hash codes impact which bucket a string is slotted into. The fix is to sort as part of enumerating the Hashtable so we always get a consistent, reproducible order.
Testing
New tests + existing tests run locally and in CI.
Risk
Low. It's just using OrderBy to sort the data being foreach'd.