Ensure parameterized tests don't produce overlapping snapshot files. #267
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The current escaping function replaces non-alphanumeric characters with the "_" character to ensure they are compatible with the filesystem's constrainted. However, this normalization can cause a problem when two separate parameterized tests point at the same snapshot file after this normalization step. This change modifies the normalization function to replace these characters with either their Unicode name or hexidecimal escape code. The resulting filenames are a bit verbose, but this approach ensures that these snapshots are always creating unique files. If you revert the normalization function change, you'll note that the "overlap]" and "overlap[" tests in `test_fmt.py` will refer to the same snapshot file and the tests will fail and/or be flaky.
Back to using the old character replacement algorithm again because filenames were getting too long. Now putting a shake_128 hash on the end of every snapshot file name to deal with any collisions. This ends up only being 8 characters, which seems like a small price to pay in order to ensure there are never any conflicts.
|
I ended up hitting issues with the old approach because filenames got too long. So now I've gone back to the same escaping algorithm that was there, but added a hash suffix of 8 characters that should break ties in the event the prefix is identical due to escaping. |
Before this change it was hashing the escaped version of the strings, which doesn't actually do anything to prevent collisions.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current escaping function replaces non-alphanumeric characters with the "_" character to ensure they are compatible with the filesystem's constraints. However, this normalization can cause a problem when two separate parameterized tests point at the same snapshot file after this normalization step.
This change modifies the normalization function to replace these characters with either their Unicode name or hexidecimal escape code. The resulting filenames are a bit verbose, but this approach ensures that these snapshots are always creating unique files.
If you revert the normalization function change, you'll note that the "overlap]" and "overlap[" tests in
test_fmt.pywill refer to the same snapshot file and the tests will fail and/or be flaky.