ensure the path regexes will accept all valid paths #48686
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, we might try to interpret the random bytes in a path as
UTF-8 and excluding \n, causing the regex match to fail or be incomplete
in some cases. But those are valid in a path, so we want PCRE2 to treat
them as transparent bytes. Accordingly, change r""a to specify all flags
needed to interpret the values simply as ASCII.
Note, this would be breaking if someone was previously trying to match a
Unicode character by `\u` while also disabling UCP matching of \w and
\s, but that seems an odd specific choice to need.
julia> match(r"\u03b1"a, "α")
ERROR: PCRE compilation error: character code point value in \u.... sequence is too large at offset 6
(this would have previously worked). Note that explicitly starting the
regex with (*UTF) or using a literal α in the regex would continue to
work as before however.
Note that `s` (DOTALL) is a more efficient matcher (if the pattern
contains `.`), as is `a`, so it is often preferable to set both when in
doubt: http://man.he.net/man3/pcre2perform
Refs: #48648
38fbd36 to
b5ce7ca
Compare
vtjnash
commented
Feb 27, 2023
| @@ -171,6 +171,9 @@ | |||
| @test string(splitdrive(S(homedir()))...) == homedir() | |||
| @test splitdrive("a\nb") == ("", "a\nb") | |||
|
|
|||
| @test splitdir("a/\xfe/\n/b/c.ext") == ("a/\xfe/\n/b", "c.ext") | |||
| @test splitext("a/\xfe/\n/b/c.ext") == ("a/\xfe/\n/b/c", ".ext") | |||
There was a problem hiding this comment.
Thought someone might be interested to note that this had previously been encountered as a bug and fixed only for splitdrive and only for windows by #32543
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously, we might try to interpret the random bytes in a path as UTF-8 and excluding \n, causing the regex match to fail or be incomplete in some cases. But those are valid in a path, so we want PCRE2 to treat them as transparent bytes. Accordingly, change r""a to specify all flags needed to interpret the values simply as ASCII.
Note, this would be breaking if someone was previously trying to match a Unicode character by
\uwhile also disabling UCP matching of \w and \s, but that seems an odd specific choice to need.(this would have previously worked). Note that explicitly starting the regex with (*UTF) or using a literal α in the regex would continue to work as before however.
Note that
s(DOTALL) is a more efficient matcher (if the pattern contains.), as isa, so it is often preferable to set both when in doubt: http://man.he.net/man3/pcre2performRefs: #48648