-
Notifications
You must be signed in to change notification settings - Fork 1.6k
<regex>: Properly parse backslashes in character classes of basic regexes
#5523
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
<regex>: Properly parse backslashes in character classes of basic regexes
#5523
Conversation
…arse flag for forbidden dash at range start
|
Reviewing now, I'll push changes soon. I updated the PR description from saying " |
Use muellerj2's superior descriptions of `_L_alt_nl` and `_L_no_nl`. Note that `_L_no_nl` is (grep, egrep). Note that `_L_esc_oct` and `_L_esc_ffnx` are (awk). `_L_esc_ffn` confusingly said "(\[fnrtv])" when other comments like `_L_ident_ERE` mean square brackets literally. Spell out "(\f \n \r \t \v)" for clarity and improved searchability. Rephrase `_L_ident_awk`'s comment for clarity. Note that `_L_anch_rstr` is (BRE) only, `_L_paren_bal` is (ERE) only, and `_L_brk_rstr` is (ERE, BRE).
|
Thanks!! 😻 I pushed some follow-up commits for additional cleanups, please double-check. I really appreciate the well-structured commit history here; ordinarily I would be nervous about mixing a refactoring and a bugfix but this was entirely reasonable. |
|
I'm mirroring this to the MSVC-internal repo - please notify me if any further changes are pushed. |
|
Thanks for the infinite bugfixes in infinite combinations, as the Vulcans say! 🖖 🐞 🛠️ |
Fixes #5379.
This renames the parser to add a new member variable describing the lexer mode: Default or inside character class. This allows the lexer to correctly process a backslash when parsing a character class/bracket expression.
I also tried to do this without renaming the parser, but this would mean we would have to pass the lexer mode (in or outside a character class) as an argument to all the functions processing escapes in any way, which is a bit of a pain. By renaming the parser, we need the least changes to the logic itself.
Since the parser is renamed, this PR is also doing a number of minor cleanups to the parser and builder (which is also renamed to do these cleanups).
The PR is split into several commits to simplify reviewing:
_Parserto_Parser2._Compile. Specifically:_L_brk_balis assigned its own bit; previously it wasSTL/stl/inc/regex
Line 1879 in cbd091e
_L_grp_escflag is added to the awk flags so that the workaround in_ClassAtomcan be removed._Lang_flagsenum and the_L_flagsmember variable tounsigned long longso that we can add more flags more easily in the future. (This already adds_L_dsh_rstrto signify that the dash-cannot appear as the starting point of a character range in BREs and EREs, but doesn't perform the parser changes to support it yet.)_Beginfrom the parser._Charis usually acharorwchar_t, so it [plus the single-byte_Modemember variable added in the last commit] can usually fit into the four bytes the compiler must add after_Mchar._Builderto_Builder2._Bmaxand_Tmaxfrom the builder.<regex>: Backslashes in character classes are sometimes not matched in basic regular expressions #5379 essentially by making_Is_esc()always return false when not in default (read: outside-bracketed-character-class) mode. Note that it matters how we change the lexer mode in_Parser2::_Alternative():_Next()and_Expect()process the first token inside or outside the square brackets, so we must change the mode before calling these functions. The tests check that we didn't get this wrong.