Normative: Add RegExp Modifiers

[rbuckton]

This adds the specification text for the Stage 3 RegExp Modifiers proposal.

Test262 tests can be found at tc39/test262#3960.

[michaelficarra]

Is there a reason we use the RegularExpressionFlags production and restrict it with an early error instead of introducing a new, more restricted production?

[rbuckton]

Is there a reason we use the RegularExpressionFlags production and restrict it with an early error instead of introducing a new, more restricted production?

Is there a reason not to? Modifiers are a strict subset of RegularExpressionFlags, and RegularExpressionFlags itself is not restricted in the grammar, only via semantics. If I were to create a RegularExpressionModifiers production, it would have the same definition as RegularExpressionFlags.

[michaelficarra]

I believe RegularExpressionFlags is not restricted in the grammar because RegularExpressionLiteral needs to not change when we add new flags. This is due to the overlap with division. We don't have that same constraint with the modifiers grammar though, so it should be able to be done with grammar restrictions and not early errors.

[rbuckton]

I believe RegularExpressionFlags is not restricted in the grammar because RegularExpressionLiteral needs to not change when we add new flags. This is due to the overlap with division. We don't have that same constraint with the modifiers grammar though, so it should be able to be done with grammar restrictions and not early errors.

How would you propose it be written so as not to require early errors? Modifiers can appear in any order, cannot be duplicated within one or both of the modifier locations, and I do plan to extend the set of allowed modifiers over time with things like x-mode, so it needs to be fairly flexible. Even if we limit it to a subset of characters like i, m, and s for now, we either need an early error for duplicates, or a complex grammar like:

RegularExpressionModifiers:
  RegularExpressionModifierChars[+IgnoreCase, +Multiline, +DotAll]

RegularExpressionModifierChars[IgnoreCase, Multiline, DotAll]:
  [empty]
  [+IgnoreCase] `i` RegularExpressionModifierChars[~IgnoreCase, ?Multiline, ?DotAll]?
  [+Multiline] `m` RegularExpressionModifierChars[?IgnoreCase, ~Multiline, ?DotAll]?
  [+DotAll] `s` RegularExpressionModifierChars[?IgnoreCase, ?Multiline, ~DotAll]?

The more modifiers we add, the more production parameters we need, and the more complex the production becomes. Plus, this doesn't avoid the need for an early error to forbid (?i-i:).

[michaelficarra]

@rbuckton I'm not saying we can't use any early errors at all on the production, just that we don't need to use early errors to enforce the allowed flag characters when an alternative grammar could do the job. I'm happy to continue using early errors to check for duplicates.

[rbuckton]

Now that both V8 and SpiderMonkey are shipping regex modifiers, I'm hoping to request Stage 4 at the October plenary. As such I'm working on updating this PR so that it is in a mergeable state.

@michaelficarra do you still want me to refactor modifiers to something other than RegularExpressionFlags?

[rbuckton]

@michaelficarra I went ahead and added a RegularExpressionModifiers production that just restricts the allowed characters to ims, and continues to use an early error to check for duplicates.

[JLHwung]

Should we just throw the early error when the second |RegularExpressionModifiers| is empty regardless of whether the first modifiers are empty? For example, currently the spec allows /(?i-:foo)/, but the implementation differs: V8 accepts this production while Babel rejects it.

[rbuckton]

Possibly? We only error here because we decided to be more pedantic about modifiers than other languages. C#, for example, seems to parse (?-:) just fine. Any change here would need consensus.

[graphemecluster]

I remembered that I was wondering the same (why the start constant is placed here) when I was reviewing my RegExp check implementation for TypeScript.
I believe that Huáng’s expectation is actually more ergonomic, otherwise it would not be so easy for the current behavior to be misimplemented.

[michaelficarra]

I guess technically we could accomplish this entirely within the grammar, but this is a fine use of early errors.

[michaelficarra]

LGTM

[rbuckton]

Outside of updating from main, is there anything preventing this from being merged now that it has Stage 4?

[michaelficarra]

@rbuckton It's waiting on editorial review from @syg.

[syg]

lgtm % question about strings