Add rules that "eslint-plugin-clean-regex" has

[ota-meshi]

Check each rule that eslint-plugin-clean-regex has, add the missing rule, and if eslint-plugin-regexp already has a similar rule, check for missing validation is needed.
Also, if we keep a note of the rule correspondence, it may be a document when creating a migration tool.

If we want to discuss one rule, we can also open a new issue.

Problems

• clean-regex/confusing-quantifier ... Warn about confusing quantifiers.
  • Add regexp/confusing-quantifier rule #93 Add new rule regexp/confusing-quantifier.
• clean-regex/disjoint-alternatives ... Disallow different alternatives that can match the same words.
  • Handled by Updated no-dupe-disjunctions #179.
• clean-regex/no-empty-alternative ... Disallow alternatives without elements.
  • Add regexp/no-empty-alternative rule #95 Add new rule regexp/no-empty-alternative.
• clean-regex/no-empty-backreference ... Disallow backreferences that will always be replaced with the empty string.
  • no-useless-backreference: Made compatible with clean-regex/no-empty-backreference #111 makes no-useless-backreference functionally equivalent to clean-regex/no-empty-backreference.
• clean-regex/no-empty-lookaround ... Disallow lookarounds that can match the empty string.
  • Similar to regexp/no-empty-lookarounds-assertion, but it doesn't catch potential empty assertions. regexp/no-empty-lookarounds-assertion needs to be changed. (no-empty-lookarounds-assertion: Better detection #113)
• clean-regex/no-lazy-ends ... Disallow lazy quantifiers at the end of an expression.
  • Add regexp/no-lazy-ends rule #96 Add new rule regexp/no-lazy-ends.
• clean-regex/no-obscure-range ... Disallow obscure ranges in character classes.
  • Added regexp/no-obscure-range rule #122 adds this rule.
• clean-regex/no-octal-escape ... Disallow octal escapes outside of character classes.
  • Similar to regexp/no-octal, but with some differences. We probably need to add options.
  • Changed no-octal to report more octal escapes #108 makes regexp/no-octal even stricter than clean-regex/no-octal-escape.
• clean-regex/no-optional-assertion ... Disallow optional assertions.
  • Add regexp/no-optional-assertion rule #135 Add new rule regexp/no-optional-assertion.
• clean-regex/no-potentially-empty-backreference ... Disallow backreferences that reference a group that might not be matched.
  • Add regexp/no-potentially-useless-backreference rule #136 Add new rule regexp/no-potentially-useless-backreference.
• clean-regex/no-unnecessary-assertions ... Disallow assertions that are known to always accept (or reject).
  • Add regexp/no-useless-assertions rule #137 Add new rule regexp/no-useless-assertions.
• clean-regex/no-zero-quantifier ... Disallow quantifiers with a maximum of 0.
  • Can check with the regexp/no-useless-exactly-quantifier rule.
  • Deprecate regexp/no-useless-exactly-quantifier rule #199 & Add no-zero-quantifier rule #200
• clean-regex/optimal-lookaround-quantifier ... Disallows the alternatives of lookarounds that end with a non-constant quantifier.
  • Add regexp/optimal-lookaround-quantifier rule #97 Add new rule regexp/optimal-lookaround-quantifier.

Suggestions

• clean-regex/consistent-match-all-characters ... Use one character class consistently whenever all characters have to be matched.
  • Fixed match-any false negatives and improved fixer #103 handles some of it but we still need to add some options.
• clean-regex/identity-escape ... How to handle identity escapes.
  • Add regexp/strict rule #220 regexp/strict rule instead
• clean-regex/no-constant-capturing-group ... Disallow capturing groups that can match only one word.
  • Improved no-assertion-capturing-group #188 brings over half of the rule.
• clean-regex/no-trivially-nested-lookaround ... Disallow lookarounds that only contain another assertion.
  • Added regexp/no-trivially-nested-assertion rule #120 adds this rule.
• clean-regex/no-trivially-nested-quantifier ... Disallow nested quantifiers that can be rewritten as one quantifier.
  • Added no-trivially-nested-quantifier rule #146 adds this rule.
• clean-regex/no-unnecessary-character-class ... Disallow unnecessary character classes.
  • Already fully covered by regexp/no-useless-character-class.
• clean-regex/no-unnecessary-flag ... Disallow unnecessary regex flags.
  • Add regexp/no-useless-flag rule #139 Add new rule regexp/no-useless-flag.
• clean-regex/no-unnecessary-group ... Disallow unnecessary non-capturing groups.
  • Improved no-useless-non-capturing-group rule #154 adds most of the functionality of clean-regex/no-unnecessary-group but it doesn't handle empty groups.
• clean-regex/no-unnecessary-lazy ... Disallow unnecessarily lazy quantifiers.
  • Updated no-useless-non-greedy rule #143
• clean-regex/no-unnecessary-quantifier ... Disallow unnecessary quantifiers.
  • Add no-useless-quantifier rule #197 adds this rule.
• clean-regex/optimal-concatenation-quantifier ... Use optimal quantifiers for concatenated quantified characters.
  • Added by Add optimal-quantifier-concatenation rule #207.
• clean-regex/optimized-character-class ... Disallows unnecessary elements in character classes.
  • Mostly covered by no-dupe-characters-character-class after Update no-dupe-characters-character-class #191.
  • The rest is already covered by no-useless-range and prefer-range.
• clean-regex/prefer-character-class ... Prefer character classes wherever possible instead of alternations.
  • Updated prefer-character-class rule #158 updates regexp/prefer-character-class.
• clean-regex/prefer-predefined-assertion ... Prefer predefined assertions over equivalent lookarounds.
  • Add prefer-predefined-assertion rule #171 adds this rule.
• clean-regex/prefer-predefined-character-set ... Prefer predefined character sets instead of their more verbose form.
  • Already covered by regexp/prefer-w and regexp/prefer-d.
• clean-regex/prefer-predefined-quantifiers ... Prefer predefined quantifiers (+*?) instead of their more verbose form.
  • This is already covered by regexp/prefer-question-quantifier, regexp/prefer-star-quantifier, and regexp/prefer-plus-quantifier.
• clean-regex/simple-constant-quantifier ... Prefer simple constant quantifiers over the range form.
  • Made no-useless-two-nums-quantifier fixable #109 makes no-useless-two-nums-quantifier functionally equivalent to clean-regex/simple-constant-quantifier.
• clean-regex/sort-flags ... Requires the regex flags to be sorted.
  • Add regexp/sort-flags rule #164 Add new rule regexp/sort-flags.

[RunDevelopment]

Note: I just found a bug in no-trivially-nested-lookaround (RunDevelopment/eslint-plugin-clean-regex#30). Whoever brings the rule over should fix the bug.

[RunDevelopment]

Regarding clean-regex/no-unnecessary-quantifier:
The rule is mostly covered by no-useless-exactly-quantifier but also does some other stuff that should probably be included in confusing-quantifier. I will need to think about that for a bit.

[ota-meshi]

I think we possible to mark no-useless-exactly-quantifier as deprecated and add new rule 😉

[RunDevelopment]

Regarding clean-regex/disjoint-alternatives:
Bringing this rule over will be tricky. It overlaps with no-dupe-disjunctions but it's not better than no-dupe-disjunctions. disjoint-alternatives is implemented using NFAs. This means that it can find non-trivial overlaps between alternatives that no-dupe-disjunctions can't find but it has 2 downside:

1. NFAs are expensive. While NFA operations are fairly optimized, they are not free.
2. Not all JS regexes can be converted into NFAs. I.e. disjoint-alternatives can't detect that ^a$ and ^a$ are the same. Assertions and backreferences can't be represented in NFAs so regexes containing them are difficult to deal with.

A hybrid approach using the current implementations of no-dupe-disjunctions and disjoint-alternatives has to be implemented.

Also want to talk about prefixes. no-dupe-disjunctions detects mistakes like a|abc (I do not understand why disallowNeverMatch is disabled by default). This is very useful but it's probably not complete (I say "probably" because the detection is so good that I couldn't find a simple example where it fails). Using NFAs, we can further improve the detection but it will be computationally expensive.

If it's ok with you @ota-meshi, then I'll bring over disjoint-alternatives some time this week

[ota-meshi]

A hybrid approach using the current implementations of no-dupe-disjunctions and disjoint-alternatives has to be implemented.

I think this idea is good!

(I do not understand why disallowNeverMatch is disabled by default)

Since disallowNeverMatch verifies patterns one by one, some patterns take a lot of time, so I turned off the default. The default behavior is that some patterns can be compared as strings, so I think it's fast in most cases.

This is very useful but it's probably not complete

I think so, too. As far as I know, I haven't verified the \p{} and i flag (This can be improved using regexp-ast-analysis).
Other than that, it should work as far as I can think of, but I'm still not as familiar with regexp as you 😅.

[RunDevelopment]

[...] some patterns take a lot of time, so I turned off the default. The default behavior is that some patterns can be compared as strings, so I think it's fast in most cases.

I ran the rule on Prism's 2k regexes this is the result:

config	time
'regexp/no-dupe-disjunctions': 'off'	2.9s
'regexp/no-dupe-disjunctions': 'warn'	3.7s
'regexp/no-dupe-disjunctions': ['warn', { disallowNeverMatch: true }]	5.4s

And then there's disjoint-alternatives:

config	time
'clean-regex/disjoint-alternatives': 'off'	2.9s
'clean-regex/disjoint-alternatives': 'warn'	5.6s

Since disjoint-alternatives doesn't even implement a disallowNeverMatch: true option, the final hybrid approach will be even slower.

That being said, I don't think that this is a huge problem. Prism has literally thousands of complex regexes, the average code base will not see a noticeable performance impact. ESLint also provides options to cache results and to only run on changed files.

We should also consider that no-dupe-disjunctions and disjoint-alternatives find real bugs that are hard to notice. These rules provide real value.

I agree that users must have the option to turn off advanced analysis if performance was a problem. But we shouldn't hide some of our best features by default.

Other than that, it should work as far as I can think of [...].

Implement disallowNeverMatch for two alternatives A|B, we need to find out whether B is a subset of A[^]* (= A followed by any string).

Don't get me wrong, the isCovered function is really good but there will be some complex cases that it simply can't detect without doing a proper subset check.
(That being said, the proper subset check is only possible in O(2^n). So we also won't be able to implement the subset check perfectly.)

[RunDevelopment]

Regarding clean-regex/no-constant-capturing-group:
With #188, we support the default configuration of clean-regex/no-constant-capturing-group. I think we don't need to bring over the rest.

There are legitimate use cases for capturing groups that only match a single string so I don't want to just disallow them.

[ota-meshi]

@RunDevelopment
I'm glad that most of the rule migrations are done 😆

The clean-regex/identity-escape rule is similar to the regexp/no-useless-escape rule, but the clean-regex/identity-escape rule seems to give user the flexibility to change the checks.

Do you think we can merge the features of the clean-regex/identity-escape rule by adding an option to the regexp/no-useless-escape rule? Or do you think we need to add new rule?

[RunDevelopment]

I'm not sure about clean-regex/identity-escape.

The problem is that the rule can also add useless escaping (i.e. you can configure it to escape all ! characters) which conflicts with ESLint's no-useless-escape rule and our regexp/no-useless-escape rule.

The reason I added clean-regex/identity-escape is that I needed something like regexp/no-useless-escape but I also wanted something that enforces the use of \{, \}, and \] outside of character classes. We don't need an extra rule just for these 3 characters and merging this into regexp/no-useless-escape is weird.

Maybe we could use the fact that Unicode parsing mode basically has the rules for escaping that I want? We could add a new rule regexp/strict that makes sure any regex without the u flag is also syntactically valid with the u flag. This rule would have to add/remove escaping in exactly the way that I want in order to achieve its goal. regexp/strict will slightly overlap with regexp/no-useless-escape but not completely (e.g. regexp/strict will allow /[\.]/).

With regexp/strict, we could make my taste for escaping into something that is actually useful. This also gets rid of conflicts with the no-useless-escape rules.

(It even gets rid of the overly complex rule option of clean-regex/identity-escape 😅 )

[ota-meshi]

The regexp/strict rule sounds good to me!

Maybe we can use RegExpValidator's strict option.
https://github.com/mysticatea/regexpp/blob/master/src/validator.ts#L127

[RunDevelopment]

Maybe we can use RegExpValidator's strict option.

That's where I got the idea for the name from :)

[RunDevelopment]

With regexp/strict now being merged, I think we are done.

Thank you very much for your help @ota-meshi!