refactor(es/lexer): Add fast lexer implementation

[kdy1]

Description:

This is a rewrite from scratch. I'll split PRs into multiple parts to track improvements. I'll not publish these codes as a separate crate. I'll merge this into swc_ecma_parser before publishing`.

Benchmark result:

Original

Gnuplot not found, using plotters backend
es/lexer/colors         time:   [4.1904 µs 4.1986 µs 4.2077 µs]
                        change: [-2.4433% -2.1954% -1.9296%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  8 (8.00%) high mild
  1 (1.00%) high severe

es/lexer/angular        time:   [2.2530 ms 2.2538 ms 2.2546 ms]
                        change: [+51.233% +51.601% +51.891%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe

es/lexer/backbone       time:   [263.75 µs 265.09 µs 266.61 µs]
                        change: [+70.699% +71.214% +71.745%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) high mild
  8 (8.00%) high severe

Benchmarking es/lexer/jquery: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.9s, enable flat sampling, or reduce sample count to 50.
es/lexer/jquery         time:   [1.5150 ms 1.5171 ms 1.5197 ms]
                        change: [+50.850% +52.439% +53.705%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

es/lexer/jquery mobile  time:   [2.2891 ms 2.2909 ms 2.2934 ms]
                        change: [+39.568% +39.823% +40.053%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

Benchmarking es/lexer/mootools: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.9s, enable flat sampling, or reduce sample count to 60.
es/lexer/mootools       time:   [1.1470 ms 1.1515 ms 1.1576 ms]
                        change: [+56.433% +57.853% +59.914%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 19 outliers among 100 measurements (19.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  16 (16.00%) high severe

es/lexer/underscore     time:   [217.32 µs 217.42 µs 217.54 µs]
                        change: [+79.825% +80.151% +80.449%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe

es/lexer/three          time:   [6.8574 ms 6.8609 ms 6.8644 ms]
                        change: [+45.520% +45.636% +45.743%] (p = 0.00 < 0.05)
                        Performance has regressed.

Benchmarking es/lexer/yui: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.3s, enable flat sampling, or reduce sample count to 60.
es/lexer/yui            time:   [1.2488 ms 1.2572 ms 1.2658 ms]
                        change: [+48.947% +49.498% +50.119%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 21 outliers among 100 measurements (21.00%)
  5 (5.00%) high mild
  16 (16.00%) high severe

New:

es/fast-lexer/angular   time:   [1.6318 ms 1.6343 ms 1.6373 ms]
Found 10 outliers among 100 measurements (10.00%)
  2 (2.00%) high mild
  8 (8.00%) high severe

es/fast-lexer/backbone  time:   [176.71 µs 177.24 µs 177.81 µs]
Found 12 outliers among 100 measurements (12.00%)
  2 (2.00%) high mild
  10 (10.00%) high severe

Benchmarking es/fast-lexer/jquery: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.5s, enable flat sampling, or reduce sample count to 60.
es/fast-lexer/jquery    time:   [1.0616 ms 1.0638 ms 1.0663 ms]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

Benchmarking es/fast-lexer/jquery mobile: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 9.1s, enable flat sampling, or reduce sample count to 50.
es/fast-lexer/jquery mobile
                        time:   [1.7711 ms 1.7734 ms 1.7767 ms]
Found 12 outliers among 100 measurements (12.00%)
  3 (3.00%) high mild
  9 (9.00%) high severe

es/fast-lexer/mootools  time:   [825.96 µs 828.12 µs 830.45 µs]
Found 9 outliers among 100 measurements (9.00%)
  7 (7.00%) high mild
  2 (2.00%) high severe

es/fast-lexer/underscore
                        time:   [138.14 µs 138.33 µs 138.59 µs]
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

es/fast-lexer/three     time:   [5.2196 ms 5.2250 ms 5.2319 ms]
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  3 (3.00%) high severe

es/fast-lexer/yui       time:   [931.84 µs 935.58 µs 940.20 µs]
Found 9 outliers among 100 measurements (9.00%)
  9 (9.00%) high mild

[changeset-bot]

⚠️ No Changeset found

Latest commit: 6d80b83

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[kdy1]

The benchmark result looks promising.

Lexer only benchmark:

Although it fails in some cases, it's much faster than the original implementation

• Original:

• New:

[kdy1]

I'll create follow-up PRs for other tasks