feat(dictgen): Add aho-corasick support

[epage]

As recommended by @BurntSushi

benches              fastest       │ slowest       │ median        │ mean          │ samples │ iters
├─ hit                             │               │               │               │         │
│  ├─ aho_corasick                 │               │               │               │         │
│  │  ╰─ finallizes  34.88 ns      │ 152 ms        │ 36.88 ns      │ 1.52 ms       │ 100     │ 100
│  ├─ cased_map                    │               │               │               │         │
│  │  ╰─ finallizes  31.88 ns      │ 5.156 µs      │ 34.38 ns      │ 85.96 ns      │ 100     │ 100
│  ├─ map                          │               │               │               │         │
│  │  ╰─ finallizes  39.88 ns      │ 3.553 µs      │ 41.88 ns      │ 77.48 ns      │ 100     │ 100
│  ├─ ordered_map                  │               │               │               │         │
│  │  ╰─ finallizes  146.8 ns      │ 3.232 µs      │ 149.8 ns      │ 205.8 ns      │ 100     │ 100
│  ╰─ trie                         │               │               │               │         │
│     ╰─ finallizes  65.88 ns      │ 6.734 µs      │ 68.88 ns      │ 136.3 ns      │ 100     │ 100
├─ miss                            │               │               │               │         │
│  ├─ aho_corasick                 │               │               │               │         │
│  │  ╰─ finalizes   6.548 ns      │ 6.99 ns       │ 6.802 ns      │ 6.781 ns      │ 100     │ 25600
│  ├─ cased_map                    │               │               │               │         │
│  │  ╰─ finalizes   11.95 ns      │ 12.14 ns      │ 12.04 ns      │ 12.04 ns      │ 100     │ 12800
│  ├─ map                          │               │               │               │         │
│  │  ╰─ finalizes   18.38 ns      │ 36.38 ns      │ 18.59 ns      │ 19.3 ns       │ 100     │ 6400
│  ├─ ordered_map                  │               │               │               │         │
│  │  ╰─ finalizes   125.4 ns      │ 198.6 ns      │ 126.8 ns      │ 128.9 ns      │ 100     │ 1600
│  ╰─ trie                         │               │               │               │         │
│     ╰─ finalizes   33.66 ns      │ 42.63 ns      │ 35.44 ns      │ 35.19 ns      │ 100     │ 3200
╰─ new                             │               │               │               │         │
   ╰─ aho_corasick   125.7 ms      │ 149.2 ms      │ 138.5 ms      │ 137.9 ms      │ 100     │ 100

Considering we should have millions of misses with only a couple of hits, this seems promising. However, the construction time is prohibitive.

[coveralls]

Pull Request Test Coverage Report for Build 12560549722

Details

• 0 of 85 (0.0%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-0.6%) to 20.029%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
crates/dictgen/src/gen.rs	0	1	0.0%
crates/typos-dict/tests/codegen.rs	0	21	0.0%
crates/dictgen/src/aho_corasick.rs	0	63	0.0%

Totals
Change from base Build 12559829744:	-0.6%
Covered Lines:	553
Relevant Lines:	2761

---

💛 - Coveralls

[BurntSushi]

Yeah building a DFA is extremely costly. Did you try using the contiguous NFA? Likely slower search times, but potentially much faster construction time.

(I have high hopes some day for adding support for serializing Aho-Corasick automata and then providing zero-copy deserialization APIs like what regex-automata has, but it's a big effort with some serious downsides.)

[BurntSushi]

Also, what are you using to produce those benchmark results? It looks nice!

[epage]

Yeah building a DFA is extremely costly. Did you try using the contiguous NFA? Likely slower search times, but potentially much faster construction time.

If its too much slower, then I might as well use PHF so unsure if its worth trying.

Also, what are you using to produce those benchmark results? It looks nice!

https://crates.io/crates/divan

It can still be jittery but appreciate the faster run time and the clean API.

[szepeviktor]

💡 https://xlinux.nist.gov/dads/HTML/ahoCorasick.html