Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
useragents.txt sample file kindly provided by @DailyMats out of DailyMotion's data (2023-04-26). The provided scripts allow: - Testing the cache hit rate of various cache configuration (algorithm and size) on sample files, this script uses a dummy parser and is thus extremely fast. - Benchmarking the average entry processing of various parser configurations (base parser + cache algoritm + cache size) on sample files, this is a much slower script but provides a a realistic evaluation, and allows using custom rules (`regexes.yaml` files) to check their impact on the performance of a given base parser. Also added a script for testing threaded parsing, as expected this gets 0 gain over the normal stuff because of the GIL (and re2 seemingly doesn't release the GIL either, though I don't know how beneficial it would be at ~30us per call). May be more useful with 3.13, or possibly with a regex-based extension releasing the GIL, at least the basis for testing things out will be here.
- Loading branch information