We need a size benchmark

[Yoric]

Something like https://github.com/binast/binjs-ref/blob/master/examples/sample_directory.rs

[Yoric]

Frameworks

Protocol

PYTHONPATH=$PYTHONPATH:. ./bench/sample_directory.py --dir ~/Documents/Code/binjs/tests/data/frameworks --binjs_encode ~/Documents/Code/binjs/target/release/binjs_encode --sampling 0.2 --dictionary-threshold 0

Result

Global ratio: 1.114944

Facebook sample set

Protocol

PYTHONPATH=$PYTHONPATH:. ./bench/sample_directory.py --dir ~/Documents/Code/binjs/tests/data/facebook/single/ --binjs_encode ~/Documents/Code/binjs/target/release/binjs_encode --sample 0.5

Result

Global ratio: 1.051008

Real js samples

Protocol

 PYTHONPATH=$PYTHONPATH:. time ./bench/sample_directory.py --dir ~/Downloads/scrap/ --binjs_encode ~/Documents/Code/binjs/target/release/binjs_encode --sample 0.2

Result

Global ratio: 1.055725

Also, 55480.71 real 16104.84 user 4696.62 sys. That was... long.

[Yoric]

If we compare to #2, the (not-really-finished) entropy-0.4 format shows:

	entropy-0.4-pre	binjs-fbssdc	entropy/fbssdc
tests/data/facebook/single	1.05	1.05	1
tests/data/frameworks	1.13	1.12	1.01
real-js-samples	1.03	1.06	0.97

Slight advantage to entropy. Not sufficient to change course now, but we probably want to merge the formats eventually to get the best of both worlds.