Rework loading benchmark to avoid v8 caching #36
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Benchmarks are quite random by themselves but there are things we must do to make them close to real numbers. As reported in #35, loading benchmark seems rigged because it uses a single process to test the init time of libraries, thus making the last one to test a clear winner.
In this PR I made it so the libraries are being tested via importing them in a separate node processes. It will take longer to test but the results seem to be more coherent. There is still some deviation but moving
picocolors
up and down does not make significant changes (comparing to the reported issue)cc @ai
Closes #35