-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
types: {Int, Uint}.MarshalTo use an expensive (math/big.Int).Bytes() to detect if isZero instead of the cheaper .BitLen() #8575
Labels
T: Performance
Performance improvements
Comments
odeke-em
added a commit
that referenced
this issue
Feb 14, 2021
Instead of using len((*math/big.Int).Bytes()) == 0, which expensively creates a byte slice and marshals a value, on the majority hot path, instead use the cheaper method .BitLen() to check if 0. Benchmarking results, just from types: name old time/op new time/op delta CoinsAdditionIntersect/sizes:_A_1,_B_1-8 132ns ± 2% 126ns ±13% -4.55% (p=0.050 n=10+10) CoinsAdditionIntersect/sizes:_A_5,_B_5-8 1.41µs ± 3% 1.41µs ± 2% ~ (p=1.000 n=10+10) CoinsAdditionIntersect/sizes:_A_5,_B_20-8 2.30µs ± 1% 2.27µs ± 3% ~ (p=0.066 n=10+10) CoinsAdditionIntersect/sizes:_A_1,_B_1000-8 30.9µs ± 3% 30.7µs ± 1% ~ (p=0.218 n=10+10) CoinsAdditionIntersect/sizes:_A_2,_B_1000-8 31.4µs ± 3% 30.8µs ± 2% -1.94% (p=0.015 n=10+10) CoinsAdditionNoIntersect/sizes:_A_1,_B_1-8 116ns ± 1% 114ns ± 4% ~ (p=0.142 n=10+10) CoinsAdditionNoIntersect/sizes:_A_5,_B_5-8 1.11µs ± 1% 1.08µs ± 3% -2.36% (p=0.003 n=8+10) CoinsAdditionNoIntersect/sizes:_A_5,_B_20-8 1.85µs ± 2% 1.82µs ± 1% -1.38% (p=0.001 n=10+9) CoinsAdditionNoIntersect/sizes:_A_1,_B_1000-8 30.7µs ± 1% 30.6µs ± 3% ~ (p=0.393 n=10+10) CoinsAdditionNoIntersect/sizes:_A_2,_B_1000-8 31.1µs ± 1% 30.7µs ± 2% -1.32% (p=0.015 n=10+10) CoinsAdditionNoIntersect/sizes:_A_1000,_B_2-8 31.0µs ± 2% 30.7µs ± 2% ~ (p=0.190 n=10+10) Bech32ifyPubKey-8 28.8µs ± 5% 28.8µs ± 3% ~ (p=0.965 n=10+8) GetPubKeyFromBech32-8 38.8µs ± 3% 39.4µs ± 2% +1.70% (p=0.013 n=9+10) ParseCoin-8 16.7µs ± 6% 15.8µs ± 4% -5.21% (p=0.001 n=10+10) MarshalTo-8 521ns ± 5% 508ns ± 3% -2.56% (p=0.029 n=10+10) UintMarshal-8 3.10µs ±17% 2.56µs ± 3% -17.45% (p=0.000 n=10+9) IntMarshal-8 2.52µs ±10% 1.94µs ± 2% -23.10% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Bech32ifyPubKey-8 4.02kB ± 0% 4.02kB ± 0% ~ (all equal) GetPubKeyFromBech32-8 2.48kB ± 0% 2.48kB ± 0% ~ (all equal) ParseCoin-8 2.21kB ± 0% 2.21kB ± 0% ~ (all equal) MarshalTo-8 80.0B ± 0% 80.0B ± 0% ~ (all equal) UintMarshal-8 440B ± 0% 392B ± 0% -10.91% (p=0.000 n=10+10) IntMarshal-8 216B ± 0% 168B ± 0% -22.22% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Bech32ifyPubKey-8 25.0 ± 0% 25.0 ± 0% ~ (all equal) GetPubKeyFromBech32-8 85.0 ± 0% 85.0 ± 0% ~ (all equal) ParseCoin-8 71.0 ± 0% 71.0 ± 0% ~ (all equal) MarshalTo-8 2.00 ± 0% 2.00 ± 0% ~ (all equal) UintMarshal-8 31.0 ± 0% 25.0 ± 0% -19.35% (p=0.000 n=10+10) IntMarshal-8 24.0 ± 0% 18.0 ± 0% -25.00% (p=0.000 n=10+10) name old speed new speed delta UintMarshal-8 2.27MB/s ±15% 2.75MB/s ± 2% +20.87% (p=0.000 n=10+8) IntMarshal-8 2.78MB/s ± 9% 3.60MB/s ± 2% +29.69% (p=0.000 n=10+10) Fixes #8575
9 tasks
odeke-em
added a commit
that referenced
this issue
Feb 15, 2021
Instead of using len((*math/big.Int).Bytes()) == 0, which expensively creates a byte slice and marshals a value, on the majority hot path, instead use the cheaper method .BitLen() to check if 0. Benchmarking results, just from types: name old time/op new time/op delta CoinsAdditionIntersect/sizes:_A_1,_B_1-8 132ns ± 2% 126ns ±13% -4.55% (p=0.050 n=10+10) CoinsAdditionIntersect/sizes:_A_5,_B_5-8 1.41µs ± 3% 1.41µs ± 2% ~ (p=1.000 n=10+10) CoinsAdditionIntersect/sizes:_A_5,_B_20-8 2.30µs ± 1% 2.27µs ± 3% ~ (p=0.066 n=10+10) CoinsAdditionIntersect/sizes:_A_1,_B_1000-8 30.9µs ± 3% 30.7µs ± 1% ~ (p=0.218 n=10+10) CoinsAdditionIntersect/sizes:_A_2,_B_1000-8 31.4µs ± 3% 30.8µs ± 2% -1.94% (p=0.015 n=10+10) CoinsAdditionNoIntersect/sizes:_A_1,_B_1-8 116ns ± 1% 114ns ± 4% ~ (p=0.142 n=10+10) CoinsAdditionNoIntersect/sizes:_A_5,_B_5-8 1.11µs ± 1% 1.08µs ± 3% -2.36% (p=0.003 n=8+10) CoinsAdditionNoIntersect/sizes:_A_5,_B_20-8 1.85µs ± 2% 1.82µs ± 1% -1.38% (p=0.001 n=10+9) CoinsAdditionNoIntersect/sizes:_A_1,_B_1000-8 30.7µs ± 1% 30.6µs ± 3% ~ (p=0.393 n=10+10) CoinsAdditionNoIntersect/sizes:_A_2,_B_1000-8 31.1µs ± 1% 30.7µs ± 2% -1.32% (p=0.015 n=10+10) CoinsAdditionNoIntersect/sizes:_A_1000,_B_2-8 31.0µs ± 2% 30.7µs ± 2% ~ (p=0.190 n=10+10) Bech32ifyPubKey-8 28.8µs ± 5% 28.8µs ± 3% ~ (p=0.965 n=10+8) GetPubKeyFromBech32-8 38.8µs ± 3% 39.4µs ± 2% +1.70% (p=0.013 n=9+10) ParseCoin-8 16.7µs ± 6% 15.8µs ± 4% -5.21% (p=0.001 n=10+10) MarshalTo-8 521ns ± 5% 508ns ± 3% -2.56% (p=0.029 n=10+10) UintMarshal-8 3.10µs ±17% 2.56µs ± 3% -17.45% (p=0.000 n=10+9) IntMarshal-8 2.52µs ±10% 1.94µs ± 2% -23.10% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Bech32ifyPubKey-8 4.02kB ± 0% 4.02kB ± 0% ~ (all equal) GetPubKeyFromBech32-8 2.48kB ± 0% 2.48kB ± 0% ~ (all equal) ParseCoin-8 2.21kB ± 0% 2.21kB ± 0% ~ (all equal) MarshalTo-8 80.0B ± 0% 80.0B ± 0% ~ (all equal) UintMarshal-8 440B ± 0% 392B ± 0% -10.91% (p=0.000 n=10+10) IntMarshal-8 216B ± 0% 168B ± 0% -22.22% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Bech32ifyPubKey-8 25.0 ± 0% 25.0 ± 0% ~ (all equal) GetPubKeyFromBech32-8 85.0 ± 0% 85.0 ± 0% ~ (all equal) ParseCoin-8 71.0 ± 0% 71.0 ± 0% ~ (all equal) MarshalTo-8 2.00 ± 0% 2.00 ± 0% ~ (all equal) UintMarshal-8 31.0 ± 0% 25.0 ± 0% -19.35% (p=0.000 n=10+10) IntMarshal-8 24.0 ± 0% 18.0 ± 0% -25.00% (p=0.000 n=10+10) name old speed new speed delta UintMarshal-8 2.27MB/s ±15% 2.75MB/s ± 2% +20.87% (p=0.000 n=10+8) IntMarshal-8 2.78MB/s ± 9% 3.60MB/s ± 2% +29.69% (p=0.000 n=10+10) Fixes #8575
mergify bot
added a commit
that referenced
this issue
Feb 15, 2021
Instead of using len((*math/big.Int).Bytes()) == 0, which expensively creates a byte slice and marshals a value, on the majority hot path, instead use the cheaper method .BitLen() to check if 0. Benchmarking results, just from types: name old time/op new time/op delta CoinsAdditionIntersect/sizes:_A_1,_B_1-8 132ns ± 2% 126ns ±13% -4.55% (p=0.050 n=10+10) CoinsAdditionIntersect/sizes:_A_5,_B_5-8 1.41µs ± 3% 1.41µs ± 2% ~ (p=1.000 n=10+10) CoinsAdditionIntersect/sizes:_A_5,_B_20-8 2.30µs ± 1% 2.27µs ± 3% ~ (p=0.066 n=10+10) CoinsAdditionIntersect/sizes:_A_1,_B_1000-8 30.9µs ± 3% 30.7µs ± 1% ~ (p=0.218 n=10+10) CoinsAdditionIntersect/sizes:_A_2,_B_1000-8 31.4µs ± 3% 30.8µs ± 2% -1.94% (p=0.015 n=10+10) CoinsAdditionNoIntersect/sizes:_A_1,_B_1-8 116ns ± 1% 114ns ± 4% ~ (p=0.142 n=10+10) CoinsAdditionNoIntersect/sizes:_A_5,_B_5-8 1.11µs ± 1% 1.08µs ± 3% -2.36% (p=0.003 n=8+10) CoinsAdditionNoIntersect/sizes:_A_5,_B_20-8 1.85µs ± 2% 1.82µs ± 1% -1.38% (p=0.001 n=10+9) CoinsAdditionNoIntersect/sizes:_A_1,_B_1000-8 30.7µs ± 1% 30.6µs ± 3% ~ (p=0.393 n=10+10) CoinsAdditionNoIntersect/sizes:_A_2,_B_1000-8 31.1µs ± 1% 30.7µs ± 2% -1.32% (p=0.015 n=10+10) CoinsAdditionNoIntersect/sizes:_A_1000,_B_2-8 31.0µs ± 2% 30.7µs ± 2% ~ (p=0.190 n=10+10) Bech32ifyPubKey-8 28.8µs ± 5% 28.8µs ± 3% ~ (p=0.965 n=10+8) GetPubKeyFromBech32-8 38.8µs ± 3% 39.4µs ± 2% +1.70% (p=0.013 n=9+10) ParseCoin-8 16.7µs ± 6% 15.8µs ± 4% -5.21% (p=0.001 n=10+10) MarshalTo-8 521ns ± 5% 508ns ± 3% -2.56% (p=0.029 n=10+10) UintMarshal-8 3.10µs ±17% 2.56µs ± 3% -17.45% (p=0.000 n=10+9) IntMarshal-8 2.52µs ±10% 1.94µs ± 2% -23.10% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Bech32ifyPubKey-8 4.02kB ± 0% 4.02kB ± 0% ~ (all equal) GetPubKeyFromBech32-8 2.48kB ± 0% 2.48kB ± 0% ~ (all equal) ParseCoin-8 2.21kB ± 0% 2.21kB ± 0% ~ (all equal) MarshalTo-8 80.0B ± 0% 80.0B ± 0% ~ (all equal) UintMarshal-8 440B ± 0% 392B ± 0% -10.91% (p=0.000 n=10+10) IntMarshal-8 216B ± 0% 168B ± 0% -22.22% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Bech32ifyPubKey-8 25.0 ± 0% 25.0 ± 0% ~ (all equal) GetPubKeyFromBech32-8 85.0 ± 0% 85.0 ± 0% ~ (all equal) ParseCoin-8 71.0 ± 0% 71.0 ± 0% ~ (all equal) MarshalTo-8 2.00 ± 0% 2.00 ± 0% ~ (all equal) UintMarshal-8 31.0 ± 0% 25.0 ± 0% -19.35% (p=0.000 n=10+10) IntMarshal-8 24.0 ± 0% 18.0 ± 0% -25.00% (p=0.000 n=10+10) name old speed new speed delta UintMarshal-8 2.27MB/s ±15% 2.75MB/s ± 2% +20.87% (p=0.000 n=10+8) IntMarshal-8 2.78MB/s ± 9% 3.60MB/s ± 2% +29.69% (p=0.000 n=10+10) Fixes #8575 Co-authored-by: Alessio Treglia <alessio@tendermint.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Summary of Bug
While auditing I noticed this code
cosmos-sdk/types/uint.go
Lines 160 to 176 in f8a6987
Results
With just this small change, we see some good benefits
and then when we apply benchmarks over the entire project there are benefits
Let's use the cheaper and more efficient alternative
For Admin Use
The text was updated successfully, but these errors were encountered: