-
Notifications
You must be signed in to change notification settings - Fork 839
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use borsh to deserialize JIT Artifact #2119
Conversation
@@ -37,9 +39,58 @@ pub struct SerializableCompilation { | |||
|
|||
/// Serializable struct that is able to serialize from and to | |||
/// a `JITArtifactInfo`. | |||
#[derive(Serialize, Deserialize)] | |||
#[derive(Serialize, Deserialize, BorshSerialize, BorshDeserialize)] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#[cfg_attr(feature = "enable-borsh", derive(BorshSerialize, BorshDeserialize))]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not under feature, because Serialize and Deserialize was always enabled, whereas crate like lib/compiler/Cargo.toml has optional serde, thus also optional borsh
lib/engine-jit/src/serialize.rs
Outdated
// JumpTableOffsets is a SecondaryMap, non trivial to impl borsh | ||
let v = bincode::serialize(&self.function_jt_offsets).map_err(|_| std::io::Error::new(std::io::ErrorKind::InvalidData, "invalid function_jt_offsets"))?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why? A SecondaryMap is a PrimaryMap without the ability to mutate the keys, which in turn is just a typesafe Vec where the index has a wrapper type instead to keep them distinct instead of plain integers. Is the problem not serialization but deserialization?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PrimaryMap is internally just a vec, but SecondaryMap is more complex to serialize:
https://github.com/bytecodealliance/wasmtime/blob/285a7feb19c1ce768d68fd421eddd1ff6b093d34/cranelift/entity/src/map.rs#L200
More proper way is implement a borshserialize/deserialize similarly to its impl of serde serialize/deserialize
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm generally in favor of conditional support for other types of serialization 👍
@@ -20,17 +20,19 @@ hashbrown = { version = "0.9", optional = true } | |||
serde = { version = "1.0", features = ["derive"], optional = true } | |||
thiserror = "1.0" | |||
serde_bytes = { version = "0.11", optional = true } | |||
smallvec = "1.6" | |||
smallvec = "1.6" | |||
borsh = { git = "https://github.com/near/borsh-rs", rev = "c62cdfbd10d4a17fc877809eba4ccb65e866d5f8", optional = true } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We shouldn't merge this while there's a git dependency as it will block us from doing releases on crates.io
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This commit haven't published, will keep this updated as we publish a new borsh
Co-authored-by: nlewycky <nicholas@mxc.ca>
Co-authored-by: nlewycky <nicholas@mxc.ca>
@Hywan @MarkMcCaskey just added an benchmark, it's a 30% increase in a larger contract, might be interested to consider borsh. But deserialization is still not fast enough for our use case, so I'm going to try some mmap based serialization approach. |
Do you mean that serialization is 30% faster? |
Both serialization (store_module) and deserialization (load_module) are 30%
faster, possibly a little more because store_module = serialization +
write_to_file and write_to_file part isn't changed. See bench numbers in PR description
…On Thu, Feb 25, 2021 at 12:04 AM Ivan Enderlin ***@***.***> wrote:
it's a 30% increase in a larger contract
Do you mean that serialization is 30% faster?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#2119 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADFFFCG7VIX5VJTPH6JUW7LTAYAARANCNFSM4XP44JAA>
.
|
close because #2190 is a faster approach |
Description
This has a little speed up on deserialization JIT Artifact, made a benchmark to store and load a 2MB lib/c-api/tests/assets/qjs.wasm from filesystem cache. To run the bench:
cargo bench -p wasmer-cache --features singlepass
. Before this PR it is:After this PR:
Review