Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trie: cache encoding of nodes on disk #2322

Closed
qdm12 opened this issue Feb 21, 2022 · 3 comments
Closed

trie: cache encoding of nodes on disk #2322

qdm12 opened this issue Feb 21, 2022 · 3 comments

Comments

@qdm12
Copy link
Contributor

qdm12 commented Feb 21, 2022

Task summary

For now we cache the encoding of trie nodes in memory for every node.
We should instead write the cached encoding to disk and fetch it from disk when needed.

  • Modify code to store node hash <-> encoding on encoding
  • Modify code to delete node hash <-> encoding when node is modified or removed
  • Modify code to fetch encoding using the node hash (kept in memory) when encoding
  • Memory usage improvement measured and documented
  • Speed performance degradation measured and documented
@noot
Copy link
Contributor

noot commented Mar 17, 2022

@qdm12 same with this, do we still need this?

@qdm12
Copy link
Contributor Author

qdm12 commented May 4, 2022

This depends on #2229 which I'll finish up soon. If the number of total nodes in the tries in the program correlate with our increased memory usage, then we probably want to do both #2323 (store the Value byte slice of a node) and this issue (store the cached encoding and digest of a node).

@qdm12
Copy link
Contributor Author

qdm12 commented Dec 7, 2022

This is no longer needed, having the whole state trie (and deltas from forks before finalization) does not use much memory (~100MB at the tip). See #1973 for an update on memory usage. Let's close this for now, we can re-open later when we need it (i.e. for parachains with superfat tries)

@qdm12 qdm12 closed this as completed Dec 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants