Implement MPT #1235
Merged
Implement MPT #1235
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #1235 +/- ##
==========================================
+ Coverage 66.74% 67.50% +0.76%
==========================================
Files 200 210 +10
Lines 17159 17900 +741
==========================================
+ Hits 11452 12083 +631
- Misses 5092 5173 +81
- Partials 615 644 +29
Continue to review full report at Codecov.
|
roman-khimov
approved these changes
Jul 28, 2020
AnnaShaleva
approved these changes
Jul 29, 2020
MPT is a trie with a branching factor = 16, i.e. it consists of sequences in 16-element alphabet. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Because there is no distinct type field in JSONized nodes, distinction is made via payload itself, thus all unmarshaling is done via NodeObject. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Because trie size is rather big, it can't be stored in memory. Thus some form of caching should also be implemented. To avoid marshaling/unmarshaling of items which are close to root and are used very frequenly we can save them across the persists. This commit implements pruning items at the specified depth, replacing them by hash nodes. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
It drastically reduces the number of allocations and hash calculations. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
It's just a waste of time. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
It should always be invalidated if something changes in the `next` (below the extension node). Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
This was differing from C# notion of PrevHash. It's not a previous root, but rather a hash of the previous serialized MPTRoot structure (that is to be signed by CNs). Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
We need to compact our in-memory MPT from time to time, otherwise it quickly fills up all available memory. This raises two obvious quesions --- when to do that and to what level do that. As for 'when', I think it's quite easy to use our regular persistence interval as an anchor (and it also frees up some memory), but we can't do that in the persistence routine itself because of synchronization issues (adding some synchronization primitives would add some cost that I'd also like to avoid), so do it indirectly by comparing persisted and current height in `storeBlock`. Choosing proper level is another problem, but if we're to roughly estimate one full branch node to use 1K of memory (usually it's way less than that) then we can easily store 1K of these nodes and that gives us a depth of 10 for our trie. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Close #851 .
P2P/Consensus/RPC are not ported.
The only difference is that storage keys and values now are serialized in a straight-forward way.