RFC: state updates #902
RFC: state updates #902
Conversation
a76cc5c to
627e478
Compare
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
Co-authored-by: teor <teor@riseup.net>
7828879 to
4b75d21
Compare
|
Rebased onto |
The rationale for this change is described in the document: it means that we write blocks only to one end of the Sled tree, and hopefully helps us with spatial access patterns. This should help alleviate a major cause of memory use in Zebra's current WIP Sled structure, which is that: - blocks are stored in random, sparse order (by hash) in the B-tree; - the `Request::GetDepth` method opens the entire block store and queries a random part of its block data to determine whether a hash is present; - if present, it deserializes the complete block data of both the given block and the current tip block, to compute the difference in block heights. This access pattern forces a large amount of B-tree data to remain resident, and could probably be avoided if we didn't do that.
|
The latest commit, 6b8a5d7, changes the RFC to store blocks by height and notes that the |
There was a problem hiding this comment.
Looks good, the recent changes clear up a lot of the outstanding issues.
Here are my remaining open questions:
Do we need to distinguish between zcashd's probabilistically final state (due to cumulative rollbacks), and Zebra's finalised state?
#902 (comment)
Do we need to store the total work of the tip in the sled state?
#902 (comment)
But I don't want to block the PR on them.
We can open up a design ticket for a separate proof of work RFC, and skip clarifying zcashd state implementation details.
But let's also check with @yaahc and @dconnolly?
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: teor <teor@riseup.net>
…into rfc-state-updates
comments have mostly been addressed, but this RFC is close enough to done that we (dierdre, henry, and jane) decided (while teor was asleep) to ship it as is and do further proofreading / updates in later PRs.
| sprout_anchor: sprout::tree::Root, | ||
| sapling_anchor: sapling::tree::Root, |
There was a problem hiding this comment.
@dconnolly does every post-Sapling block have a sprout and a sapling anchor?
There was a problem hiding this comment.
'Yes'. What it is depends on the contents of the transactions in the block, joinsplits for sprout and spends/outputs for sapling.
If there are no joinsplits, then the input sprout treestate from the parent block has not been changed, and I think (but staring at the spec to confirm) that means that the output Sprout treestate == the input Sprout treestate. Same for Sapling, if there are no changes to the state, I assume the final treestate (anchor) is the output state from the last transaction (a no-op).
There was a problem hiding this comment.
Ok, that sounds like we'll always have both Roots in CommitBlock, because it only accepts post-Sapling blocks.
But I just realised that CommitFinalizedBlock accepts all blocks, including Genesis blocks.
Do genesis blocks have a Sprout root?
Do pre-Sapling blocks have a Sapling root?
There was a problem hiding this comment.
For the genesis block, the input treestate 'for each of Sprout and Sapling', is the empty treestate, which we can compute with my changes in #1045 (at least for Sapling for now, I'm still working on Sprout).
So for pre-Sapling, I think they literally compute and pass along the empty Sapling treestate, from block to block, until Sapling activation.
There was a problem hiding this comment.
Technically there is an unspendable block subsidy in the genesis block (as with Bitcoin); it just happens that due to the slow start, it has a value of zero. In a chain where ZIP 213 is active from genesis, there could therefore be a Sapling output in the genesis block; if that chain also disabled slow start, the output would have value. It's not immediately clear to me at 4am whether that output should be spendable; under the current consensus rules in the spec, I think it would be added to the commitment tree and therefore be spendable (whereas genesis transparent UTXOs are not added to the UTXO set), but IDK what zcashd would do here (again, special genesis block).
In practice the genesis block never contains shielded outputs in zcashd due to the way that the genesis block is side loaded, making it hard to generate on the fly and thus simpler to hard-coded for regtest. In our RPC tests we currently activate the NUs up to Sapling at height 1.
There was a problem hiding this comment.
Also, transparent coinbase UTXOs can't be spent until 100 blocks after they are created :-)
| sprout_anchor: sprout::tree::Root, | ||
| sapling_anchor: sapling::tree::Root, |
There was a problem hiding this comment.
If we make changes above, we'll also need to make them here.
WIP, to include content from #871
Rendered