World State Improvements #9
Conversation
|
|
||
| ### Snapshotting | ||
|
|
||
| The current snapshot implementation provides 2 options. Either a node only offers `current` state or it offers both `current` state and **all** historical state. The intention being that only full archive nodes would need to offer historical state. This has turned out to be an invalid assumption as most nodes will probably want to provide a short history of snapshotted state leaving archive nodes to provide the full history. We will therefore need to provide the ability to configure a window of blocks for which state is available, up the current latest block number. Currently, it is very hard to prune historical state. |
There was a problem hiding this comment.
The intention being that only full archive nodes would need to offer historical state. This has turned out to be an invalid assumption as most nodes will probably want to provide a short history of snapshotted state leaving archive nodes to provide the full history.
This is especially relevant when a user is proving their client-side tx and they need to request merkle paths against the state root they're working with. If the node advances to a new block while the user is still simulating their tx, they'd have to throw everything out and start over to use the new root, unless they can request merkle paths for a given recent root.
There was a problem hiding this comment.
Really nice write up - very clever ideas! This should work nicely! Thanks for sharing. I've only added one small question.
Oh, and this question: I recall in aztec connect we were storing tree stumps (I think that's what they were called?), as a way of reducing the amount of storage. Have we dismissed that as being too complicated?
|
|
||
| ## Images | ||
|
|
||
| Taking the 3-stage (uncommitted, committed, snapshotted) state architecture outlined above, we can make the concept generic and introduce the term `image` representing a cahe of 'pending' state accumulated through a selection of blocks on top of the `finalized` chain state. The world state module will create an `image` for every tree at every block on the pending chain as it is read from L1. Additionally, images can be requested on demand for block-building activities and later destroyed. Images that extend beyond the previous pending epoch will not duplicate the state of that epoch. Instead they will reference the image at the tip of that epoch. All images will reference the `finalized` store. |
There was a problem hiding this comment.
as it is read from L1
Perhaps we also want to enable nodes to 'optimistically' create images from not-yet-proposed-to-L1 blocks that they've seen in the mempool. Even if a proposal hasn't been finalised on L1, a user might wish to act on their view of the mempool. E.g. the next sequencer might take a risk and make a head start on preparing their block, or an (advanced) user might wish to optimistically begin generating a tx from the state root of the not-yet-proposed-to-L1 block.
There was a problem hiding this comment.
Yes, a sequencer who is next in line will most definitely want to do this. If they wait until the prior proposal is confirmed on L1 they will only have a short period to build their own proposal.
|
@spalladino are we good to merge this one? |
That's for @PhilWindle to decide! |
# Goal We want to be able to kick off more than one `prove(blocknumber)` job in the prover-node at the same time. We currently cannot do it because the prover-node has a single world-state, and building a proof modifies world-state. In particular, preparing the inputs for base rollups modifies state trees, and finalising the block modifies the archive tree. # Why? This'll be needed for the proving integration contest, in case we generate blocks faster than the time it takes to prove them. It may still be useful once we move to epoch proving and sequencer-prover coordination, since the same prover could be picked for generating proofs for two consecutive epochs. # How? ## The easy way Easiest approach is to keep everything as-is today, and clone the world state before kicking off a job. Eventually, once we implement [Phil's world-state](AztecProtocol/engineering-designs#9), we can use the writeable world-state snapshots for this. **This is what we're doing on this PR.** ## The not-so-easy way Another approach is to decouple input-generation from proving. Today the prover-orchestrator is responsible for computing all inputs, but this is not strictly needed. We can have one component that generates all inputs, modifies world-state, and outputs a graph of proving jobs (as Mitch commented [here](https://aztecprotocol.slack.com/archives/C04BTJAA694/p1722195399887399?thread_ts=1722195378.794149&cid=C04BTJAA694)). And then another component that orchestrates the execution of proving jobs exclusively based on their dependencies. Note that this new component would be a good fit for generating a block in the sequencer, which today runs an orchestrator without proving enabled to get to a block header. It's unclear whether this component should run everything serially (like [the old block builder](https://aztecprotocol.slack.com/archives/C04BTJAA694/p1722195399887399?thread_ts=1722195378.794149&cid=C04BTJAA694) did), or it makes more sense to fan out circuit simulation jobs to workers (like the proving orchestrator can do now).
No description provided.