feat: state: Fast migration for v15 #7901
Conversation
|
Some more experimenting and tweaking is needed, but this gets the job done well. Will share latest numbers soon (roughly 35 mins for premigration, 3 mins for migration on my box). |
3fe7765 to
ebeb7b0
Compare
| @@ -1245,8 +1247,15 @@ func PreUpgradeActorsV7(ctx context.Context, sm *stmgr.StateManager, cache stmgr | |||
| workerCount /= 2 | |||
| } | |||
|
|
|||
| config := nv15.Config{MaxWorkers: uint(workerCount)} | |||
| _, err := upgradeActorsV7Common(ctx, sm, cache, root, epoch, ts, config) | |||
| lbts, lbRoot, err := stmgr.GetLookbackTipSetForRound(ctx, sm, ts, epoch) | |||
There was a problem hiding this comment.
This migration requires us to be reorg-insensitive, so the premigration has to be based on finality.
| @@ -1255,7 +1264,12 @@ func upgradeActorsV7Common( | |||
| root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet, | |||
| config nv15.Config, | |||
| ) (cid.Cid, error) { | |||
| buf := blockstore.NewTieredBstore(sm.ChainStore().StateBlockstore(), blockstore.NewMemorySync()) | |||
| writeStore, err := autobatch.NewBlockstore(sm.ChainStore().StateBlockstore(), blockstore.NewMemorySync(), 100_000, 100, true) | |||
There was a problem hiding this comment.
100_000 here wasn't chosen very scientifically...
There was a problem hiding this comment.
This blockstore:
- Doesn't guarantee a flush order.
- Doesn't actually wait in
Flushfor the all data to be written. - Definitely wasn't written for this use-case.
3 isn't really an issue, but 1 and 2 are. Ideally the node won't be borked if you crash during a pre-migration.
|
Update to the tagged actors RC before landing this. |
Codecov Report
@@ Coverage Diff @@
## master #7901 +/- ##
==========================================
- Coverage 39.38% 39.33% -0.06%
==========================================
Files 655 656 +1
Lines 70254 70331 +77
==========================================
- Hits 27673 27662 -11
- Misses 37818 37903 +85
- Partials 4763 4766 +3
Continue to review full report at Codecov.
|
| {{if (ge .v 7)}} | ||
| Sv: toV{{.v}}SignedVoucher(*sv), | ||
| {{else}} | ||
| Sv: *sv, | ||
| {{end}} |
There was a problem hiding this comment.
nit: whitespace?
| {{if (ge .v 7)}} | |
| Sv: toV{{.v}}SignedVoucher(*sv), | |
| {{else}} | |
| Sv: *sv, | |
| {{end}} | |
| {{if (ge .v 7) -}} | |
| Sv: toV{{.v}}SignedVoucher(*sv), | |
| {{- else -}} | |
| Sv: *sv, | |
| {{- end}} |
| @@ -1255,7 +1264,12 @@ func upgradeActorsV7Common( | |||
| root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet, | |||
| config nv15.Config, | |||
| ) (cid.Cid, error) { | |||
| buf := blockstore.NewTieredBstore(sm.ChainStore().StateBlockstore(), blockstore.NewMemorySync()) | |||
| writeStore, err := autobatch.NewBlockstore(sm.ChainStore().StateBlockstore(), blockstore.NewMemorySync(), 100_000, 100, true) | |||
There was a problem hiding this comment.
This blockstore:
- Doesn't guarantee a flush order.
- Doesn't actually wait in
Flushfor the all data to be written. - Definitely wasn't written for this use-case.
3 isn't really an issue, but 1 and 2 are. Ideally the node won't be borked if you crash during a pre-migration.
I'd just implement a much simpler version of this. Basically, you need something with 3 stages:
Writes go into buffered. When it fills, Gets check buffered, then flushing, then backing. The background goroutine ensures that data is always flushed in-order. |
|
Superseded by #7933 |
Related Issues
Fixes #7870
Proposed Changes
Additional Info
Checklist
Before you mark the PR ready for review, please make sure that:
<PR type>: <#issue number> <area>: <change being made>fix: #1234 mempool: Introduce a cache for valid signaturesPR type: fix, feat, INTERFACE BREAKING CHANGE, CONSENSUS BREAKING, build, chore, ci, docs, misc,perf, refactor, revert, style, testarea: api, chain, state, vm, data transfer, market, mempool, message, block production, multisig, networking, paychan, proving, sealing, wallet