From 5f3a69e8f184fc62937beedf29d1a7c99364bc32 Mon Sep 17 00:00:00 2001 From: Phi Date: Tue, 11 Jul 2023 15:13:43 +0200 Subject: [PATCH 1/9] nv-skeleton documentation Add a tutorial for how one can create a nv-skeleton in Lotus --- .../misc/Building_a_network_skeleton.md | 220 ++++++++++++++++++ 1 file changed, 220 insertions(+) create mode 100644 documentation/misc/Building_a_network_skeleton.md diff --git a/documentation/misc/Building_a_network_skeleton.md b/documentation/misc/Building_a_network_skeleton.md new file mode 100644 index 00000000000..06eebc778cc --- /dev/null +++ b/documentation/misc/Building_a_network_skeleton.md @@ -0,0 +1,220 @@ +# Network Upgrade Skeleton in Lotus + +Follow these steps to create a skeleton for a network upgrade in Lotus: + +## Setup + +1. Clone the [go-state-types](https://github.com/filecoin-project/go-state-types) repository. + +2. In the Lotus repository, add `replace github.com/filecoin-project/go-state-types => ../go-state-types` to the very end of your Lotus `go.mod` file. + - This ensures that your local clone copy of `go-state-types` is used. Any changes you make there will be reflected in your Lotus project. + +## Actor Version Checklist + +1. Follow the [go-state-types actor version checklist](https://github.com/filecoin-project/go-state-types/blob/master/actors_version_checklist.md): + + - Copy `go-state-types/builtin/vX` to `go-state-types/builtin/v(X+1)`. + - Change all references from vX to v(X+1) in the new files. + - Add new network version to `network/version.go`. + - Add new actors version to `actors/version.go`. + - Add `Version(XX+1) Version = XX+1` as a constant. + - In `func VersionForNetwork` add `case network.Version(XX+1): return Version(XX+1), nil`. + - Add the new version to the gen step of the makefile. + - Add `$(GO_BIN) run ./builtin/v(XX+1)/gen/gen.go`. + +## Actor Version Lotus Integration Checklist + +1. Import new actors: + + - Create a mock actor-bundle for the new network version. + - In `/build/actors` run `./pack.sh vXX+1 vXX.0.0` where XX is the current actor bundle version. + +2. Define upgrade heights in `build/params_`: + + - Update the following files: + - `params_2k.go` + - Set previous `UpgradeXxxxxHeight = abi.ChainEpoch(-xx-1)` + - Add `var UpgradeXxxxxHeight = abi.ChainEpoch(200)` + - Add `UpgradeXxxxxHeight = getUpgradeHeight("LOTUS_XXXXX_HEIGHT", UpgradeXXXXHeight)` + - Set `const GenesisNetworkVersion = network.VersionXX` where XX is the network version you are upgrading from. + - `params_butterfly.go` + - Add comment with ?????? signaling that the new upgrade date is unkown + - Add `const UpgradeXxxxxHeight = 999999999999999` + - `params_calibnet.go` + - Add comment with `??????` signaling that the new upgrade date is unkown + - Add `const UpgradeXxxxxHeight = 999999999999999` + - `params_interop.go` + - set previous upgrade to `var UpgradeXxxxxHeigh = abi.ChainEpoch(-xx-1)` + - Add `const UpgradeXxxxxHeight = 50` + - `params_mainnet.go` + - Set previous upgrade to `const UpgradeXxxxxHeight = XX` + - Add comment with ???? signaling that the new upgrade date is unkown + - Add `var UpgradeXxxxxxHeight = abi.ChainEpoch(9999999999)` + - Change the `LOTUS_DISABLE_XXXX` env variable to the new network name + - `params_testground.go` + - Add `UpgradeXxxxxHeight abi.ChainEpoch = (-xx-1)` + +3. Generate adapters: + + - Update `gen/inlinegen-data.json`. + - Add `XX+1` to "actorVersions" and set "latestActorsVersion" to `XX+1`. + - Add `XX+1` to "networkVersions" and set "latestNetworkVersion" to `XX+1`. + + - Run `make actors-gen`. This generates the `/chain/actors/builtin/*` code, `/chain/actors/policy/policy.go` code, `/chain/actors/version.go`, and `/itest/kit/ensemble_opts_nv.go`. + +4. Update `chain/consensus/filcns/upgrades.go`. + - Import `nv(XX+1) "github.com/filecoin-project/go-state-types/builtin/v(XX+1)/migration`. + - Add Schedule. [^1] + - Add Migration. + +5. Add actorstype to the NewActorRegistry in `/chain/consensus/computestate.go`. + - Add `inv.Register(actorstypes.Version(XX+1), vm.ActorsVersionPredicate(actorstypes.Version(XX+1)), builtin.MakeRegistry(actorstypes.Version(XX+1))`. + +6. Add upgrade field to `api/types.go/ForkUpgradeParams`. + - Add `UpgradeXxxxxHeight abi.ChainEpoch` to `ForkUpgradeParams` struct. + +7. Add upgrade to `node/impl/full/state.go`. + - Add `UpgradeXxxxxHeight: build.UpgradeXxxxxHeight,`. + +8. Add network version to `chain/state/statetree.go`. + - Add `network.VersionXX+1` to `VersionForNetwork` function. + +9. Run `make gen`. + +10. Run `make docsgen-cli`. + +And you're done! This should create a network upgrade skeleton that you are able to run locally with your local go-state-types clones, and a mock Actors-bundle. This will allow you to: + +- Have a local developer network that starts at the current network version. +- Be able to see the Actor CIDs/Actor version for the mock v12-bundle through `lotus state actors-cids --network-version XX+1` +- Have a successful pre-migration. +- Complete Migration at upgrade epoch, but fail immidiately after the upgrade. + +At this point you are blocked on FVM/Actors work landing. + +// TODO: Create a video-tutorial going through all the steps + +[^1]: Here is an example of how you can add a schedule: + + ```go + { + Height: build.UpgradeXxxxHeight, + Network: network.Version(XX+1), + Migration: UpgradeActorsV(XX+1), + PreMigrations: []stmgr.PreMigration{{ + PreMigration: PreUpgradeActors(VXX+1), + StartWithin: 120, + DontStartWithin: 15, + StopWithin: 10, + }}, + Expensive: true, + }, + ``` + + This schedule should be added to the `DefaultUpgradeSchedule` function, specifically within the `updates` array. + +[^2]: Here is an example of how you can add a migration: + +```go +func PreUpgradeActorsV(XX+1)(ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache, root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) error { + // Use half the CPUs for pre-migration, but leave at least 3. + workerCount := MigrationMaxWorkerCount + if workerCount <= 4 { + workerCount = 1 + } else { + workerCount /= 2 + } + + lbts, lbRoot, err := stmgr.GetLookbackTipSetForRound(ctx, sm, ts, epoch) + if err != nil { + return xerrors.Errorf("error getting lookback ts for premigration: %w", err) + } + + config := migration.Config{ + MaxWorkers: uint(workerCount), + ProgressLogPeriod: time.Minute * 5, + } + + _, err = upgradeActorsV(XX+1)Common(ctx, sm, cache, lbRoot, epoch, lbts, config) + return err +} + +func UpgradeActorsV(XX+1)(ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache, cb stmgr.ExecMonitor, + root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) { + // Use all the CPUs except 2. + workerCount := MigrationMaxWorkerCount - 3 + if workerCount <= 0 { + workerCount = 1 + } + config := migration.Config{ + MaxWorkers: uint(workerCount), + JobQueueSize: 1000, + ResultQueueSize: 100, + ProgressLogPeriod: 10 * time.Second, + } + newRoot, err := upgradeActorsV(XX+1)Common(ctx, sm, cache, root, epoch, ts, config) + if err != nil { + return cid.Undef, xerrors.Errorf("migrating actors v11 state: %w", err) + } + return newRoot, nil +} + +func upgradeActorsV(XX+1)Common( + ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache, + root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet, + config migration.Config, +) (cid.Cid, error) { + writeStore := blockstore.NewAutobatch(ctx, sm.ChainStore().StateBlockstore(), units.GiB/4) + adtStore := store.ActorStore(ctx, writeStore) + // ensure that the manifest is loaded in the blockstore + if err := bundle.LoadBundles(ctx, writeStore, actorstypes.Version(XX+1)); err != nil { + return cid.Undef, xerrors.Errorf("failed to load manifest bundle: %w", err) + } + + // Load the state root. + var stateRoot types.StateRoot + if err := adtStore.Get(ctx, root, &stateRoot); err != nil { + return cid.Undef, xerrors.Errorf("failed to decode state root: %w", err) + } + + if stateRoot.Version != types.StateTreeVersion5 { + return cid.Undef, xerrors.Errorf( + "expected state root version 5 for actors v(XX+1) upgrade, got %d", + stateRoot.Version, + ) + } + + manifest, ok := actors.GetManifest(actorstypes.Version(XX+1)) + if !ok { + return cid.Undef, xerrors.Errorf("no manifest CID for v(XX+1) upgrade") + } + + // Perform the migration + newHamtRoot, err := nv(XX+1).MigrateStateTree(ctx, adtStore, manifest, stateRoot.Actors, epoch, config, + migrationLogger{}, cache) + if err != nil { + return cid.Undef, xerrors.Errorf("upgrading to actors v11: %w", err) + } + + // Persist the result. + newRoot, err := adtStore.Put(ctx, &types.StateRoot{ + Version: types.StateTreeVersion5, + Actors: newHamtRoot, + Info: stateRoot.Info, + }) + if err != nil { + return cid.Undef, xerrors.Errorf("failed to persist new state root: %w", err) + } + + // Persists the new tree and shuts down the flush worker + if err := writeStore.Flush(ctx); err != nil { + return cid.Undef, xerrors.Errorf("writeStore flush failed: %w", err) + } + + if err := writeStore.Shutdown(ctx); err != nil { + return cid.Undef, xerrors.Errorf("writeStore shutdown failed: %w", err) + } + + return newRoot, nil +} +``` \ No newline at end of file From 3c3a4505042cf48d1cea0cef7bd8b7b04a0e4a10 Mon Sep 17 00:00:00 2001 From: Phi Date: Tue, 11 Jul 2023 15:19:32 +0200 Subject: [PATCH 2/9] Add footnote for `Add migration` step Add footnote for `Add migration` step --- documentation/misc/Building_a_network_skeleton.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/misc/Building_a_network_skeleton.md b/documentation/misc/Building_a_network_skeleton.md index 06eebc778cc..c02388dd8ca 100644 --- a/documentation/misc/Building_a_network_skeleton.md +++ b/documentation/misc/Building_a_network_skeleton.md @@ -65,7 +65,7 @@ Follow these steps to create a skeleton for a network upgrade in Lotus: 4. Update `chain/consensus/filcns/upgrades.go`. - Import `nv(XX+1) "github.com/filecoin-project/go-state-types/builtin/v(XX+1)/migration`. - Add Schedule. [^1] - - Add Migration. + - Add Migration. [^2] 5. Add actorstype to the NewActorRegistry in `/chain/consensus/computestate.go`. - Add `inv.Register(actorstypes.Version(XX+1), vm.ActorsVersionPredicate(actorstypes.Version(XX+1)), builtin.MakeRegistry(actorstypes.Version(XX+1))`. From 5c5e0a50edc0cc3a82e1267f1120e747b77e3dde Mon Sep 17 00:00:00 2001 From: Phi Date: Tue, 11 Jul 2023 15:40:28 +0200 Subject: [PATCH 3/9] Indent migration-code Indent migration-code to make it show properly as a footnote. --- .../misc/Building_a_network_skeleton.md | 206 +++++++++--------- 1 file changed, 103 insertions(+), 103 deletions(-) diff --git a/documentation/misc/Building_a_network_skeleton.md b/documentation/misc/Building_a_network_skeleton.md index c02388dd8ca..c9830d7bfe6 100644 --- a/documentation/misc/Building_a_network_skeleton.md +++ b/documentation/misc/Building_a_network_skeleton.md @@ -115,106 +115,106 @@ At this point you are blocked on FVM/Actors work landing. [^2]: Here is an example of how you can add a migration: -```go -func PreUpgradeActorsV(XX+1)(ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache, root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) error { - // Use half the CPUs for pre-migration, but leave at least 3. - workerCount := MigrationMaxWorkerCount - if workerCount <= 4 { - workerCount = 1 - } else { - workerCount /= 2 - } - - lbts, lbRoot, err := stmgr.GetLookbackTipSetForRound(ctx, sm, ts, epoch) - if err != nil { - return xerrors.Errorf("error getting lookback ts for premigration: %w", err) - } - - config := migration.Config{ - MaxWorkers: uint(workerCount), - ProgressLogPeriod: time.Minute * 5, - } - - _, err = upgradeActorsV(XX+1)Common(ctx, sm, cache, lbRoot, epoch, lbts, config) - return err -} - -func UpgradeActorsV(XX+1)(ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache, cb stmgr.ExecMonitor, - root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) { - // Use all the CPUs except 2. - workerCount := MigrationMaxWorkerCount - 3 - if workerCount <= 0 { - workerCount = 1 - } - config := migration.Config{ - MaxWorkers: uint(workerCount), - JobQueueSize: 1000, - ResultQueueSize: 100, - ProgressLogPeriod: 10 * time.Second, - } - newRoot, err := upgradeActorsV(XX+1)Common(ctx, sm, cache, root, epoch, ts, config) - if err != nil { - return cid.Undef, xerrors.Errorf("migrating actors v11 state: %w", err) - } - return newRoot, nil -} - -func upgradeActorsV(XX+1)Common( - ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache, - root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet, - config migration.Config, -) (cid.Cid, error) { - writeStore := blockstore.NewAutobatch(ctx, sm.ChainStore().StateBlockstore(), units.GiB/4) - adtStore := store.ActorStore(ctx, writeStore) - // ensure that the manifest is loaded in the blockstore - if err := bundle.LoadBundles(ctx, writeStore, actorstypes.Version(XX+1)); err != nil { - return cid.Undef, xerrors.Errorf("failed to load manifest bundle: %w", err) - } - - // Load the state root. - var stateRoot types.StateRoot - if err := adtStore.Get(ctx, root, &stateRoot); err != nil { - return cid.Undef, xerrors.Errorf("failed to decode state root: %w", err) - } - - if stateRoot.Version != types.StateTreeVersion5 { - return cid.Undef, xerrors.Errorf( - "expected state root version 5 for actors v(XX+1) upgrade, got %d", - stateRoot.Version, - ) - } - - manifest, ok := actors.GetManifest(actorstypes.Version(XX+1)) - if !ok { - return cid.Undef, xerrors.Errorf("no manifest CID for v(XX+1) upgrade") - } - - // Perform the migration - newHamtRoot, err := nv(XX+1).MigrateStateTree(ctx, adtStore, manifest, stateRoot.Actors, epoch, config, - migrationLogger{}, cache) - if err != nil { - return cid.Undef, xerrors.Errorf("upgrading to actors v11: %w", err) - } - - // Persist the result. - newRoot, err := adtStore.Put(ctx, &types.StateRoot{ - Version: types.StateTreeVersion5, - Actors: newHamtRoot, - Info: stateRoot.Info, - }) - if err != nil { - return cid.Undef, xerrors.Errorf("failed to persist new state root: %w", err) - } - - // Persists the new tree and shuts down the flush worker - if err := writeStore.Flush(ctx); err != nil { - return cid.Undef, xerrors.Errorf("writeStore flush failed: %w", err) - } - - if err := writeStore.Shutdown(ctx); err != nil { - return cid.Undef, xerrors.Errorf("writeStore shutdown failed: %w", err) - } - - return newRoot, nil -} -``` \ No newline at end of file + ```go + func PreUpgradeActorsV(XX+1)(ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache, root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) error { + // Use half the CPUs for pre-migration, but leave at least 3. + workerCount := MigrationMaxWorkerCount + if workerCount <= 4 { + workerCount = 1 + } else { + workerCount /= 2 + } + + lbts, lbRoot, err := stmgr.GetLookbackTipSetForRound(ctx, sm, ts, epoch) + if err != nil { + return xerrors.Errorf("error getting lookback ts for premigration: %w", err) + } + + config := migration.Config{ + MaxWorkers: uint(workerCount), + ProgressLogPeriod: time.Minute * 5, + } + + _, err = upgradeActorsV(XX+1)Common(ctx, sm, cache, lbRoot, epoch, lbts, config) + return err + } + + func UpgradeActorsV(XX+1)(ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache, cb stmgr.ExecMonitor, + root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) { + // Use all the CPUs except 2. + workerCount := MigrationMaxWorkerCount - 3 + if workerCount <= 0 { + workerCount = 1 + } + config := migration.Config{ + MaxWorkers: uint(workerCount), + JobQueueSize: 1000, + ResultQueueSize: 100, + ProgressLogPeriod: 10 * time.Second, + } + newRoot, err := upgradeActorsV(XX+1)Common(ctx, sm, cache, root, epoch, ts, config) + if err != nil { + return cid.Undef, xerrors.Errorf("migrating actors v11 state: %w", err) + } + return newRoot, nil + } + + func upgradeActorsV(XX+1)Common( + ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache, + root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet, + config migration.Config, + ) (cid.Cid, error) { + writeStore := blockstore.NewAutobatch(ctx, sm.ChainStore().StateBlockstore(), units.GiB/4) + adtStore := store.ActorStore(ctx, writeStore) + // ensure that the manifest is loaded in the blockstore + if err := bundle.LoadBundles(ctx, writeStore, actorstypes.Version(XX+1)); err != nil { + return cid.Undef, xerrors.Errorf("failed to load manifest bundle: %w", err) + } + + // Load the state root. + var stateRoot types.StateRoot + if err := adtStore.Get(ctx, root, &stateRoot); err != nil { + return cid.Undef, xerrors.Errorf("failed to decode state root: %w", err) + } + + if stateRoot.Version != types.StateTreeVersion5 { + return cid.Undef, xerrors.Errorf( + "expected state root version 5 for actors v(XX+1) upgrade, got %d", + stateRoot.Version, + ) + } + + manifest, ok := actors.GetManifest(actorstypes.Version(XX+1)) + if !ok { + return cid.Undef, xerrors.Errorf("no manifest CID for v(XX+1) upgrade") + } + + // Perform the migration + newHamtRoot, err := nv(XX+1).MigrateStateTree(ctx, adtStore, manifest, stateRoot.Actors, epoch, config, + migrationLogger{}, cache) + if err != nil { + return cid.Undef, xerrors.Errorf("upgrading to actors v11: %w", err) + } + + // Persist the result. + newRoot, err := adtStore.Put(ctx, &types.StateRoot{ + Version: types.StateTreeVersion5, + Actors: newHamtRoot, + Info: stateRoot.Info, + }) + if err != nil { + return cid.Undef, xerrors.Errorf("failed to persist new state root: %w", err) + } + + // Persists the new tree and shuts down the flush worker + if err := writeStore.Flush(ctx); err != nil { + return cid.Undef, xerrors.Errorf("writeStore flush failed: %w", err) + } + + if err := writeStore.Shutdown(ctx); err != nil { + return cid.Undef, xerrors.Errorf("writeStore shutdown failed: %w", err) + } + + return newRoot, nil + } + ``` \ No newline at end of file From aba647f61d889917cce9114efa3e621066cf2a59 Mon Sep 17 00:00:00 2001 From: Phi Date: Mon, 23 Oct 2023 09:03:24 +0100 Subject: [PATCH 4/9] Add ref-fvm and filecoin-ffi checklist Add ref-fvm and filecoin-ffi checklist --- .../misc/Building_a_network_skeleton.md | 27 ++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/documentation/misc/Building_a_network_skeleton.md b/documentation/misc/Building_a_network_skeleton.md index c9830d7bfe6..013030979cc 100644 --- a/documentation/misc/Building_a_network_skeleton.md +++ b/documentation/misc/Building_a_network_skeleton.md @@ -1,15 +1,36 @@ # Network Upgrade Skeleton in Lotus -Follow these steps to create a skeleton for a network upgrade in Lotus: +This guide will walk you through the process of creating a skeleton for a network upgrade in Lotus. The process involves making changes in multiple repositories in the following order: + +1. [`ref-fvm`](#ref-fvm-checklist) +2. [`filecoin-ffi`](#filecoin-ffi-checklist) +3. [`go-state-types`](#go-state-types-skeleton-checklist) +4. [`lotus`](#lotus-checklist) + +Each repository has its own set of steps that need to be followed. This guide will provide detailed instructions for each repository. ## Setup +1. Clone the [ref-fvm](https://github.com/filecoin-project/ref-fvm.git) repository. + +1. Clone the [filecoin-ffi](https://github.com/filecoin-project/filecoin-ffi.git) repository. + 1. Clone the [go-state-types](https://github.com/filecoin-project/go-state-types) repository. -2. In the Lotus repository, add `replace github.com/filecoin-project/go-state-types => ../go-state-types` to the very end of your Lotus `go.mod` file. +2. In your Lotus repository, add `replace github.com/filecoin-project/go-state-types => ../go-state-types` to the very end of your Lotus `go.mod` file. - This ensures that your local clone copy of `go-state-types` is used. Any changes you make there will be reflected in your Lotus project. -## Actor Version Checklist +## Ref-FVM Checklist + +1. Add support for the new network version in Ref-FVM: + + - In `fvm/src/gas/price_list.rs` add the new network version to the `price_list_by_network_version` function. + - In `fvm/src/machine/default.rs` in the new function of your machine context, you will find a `SUPPORTED_VERSIONS` constant that defines the range of network versions (for networks that are not Hyperspace) supported. Bump this range to support your new network version. + - In `shared/src/version/mod.rs`, in the `NetworkVersion` implementation, you will find a series of constants representing different network versions. To add a new network version, you need to declare a new constant: `pub const (VXX+1): Self = Self(XX+1);` + +## Filecoin-FFI Checklist + +## Go-State-Types Skeleton Checklist 1. Follow the [go-state-types actor version checklist](https://github.com/filecoin-project/go-state-types/blob/master/actors_version_checklist.md): From 861c42c0d756f300f83a0421995750ca5c1739d5 Mon Sep 17 00:00:00 2001 From: Phi Date: Tue, 24 Oct 2023 09:22:18 +0100 Subject: [PATCH 5/9] Add Filecoin-FFI steps Add Filecoin-FFI steps --- documentation/misc/Building_a_network_skeleton.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/documentation/misc/Building_a_network_skeleton.md b/documentation/misc/Building_a_network_skeleton.md index 013030979cc..1e1347025dd 100644 --- a/documentation/misc/Building_a_network_skeleton.md +++ b/documentation/misc/Building_a_network_skeleton.md @@ -4,7 +4,7 @@ This guide will walk you through the process of creating a skeleton for a networ 1. [`ref-fvm`](#ref-fvm-checklist) 2. [`filecoin-ffi`](#filecoin-ffi-checklist) -3. [`go-state-types`](#go-state-types-skeleton-checklist) +3. [`go-state-types`](#go-state-types-checklist) 4. [`lotus`](#lotus-checklist) Each repository has its own set of steps that need to be followed. This guide will provide detailed instructions for each repository. @@ -30,7 +30,13 @@ Each repository has its own set of steps that need to be followed. This guide wi ## Filecoin-FFI Checklist -## Go-State-Types Skeleton Checklist +1. Update the `TryFrom` implementation for `EngineVersion` in `rust/src/fvm/engine.rs` + - Add the new network version number (XX+1) to the existing match arm for the network version. + +2. Patch the FVM-dependency (fvm3) in `rust/cargo.toml` to use the custom branch of the FVM created in the [Ref-FVM Checklist](#ref-fvm-checklist)) + - Add `features = ["your-ref-fvm-branch"]` to tell Cargo to use you Ref-FVM branch. + +## Go-State-Types Checklist 1. Follow the [go-state-types actor version checklist](https://github.com/filecoin-project/go-state-types/blob/master/actors_version_checklist.md): From f5d6fc1e9b82f41e30773724c9277d8182d314b7 Mon Sep 17 00:00:00 2001 From: Phi Date: Wed, 8 Nov 2023 08:19:45 +0100 Subject: [PATCH 6/9] Add step to params_butterfly.go Add step to params_butterfly.go --- documentation/misc/Building_a_network_skeleton.md | 1 + 1 file changed, 1 insertion(+) diff --git a/documentation/misc/Building_a_network_skeleton.md b/documentation/misc/Building_a_network_skeleton.md index 1e1347025dd..b40a2be1584 100644 --- a/documentation/misc/Building_a_network_skeleton.md +++ b/documentation/misc/Building_a_network_skeleton.md @@ -65,6 +65,7 @@ Each repository has its own set of steps that need to be followed. This guide wi - Add `UpgradeXxxxxHeight = getUpgradeHeight("LOTUS_XXXXX_HEIGHT", UpgradeXXXXHeight)` - Set `const GenesisNetworkVersion = network.VersionXX` where XX is the network version you are upgrading from. - `params_butterfly.go` + - set previous upgrade to `var UpgradeXxxxxHeigh = abi.ChainEpoch(-xx-1)` - Add comment with ?????? signaling that the new upgrade date is unkown - Add `const UpgradeXxxxxHeight = 999999999999999` - `params_calibnet.go` From 6bcdd1e7953f0fced8ceb0ce8fc12f95340eceaf Mon Sep 17 00:00:00 2001 From: Phi Date: Mon, 20 Nov 2023 20:31:12 +0100 Subject: [PATCH 7/9] Fix typo Fix typo --- documentation/misc/Building_a_network_skeleton.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/misc/Building_a_network_skeleton.md b/documentation/misc/Building_a_network_skeleton.md index b40a2be1584..d1af4aaaba8 100644 --- a/documentation/misc/Building_a_network_skeleton.md +++ b/documentation/misc/Building_a_network_skeleton.md @@ -114,7 +114,7 @@ Each repository has its own set of steps that need to be followed. This guide wi And you're done! This should create a network upgrade skeleton that you are able to run locally with your local go-state-types clones, and a mock Actors-bundle. This will allow you to: - Have a local developer network that starts at the current network version. -- Be able to see the Actor CIDs/Actor version for the mock v12-bundle through `lotus state actors-cids --network-version XX+1` +- Be able to see the Actor CIDs/Actor version for the mock v12-bundle through `lotus state actor-cids --network-version XX+1` - Have a successful pre-migration. - Complete Migration at upgrade epoch, but fail immidiately after the upgrade. From 72fcd414a8e4c145b5caf8545b1f0ae0a544a97c Mon Sep 17 00:00:00 2001 From: Phi Date: Tue, 21 Nov 2023 12:00:56 +0100 Subject: [PATCH 8/9] Add links to reference PRs Add links to reference PRs --- .../misc/Building_a_network_skeleton.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/documentation/misc/Building_a_network_skeleton.md b/documentation/misc/Building_a_network_skeleton.md index d1af4aaaba8..132b4ea3116 100644 --- a/documentation/misc/Building_a_network_skeleton.md +++ b/documentation/misc/Building_a_network_skeleton.md @@ -13,11 +13,11 @@ Each repository has its own set of steps that need to be followed. This guide wi 1. Clone the [ref-fvm](https://github.com/filecoin-project/ref-fvm.git) repository. -1. Clone the [filecoin-ffi](https://github.com/filecoin-project/filecoin-ffi.git) repository. +2. Clone the [filecoin-ffi](https://github.com/filecoin-project/filecoin-ffi.git) repository. -1. Clone the [go-state-types](https://github.com/filecoin-project/go-state-types) repository. +3. Clone the [go-state-types](https://github.com/filecoin-project/go-state-types) repository. -2. In your Lotus repository, add `replace github.com/filecoin-project/go-state-types => ../go-state-types` to the very end of your Lotus `go.mod` file. +4. In your Lotus repository, add `replace github.com/filecoin-project/go-state-types => ../go-state-types` to the very end of your Lotus `go.mod` file. - This ensures that your local clone copy of `go-state-types` is used. Any changes you make there will be reflected in your Lotus project. ## Ref-FVM Checklist @@ -28,6 +28,8 @@ Each repository has its own set of steps that need to be followed. This guide wi - In `fvm/src/machine/default.rs` in the new function of your machine context, you will find a `SUPPORTED_VERSIONS` constant that defines the range of network versions (for networks that are not Hyperspace) supported. Bump this range to support your new network version. - In `shared/src/version/mod.rs`, in the `NetworkVersion` implementation, you will find a series of constants representing different network versions. To add a new network version, you need to declare a new constant: `pub const (VXX+1): Self = Self(XX+1);` +You can take a look at [this Ref-FVM PR as a reference](https://github.com/filecoin-project/ref-fvm/pull/1929), which added the skeleton for network version 22. + ## Filecoin-FFI Checklist 1. Update the `TryFrom` implementation for `EngineVersion` in `rust/src/fvm/engine.rs` @@ -36,6 +38,8 @@ Each repository has its own set of steps that need to be followed. This guide wi 2. Patch the FVM-dependency (fvm3) in `rust/cargo.toml` to use the custom branch of the FVM created in the [Ref-FVM Checklist](#ref-fvm-checklist)) - Add `features = ["your-ref-fvm-branch"]` to tell Cargo to use you Ref-FVM branch. +You can take a look at this [Filecoin-FFI PR as a reference](https://github.com/filecoin-project/filecoin-ffi/pull/438), which added the skeleton for network version 22. + ## Go-State-Types Checklist 1. Follow the [go-state-types actor version checklist](https://github.com/filecoin-project/go-state-types/blob/master/actors_version_checklist.md): @@ -48,8 +52,10 @@ Each repository has its own set of steps that need to be followed. This guide wi - In `func VersionForNetwork` add `case network.Version(XX+1): return Version(XX+1), nil`. - Add the new version to the gen step of the makefile. - Add `$(GO_BIN) run ./builtin/v(XX+1)/gen/gen.go`. + +You can take a look at this [Go-State-Types PR as a reference](https://github.com/filecoin-project/go-state-types/pull/232), which added the skeleton for network version 22. -## Actor Version Lotus Integration Checklist +## Lotus Checklist 1. Import new actors: @@ -118,7 +124,7 @@ And you're done! This should create a network upgrade skeleton that you are able - Have a successful pre-migration. - Complete Migration at upgrade epoch, but fail immidiately after the upgrade. -At this point you are blocked on FVM/Actors work landing. +You can take a look at this [Lotus PR as a reference](https://github.com/filecoin-project/lotus/pull/11432), which added the skeleton for network version 22. // TODO: Create a video-tutorial going through all the steps From 36f7ced05fe67fa59b6e02320c5b6402c7f36c8a Mon Sep 17 00:00:00 2001 From: Phi Date: Wed, 29 Nov 2023 09:51:02 +0100 Subject: [PATCH 9/9] Update ref-fvm list Update ref-fvm list --- documentation/misc/Building_a_network_skeleton.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/documentation/misc/Building_a_network_skeleton.md b/documentation/misc/Building_a_network_skeleton.md index 132b4ea3116..5aea6f706f8 100644 --- a/documentation/misc/Building_a_network_skeleton.md +++ b/documentation/misc/Building_a_network_skeleton.md @@ -24,8 +24,9 @@ Each repository has its own set of steps that need to be followed. This guide wi 1. Add support for the new network version in Ref-FVM: - - In `fvm/src/gas/price_list.rs` add the new network version to the `price_list_by_network_version` function. - - In `fvm/src/machine/default.rs` in the new function of your machine context, you will find a `SUPPORTED_VERSIONS` constant that defines the range of network versions (for networks that are not Hyperspace) supported. Bump this range to support your new network version. + - In `fvm/Cargo.toml` add `nvXX-dev` as a feature flag in the [features]-section. + - In `fvm/src/gas/price_list.rs`, extend the `price_list_by_network_version` function to support the new network version with the `nvXX-dev` feature flag. + - In fvm/src/machine/default.rs, locate the new function within your machine context. You'll find a SUPPORTED_VERSIONS constant that sets the range of supported network versions. Update this range to include the new network version. Do this by replacing the existing feature flag nvXX-dev and NetworkVersion::VXX with the new ones corresponding to your new network version. - In `shared/src/version/mod.rs`, in the `NetworkVersion` implementation, you will find a series of constants representing different network versions. To add a new network version, you need to declare a new constant: `pub const (VXX+1): Self = Self(XX+1);` You can take a look at [this Ref-FVM PR as a reference](https://github.com/filecoin-project/ref-fvm/pull/1929), which added the skeleton for network version 22.