chore: merge release/v1.18.0 into master #9597

arajasek · 2022-11-06T19:31:56Z

Related Issues

This was requested by @jennijuju for users wanting to sync the reset calibration testnet with master.

Proposed Changes

Merges the release/v1.18.0 branch into master, updating the build version to 1.19.0-dev.

Additional Info

Checklist

Before you mark the PR ready for review, please make sure that:

Commits have a clear commit message.
PR title is in the form of of <PR type>: <area>: <change being made>
- example: fix: mempool: Introduce a cache for valid signatures
- PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
- area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
New features have usage guidelines and / or documentation updates in
- Lotus Documentation
- Discussion Tutorials
Tests exist for new functionality or change in behavior
CI is green

* Allow builtin-actors to return a map of methods * go mod * Fix tests * Fix tests, check carefully please

feat: refactor: remove NewestNetworkVersion

* Integrate datacap actor * Implement datacap actor in chain/builtin

feat: API: support typed errors over RPC

… recovery sectors

feat: wdpost: Add ability to only have single partition per msg for partitions with recovery sectors

…-for-beneficiary fix: state: Return beneficiary info from miner state Info()

…ges in progress

…ly-upgrade-heights chore: butterfly: set new upgrade height for butterflynet

…t-upgrade-heights chore: calibnet: update calibnet upgrade heights

chore: release: bump version to rc5

chore: butterfly: artifacts

chore: artifacts: update calibnet artifacts

feat: shed: Add invariance tests for v9

jennijuju · 2022-11-07T15:46:18Z

CHANGELOG.md

@@ -1,5 +1,92 @@
 # Lotus changelog

+# 1.18.0-rc5 / 2022-11-1


not meant to be here ?

I think it is? Because the new version is v1.19.0-dev, so it's a bit odd if changelog is still 1.17.2?

So i would say merge in 1.18.0-rc5 changelog now, and when we have the final release, update to 1.18.0?

chain/actors/builtin/market/v3.go

chain/actors/builtin/market/v2.go

chain/actors/builtin/verifreg/v2.go

chain/consensus/filcns/upgrades.go

magik6k · 2022-11-07T16:59:36Z

blockstore/autobatch.go

-	return bs.Get(ctx, c)
+	return nil, ipld.ErrNotFound{Cid: c}


Shouldn't this technically go to backingBs again?

magik6k · 2022-11-07T17:01:31Z

build/params_2k.go

@@ -56,7 +56,7 @@ var UpgradeOhSnapHeight = abi.ChainEpoch(-18)

 var UpgradeSkyrHeight = abi.ChainEpoch(-19)

-var UpgradeV17Height = abi.ChainEpoch(99999999999999)
+var UpgradeSharkHeight = abi.ChainEpoch(100)


Can we keep this consistent with other upgrades in 2k devnets and set this to -20?

Eventually? I like starting on the prev version, and upgrading early as we're testing a new nv. After Shark sharks, I would support bumping devnet to starting from v17 by default.

chain/actors/builtin/datacap/actor.go.template

magik6k · 2022-11-07T17:19:34Z

chain/actors/builtin/registry.go

+	// method names always match the field names in the
+	// `builtin.Method*` structs (tested in the specs-actors
+	// tests).
+	fnName := runtime.FuncForPC(ev.Pointer()).Name()


That's mildly scary, but I guess fiine? Would be really nice to have a unit test in lotus making sure we notice when this breaks

Really, we should get rid of this. This is lotus-vm legacy stuff.

We do have integ tests that break (cuz they depend on the MethodMeta being setup correctly)...which is also the only reason anyone notices when we break this...

magik6k · 2022-11-07T17:27:51Z

chain/consensus/filcns/upgrades.go

 		Network:   network.Version17,
 		Migration: UpgradeActorsV9,
 		PreMigrations: []stmgr.PreMigration{{
 			PreMigration:    PreUpgradeActorsV9,
-			StartWithin:     180,
+			StartWithin:     240,


Do we have numbers for premigrations? I'm assuming that we do, but want to sanity check

Yeah, we're looking at roughly 40 mins for the premigration, but we really want this to finish, so 2 hours is the window I'm going with.

magik6k · 2022-11-07T17:39:57Z

node/impl/full/state.go

@@ -769,6 +771,135 @@ func (m *StateModule) StateMarketStorageDeal(ctx context.Context, dealId abi.Dea
 	return stmgr.GetStorageDeal(ctx, m.StateManager, dealId, ts)
 }

+func (a *StateAPI) StateGetAllocationForPendingDeal(ctx context.Context, dealId abi.DealID, tsk types.TipSetKey) (*verifregtypes.Allocation, error) {


Not something to do now, but looking at this code it really looks like we could make it a lot less duplicated by sprinkling some helpers with generics here

magik6k · 2022-11-07T17:50:22Z

testplans/lotus-soup/go.mod

@@ -86,6 +86,7 @@ require (
 	github.com/filecoin-project/go-bitfield v0.2.4 // indirect
 	github.com/filecoin-project/go-cbor-util v0.0.1 // indirect
 	github.com/filecoin-project/go-commp-utils v0.1.3 // indirect
+	github.com/filecoin-project/go-commp-utils/nonffi v0.0.0-20220905160352-62059082a837 // indirect


Yeee, good catch.

Stebalien · 2022-11-07T18:47:08Z

blockstore/autobatch.go

-	return bs.Get(ctx, c)
+	return nil, ipld.ErrNotFound{Cid: c}


Stebalien · 2022-11-07T18:48:57Z

build/builtin_actors_test.go

@@ -25,7 +25,7 @@ func TestRegistration(t *testing.T) {
 	require.True(t, found)
 	require.True(t, manifestCid.Defined())

-	for _, key := range actors.GetBuiltinActorsKeys() {
+	for _, key := range actors.GetBuiltinActorsKeys(actorstypes.Version8) {


hm. latest version?

Stebalien · 2022-11-07T18:53:05Z

chain/actors/builtin/registry.go

+	// method names always match the field names in the
+	// `builtin.Method*` structs (tested in the specs-actors
+	// tests).
+	fnName := runtime.FuncForPC(ev.Pointer()).Name()


Really, we should get rid of this. This is lotus-vm legacy stuff.

Stebalien · 2022-11-07T18:56:57Z

chain/messagepool/messagepool.go

 	epoch := curTs.Height() + 1
 	minGas := vm.PricelistByEpoch(epoch).OnChainMessage(m.ChainLength())

-	if err := m.VMMessage().ValidForBlockInclusion(minGas.Total(), build.NewestNetworkVersion); err != nil {
+	if err := m.VMMessage().ValidForBlockInclusion(minGas.Total(), mp.api.StateNetworkVersion(ctx, epoch)); err != nil {


So, the old code here was to avoid adding message to the pool that would be invalid very shortly. Is there a good reason to change this?

Are you suggesting dropping the check? I'd be okay with that.

The reason it's changing is because we're getting rid of NewestNetworkVersion, which is an overall terrible variable.

arajasek and others added 30 commits September 20, 2022 15:56

build: set version to v1.18.0-dev

99db94f

chore: actors: Allow builtin-actors to return a map of methods (#9342)

94add97

* Allow builtin-actors to return a map of methods * go mod * Fix tests * Fix tests, check carefully please

Delete lotus-pond (#9352)

9e94dc5

feat: add StateNetworkVersion to mpool API

347d92b

chore: refactor: rename NewestNetworkVersion

32670e8

Merge pull request #9351 from filecoin-project/asr/remove-newest-network

ff99415

feat: refactor: remove NewestNetworkVersion

feat: actors: Integrate datacap actor into lotus (#9348)

37be576

* Integrate datacap actor * Implement datacap actor in chain/builtin

feat: support typed errors over RPC

93f94d4

chore: deps: update to go-jsonrpc 0.1.8

6189932

remove duplicate import

5b96496

fix: itest: check for closed connection

0626286

chore: refactor: move retry test to API

3c195e4

address magik supernit

3e81c56

Merge pull request #9061 from filecoin-project/asr/rpc-errors

d89b196

feat: API: support typed errors over RPC

Add ability to only have single partition per msg for partitions with…

6696de6

… recovery sectors

doc gen

eb82d55

Address comments

4904411

Merge pull request #9427 from filecoin-project/sbansal/backport-9413

fd2b827

feat: wdpost: Add ability to only have single partition per msg for partitions with recovery sectors

Return beneficiary info from miner state Info()

fa64b2d

Merge pull request #9440 from filecoin-project/gstuart/fix-miner-info…

fc59d2b

…-for-beneficiary fix: state: Return beneficiary info from miner state Info()

Update builtin-actors to dev/20220922-v9 which includes FIP-0045 chan…

4218612

…ges in progress

Integrate verifreg changes to lotus

bc4f9bb

Setup datacap actor

530e6a0

Update builtin-actors to dev/20220922-v9-1

7c62f9e

Update datacap actor to query datacap instead of verifreg

1973e44

update gst

558a8bc

update markets

b4c04ad

update actors with hamt fix

64b1a52

update gst

c246366

Update datacap to parse tokens

a604a60

geoff-vball and others added 12 commits November 1, 2022 12:39

set new upgrade height for butterfly

b0a2863

Merge pull request #9585 from filecoin-project/gstuart/update-butterf…

8b65834

…ly-upgrade-heights chore: butterfly: set new upgrade height for butterflynet

tweak calibnet upgrade schedule

ea5b60d

Merge pull request #9584 from filecoin-project/gstuart/update-calibne…

1608c7c

…t-upgrade-heights chore: calibnet: update calibnet upgrade heights

Merge branch 'release/v1.18.0' into gstuart/bump-version-rc

1e09eff

clarify changelog note about calibnet

546b656

Merge pull request #9583 from filecoin-project/gstuart/bump-version-rc

fe8b801

chore: release: bump version to rc5

butterfly

d186168

Merge pull request #9586 from filecoin-project/jen/buttttttt

694db60

chore: butterfly: artifacts

update calibnet artifacts

065e87c

Merge pull request #9588 from filecoin-project/gstuart/calib-atrifacts

fa9d34f

chore: artifacts: update calibnet artifacts

Merge pull request #9521 from filecoin-project/gstuart/invariance-test

c1e1d00

feat: shed: Add invariance tests for v9

arajasek requested a review from a team as a code owner November 6, 2022 19:31

arajasek force-pushed the asr/merge-release-into-master branch 2 times, most recently from a728636 to 2892ce0 Compare November 6, 2022 19:39

Merge branch 'release/v1.18.0' into asr/merge-release-into-master

c0b7343

arajasek force-pushed the asr/merge-release-into-master branch from 2892ce0 to c0b7343 Compare November 6, 2022 19:40

jennijuju reviewed Nov 7, 2022

View reviewed changes

chain/actors/builtin/market/v3.go Show resolved Hide resolved

jennijuju reviewed Nov 7, 2022

View reviewed changes

chain/actors/builtin/market/v2.go Show resolved Hide resolved

jennijuju reviewed Nov 7, 2022

View reviewed changes

chain/actors/builtin/verifreg/v2.go Show resolved Hide resolved

jennijuju reviewed Nov 7, 2022

View reviewed changes

chain/consensus/filcns/upgrades.go Show resolved Hide resolved

magik6k mentioned this pull request Nov 7, 2022

feat: Retrieval into remote blockstores #9565

Merged

magik6k reviewed Nov 7, 2022

View reviewed changes

jennijuju approved these changes Nov 7, 2022

View reviewed changes

Stebalien reviewed Nov 7, 2022

View reviewed changes

arajasek added 2 commits November 7, 2022 15:34

address merge review

f77c849

Merge branch 'master' into asr/merge-release-into-master

be5adc7

arajasek merged commit 65ee059 into master Nov 7, 2022

arajasek deleted the asr/merge-release-into-master branch November 7, 2022 20:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: merge release/v1.18.0 into master #9597

chore: merge release/v1.18.0 into master #9597

arajasek commented Nov 6, 2022

jennijuju Nov 7, 2022

arajasek Nov 7, 2022

magik6k Nov 7, 2022

Stebalien Nov 7, 2022

magik6k Nov 7, 2022

arajasek Nov 7, 2022

magik6k Nov 7, 2022

Stebalien Nov 7, 2022

arajasek Nov 7, 2022

magik6k Nov 7, 2022

arajasek Nov 7, 2022

magik6k Nov 7, 2022

magik6k Nov 7, 2022

arajasek Nov 7, 2022

Stebalien Nov 7, 2022

Stebalien Nov 7, 2022

arajasek Nov 7, 2022

Stebalien Nov 7, 2022

Stebalien Nov 7, 2022

arajasek Nov 7, 2022

chore: merge release/v1.18.0 into master #9597

chore: merge release/v1.18.0 into master #9597

Conversation

arajasek commented Nov 6, 2022

Related Issues

Proposed Changes

Additional Info

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment