Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(store/v2): simplify genesis flow #22435

Merged
merged 27 commits into from
Nov 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
03d2595
initial commit of genesis refactor
kocubinski Nov 5, 2024
31078b0
go mod tidy all
kocubinski Nov 5, 2024
86a218b
go mod tidy all
kocubinski Nov 5, 2024
26f56ad
add err handling
kocubinski Nov 5, 2024
96d5904
upgrade to iavl@v1.3.1
kocubinski Nov 5, 2024
05ffc7d
chore: upgrade to iavl@v1.3.1
kocubinski Nov 5, 2024
03ba454
maybe fix BaseApp assumptions?
kocubinski Nov 5, 2024
c9c31b2
try a store/v1 fix
kocubinski Nov 5, 2024
ea177e6
revert change to baseapp/abci.go
kocubinski Nov 5, 2024
b5274ad
fix doc comment
kocubinski Nov 5, 2024
eab9201
Merge branch 'kocu/iavl-v1.3.1' into kocu/genesis-rf
kocubinski Nov 5, 2024
6188010
Merge branch 'main' of github.com:cosmos/cosmos-sdk into kocu/genesis-rf
kocubinski Nov 5, 2024
18985f4
Merge branch 'main' into kocu/genesis-rf
tac0turtle Nov 6, 2024
864c549
Merge branch 'main' of github.com:cosmos/cosmos-sdk into kocu/genesis-rf
kocubinski Nov 7, 2024
2b05e28
Merge branch 'kocu/genesis-rf' of github.com:cosmos/cosmos-sdk into k…
kocubinski Nov 7, 2024
342c547
go mod tidy all
kocubinski Nov 7, 2024
bef693d
rm unused import
kocubinski Nov 7, 2024
ef3b9ae
call Hash() before turning commit info
kocubinski Nov 7, 2024
461705e
fix integration tests to work with new genesis flow
kocubinski Nov 7, 2024
2ca44a0
add replace
kocubinski Nov 7, 2024
fd648e4
add another replace
kocubinski Nov 7, 2024
c2d0d8c
try fix rocksdb tests
kocubinski Nov 7, 2024
d38885d
fix tests, rm unused code
kocubinski Nov 7, 2024
7089eab
rm WorkingHash() from root
kocubinski Nov 7, 2024
bae9ee7
fix cometbft tests
kocubinski Nov 7, 2024
5e03575
core changelog
kocubinski Nov 7, 2024
6fa61a1
fix simapp/v2 tests
kocubinski Nov 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions core/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ Ref: https://keepachangelog.com/en/1.0.0/

## [Unreleased]

### API Breaking

* [#22435](https://github.com/cosmos/cosmos-sdk/pull/22435) Add `Version uint64` field to `store.Changeset` and update `Changeset` constructors to accept a `version uint64` as their first argument.

### Features

* [#22326](https://github.com/cosmos/cosmos-sdk/pull/22326) Introduce codec package in order to facilitate removal of Cosmos SDK dependency in modules.
Expand Down
8 changes: 5 additions & 3 deletions core/store/changeset.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (

// Changeset is a list of changes to be written to disk
type Changeset struct {
Version uint64
Changes []StateChanges
}

Expand All @@ -29,11 +30,11 @@ type KVPair = struct {
Remove bool
}

func NewChangeset() *Changeset {
return &Changeset{}
func NewChangeset(version uint64) *Changeset {
return &Changeset{Version: version}
}

func NewChangesetWithPairs(pairs map[string]KVPairs) *Changeset {
func NewChangesetWithPairs(version uint64, pairs map[string]KVPairs) *Changeset {
changes := make([]StateChanges, len(pairs))
i := 0
for storeKey, kvPairs := range pairs {
Expand All @@ -44,6 +45,7 @@ func NewChangesetWithPairs(pairs map[string]KVPairs) *Changeset {
i++
}
return &Changeset{
Version: version,
Changes: changes,
}
}
Expand Down
1 change: 1 addition & 0 deletions runtime/v2/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ go 1.23
// server v2 integration
replace (
cosmossdk.io/api => ../../api
cosmossdk.io/core => ../../core
cosmossdk.io/core/testing => ../../core/testing
cosmossdk.io/server/v2/appmanager => ../../server/v2/appmanager
cosmossdk.io/server/v2/stf => ../../server/v2/stf
Expand Down
2 changes: 0 additions & 2 deletions runtime/v2/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ buf.build/gen/go/cometbft/cometbft/protocolbuffers/go v1.35.1-20240701160653-fed
buf.build/gen/go/cometbft/cometbft/protocolbuffers/go v1.35.1-20240701160653-fedbb9acfd2f.1/go.mod h1:JTBMfyi+qAXUHumX+rcD2WIq9FNWmdcNh5MjBnSw0L0=
buf.build/gen/go/cosmos/gogo-proto/protocolbuffers/go v1.35.1-20240130113600-88ef6483f90f.1 h1:F78ecjvMtgd1aZ1Aj9cvBjURxVGCYvRM+OOy5eR+pjw=
buf.build/gen/go/cosmos/gogo-proto/protocolbuffers/go v1.35.1-20240130113600-88ef6483f90f.1/go.mod h1:zqi/LZjZhyvjCMTEVIwAf5VRlkLduuCfqmZxgoormq0=
cosmossdk.io/core v1.0.0-alpha.5 h1:McjYXAQ6XcT20v2uHyH7PhoWH8V+mebzfVFqT3GinsI=
cosmossdk.io/core v1.0.0-alpha.5/go.mod h1:3u9cWq1FAVtiiCrDPpo4LhR+9V6k/ycSG4/Y/tREWCY=
cosmossdk.io/depinject v1.1.0 h1:wLan7LG35VM7Yo6ov0jId3RHWCGRhe8E8bsuARorl5E=
cosmossdk.io/depinject v1.1.0/go.mod h1:kkI5H9jCGHeKeYWXTqYdruogYrEeWvBQCw1Pj4/eCFI=
cosmossdk.io/errors/v2 v2.0.0-20240731132947-df72853b3ca5 h1:IQNdY2kB+k+1OM2DvqFG1+UgeU1JzZrWtwuWzI3ZfwA=
Expand Down
30 changes: 9 additions & 21 deletions server/v2/cometbft/abci.go
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ func (c *Consensus[T]) InitChain(ctx context.Context, req *abciproto.InitChainRe
IsGenesis: true,
}

blockresponse, genesisState, err := c.app.InitGenesis(
blockResponse, genesisState, err := c.app.InitGenesis(
ctx,
br,
req.AppStateBytes,
Expand All @@ -338,17 +338,16 @@ func (c *Consensus[T]) InitChain(ctx context.Context, req *abciproto.InitChainRe
return nil, fmt.Errorf("genesis state init failure: %w", err)
}

for _, txRes := range blockresponse.TxResults {
for _, txRes := range blockResponse.TxResults {
if err := txRes.Error; err != nil {
space, code, log := errorsmod.ABCIInfo(err, c.cfg.AppTomlConfig.Trace)
c.logger.Warn("genesis tx failed", "codespace", space, "code", code, "log", log)
space, code, txLog := errorsmod.ABCIInfo(err, c.cfg.AppTomlConfig.Trace)
c.logger.Warn("genesis tx failed", "codespace", space, "code", code, "log", txLog)
}
}

validatorUpdates := intoABCIValidatorUpdates(blockresponse.ValidatorUpdates)
validatorUpdates := intoABCIValidatorUpdates(blockResponse.ValidatorUpdates)

// set the initial version of the store
if err := c.store.SetInitialVersion(uint64(req.InitialHeight)); err != nil {
if err := c.store.SetInitialVersion(uint64(req.InitialHeight - 1)); err != nil {
return nil, fmt.Errorf("failed to set initial version: %w", err)
}

Expand All @@ -357,9 +356,10 @@ func (c *Consensus[T]) InitChain(ctx context.Context, req *abciproto.InitChainRe
return nil, err
}
cs := &store.Changeset{
Version: uint64(req.InitialHeight - 1),
Changes: stateChanges,
}
stateRoot, err := c.store.WorkingHash(cs)
stateRoot, err := c.store.Commit(cs)
if err != nil {
return nil, fmt.Errorf("unable to write the changeset: %w", err)
}
Expand Down Expand Up @@ -455,18 +455,6 @@ func (c *Consensus[T]) FinalizeBlock(
return nil, err
}

// we don't need to deliver the block in the genesis block
if req.Height == int64(c.initialHeight) {
appHash, err := c.store.Commit(store.NewChangeset())
if err != nil {
return nil, fmt.Errorf("unable to commit the changeset: %w", err)
}
c.lastCommittedHeight.Store(req.Height)
return &abciproto.FinalizeBlockResponse{
AppHash: appHash,
}, nil
}

// TODO(tip): can we expect some txs to not decode? if so, what we do in this case? this does not seem to be the case,
// considering that prepare and process always decode txs, assuming they're the ones providing txs we should never
// have a tx that fails decoding.
Expand Down Expand Up @@ -507,7 +495,7 @@ func (c *Consensus[T]) FinalizeBlock(
if err != nil {
return nil, err
}
appHash, err := c.store.Commit(&store.Changeset{Changes: stateChanges})
appHash, err := c.store.Commit(&store.Changeset{Version: uint64(req.Height), Changes: stateChanges})
if err != nil {
return nil, fmt.Errorf("unable to commit the changeset: %w", err)
}
Expand Down
35 changes: 22 additions & 13 deletions server/v2/cometbft/abci_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import (

var (
sum = sha256.Sum256([]byte("test-hash"))
emptyHash = sha256.Sum256([]byte(""))
DefaulConsensusParams = &v1.ConsensusParams{
Block: &v1.BlockParams{
MaxGas: 5000000,
Expand Down Expand Up @@ -124,6 +125,7 @@ func TestConsensus_InitChain_Without_UpdateParam(t *testing.T) {
_, err = c.FinalizeBlock(context.Background(), &abciproto.FinalizeBlockRequest{
Time: time.Now(),
Height: 1,
Hash: emptyHash[:],
})
require.NoError(t, err)
assertStoreLatestVersion(t, mockStore, 1)
Expand All @@ -144,6 +146,7 @@ func TestConsensus_InitChain_With_UpdateParam(t *testing.T) {
_, err = c.FinalizeBlock(context.Background(), &abciproto.FinalizeBlockRequest{
Time: time.Now(),
Height: 1,
Hash: emptyHash[:],
})
require.NoError(t, err)

Expand All @@ -159,15 +162,16 @@ func TestConsensus_InitChain_Invalid_Height(t *testing.T) {
InitialHeight: 2,
})
require.NoError(t, err)
assertStoreLatestVersion(t, mockStore, 0)
assertStoreLatestVersion(t, mockStore, 1)

// Shouldn't be able to commit genesis block 2
// Shouldn't be able to commit genesis block 3
_, err = c.FinalizeBlock(context.Background(), &abciproto.FinalizeBlockRequest{
Time: time.Now(),
Height: 2,
Height: 3,
Hash: emptyHash[:],
})
require.Error(t, err)
require.True(t, strings.Contains(err.Error(), "unable to commit the changeset"))
require.True(t, strings.Contains(err.Error(), "invalid height"))
}

func TestConsensus_FinalizeBlock_Invalid_Height(t *testing.T) {
Expand All @@ -182,12 +186,14 @@ func TestConsensus_FinalizeBlock_Invalid_Height(t *testing.T) {
_, err = c.FinalizeBlock(context.Background(), &abciproto.FinalizeBlockRequest{
Time: time.Now(),
Height: 1,
Hash: emptyHash[:],
})
require.NoError(t, err)

_, err = c.FinalizeBlock(context.Background(), &abciproto.FinalizeBlockRequest{
Time: time.Now(),
Height: 3,
Hash: emptyHash[:],
})
require.Error(t, err)
}
Expand All @@ -206,6 +212,7 @@ func TestConsensus_FinalizeBlock_NoTxs(t *testing.T) {
_, err = c.FinalizeBlock(context.Background(), &abciproto.FinalizeBlockRequest{
Time: time.Now(),
Height: 1,
Hash: emptyHash[:],
})
require.NoError(t, err)

Expand Down Expand Up @@ -236,6 +243,7 @@ func TestConsensus_FinalizeBlock_MultiTxs_OutOfGas(t *testing.T) {
_, err = c.FinalizeBlock(context.Background(), &abciproto.FinalizeBlockRequest{
Time: time.Now(),
Height: 1,
Hash: emptyHash[:],
})
require.NoError(t, err)

Expand Down Expand Up @@ -267,6 +275,7 @@ func TestConsensus_FinalizeBlock_MultiTxs(t *testing.T) {
_, err = c.FinalizeBlock(context.Background(), &abciproto.FinalizeBlockRequest{
Time: time.Now(),
Height: 1,
Hash: emptyHash[:],
})
require.NoError(t, err)

Expand Down Expand Up @@ -554,6 +563,7 @@ func TestConsensus_Info(t *testing.T) {
_, err = c.FinalizeBlock(context.Background(), &abciproto.FinalizeBlockRequest{
Time: time.Now(),
Height: 1,
Hash: emptyHash[:],
})
require.NoError(t, err)

Expand All @@ -570,7 +580,8 @@ func TestConsensus_Query(t *testing.T) {
c := setUpConsensus(t, 100_000, cometmock.MockMempool[mock.Tx]{})

// Write data to state storage
err := c.store.GetStateStorage().ApplyChangeset(1, &store.Changeset{
err := c.store.GetStateStorage().ApplyChangeset(&store.Changeset{
Version: 1,
Changes: []store.StateChanges{
{
Actor: actorName,
Expand All @@ -597,6 +608,7 @@ func TestConsensus_Query(t *testing.T) {
Time: time.Now(),
Height: 1,
Txs: [][]byte{mockTx.Bytes()},
Hash: emptyHash[:],
})
require.NoError(t, err)

Expand Down Expand Up @@ -687,22 +699,19 @@ func setUpConsensus(t *testing.T, gasLimit uint64, mempool mempool.Mempool[mock.
nil,
)

return NewConsensus[mock.Tx](log.NewNopLogger(), "testing-app", am, func() error { return nil }, mempool, map[string]struct{}{}, nil, mockStore, Config{AppTomlConfig: DefaultAppTomlConfig()}, mock.TxCodec{}, "test")
return NewConsensus[mock.Tx](log.NewNopLogger(), "testing-app", am, func() error { return nil },
mempool, map[string]struct{}{}, nil, mockStore,
Config{AppTomlConfig: DefaultAppTomlConfig()}, mock.TxCodec{}, "test")
}

// Check target version same with store's latest version
// And should have commit info of target version
// If block 0, commitInfo returned should be nil
func assertStoreLatestVersion(t *testing.T, store types.Store, target uint64) {
t.Helper()
version, err := store.GetLatestVersion()
require.NoError(t, err)
require.Equal(t, version, target)
require.Equal(t, target, version)
commitInfo, err := store.GetStateCommitment().GetCommitInfo(version)
require.NoError(t, err)
if target != 0 {
require.Equal(t, commitInfo.Version, target)
} else {
require.Nil(t, commitInfo)
}
require.Equal(t, target, commitInfo.Version)
}
1 change: 1 addition & 0 deletions server/v2/cometbft/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ go 1.23.1

replace (
cosmossdk.io/api => ../../../api
cosmossdk.io/core => ../../../core
cosmossdk.io/core/testing => ../../../core/testing
cosmossdk.io/server/v2 => ../
cosmossdk.io/server/v2/appmanager => ../appmanager
Expand Down
2 changes: 0 additions & 2 deletions server/v2/cometbft/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMT
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cosmossdk.io/collections v0.4.0 h1:PFmwj2W8szgpD5nOd8GWH6AbYNi1f2J6akWXJ7P5t9s=
cosmossdk.io/collections v0.4.0/go.mod h1:oa5lUING2dP+gdDquow+QjlF45eL1t4TJDypgGd+tv0=
cosmossdk.io/core v1.0.0-alpha.5 h1:McjYXAQ6XcT20v2uHyH7PhoWH8V+mebzfVFqT3GinsI=
cosmossdk.io/core v1.0.0-alpha.5/go.mod h1:3u9cWq1FAVtiiCrDPpo4LhR+9V6k/ycSG4/Y/tREWCY=
cosmossdk.io/depinject v1.1.0 h1:wLan7LG35VM7Yo6ov0jId3RHWCGRhe8E8bsuARorl5E=
cosmossdk.io/depinject v1.1.0/go.mod h1:kkI5H9jCGHeKeYWXTqYdruogYrEeWvBQCw1Pj4/eCFI=
cosmossdk.io/errors v1.0.1 h1:bzu+Kcr0kS/1DuPBtUFdWjzLqyUuCiyHjyJB6srBV/0=
Expand Down
20 changes: 2 additions & 18 deletions server/v2/cometbft/internal/mock/mock_store.go
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,7 @@ func (s *MockStore) StateLatest() (uint64, corestore.ReaderMap, error) {
}

func (s *MockStore) Commit(changeset *corestore.Changeset) (corestore.Hash, error) {
v, _, _ := s.StateLatest()
err := s.Storage.ApplyChangeset(v, changeset)
err := s.Storage.ApplyChangeset(changeset)
if err != nil {
return []byte{}, err
}
Expand All @@ -70,8 +69,7 @@ func (s *MockStore) Commit(changeset *corestore.Changeset) (corestore.Hash, erro
return []byte{}, err
}

commitInfo, err := s.Committer.Commit(v + 1)
fmt.Println("commitInfo", commitInfo, err)
_, err = s.Committer.Commit(changeset.Version)
return []byte{}, err
}

Expand Down Expand Up @@ -127,17 +125,3 @@ func (s *MockStore) LastCommitID() (proof.CommitID, error) {
func (s *MockStore) SetInitialVersion(v uint64) error {
return s.Committer.SetInitialVersion(v)
}

func (s *MockStore) WorkingHash(changeset *corestore.Changeset) (corestore.Hash, error) {
v, _, _ := s.StateLatest()
err := s.Storage.ApplyChangeset(v, changeset)
if err != nil {
return []byte{}, err
}

err = s.Committer.WriteChangeset(changeset)
if err != nil {
return []byte{}, err
}
return []byte{}, nil
}
4 changes: 0 additions & 4 deletions server/v2/cometbft/types/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,6 @@ type Store interface {
// SetInitialVersion sets the initial version of the store.
SetInitialVersion(uint64) error

// WorkingHash writes the provided changeset to the state and returns
// the working hash of the state.
WorkingHash(*store.Changeset) (store.Hash, error)

// Commit commits the provided changeset and returns
// the new state root of the state.
Commit(*store.Changeset) (store.Hash, error)
Expand Down
1 change: 1 addition & 0 deletions server/v2/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ go 1.23

replace (
cosmossdk.io/api => ../../api
cosmossdk.io/core => ../../core
cosmossdk.io/server/v2/appmanager => ./appmanager
cosmossdk.io/server/v2/stf => ./stf
cosmossdk.io/store/v2 => ../../store/v2
Expand Down
2 changes: 0 additions & 2 deletions server/v2/go.sum
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cosmossdk.io/core v1.0.0-alpha.5 h1:McjYXAQ6XcT20v2uHyH7PhoWH8V+mebzfVFqT3GinsI=
cosmossdk.io/core v1.0.0-alpha.5/go.mod h1:3u9cWq1FAVtiiCrDPpo4LhR+9V6k/ycSG4/Y/tREWCY=
cosmossdk.io/core/testing v0.0.0-20240923163230-04da382a9f29 h1:NxxUo0GMJUbIuVg0R70e3cbn9eFTEuMr7ev1AFvypdY=
cosmossdk.io/core/testing v0.0.0-20240923163230-04da382a9f29/go.mod h1:8s2tPeJtSiQuoyPmr2Ag7meikonISO4Fv4MoO8+ORrs=
cosmossdk.io/errors/v2 v2.0.0-20240731132947-df72853b3ca5 h1:IQNdY2kB+k+1OM2DvqFG1+UgeU1JzZrWtwuWzI3ZfwA=
Expand Down
2 changes: 1 addition & 1 deletion server/v2/store/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ func (s *Server[T]) Start(context.Context) error {
}

func (s *Server[T]) Stop(context.Context) error {
return s.store.Close()
return nil
}

func (s *Server[T]) CLICommands() serverv2.CLIConfig {
Expand Down
Loading
Loading