Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get signature endpoint: alternative PR with different packaging #507

Merged
merged 76 commits into from
Feb 10, 2023
Merged
Show file tree
Hide file tree
Changes from 73 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
e6219a2
base warp backend
Jan 19, 2023
b054615
add signature caching
Jan 19, 2023
183a5cd
add docs
Jan 20, 2023
6043c13
error handling
Jan 20, 2023
24fe40e
pr fixes
Jan 20, 2023
dc3e441
basic signature request
Jan 20, 2023
7820ec1
hash unsigned message for key
Jan 20, 2023
a2fa3a3
Merge branch 'warp-backend' into signature-handler
Jan 20, 2023
ff4d54d
implement new Request and RequestHandler interfaces
Jan 20, 2023
4f0403e
signature handler impl without constructing one
Jan 20, 2023
c729040
fix import
Jan 20, 2023
618a0de
Merge remote-tracking branch 'origin/master' into signature-handler
Jan 23, 2023
738d19e
quick pr fixes and merge
Jan 23, 2023
8d33345
quick pr fixes and merge
Jan 23, 2023
be5a4d6
Merge remote-tracking branch 'origin/master' into warp-backend
Jan 23, 2023
e8ac670
save signature instead of whole msg
Jan 23, 2023
bd75e6d
use avaGO cache
Jan 23, 2023
5d61bb0
rename warpBackend and docs
Jan 23, 2023
7b650b6
fix nits
Jan 23, 2023
41b86bf
Merge remote-tracking branch 'origin/master' into warp-backend
Jan 23, 2023
2cdd440
Merge branch 'warp-backend' into signature-handler
Jan 23, 2023
887fd6f
Update plugin/evm/warp_backend.go
Jan 23, 2023
7c8e06f
Update plugin/evm/warp_backend.go
Jan 23, 2023
b0f2ff3
fix pr nits
Jan 23, 2023
8fc39cd
pr fixes and testing
Jan 24, 2023
76a8a0e
type check for caching
Jan 24, 2023
4dc1d23
Merge branch 'warp-backend' into signature-handler
Jan 24, 2023
c745420
handlers and request before tests
Jan 24, 2023
17d1101
fix imports
Jan 24, 2023
4582f47
signature handler with stats and test
Jan 25, 2023
6a44563
use memdb and remove extra test
Jan 25, 2023
fb76223
remove unused
Jan 26, 2023
b7bfa17
fix imports
Jan 26, 2023
3d47a86
fix imports
Jan 26, 2023
ba6ee49
Merge remote-tracking branch 'origin/warp-backend' into signature-han…
Jan 26, 2023
cffb9cc
nit
Jan 26, 2023
005d90b
update license year
Jan 27, 2023
66a8238
use require noError
Jan 27, 2023
1389af4
Merge remote-tracking branch 'origin/master' into warp-backend
Jan 30, 2023
9a99797
saving message in db and pr fixes
Jan 30, 2023
67b7e6f
Merge branch 'warp-backend' into signature-handler
Jan 31, 2023
70e1d00
create noop signature handler and refactor code handler
Jan 31, 2023
de89d5b
get signature endpoint
cam-schultz Feb 1, 2023
562c8f0
add api arg to evm client
cam-schultz Feb 1, 2023
6d86d97
Merge remote-tracking branch 'origin/master' into signature-handler
Feb 2, 2023
76febba
Update sync/handlers/handler.go
Feb 2, 2023
48513b1
Merge branch 'signature-handler' of github.com:ava-labs/subnet-evm in…
Feb 2, 2023
ce25575
update backend return value
Feb 2, 2023
9d93fd1
refactor handlers to network handler
Feb 2, 2023
cfd843e
change constructor of handler stats
Feb 2, 2023
4b636e9
pr cleanups
Feb 2, 2023
9755d4f
warp api
cam-schultz Feb 6, 2023
cf5c907
initialize warp backend
cam-schultz Feb 6, 2023
656f286
Merge branch 'signature-handler' into get-signature-endpoint
cam-schultz Feb 6, 2023
fc668f9
build fix
cam-schultz Feb 6, 2023
dfcc95e
wip
cam-schultz Feb 7, 2023
cfcb650
warp api follows eth api pattern
cam-schultz Feb 7, 2023
0d4e895
cleanup and comments
cam-schultz Feb 7, 2023
cee8aa0
clean up response
cam-schultz Feb 8, 2023
93a43d0
fix warp client return type
cam-schultz Feb 8, 2023
453e543
nits for get-signature-endpoint (#502)
darioush Feb 9, 2023
ae7cdcf
Merge branch 'master' of github.com:ava-labs/subnet-evm into get-sign…
darioush Feb 9, 2023
6b46ff4
Merge branch 'get-signature-endpoint' of github.com:ava-labs/subnet-e…
darioush Feb 9, 2023
329dcb8
Merge branch 'master' into get-signature-endpoint
aaronbuchwald Feb 9, 2023
b0efb9d
resolve merge conflict
cam-schultz Feb 9, 2023
8626fc4
Merge branch 'get-signature-endpoint' of github.com:ava-labs/subnet-e…
cam-schultz Feb 9, 2023
caa66f7
warp: Group packages for symmetry w/ sync
darioush Feb 9, 2023
a3430ad
more reshuffle
darioush Feb 9, 2023
5c441cb
more shuffle
darioush Feb 9, 2023
2eed37f
pr comments
darioush Feb 9, 2023
587559c
fix
darioush Feb 9, 2023
8428f03
update to []byte
darioush Feb 9, 2023
555da6e
update svc return type
darioush Feb 9, 2023
b61f981
rename arg
darioush Feb 10, 2023
1a7c1e7
fix type
darioush Feb 10, 2023
99229ac
add stats pkg
darioush Feb 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 0 additions & 22 deletions handlers/stats/mock_stats.go

This file was deleted.

38 changes: 0 additions & 38 deletions handlers/stats/stats.go

This file was deleted.

1 change: 1 addition & 0 deletions plugin/evm/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ type Config struct {

// Subnet EVM APIs
SnowmanAPIEnabled bool `json:"snowman-api-enabled"`
WarpAPIEnabled bool `json:"warp-api-enabled"`
AdminAPIEnabled bool `json:"admin-api-enabled"`
AdminAPIDir string `json:"admin-api-dir"`

Expand Down
21 changes: 10 additions & 11 deletions handlers/handler.go → plugin/evm/network_handler.go
Original file line number Diff line number Diff line change
@@ -1,20 +1,19 @@
// (c) 2023, Ava Labs, Inc. All rights reserved.
// See the file LICENSE for licensing terms.

package handlers
package evm

import (
"context"

"github.com/ava-labs/avalanchego/codec"
"github.com/ava-labs/avalanchego/ids"
"github.com/ava-labs/subnet-evm/handlers/stats"
warpHandlers "github.com/ava-labs/subnet-evm/handlers/warp"
"github.com/ava-labs/subnet-evm/metrics"
"github.com/ava-labs/subnet-evm/plugin/evm/message"
"github.com/ava-labs/subnet-evm/sync/handlers"
syncHandlers "github.com/ava-labs/subnet-evm/sync/handlers"
syncStats "github.com/ava-labs/subnet-evm/sync/handlers/stats"
"github.com/ava-labs/subnet-evm/trie"
warpHandlers "github.com/ava-labs/subnet-evm/warp/handlers"
)

var _ message.RequestHandler = &networkHandler{}
Expand All @@ -26,18 +25,18 @@ type networkHandler struct {
signatureRequestHandler warpHandlers.SignatureRequestHandler
}

// NewNetworkHandler constructs the handler for serving network requests.
func NewNetworkHandler(
provider handlers.SyncDataProvider,
// newNetworkHandler constructs the handler for serving network requests.
func newNetworkHandler(
provider syncHandlers.SyncDataProvider,
evmTrieDB *trie.Database,
networkCodec codec.Manager,
) message.RequestHandler {
handlerStats := stats.NewHandlerStats(metrics.Enabled)
syncStats := syncStats.NewHandlerStats(metrics.Enabled)
return &networkHandler{
// State sync handlers
stateTrieLeafsRequestHandler: syncHandlers.NewLeafsRequestHandler(evmTrieDB, provider, networkCodec, handlerStats),
blockRequestHandler: syncHandlers.NewBlockRequestHandler(provider, networkCodec, handlerStats),
codeRequestHandler: syncHandlers.NewCodeRequestHandler(evmTrieDB.DiskDB(), networkCodec, handlerStats),
stateTrieLeafsRequestHandler: syncHandlers.NewLeafsRequestHandler(evmTrieDB, provider, networkCodec, syncStats),
blockRequestHandler: syncHandlers.NewBlockRequestHandler(provider, networkCodec, syncStats),
codeRequestHandler: syncHandlers.NewCodeRequestHandler(evmTrieDB.DiskDB(), networkCodec, syncStats),

// TODO: initialize actual signature request handler when warp is ready
signatureRequestHandler: &warpHandlers.NoopSignatureRequestHandler{},
Expand Down
31 changes: 26 additions & 5 deletions plugin/evm/vm.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ import (
"time"

avalanchegoMetrics "github.com/ava-labs/avalanchego/api/metrics"
"github.com/ava-labs/subnet-evm/handlers"
"github.com/prometheus/client_golang/prometheus"

"github.com/ava-labs/subnet-evm/commontype"
Expand All @@ -36,6 +35,7 @@ import (
statesyncclient "github.com/ava-labs/subnet-evm/sync/client"
"github.com/ava-labs/subnet-evm/sync/client/stats"
"github.com/ava-labs/subnet-evm/trie"
"github.com/ava-labs/subnet-evm/warp"

// Force-load tracer engine to trigger registration
//
Expand Down Expand Up @@ -81,9 +81,10 @@ const (
// and fail verification
maxFutureBlockTime = 10 * time.Second

decidedCacheSize = 100
missingCacheSize = 50
unverifiedCacheSize = 50
decidedCacheSize = 100
missingCacheSize = 50
unverifiedCacheSize = 50
warpSignatureCacheSize = 500

// Prefixes for metrics gatherers
ethMetricsPrefix = "eth"
Expand All @@ -102,6 +103,7 @@ var (
lastAcceptedKey = []byte("last_accepted_key")
acceptedPrefix = []byte("snowman_accepted")
metadataPrefix = []byte("metadata")
warpPrefix = []byte("warp")
ethDBPrefix = []byte("ethdb")
)

Expand Down Expand Up @@ -177,6 +179,10 @@ type VM struct {
// block.
acceptedBlockDB database.Database

// [warpDB] is used to store warp message signatures
// set to a prefixDB with the prefix [warpPrefix]
warpDB database.Database

toEngine chan<- commonEng.Message

syntacticBlockValidator BlockValidator
Expand Down Expand Up @@ -206,6 +212,10 @@ type VM struct {
// State sync server and client
StateSyncServer
StateSyncClient

// Avalanche Warp Messaging backend
// Used to serve BLS signatures of warp messages over RPC
warpBackend warp.WarpBackend
}

/*
Expand Down Expand Up @@ -274,6 +284,7 @@ func (vm *VM) Initialize(
vm.db = versiondb.New(baseDB)
vm.acceptedBlockDB = prefixdb.New(acceptedPrefix, vm.db)
vm.metadataDB = prefixdb.New(metadataPrefix, vm.db)
vm.warpDB = prefixdb.New(warpPrefix, vm.db)

if vm.config.InspectDatabase {
start := time.Now()
Expand Down Expand Up @@ -415,6 +426,9 @@ func (vm *VM) Initialize(
vm.Network = peer.NewNetwork(appSender, vm.networkCodec, message.CrossChainCodec, chainCtx.NodeID, vm.config.MaxOutboundActiveRequests, vm.config.MaxOutboundActiveCrossChainRequests)
vm.client = peer.NewNetworkClient(vm.Network)

// initialize warp backend
vm.warpBackend = warp.NewWarpBackend(vm.ctx, vm.warpDB, warpSignatureCacheSize)

if err := vm.initializeChain(lastAcceptedHash, vm.ethConfig); err != nil {
return err
}
Expand Down Expand Up @@ -606,7 +620,7 @@ func (vm *VM) setAppRequestHandlers() {
},
)

networkHandler := handlers.NewNetworkHandler(vm.blockChain, evmTrieDB, vm.networkCodec)
networkHandler := newNetworkHandler(vm.blockChain, evmTrieDB, vm.networkCodec)
vm.Network.SetRequestHandler(networkHandler)
}

Expand Down Expand Up @@ -795,6 +809,13 @@ func (vm *VM) CreateHandlers(context.Context) (map[string]*commonEng.HTTPHandler
enabledAPIs = append(enabledAPIs, "snowman")
}

if vm.config.WarpAPIEnabled {
if err := handler.RegisterName("warp", &warp.WarpAPI{Backend: vm.warpBackend}); err != nil {
return nil, err
}
enabledAPIs = append(enabledAPIs, "warp")
}

log.Info(fmt.Sprintf("Enabled APIs: %s", strings.Join(enabledAPIs, ", ")))
apis[ethRPCEndpoint] = &commonEng.HTTPHandler{
LockOptions: commonEng.NoLock,
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
// (c) 2023, Ava Labs, Inc. All rights reserved.
// See the file LICENSE for licensing terms.

package warp
package handlers

import (
"context"
"time"

"github.com/ava-labs/avalanchego/codec"
"github.com/ava-labs/avalanchego/ids"
"github.com/ava-labs/subnet-evm/handlers/warp/stats"
"github.com/ava-labs/subnet-evm/plugin/evm/message"
"github.com/ava-labs/subnet-evm/plugin/evm/warp"
"github.com/ava-labs/subnet-evm/warp"
"github.com/ethereum/go-ethereum/log"
)

Expand All @@ -25,10 +24,10 @@ type SignatureRequestHandler interface {
type signatureRequestHandler struct {
backend warp.WarpBackend
codec codec.Manager
stats stats.SignatureRequestHandlerStats
stats SignatureRequestHandlerStats
}

func NewSignatureRequestHandler(backend warp.WarpBackend, codec codec.Manager, stats stats.SignatureRequestHandlerStats) SignatureRequestHandler {
func NewSignatureRequestHandler(backend warp.WarpBackend, codec codec.Manager, stats SignatureRequestHandlerStats) SignatureRequestHandler {
return &signatureRequestHandler{
backend: backend,
codec: codec,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// (c) 2023, Ava Labs, Inc. All rights reserved.
// See the file LICENSE for licensing terms.

package warp
package handlers

import (
"context"
Expand All @@ -14,9 +14,8 @@ import (
"github.com/ava-labs/avalanchego/utils/crypto/bls"
"github.com/ava-labs/avalanchego/utils/hashing"
"github.com/ava-labs/avalanchego/vms/platformvm/teleporter"
"github.com/ava-labs/subnet-evm/handlers/stats"
"github.com/ava-labs/subnet-evm/plugin/evm/message"
"github.com/ava-labs/subnet-evm/plugin/evm/warp"
"github.com/ava-labs/subnet-evm/warp"
"github.com/stretchr/testify/require"
)

Expand All @@ -38,20 +37,20 @@ func TestSignatureHandler(t *testing.T) {
require.NoError(t, err)
unknownMessageID := ids.GenerateTestID()

mockHandlerStats := &stats.MockHandlerStats{}
mockHandlerStats := &MockSignatureRequestHandlerStats{}
signatureRequestHandler := NewSignatureRequestHandler(warpBackend, message.Codec, mockHandlerStats)

tests := map[string]struct {
setup func() (request message.SignatureRequest, expectedResponse []byte)
verifyStats func(t *testing.T, stats *stats.MockHandlerStats)
verifyStats func(t *testing.T, stats *MockSignatureRequestHandlerStats)
}{
"normal": {
setup: func() (request message.SignatureRequest, expectedResponse []byte) {
return message.SignatureRequest{
MessageID: messageID,
}, signature[:]
},
verifyStats: func(t *testing.T, stats *stats.MockHandlerStats) {
verifyStats: func(t *testing.T, stats *MockSignatureRequestHandlerStats) {
require.EqualValues(t, 1, mockHandlerStats.SignatureRequestCount)
require.EqualValues(t, 1, mockHandlerStats.SignatureRequestHit)
require.EqualValues(t, 0, mockHandlerStats.SignatureRequestMiss)
Expand All @@ -64,7 +63,7 @@ func TestSignatureHandler(t *testing.T) {
MessageID: unknownMessageID,
}, nil
},
verifyStats: func(t *testing.T, stats *stats.MockHandlerStats) {
verifyStats: func(t *testing.T, stats *MockSignatureRequestHandlerStats) {
require.EqualValues(t, 1, mockHandlerStats.SignatureRequestCount)
require.EqualValues(t, 1, mockHandlerStats.SignatureRequestMiss)
require.EqualValues(t, 0, mockHandlerStats.SignatureRequestHit)
Expand Down
2 changes: 1 addition & 1 deletion handlers/warp/stats/stats.go → warp/handlers/stats.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// (c) 2023, Ava Labs, Inc. All rights reserved.
// See the file LICENSE for licensing terms.

package stats

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wondering if we should keep the packaging to stats for consistency with the sync stats.

package handlers

import (
"sync"
Expand Down
45 changes: 45 additions & 0 deletions warp/warp_client.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
// (c) 2023, Ava Labs, Inc. All rights reserved.
// See the file LICENSE for licensing terms.

package warp

import (
"context"
"fmt"

"github.com/ava-labs/avalanchego/ids"
"github.com/ava-labs/subnet-evm/rpc"
"github.com/ethereum/go-ethereum/common/hexutil"
)

var _ WarpClient = (*warpClient)(nil)

type WarpClient interface {
GetSignature(ctx context.Context, signatureRequest ids.ID) ([]byte, error)
darioush marked this conversation as resolved.
Show resolved Hide resolved
}

// warpClient implementation for interacting with EVM [chain]
type warpClient struct {
client *rpc.Client
}

// NewWarpClient returns a WarpClient for interacting with EVM [chain]
func NewWarpClient(uri, chain string) (WarpClient, error) {
client, err := rpc.Dial(fmt.Sprintf("%s/ext/bc/%s/rpc", uri, chain))
if err != nil {
return nil, fmt.Errorf("failed to dial client. err: %w", err)
}
return &warpClient{
client: client,
}, nil
}

// GetSignature requests the BLS signature associated with a messageID
func (c *warpClient) GetSignature(ctx context.Context, messageID ids.ID) ([]byte, error) {
var res hexutil.Bytes
err := c.client.CallContext(ctx, &res, "warp_getSignature", messageID[:])
if err != nil {
return nil, fmt.Errorf("call to warp_getSignature failed. err: %w", err)
}
return res, err
}
26 changes: 26 additions & 0 deletions warp/warp_service.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
// (c) 2023, Ava Labs, Inc. All rights reserved.
// See the file LICENSE for licensing terms.

package warp

import (
"context"
"fmt"

"github.com/ava-labs/avalanchego/ids"
"github.com/ethereum/go-ethereum/common/hexutil"
)

// WarpAPI introduces snowman specific functionality to the evm
type WarpAPI struct {
Backend WarpBackend
}

// GetSignature returns the BLS signature associated with a messageID.
func (api *WarpAPI) GetSignature(ctx context.Context, messageID ids.ID) (hexutil.Bytes, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just confirming that the params argument in the JSON request can be hex encoded with this signature. Similarly, can you confirm that returning hexutil.Bytes directly gives a hex encoded output?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing locally with modified code to get a response:

curl --location --request POST 'http://127.0.0.1:9652/ext/bc/2ioeFdBb3qMEfnTQc9kd3CqXBmTmxqh96GyCY49b8X3UmZx5fZ/rpc' \
--header 'Content-Type: application/json' \
--data-raw '{
    "jsonrpc": "2.0",
    "method": "warp_getSignature",
    "params": [
        "11111111111111111111111111111111LpoYY"
    ],
    "id": 1
}'

Output:

{
    "jsonrpc": "2.0",
    "id": 1,
    "result": "0x0000000000000000000000000000000000000000000000000000000000000000"
}

with a different response output:

{
    "jsonrpc": "2.0",
    "id": 1,
    "result": "0x6d0d36349f470af56409c02b53d5dfcc1a78071961f456321d9c707489d7116f"
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does c.client.CallContext(ctx, &res, "warp_getSignature", messageID[:]) in warpClient.GetSignature encode the bytes as cb58? I had done that manually in #475, but did not test if it did that cb58 coding automatically.

Copy link
Collaborator Author

@darioush darioush Feb 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it needs to be ids.ID so the type has the json encoding attached.

signature, err := api.Backend.GetSignature(ctx, messageID)
if err != nil {
return nil, fmt.Errorf("failed to get signature for with error %w", err)
}
return signature[:], nil
}