Refactor e2e tests

.github/workflows/lint-tests-release.yml

@@ -63,29 +63,15 @@ jobs:
       - name: Yarn install
         run: yarn
         working-directory: ./contract-examples
-      - name: Run e2e tests
+      - name: Install AvalancheGo Release
         shell: bash
-        run: SKIP_NETWORK_RUNNER_START=true SKIP_NETWORK_RUNNER_SHUTDOWN=true ENABLE_SOLIDITY_TESTS=true scripts/run.sh
-
-  simulator_test:
-    name: Load testing with simulator
-    runs-on: ubuntu-latest
-    steps:
-      - name: Git checkout
-        uses: actions/checkout@v3
-        with:
-          fetch-depth: 0
-      - name: Set up Go
-        uses: actions/setup-go@v3
-        with:
-          go-version: 1.18
-      - name: Install dependencies with go module
+        run: BASEDIR=/tmp/e2e-test AVALANCHEGO_BUILD_PATH=/tmp/e2e-test/avalanchego ./scripts/install_avalanchego_release.sh
+      - name: Build Subnet-EVM Plugin Binary
         shell: bash
-        run: go mod download
-      - name: Run simulator tests
+        run: ./scripts/build.sh /tmp/e2e-test/avalanchego/plugins/srEXiWaHuhNyGwPUi444Tu47ZEDwxTWrbQiuD7FmgSAQ6X7Dy
+      - name: Run E2E Tests
         shell: bash
-        # skip shutdown so external simulator binary can run against the existing cluster
-        run: SKIP_NETWORK_RUNNER_SHUTDOWN=true RUN_SIMULATOR=true scripts/run.sh
+        run: AVALANCHEGO_BUILD_PATH=/tmp/e2e-test/avalanchego DATA_DIR=/tmp/e2e-test/data ./scripts/run_ginkgo.sh
 
   release:
     # needs: [lint_test, unit_test, e2e_test, simulator_test]

accounts/abi/bind/precompile_template.go

@@ -38,7 +38,7 @@ Typically, custom codes are required in only those areas.
 7- Write solidity tests for your precompile in contract-examples/test
 8- Create your genesis with your precompile enabled in tests/e2e/genesis/
 9- Create e2e test for your solidity test in tests/e2e/solidity/suites.go
-10- Run your e2e precompile Solidity tests with 'E2E=true ./scripts/run.sh'
+10- Run your e2e precompile Solidity tests with './scripts/run_ginkgo.sh'
 
 */
 

cmd/simulator/.gitignore

@@ -0,0 +1,34 @@
+simulator
+
+*.log
+*~
+.DS_Store
+
+# Binaries for programs and plugins
+*.exe
+*.exe~
+*.dll
+*.so
+*.dylib
+*.profile
+
+# Test binary, build with `go test -c`
+*.test
+
+# Output of the go coverage tool, specifically when used with LiteIDE
+*.out
+
+# ignore GoLand metafiles directory
+.idea/
+
+*logs/
+
+.vscode*
+
+.coverage
+
+bin/
+build/
+
+# goreleaser
+dist/

cmd/simulator/README.md

@@ -0,0 +1,89 @@
+# Load Simulator
+
+When building developing your own blockchain using `subnet-evm`, you may want to analyze how your fee parameterization behaves and/or how many resources your VM uses under different load patterns. For this reason, we developed `cmd/simulator`. `cmd/simulator` lets you drive arbitrary load across any number of [endpoints] with a user-specified `keys` directory (insecure) `timeout`, `concurrency`, `base-fee`, and `priority-fee`.
+
+## Building the Load Simulator
+
+To build the load simulator, navigate to the base of the simulator directory:
+
+```bash
+cd $GOPATH/src/github.com/ava-labs/subnet-evm/cmd/simulator
+```
+
+Build the simulator:
+
+```bash
+go build -o ./simulator *.go
+```
+
+To confirm that you built successfully, run the simulator and print the version:
+
+```bash
+./simulator -v
+```
+
+This should give the following output:
+
+```
+v0.0.1
+```
+
+To run the load simulator, you must first start an EVM based network. The load simulator works on both the C-Chain and Subnet-EVM, so we will start a single node network and run the load simulator on the C-Chain.
+
+To start a single node network, follow the instructions from the AvalancheGo [README](https://github.com/ava-labs/avalanchego#building-avalanchego) to build from source.
+
+Once you've built AvalancheGo, open the AvalancheGo directory in a separate terminal window and run a single node non-staking network with the following command:
+
+```bash
+./build/avalanchego --staking-enabled=false --network-id=local
+```
+
+:::warning
+The staking-enabled flag is only for local testing. Disabling staking serves two functions explicitly for testing purposes:
+
+1. Ignore stake weight on the P-Chain and count each connected peer as having a stake weight of 1
+2. Automatically opts in to validate every Subnet
+:::
+
+Once you have AvalancheGo running locally, it will be running an HTTP Server on the default port `9650`. This means that the RPC Endpoint for the C-Chain will be http://127.0.0.1:9650/ext/bc/C/rpc.
+
+Now, we can run the simulator command to simulate some load on the local C-Chain for 30s:
+
+```bash
+RPC_ENDPOINTS=http://127.0.0.1:9650/ext/bc/C/rpc
+./simulator --rpc-endpoints=$RPC_ENDPOINTS --keys=./.simulator/keys --timeout=30s --concurrency=10 --base-fee=300 --priority-fee=100
+```
+
+## Command Line Flags
+
+### `rpc-endpoints` (string)
+
+`rpc-endpoints` is a comma separated list of RPC endpoints to hit during the load test.
+
+### `keys` (string)
+
+`keys` specifies the directory to find the private keys to use throughout the test. The directory should contain files with the hex address as the name and the corresponding private key as the only content.
+
+If the test needs to generate more keys (to meet the number of workers specified by `concurrency`), it will save them in this directory to ensure that the private keys holding funds at the end of the test are preserved.
+
+:::warning
+The `keys` directory is not a secure form of storage for private keys. This should only be used in local network and short lived network testing where losing or compromising the keys is not an issue.
+:::
+
+Note: if none of the keys in this directory have any funds, then the simulator will log an address that it expects to receive funds in order to fund the load test and wait for those funds to arrive.
+
+### `timeout` (duration)
+
+`timeout` specifies the duration to simulate load on the network for this test.
+
+### `concurrency` (int)
+
+`concurrency` specifies the number of concurrent workers that should send transactions to the network throughout the test. Each worker in the load test is a pairing of a private key and an RPC Endpoint. The private key is used to generate a stream of transactions, which are issued to the RPC Endpoint.
+
+### `base-fee` (int)
+
+`base-fee` specifies the base fee (denominated in GWei) to use for every transaction generated during the load test (generates Dynamic Fee Transactions).
+
+### `priority-fee` (int)
+
+`priority-fee` specifies the priority fee (denominated in GWei) to use for every transaction generated during the load test (generates Dynamic Fee Transactions).

cmd/simulator/go.mod

@@ -3,78 +3,43 @@ module github.com/ava-labs/subnet-evm/cmd/simulator
 go 1.18
 
 require (
-	github.com/ava-labs/subnet-evm v0.0.0-00010101000000-000000000000
 	github.com/ethereum/go-ethereum v1.10.26
 	github.com/spf13/cobra v1.5.0
 	golang.org/x/sync v0.1.0
 	sigs.k8s.io/yaml v1.3.0
 )
 
-// always depend on the local version
-replace github.com/ava-labs/subnet-evm => ../..
-
 require (
 	github.com/VictoriaMetrics/fastcache v1.10.0 // indirect
-	github.com/ava-labs/avalanchego v1.9.7 // indirect
-	github.com/beorn7/perks v1.0.1 // indirect
 	github.com/btcsuite/btcd/btcec/v2 v2.3.2 // indirect
-	github.com/cenkalti/backoff/v4 v4.1.3 // indirect
-	github.com/cespare/xxhash/v2 v2.1.2 // indirect
-	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/deckarep/golang-set v1.8.0 // indirect
 	github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1 // indirect
-	github.com/go-logr/logr v1.2.3 // indirect
-	github.com/go-logr/stdr v1.2.2 // indirect
+	github.com/gballet/go-libpcsclite v0.0.0-20191108122812-4678299bea08 // indirect
 	github.com/go-ole/go-ole v1.2.6 // indirect
 	github.com/go-stack/stack v1.8.0 // indirect
-	github.com/golang/mock v1.6.0 // indirect
-	github.com/golang/protobuf v1.5.2 // indirect
-	github.com/golang/snappy v0.0.4 // indirect
-	github.com/google/uuid v1.2.0 // indirect
-	github.com/gorilla/rpc v1.2.0 // indirect
+	github.com/google/uuid v1.3.0 // indirect
 	github.com/gorilla/websocket v1.4.2 // indirect
-	github.com/grpc-ecosystem/grpc-gateway/v2 v2.12.0 // indirect
 	github.com/inconshreveable/mousetrap v1.0.0 // indirect
-	github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
-	github.com/mr-tron/base58 v1.2.0 // indirect
-	github.com/nbutton23/zxcvbn-go v0.0.0-20180912185939-ae427f1e4c1d // indirect
-	github.com/pmezard/go-difflib v1.0.0 // indirect
-	github.com/prometheus/client_golang v1.13.0 // indirect
-	github.com/prometheus/client_model v0.2.0 // indirect
-	github.com/prometheus/common v0.37.0 // indirect
-	github.com/prometheus/procfs v0.8.0 // indirect
+	github.com/kr/pretty v0.1.0 // indirect
+	github.com/mattn/go-colorable v0.1.12 // indirect
+	github.com/mattn/go-isatty v0.0.14 // indirect
+	github.com/mitchellh/mapstructure v1.5.0 // indirect
+	github.com/prometheus/tsdb v0.10.0 // indirect
 	github.com/rjeczalik/notify v0.9.2 // indirect
 	github.com/shirou/gopsutil v3.21.11+incompatible // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
+	github.com/status-im/keycard-go v0.0.0-20200402102358-957c09536969 // indirect
 	github.com/stretchr/testify v1.8.1 // indirect
-	github.com/supranational/blst v0.3.11-0.20220920110316-f72618070295 // indirect
 	github.com/syndtr/goleveldb v1.0.1-0.20220614013038-64ee5596c38a // indirect
 	github.com/tklauser/go-sysconf v0.3.5 // indirect
 	github.com/tklauser/numcpus v0.2.2 // indirect
+	github.com/tyler-smith/go-bip39 v1.0.2 // indirect
 	github.com/yusufpapurcu/wmi v1.2.2 // indirect
-	go.opentelemetry.io/otel v1.11.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.11.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.11.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.11.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.11.0 // indirect
-	go.opentelemetry.io/otel/sdk v1.11.0 // indirect
-	go.opentelemetry.io/otel/trace v1.11.0 // indirect
-	go.opentelemetry.io/proto/otlp v0.19.0 // indirect
-	go.uber.org/atomic v1.10.0 // indirect
-	go.uber.org/multierr v1.8.0 // indirect
-	go.uber.org/zap v1.24.0 // indirect
 	golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa // indirect
-	golang.org/x/exp v0.0.0-20220426173459-3bcf042a4bf5 // indirect
-	golang.org/x/net v0.1.0 // indirect
-	golang.org/x/sys v0.1.0 // indirect
-	golang.org/x/term v0.1.0 // indirect
-	golang.org/x/text v0.4.0 // indirect
+	golang.org/x/sys v0.3.0 // indirect
+	golang.org/x/text v0.5.0 // indirect
 	golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac // indirect
-	gonum.org/v1/gonum v0.11.0 // indirect
-	google.golang.org/genproto v0.0.0-20221027153422-115e99e71e1c // indirect
-	google.golang.org/grpc v1.51.0-dev // indirect
-	google.golang.org/protobuf v1.28.1 // indirect
-	gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect
+	gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
+	gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
-	gopkg.in/yaml.v3 v3.0.1 // indirect
 )