tests: update and re-generate cross-client test-suite and tests

Tests are generated on top of the latest
ethereum/go-ethereum suite version: a380655.

tests: tidy up code

Date: 2022-08-17 08:17:36-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: generate state subtests

This run was run against a clean version
of the upstream versions. It passed all tests
first try.
A keeper.

Date: 2022-08-17 08:06:32-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: (experiment) remove test runner gasprice=1 default

Date: 2022-08-17 07:39:20-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: refactor and clean up code (noop)

Date: 2022-08-17 07:37:28-05:00
Signed-off-by: meows <b5c6@protonmail.com>

make.test.out:

Date: 2022-08-16 17:53:18-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: generate state tests w/ no fails

Date: 2022-08-16 17:52:14-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: again, again, fixup maybe the stupid test thingy

Date: 2022-08-16 17:08:29-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: yet another patch to figure out the new exception handling biz

Date: 2022-08-16 16:15:13-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: generate state tests

Date: 2022-08-16 15:54:39-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: fixup previous commit; I forgot the post data besides the exception

Date: 2022-08-16 15:01:13-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: fix error handling/tx types for London:Mystique test gen

Date: 2022-08-16 15:00:09-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: (wip) generate state tests again

These FAILED but with the 'transaction type
not supported' for a London:Mystique conversion.

Date: 2022-08-16 14:35:34-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: fix gen function to copy the new TxBytes post state field

These were missing, apparently inconsequentially,
but now they seem to be where they should be.

Date: 2022-08-16 14:26:41-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: handle expected exceptions for test filling

stTransactionTest/ValueOverflow.json was failing
because it expects an exception, but the parse error
was getting thrown like it was unexpected.

Now the gen functions handle ExpectExceptions
better and the issue is not an issue anymore.

Date: 2022-08-16 14:25:00-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: generate state tests, bump submodule

fixes gasprice issue cited in previous commit

Date: 2022-08-16 14:07:26-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: generate tests, bump submodule

Date: 2022-08-16 14:06:27-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: set test Tx gasPrice if not set

ETC_Mystique does not adopt EIP1559.
Thus, it does not configure baseFee,
and txes require gas prices.

An example test (stBadOpcode/opc0CDiffPlaces.json)
does not configure gas price.
So we set a default as 1 in case the value is not set.
This allows the test data to get parsed toMesssage
properly, and we can generate the tests right.

Date: 2022-08-16 13:39:57-05:00
Signed-off-by: meows <b5c6@protonmail.com>

generate state tests again

This writes now to the new LegacyTests submodule.

Signed-off-by: meows <b5c6@protonmail.com>

tests: update testdata submodule to generated state test version

Date: 2022-08-16 09:56:20-05:00
Signed-off-by: meows <b5c6@protonmail.com>

:eyeglasses: :pen:

Signed-off-by: meows <b5c6@protonmail.com>

tests: use .ExpectException instead of handrolling expections

This field was added somewhat recently,
and is already implemented in the state_test.go
file and runner.

Date: 2022-08-16 09:03:53-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: add more expected failures for accessLists

Date: 2022-08-16 08:57:25-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: skip accessList tests for Homestead

The command 'make tests-generate-state' paniced:

    --- PASS: TestGenStateAll/testdata/GeneralStateTests/stEIP3607/transactionCollidingWithNonEmptyAccount_send.json (0.00s)
        --- PASS: TestGenStateAll/testdata/GeneralStateTests/stEIP3607/transactionCollidingWithNonEmptyAccount_send.json/ETC_Mystique/0/trie (0.00s)
        --- PASS: TestGenStateAll/testdata/GeneralStateTests/stEIP3607/transactionCollidingWithNonEmptyAccount_send.json/London/0/trie (0.00s)
    --- FAIL: TestGenStateAll/testdata/GeneralStateTests/stExample/accessListExample.json (0.01s)
        --- PASS: TestGenStateAll/testdata/GeneralStateTests/stExample/accessListExample.json/ETC_Agharta/0/trie (0.00s)
        --- PASS: TestGenStateAll/testdata/GeneralStateTests/stExample/accessListExample.json/ETC_Agharta/1/trie (0.00s)
        --- FAIL: TestGenStateAll/testdata/GeneralStateTests/stExample/accessListExample.json/Homestead/0/trie (0.00s)
panic: transaction type not supported [recovered]
        panic: transaction type not supported

goroutine 72552 [running]:
testing.tRunner.func1.2({0xb07800, 0xc00007a8b0})
        /home/ia/go1.18.3.linux-amd64/src/testing/testing.go:1389 +0x24e
testing.tRunner.func1()
        /home/ia/go1.18.3.linux-amd64/src/testing/testing.go:1392 +0x39f
panic({0xb07800, 0xc00007a8b0})
        /home/ia/go1.18.3.linux-amd64/src/runtime/panic.go:838 +0x207
github.com/ethereum/go-ethereum/tests.(*testMatcherGen).stateTestRunner.func1(0xc000821d40)
        /home/ia/go/src/github.com/ethereum/go-ethereum/tests/state_mgen_test.go:331 +0x209
testing.tRunner(0xc000821d40, 0xc0004ab1a0)
        /home/ia/go1.18.3.linux-amd64/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
        /home/ia/go1.18.3.linux-amd64/src/testing/testing.go:1486 +0x35f
FAIL    github.com/ethereum/go-ethereum/tests   293.108s
FAIL
Makefile:101: recipe for target 'tests-generate-state' failed
make: *** [tests-generate-state] Error 1

Date: 2022-08-16 08:44:48-05:00
Signed-off-by: meows <b5c6@protonmail.com>

params/coregeth.json.d,tests: make tests-generate-difficulty; yields testdata 78c4480bab

Date: 2022-08-16 08:24:16-05:00
Signed-off-by: meows <b5c6@protonmail.com>

tests: difficulty tests to use BasicTests dir (not .core-geth)

Date: 2022-08-16 08:20:50-05:00
Signed-off-by: meows <b5c6@protonmail.com>

params/coregeth.json.d/custom_mainnetwork_difficulty_test.json

@@ -1,6 +1,9 @@
 {
     "networkId": 1,
     "chainId": 1,
+    "supportedProtocolVersions": [
+        66
+    ],
     "eip2FBlock": 1150000,
     "eip7FBlock": 1150000,
     "daoForkBlock": 1920000,

params/coregeth.json.d/difficulty_json_difficulty_test.json

@@ -2,8 +2,7 @@
     "networkId": 1,
     "chainId": 1,
     "supportedProtocolVersions": [
-        66,
-        65
+        66
     ],
     "eip2FBlock": 1150000,
     "eip7FBlock": 1150000,

params/coregeth.json.d/mainnetwork_difficulty_test.json

@@ -2,8 +2,7 @@
     "networkId": 1,
     "chainId": 1,
     "supportedProtocolVersions": [
-        66,
-        65
+        66
     ],
     "eip2FBlock": 1150000,
     "eip7FBlock": 1150000,

params/coregeth.json.d/ropsten_difficulty_test.json

@@ -2,8 +2,7 @@
     "networkId": 3,
     "chainId": 3,
     "supportedProtocolVersions": [
-        66,
-        65
+        66
     ],
     "eip2FBlock": 0,
     "eip7FBlock": 0,
@@ -42,6 +41,7 @@
     "eip3541FBlock": 10499401,
     "eip3529FBlock": 10499401,
     "ethash": {},
+    "terminalTotalDifficulty": 50000000000000000,
     "difficultyBombDelays": {
         "0x19f0a0": "0x2dc6c0",
         "0x408b70": "0x1e8480",

tests/init_test.go

@@ -65,8 +65,9 @@ var (
 	legacyStateTestDir = filepath.Join(baseDir, "LegacyTests", "Constantinople", "GeneralStateTests")
 	transactionTestDir = filepath.Join(baseDir, "TransactionTests")
 	rlpTestDir         = filepath.Join(baseDir, "RLPTests")
-	difficultyTestDir  = filepath.Join(baseDir, "BasicTests.core-geth")
-	benchmarksDir      = filepath.Join(".", "evm-benchmarks", "benchmarks")
+	// difficultyTestDir  = filepath.Join(baseDir, "BasicTests.core-geth")
+	difficultyTestDir = filepath.Join(baseDir, "BasicTests")
+	benchmarksDir     = filepath.Join(".", "evm-benchmarks", "benchmarks")
 )
 
 func readJSON(reader io.Reader, value interface{}) error {

tests/state_mgen_test.go

@@ -132,7 +132,7 @@ func TestGenStateAll(t *testing.T) {
 // For production use, use TestGenStateAll.
 func TestGenStateSingles(t *testing.T) {
 	if os.Getenv(CG_GENERATE_STATE_TESTS_KEY) == "" {
-		t.Skip()
+		t.Skip("Use CG_GENERATE_STATE_TESTS=1 to generate state tests.")
 	}
 	if os.Getenv(CG_CHAINCONFIG_FEATURE_EQ_COREGETH_KEY) == "" {
 		t.Fatal("Must use core-geth chain configs for test generation, converting if necessary.")
@@ -141,9 +141,10 @@ func TestGenStateSingles(t *testing.T) {
 	head := build.RunGit("rev-parse", "HEAD")
 	head = strings.TrimSpace(head)
 
-	files := []string{
-		filepath.Join(stateTestDir, "stStaticFlagEnabled/DelegatecallToPrecompileFromContractInitialization.json"),
-		filepath.Join(stateTestDir, "stStaticCall/StaticcallToPrecompileFromCalledContract.json"),
+	filesEnv := os.Getenv("TEST_FILES")
+	files := []string{}
+	for _, f := range strings.Split(filesEnv, ",") {
+		files = append(files, filepath.Join(stateTestDir, f))
 	}
 
 	tm := new(testMatcherGen)
@@ -157,6 +158,7 @@ func TestGenStateSingles(t *testing.T) {
 	tm.generateFromReference("ConstantinopleFix", "ETC_Agharta")
 	tm.generateFromReference("Berlin", "ETC_Magneto")
 	tm.generateFromReference("Istanbul", "ETC_Phoenix")
+	tm.generateFromReference("London", "ETC_Mystique")
 
 	for _, f := range files {
 		tm.runTestFile(t, f, f, tm.testWriteTest)
@@ -243,11 +245,6 @@ func (tm *testMatcherGen) stateTestsGen(w io.WriteCloser, writeCallback, skipCal
 				targets[targetFork] = make([]stPostState, subtestsLen)
 			}
 
-			targetSubtest := StateSubtest{
-				Fork:  targetFork,
-				Index: s.Index,
-			}
-
 			refPostState := test.json.Post[referenceFork]
 
 			// Initialize the post state with reference indexes.
@@ -259,28 +256,65 @@ func (tm *testMatcherGen) stateTestsGen(w io.WriteCloser, writeCallback, skipCal
 					Gas:   refPostState[s.Index].Indexes.Gas,
 					Value: refPostState[s.Index].Indexes.Value,
 				},
+				TxBytes:         refPostState[s.Index].TxBytes,
+				ExpectException: refPostState[s.Index].ExpectException,
+				// Root: <This is set pending the results of the Run under the target subtests.>
+				// Logs: <This is set pending the results of the Run under the target subtests.>
 			}
 
 			// vmConfig is constructed using global variables for possible EVM and EWASM interpreters.
 			// These interpreters are configured with environment variables and are assigned in an init() function.
 			vmConfig := vm.Config{EVMInterpreter: *testEVM, EWASMInterpreter: *testEWASM}
 
-			// Since we know that tests run with and without the snapshotter features are equivalent, either boolean state
-			// is valid and 'false' is arbitrary.
+			// // Since we know that tests run with and without the snapshotter features are equivalent, either boolean state
+			// // is valid and 'false' is arbitrary.
+			// if test.json.Tx.GasPrice == nil {
+			// 	test.json.Tx.GasPrice = big.NewInt(1)
+			// }
+
+			targetSubtest := StateSubtest{
+				Fork:  targetFork,
+				Index: s.Index,
+			}
+
 			_, statedb, root, err := test.RunNoVerifyWithPost(targetSubtest, vmConfig, false, stPost)
 			if err != nil {
-				t.Fatalf("Error encountered at RunSetPost: %v", err)
+				// Our runner has returned an error.
+				// This can either be an intentional error (testing for the error), or an "unexpected" error,
+				// which, since we're transposing tests, could still be a desirable error.
+				// For example, London adopts EIP1559, while its counterpart, Mystique, does not.
+				// In this case, the London test might return no error, while Mystique would return an error
+				// (eg "unsupported transaction type").
+
+				// An error was returned, but none defined for this test.
+				// We write the error to the generated test.
+				if refPostState[s.Index].ExpectException == "" {
+					// TODO: Turn this error into the kind of error constants that upstream uses, eg. TR_TypeNotSupported.
+					stPost.ExpectException = err.Error()
+				}
+
+				// Either way, we maintain the incumbent post state values,
+				// although we do not expect our runner to return these for us.
+				stPost.Root = refPostState[s.Index].Root
+				stPost.Logs = refPostState[s.Index].Logs
 			}
 
-			// Assign the generated testable values.
-			stPost.Root = common.UnprefixedHash(root)
-			stPost.Logs = common.UnprefixedHash(rlpHash(statedb.Logs()))
+			if err == nil {
+				if refPostState[s.Index].ExpectException != "" {
+					// An error was expected, but none returned.
+					// We overwrite the expected error to a zero value, because it didn't fail under our target configuration.
+					stPost.ExpectException = ""
+				}
+				// If no error was returned, we can safely expect the root and statedb value to exist for us.
+				stPost.Root = common.UnprefixedHash(root)
+				stPost.Logs = common.UnprefixedHash(rlpHash(statedb.Logs()))
+			}
 
 			targets[targetFork][s.Index] = stPost
 		}
 
 		if len(targets) == 0 {
-			t.Skip()
+			t.Skip("No targets found for this test")
 			skipCallback()
 			return
 		}
@@ -322,13 +356,17 @@ func (tm *testMatcherGen) stateTestRunner(t *testing.T, name string, test *State
 			// These interpreters are configured with environment variables and are assigned in an init() function.
 			vmConfig := vm.Config{EVMInterpreter: *testEVM, EWASMInterpreter: *testEWASM}
 			_, _, err := test.Run(st, vmConfig, false)
+			if err != nil && len(test.json.Post[st.Fork][st.Index].ExpectException) > 0 {
+				// Ignore expected errors (TODO MariusVanDerWijden check error string)
+				return
+			}
 			checkedErr := tm.checkFailure(t, err)
 			if checkedErr != nil && *testEWASM != "" {
 				checkedErr = fmt.Errorf("%w ewasm=%s", checkedErr, *testEWASM)
 			}
 			if checkedErr != nil {
 				if tm.errorPanics {
-					panic(err)
+					panic(checkedErr)
 				} else {
 					t.Fatal(err)
 				}
@@ -346,7 +384,10 @@ func TestGenStateCoreGethConfigs(t *testing.T) {
 
 	// FYI: Possibly the slowest and stupidest way to write 14 files: read 42189 test to do it
 	// and write each file 1486 times.
-	for _, d := range []string{stateTestDir, legacyStateTestDir} {
+	for _, d := range []string{
+		stateTestDir,
+		legacyStateTestDir,
+	} {
 		st.walkFullName(t, d, func(t *testing.T, name string, test *StateTest) {
 			subtests := test.Subtests(nil)
 			for _, subtest := range subtests {

tests/state_test_util.go

@@ -235,15 +235,34 @@ func (t *StateTest) RunNoVerifyWithPost(subtest StateSubtest, vmconfig vm.Config
 	}
 	msg, err := t.json.Tx.toMessage(post, baseFee)
 	if err != nil {
-		return nil, nil, common.Hash{}, err
+		return nil, nil, common.Hash{}, fmt.Errorf("%w: toMessage: %v", err, t.json.Tx)
+	}
+
+	// Try to recover tx with current signer
+	if len(post.TxBytes) != 0 {
+		var ttx types.Transaction
+		err := ttx.UnmarshalBinary(post.TxBytes)
+		if err != nil {
+			return nil, nil, common.Hash{}, err
+		}
+
+		if _, err := types.Sender(types.LatestSigner(config), &ttx); err != nil {
+			return nil, nil, common.Hash{}, err
+		}
 	}
 
 	// Prepare the EVM.
 	txContext := core.NewEVMTxContext(msg)
 	context := core.NewEVMBlockContext(block.Header(), nil, &t.json.Env.Coinbase)
 	context.GetHash = vmTestBlockHash
+	context.BaseFee = baseFee
+	context.Random = nil
+	if config.IsEnabled(config.GetEIP1559Transition, new(big.Int)) && t.json.Env.Random != nil {
+		rnd := common.BigToHash(t.json.Env.Random)
+		context.Random = &rnd
+		context.Difficulty = big.NewInt(0)
+	}
 	evm := vm.NewEVM(context, txContext, statedb, config, vmconfig)
-
 	// Execute the message.
 	snapshot := statedb.Snapshot()
 	gaspool := new(core.GasPool)