diff --git a/CHANGELOG.md b/CHANGELOG.md index 2c776df174c..ee5616c4e63 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,21 @@ # Changelog +## 23.10.1 +- Cache last n blocks by using a new Besu flag --cache-last-blocks=n [#6009](https://github.com/hyperledger/besu/pull/6009) +- Optimize performances of RPC method Eth_feeHistory [#6011](https://github.com/hyperledger/besu/pull/6011) + +### Breaking Changes + +### Deprecations +- `--tx-pool-disable-locals` has been deprecated for removal in favor of `--tx-pool-no-local-priority`, no semantic change, only a renaming [#5959](https://github.com/hyperledger/besu/pull/5959) + +### Additions and Improvements +- New option `--tx-pool-priority-senders` to specify a list of senders, that has the effect to prioritize any transactions sent by these senders from any source [#5959](https://github.com/hyperledger/besu/pull/5959) + +### Bug Fixes + +### Download Links + ## 23.10.0 ### Layered Transaction Pool: the new default transaction pool implementation With this release the previously experimental Layered txpool is marked stable and enabled by default, so please read the following instructions if you used to tune txpool behaviour, @@ -25,16 +41,16 @@ By default, the txpool is tuned for mainnet usage, but if you are using private ### Breaking Changes - Removed support for Kotti network (ETC) [#5816](https://github.com/hyperledger/besu/pull/5816) -- Layered transaction pool implementation is now stable and enabled by default, so the following changes to experimental options have been done [#5772](https://github.com/hyperledger/besu): +- Layered transaction pool implementation is now stable and enabled by default, so the following changes to experimental options have been done [#5772](https://github.com/hyperledger/besu/pull/5772): - `--Xlayered-tx-pool` is gone, to select the implementation use the new `--tx-pool` option with values `layered` (default) or `legacy` - - `--Xlayered-tx-pool-layer-max-capacity`, `--Xlayered-tx-pool-max-prioritized` and `--Xlayered-tx-pool-max-future-by-sender` just drop the `X` and keep the same behavior + - `--Xlayered-tx-pool-layer-max-capacity`, `--Xlayered-tx-pool-max-prioritized` and `--Xlayered-tx-pool-max-future-by-sender` just drop the `Xlayered-` and keep the same behavior ### Additions and Improvements - Add access to an immutable world view to start/end transaction hooks in the tracing API[#5836](https://github.com/hyperledger/besu/pull/5836) - Layered transaction pool implementation is now stable and enabled by default. If you want still to use the legacy implementation, use `--tx-pool=legacy`. - By default, the new transaction pool is capped at using 25MB of memory, this limit can be raised using `--layered-tx-pool-layer-max-capacity` options [#5772](https://github.com/hyperledger/besu) + By default, the new transaction pool is capped at using 25MB of memory, this limit can be raised using `--layered-tx-pool-layer-max-capacity` options [#5772](https://github.com/hyperledger/besu/pull/5772) - Tune G1GC to reduce Besu memory footprint, and new `besu-untuned` start scripts to run without any specific G1GC flags [#5879](https://github.com/hyperledger/besu/pull/5879) -- Reduce `engine_forkchoiceUpdatedV?` response time by asynchronously process block added events in the transaction pool [#5909](https://github.com/hyperledger/besu/pull/5909) +- Reduce `engine_forkchoiceUpdatedV?` response time by asynchronously process block added events in the transaction pool [#5909](https://github.com/hyperledger/besu/pull/5909) ### Bug Fixes - do not create ignorable storage on revert storage-variables subcommand [#5830](https://github.com/hyperledger/besu/pull/5830) @@ -42,7 +58,8 @@ By default, the txpool is tuned for mainnet usage, but if you are using private - Don't put control characters, escaped or otherwise, in t8n stacktraces [#5910](https://github.com/hyperledger/besu/pull/5910) ### Download Links - +https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/23.10.0/besu-23.10.0.tar.gz / sha256: 3c75f3792bfdb0892705b378f0b8bfc14ef6cecf1d8afe711d8d8687ed6687cf +https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/23.10.0/besu-23.10.0.zip / sha256: d5dafff4c3cbf104bf75b34a9f108dcdd7b08d2759de75ec65cd997f38f52866 ## 23.7.3 diff --git a/KNOWN_ISSUES.md b/KNOWN_ISSUES.md index 89c67789eda..47c17ea149a 100644 --- a/KNOWN_ISSUES.md +++ b/KNOWN_ISSUES.md @@ -1,6 +1,6 @@ # Known Issues -Details on previously identified known issues are provided below. Details on known issues identfied +Details on previously identified known issues are provided below. Details on known issues identified in the current release are provided in the [Changelog](CHANGELOG.md). Known issues are open issues categorized as [Very High or High impact](https://wiki.hyperledger.org/display/BESU/Defect+Prioritisation+Policy). diff --git a/README.md b/README.md index 38eb3e7725f..f0a8690d0f2 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,7 @@ [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3174/badge)](https://bestpractices.coreinfrastructure.org/projects/3174) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/hyperledger/besu/blob/main/LICENSE) [![Discord](https://img.shields.io/discord/905194001349627914?logo=Hyperledger&style=plastic)](https://discord.gg/hyperledger) + [![Twitter Follow](https://img.shields.io/twitter/follow/HyperledgerBesu)](https://twitter.com/HyperledgerBesu) [Download](https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/) @@ -15,7 +16,7 @@ Besu is an Apache 2.0 licensed, MainNet compatible, Ethereum client written in J * [Besu Issues] * [Besu Wiki](https://wiki.hyperledger.org/display/BESU/Hyperledger+Besu) * [How to Contribute to Besu](https://wiki.hyperledger.org/display/BESU/How+to+Contribute) -* [Besu Roadmap](https://wiki.hyperledger.org/display/BESU/Roadmap) +* [Besu Roadmap & Planning](https://wiki.hyperledger.org/pages/viewpage.action?pageId=24781786) ## Issues diff --git a/acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/node/ThreadBesuNodeRunner.java b/acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/node/ThreadBesuNodeRunner.java index e06aa8c9641..0c5f170931b 100644 --- a/acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/node/ThreadBesuNodeRunner.java +++ b/acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/node/ThreadBesuNodeRunner.java @@ -48,7 +48,7 @@ import org.hyperledger.besu.plugin.services.StorageService; import org.hyperledger.besu.plugin.services.TransactionSelectionService; import org.hyperledger.besu.plugin.services.storage.rocksdb.RocksDBPlugin; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory; import org.hyperledger.besu.plugin.services.txvalidator.PluginTransactionValidatorFactory; import org.hyperledger.besu.services.BesuConfigurationImpl; import org.hyperledger.besu.services.BesuEventsImpl; @@ -185,7 +185,7 @@ public void startNode(final BesuNode node) { final int maxPeers = 25; - final Optional transactionSelectorFactory = + final Optional transactionSelectorFactory = getTransactionSelectorFactory(besuPluginContext); final PluginTransactionValidatorFactory pluginTransactionValidatorFactory = @@ -323,7 +323,7 @@ public String getConsoleContents() { throw new RuntimeException("Console contents can only be captured in process execution"); } - private Optional getTransactionSelectorFactory( + private Optional getTransactionSelectorFactory( final BesuPluginContextImpl besuPluginContext) { final Optional txSelectionService = besuPluginContext.getService(TransactionSelectionService.class); diff --git a/acceptance-tests/tests/src/test/java/org/hyperledger/besu/tests/acceptance/jsonrpc/ExecutionEngineEip6110AcceptanceTest.java b/acceptance-tests/tests/src/test/java/org/hyperledger/besu/tests/acceptance/jsonrpc/ExecutionEngineEip6110AcceptanceTest.java index fb2abde25d4..d1771ac5990 100644 --- a/acceptance-tests/tests/src/test/java/org/hyperledger/besu/tests/acceptance/jsonrpc/ExecutionEngineEip6110AcceptanceTest.java +++ b/acceptance-tests/tests/src/test/java/org/hyperledger/besu/tests/acceptance/jsonrpc/ExecutionEngineEip6110AcceptanceTest.java @@ -20,12 +20,10 @@ import org.junit.AfterClass; import org.junit.BeforeClass; -import org.junit.Ignore; import org.junit.runner.RunWith; import org.junit.runners.Parameterized; @RunWith(Parameterized.class) -@Ignore("EIP-6110 is not yet implemented") public class ExecutionEngineEip6110AcceptanceTest extends AbstractJsonRpcTest { private static final String GENESIS_FILE = "/jsonrpc/engine/eip6110/genesis.json"; private static final String TEST_CASE_PATH = "/jsonrpc/engine/eip6110/test-cases/"; diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/genesis.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/genesis.json index 12f58a81bfa..232c577f6c8 100644 --- a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/genesis.json +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/genesis.json @@ -1,6 +1,6 @@ { "config": { - "chainId":1, + "chainId":6110, "homesteadBlock":0, "eip150Block":0, "eip155Block":0, @@ -19,7 +19,7 @@ "period": 5, "epoch": 30000 }, - "depositContractAddress": "0x00000000219ab540356cbb839cbe05303d7705fa" + "depositContractAddress": "0x4242424242424242424242424242424242424242" }, "nonce":"0x42", "timestamp":"0x0", @@ -29,7 +29,46 @@ "mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000", "coinbase":"0x0000000000000000000000000000000000000000", "alloc":{ - "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b":{"balance":"0x6d6172697573766477000000"} + "0xa4664C40AACeBD82A2Db79f0ea36C06Bc6A19Adb": { + "balance": "1000000000000000000000000000" + }, + "0x4242424242424242424242424242424242424242": { + "balance": "0", + "code": "0x60806040526004361061003f5760003560e01c806301ffc9a71461004457806322895118146100a4578063621fd130146101ba578063c5f2892f14610244575b600080fd5b34801561005057600080fd5b506100906004803603602081101561006757600080fd5b50357fffffffff000000000000000000000000000000000000000000000000000000001661026b565b604080519115158252519081900360200190f35b6101b8600480360360808110156100ba57600080fd5b8101906020810181356401000000008111156100d557600080fd5b8201836020820111156100e757600080fd5b8035906020019184600183028401116401000000008311171561010957600080fd5b91939092909160208101903564010000000081111561012757600080fd5b82018360208201111561013957600080fd5b8035906020019184600183028401116401000000008311171561015b57600080fd5b91939092909160208101903564010000000081111561017957600080fd5b82018360208201111561018b57600080fd5b803590602001918460018302840111640100000000831117156101ad57600080fd5b919350915035610304565b005b3480156101c657600080fd5b506101cf6110b5565b6040805160208082528351818301528351919283929083019185019080838360005b838110156102095781810151838201526020016101f1565b50505050905090810190601f1680156102365780820380516001836020036101000a031916815260200191505b509250505060405180910390f35b34801561025057600080fd5b506102596110c7565b60408051918252519081900360200190f35b60007fffffffff0000000000000000000000000000000000000000000000000000000082167f01ffc9a70000000000000000000000000000000000000000000000000000000014806102fe57507fffffffff0000000000000000000000000000000000000000000000000000000082167f8564090700000000000000000000000000000000000000000000000000000000145b92915050565b6030861461035d576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260268152602001806118056026913960400191505060405180910390fd5b602084146103b6576040517f08c379a000000000000000000000000000000000000000000000000000000000815260040180806020018281038252603681526020018061179c6036913960400191505060405180910390fd5b6060821461040f576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260298152602001806118786029913960400191505060405180910390fd5b670de0b6b3a7640000341015610470576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260268152602001806118526026913960400191505060405180910390fd5b633b9aca003406156104cd576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260338152602001806117d26033913960400191505060405180910390fd5b633b9aca00340467ffffffffffffffff811115610535576040517f08c379a000000000000000000000000000000000000000000000000000000000815260040180806020018281038252602781526020018061182b6027913960400191505060405180910390fd5b6060610540826114ba565b90507f649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c589898989858a8a6105756020546114ba565b6040805160a0808252810189905290819060208201908201606083016080840160c085018e8e80828437600083820152601f017fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe01690910187810386528c815260200190508c8c808284376000838201819052601f9091017fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe01690920188810386528c5181528c51602091820193918e019250908190849084905b83811015610648578181015183820152602001610630565b50505050905090810190601f1680156106755780820380516001836020036101000a031916815260200191505b5086810383528881526020018989808284376000838201819052601f9091017fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0169092018881038452895181528951602091820193918b019250908190849084905b838110156106ef5781810151838201526020016106d7565b50505050905090810190601f16801561071c5780820380516001836020036101000a031916815260200191505b509d505050505050505050505050505060405180910390a1600060028a8a600060801b604051602001808484808284377fffffffffffffffffffffffffffffffff0000000000000000000000000000000090941691909301908152604080517ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0818403018152601090920190819052815191955093508392506020850191508083835b602083106107fc57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe090920191602091820191016107bf565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610859573d6000803e3d6000fd5b5050506040513d602081101561086e57600080fd5b5051905060006002806108846040848a8c6116fe565b6040516020018083838082843780830192505050925050506040516020818303038152906040526040518082805190602001908083835b602083106108f857805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe090920191602091820191016108bb565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610955573d6000803e3d6000fd5b5050506040513d602081101561096a57600080fd5b5051600261097b896040818d6116fe565b60405160009060200180848480828437919091019283525050604080518083038152602092830191829052805190945090925082918401908083835b602083106109f457805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe090920191602091820191016109b7565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610a51573d6000803e3d6000fd5b5050506040513d6020811015610a6657600080fd5b5051604080516020818101949094528082019290925280518083038201815260609092019081905281519192909182918401908083835b60208310610ada57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610a9d565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610b37573d6000803e3d6000fd5b5050506040513d6020811015610b4c57600080fd5b50516040805160208101858152929350600092600292839287928f928f92018383808284378083019250505093505050506040516020818303038152906040526040518082805190602001908083835b60208310610bd957805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610b9c565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610c36573d6000803e3d6000fd5b5050506040513d6020811015610c4b57600080fd5b50516040518651600291889160009188916020918201918291908601908083835b60208310610ca957805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610c6c565b6001836020036101000a0380198251168184511680821785525050505050509050018367ffffffffffffffff191667ffffffffffffffff1916815260180182815260200193505050506040516020818303038152906040526040518082805190602001908083835b60208310610d4e57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610d11565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610dab573d6000803e3d6000fd5b5050506040513d6020811015610dc057600080fd5b5051604080516020818101949094528082019290925280518083038201815260609092019081905281519192909182918401908083835b60208310610e3457805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610df7565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610e91573d6000803e3d6000fd5b5050506040513d6020811015610ea657600080fd5b50519050858114610f02576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260548152602001806117486054913960600191505060405180910390fd5b60205463ffffffff11610f60576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260218152602001806117276021913960400191505060405180910390fd5b602080546001019081905560005b60208110156110a9578160011660011415610fa0578260008260208110610f9157fe5b0155506110ac95505050505050565b600260008260208110610faf57fe5b01548460405160200180838152602001828152602001925050506040516020818303038152906040526040518082805190602001908083835b6020831061102557805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610fe8565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015611082573d6000803e3d6000fd5b5050506040513d602081101561109757600080fd5b50519250600282049150600101610f6e565b50fe5b50505050505050565b60606110c26020546114ba565b905090565b6020546000908190815b60208110156112f05781600116600114156111e6576002600082602081106110f557fe5b01548460405160200180838152602001828152602001925050506040516020818303038152906040526040518082805190602001908083835b6020831061116b57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0909201916020918201910161112e565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa1580156111c8573d6000803e3d6000fd5b5050506040513d60208110156111dd57600080fd5b505192506112e2565b600283602183602081106111f657fe5b015460405160200180838152602001828152602001925050506040516020818303038152906040526040518082805190602001908083835b6020831061126b57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0909201916020918201910161122e565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa1580156112c8573d6000803e3d6000fd5b5050506040513d60208110156112dd57600080fd5b505192505b6002820491506001016110d1565b506002826112ff6020546114ba565b600060401b6040516020018084815260200183805190602001908083835b6020831061135a57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0909201916020918201910161131d565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790527fffffffffffffffffffffffffffffffffffffffffffffffff000000000000000095909516920191825250604080518083037ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff8018152601890920190819052815191955093508392850191508083835b6020831061143f57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101611402565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa15801561149c573d6000803e3d6000fd5b5050506040513d60208110156114b157600080fd5b50519250505090565b60408051600880825281830190925260609160208201818036833701905050905060c082901b8060071a60f81b826000815181106114f457fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060061a60f81b8260018151811061153757fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060051a60f81b8260028151811061157a57fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060041a60f81b826003815181106115bd57fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060031a60f81b8260048151811061160057fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060021a60f81b8260058151811061164357fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060011a60f81b8260068151811061168657fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060001a60f81b826007815181106116c957fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a90535050919050565b6000808585111561170d578182fd5b83861115611719578182fd5b505082019391909203915056fe4465706f736974436f6e74726163743a206d65726b6c6520747265652066756c6c4465706f736974436f6e74726163743a207265636f6e7374727563746564204465706f7369744461746120646f6573206e6f74206d6174636820737570706c696564206465706f7369745f646174615f726f6f744465706f736974436f6e74726163743a20696e76616c6964207769746864726177616c5f63726564656e7469616c73206c656e6774684465706f736974436f6e74726163743a206465706f7369742076616c7565206e6f74206d756c7469706c65206f6620677765694465706f736974436f6e74726163743a20696e76616c6964207075626b6579206c656e6774684465706f736974436f6e74726163743a206465706f7369742076616c756520746f6f20686967684465706f736974436f6e74726163743a206465706f7369742076616c756520746f6f206c6f774465706f736974436f6e74726163743a20696e76616c6964207369676e6174757265206c656e677468a26469706673582212201dd26f37a621703009abf16e77e69c93dc50c79db7f6cc37543e3e0e3decdc9764736f6c634300060b0033", + "storage": { + "0x0000000000000000000000000000000000000000000000000000000000000022": "0xf5a5fd42d16a20302798ef6ed309979b43003d2320d9f0e8ea9831a92759fb4b", + "0x0000000000000000000000000000000000000000000000000000000000000023": "0xdb56114e00fdd4c1f85c892bf35ac9a89289aaecb1ebd0a96cde606a748b5d71", + "0x0000000000000000000000000000000000000000000000000000000000000024": "0xc78009fdf07fc56a11f122370658a353aaa542ed63e44c4bc15ff4cd105ab33c", + "0x0000000000000000000000000000000000000000000000000000000000000025": "0x536d98837f2dd165a55d5eeae91485954472d56f246df256bf3cae19352a123c", + "0x0000000000000000000000000000000000000000000000000000000000000026": "0x9efde052aa15429fae05bad4d0b1d7c64da64d03d7a1854a588c2cb8430c0d30", + "0x0000000000000000000000000000000000000000000000000000000000000027": "0xd88ddfeed400a8755596b21942c1497e114c302e6118290f91e6772976041fa1", + "0x0000000000000000000000000000000000000000000000000000000000000028": "0x87eb0ddba57e35f6d286673802a4af5975e22506c7cf4c64bb6be5ee11527f2c", + "0x0000000000000000000000000000000000000000000000000000000000000029": "0x26846476fd5fc54a5d43385167c95144f2643f533cc85bb9d16b782f8d7db193", + "0x000000000000000000000000000000000000000000000000000000000000002a": "0x506d86582d252405b840018792cad2bf1259f1ef5aa5f887e13cb2f0094f51e1", + "0x000000000000000000000000000000000000000000000000000000000000002b": "0xffff0ad7e659772f9534c195c815efc4014ef1e1daed4404c06385d11192e92b", + "0x000000000000000000000000000000000000000000000000000000000000002c": "0x6cf04127db05441cd833107a52be852868890e4317e6a02ab47683aa75964220", + "0x000000000000000000000000000000000000000000000000000000000000002d": "0xb7d05f875f140027ef5118a2247bbb84ce8f2f0f1123623085daf7960c329f5f", + "0x000000000000000000000000000000000000000000000000000000000000002e": "0xdf6af5f5bbdb6be9ef8aa618e4bf8073960867171e29676f8b284dea6a08a85e", + "0x000000000000000000000000000000000000000000000000000000000000002f": "0xb58d900f5e182e3c50ef74969ea16c7726c549757cc23523c369587da7293784", + "0x0000000000000000000000000000000000000000000000000000000000000030": "0xd49a7502ffcfb0340b1d7885688500ca308161a7f96b62df9d083b71fcc8f2bb", + "0x0000000000000000000000000000000000000000000000000000000000000031": "0x8fe6b1689256c0d385f42f5bbe2027a22c1996e110ba97c171d3e5948de92beb", + "0x0000000000000000000000000000000000000000000000000000000000000032": "0x8d0d63c39ebade8509e0ae3c9c3876fb5fa112be18f905ecacfecb92057603ab", + "0x0000000000000000000000000000000000000000000000000000000000000033": "0x95eec8b2e541cad4e91de38385f2e046619f54496c2382cb6cacd5b98c26f5a4", + "0x0000000000000000000000000000000000000000000000000000000000000034": "0xf893e908917775b62bff23294dbbe3a1cd8e6cc1c35b4801887b646a6f81f17f", + "0x0000000000000000000000000000000000000000000000000000000000000035": "0xcddba7b592e3133393c16194fac7431abf2f5485ed711db282183c819e08ebaa", + "0x0000000000000000000000000000000000000000000000000000000000000036": "0x8a8d7fe3af8caa085a7639a832001457dfb9128a8061142ad0335629ff23ff9c", + "0x0000000000000000000000000000000000000000000000000000000000000037": "0xfeb3c337d7a51a6fbf00b9e34c52e1c9195c969bd4e7a0bfd51d5c5bed9c1167", + "0x0000000000000000000000000000000000000000000000000000000000000038": "0xe71f0aa83cc32edfbefa9f4d3e0174ca85182eec9f3a09f6a6c0df6377a510d7", + "0x0000000000000000000000000000000000000000000000000000000000000039": "0x31206fa80a50bb6abe29085058f16212212a60eec8f049fecb92d8c8e0a84bc0", + "0x000000000000000000000000000000000000000000000000000000000000003a": "0x21352bfecbeddde993839f614c3dac0a3ee37543f9b412b16199dc158e23b544", + "0x000000000000000000000000000000000000000000000000000000000000003b": "0x619e312724bb6d7c3153ed9de791d764a366b389af13c58bf8a8d90481a46765", + "0x000000000000000000000000000000000000000000000000000000000000003c": "0x7cdd2986268250628d0c10e385c58c6191e6fbe05191bcc04f133f2cea72c1c4", + "0x000000000000000000000000000000000000000000000000000000000000003d": "0x848930bd7ba8cac54661072113fb278869e07bb8587f91392933374d017bcbe1", + "0x000000000000000000000000000000000000000000000000000000000000003e": "0x8869ff2c22b28cc10510d9853292803328be4fb0e80495e8bb8d271f5b889636", + "0x000000000000000000000000000000000000000000000000000000000000003f": "0xb5fe28e79f1b850f8658246ce9b6a1e7b49fc06db7143e8fe0b4f2b0c5523a5c", + "0x0000000000000000000000000000000000000000000000000000000000000040": "0x985e929f70af28d0bdd1a90a808f977f597c7c778c489e98d3bd8910d31ac0f7" + } + } }, "number":"0x0", "gasUsed":"0x0", diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/01_cancun_prepare_payload.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/01_cancun_prepare_payload.json new file mode 100644 index 00000000000..b398cda18af --- /dev/null +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/01_cancun_prepare_payload.json @@ -0,0 +1,34 @@ +{ + "request": { + "jsonrpc": "2.0", + "method": "engine_forkchoiceUpdatedV3", + "params": [ + { + "headBlockHash": "0x26118cf71453320edcebbc4ebb34af5b578087a32385b80108bf691fa23efc42", + "safeBlockHash": "0x26118cf71453320edcebbc4ebb34af5b578087a32385b80108bf691fa23efc42", + "finalizedBlockHash": "0x0000000000000000000000000000000000000000000000000000000000000000" + }, + { + "timestamp": "0x10", + "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", + "suggestedFeeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", + "withdrawals": [], + "parentBeaconBlockRoot": "0x0000000000000000000000000000000000000000000000000000000000000000" + } + ], + "id": 67 + }, + "response": { + "jsonrpc": "2.0", + "id": 67, + "result": { + "payloadStatus": { + "status": "VALID", + "latestValidHash": "0x26118cf71453320edcebbc4ebb34af5b578087a32385b80108bf691fa23efc42", + "validationError": null + }, + "payloadId": "0x282643c14de2dfef" + } + }, + "statusCode" : 200 +} \ No newline at end of file diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/01_shanghai_prepare_payload.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/01_shanghai_prepare_payload.json deleted file mode 100644 index 960194fcae0..00000000000 --- a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/01_shanghai_prepare_payload.json +++ /dev/null @@ -1,46 +0,0 @@ -{ - "request": { - "jsonrpc": "2.0", - "method": "engine_forkchoiceUpdatedV2", - "params": [ - { - "headBlockHash": "0xfe950635b1bd2a416ff6283b0bbd30176e1b1125ad06fa729da9f3f4c1c61710", - "safeBlockHash": "0xfe950635b1bd2a416ff6283b0bbd30176e1b1125ad06fa729da9f3f4c1c61710", - "finalizedBlockHash": "0x0000000000000000000000000000000000000000000000000000000000000000" - }, - { - "timestamp": "0x10", - "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", - "suggestedFeeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "withdrawals": [ - { - "index": "0x0", - "validatorIndex": "0x0", - "address": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "amount": "0x3" - }, - { - "index": "0x1", - "validatorIndex": "0x1", - "address": "0xfe3b557e8fb62b89f4916b721be55ceb828dbd73", - "amount": "0x4" - } - ] - } - ], - "id": 67 - }, - "response": { - "jsonrpc": "2.0", - "id": 67, - "result": { - "payloadStatus": { - "status": "VALID", - "latestValidHash": "0xfe950635b1bd2a416ff6283b0bbd30176e1b1125ad06fa729da9f3f4c1c61710", - "validationError": null - }, - "payloadId": "0x0065bd1bbeaff359" - } - }, - "statusCode" : 200 -} \ No newline at end of file diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/02_shanghai_getPayloadV2.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/02_cancun_getPayloadV3.json similarity index 58% rename from acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/02_shanghai_getPayloadV2.json rename to acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/02_cancun_getPayloadV3.json index 9d25e3f5e84..73cbe51bdeb 100644 --- a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/02_shanghai_getPayloadV2.json +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/02_cancun_getPayloadV3.json @@ -1,9 +1,9 @@ { "request": { "jsonrpc": "2.0", - "method": "engine_getPayloadV2", + "method": "engine_getPayloadV3", "params": [ - "0x0065bd1bbeaff359" + "0x282643c14de2dfef" ], "id": 67 }, @@ -12,9 +12,9 @@ "id": 67, "result": { "executionPayload": { - "parentHash": "0xfe950635b1bd2a416ff6283b0bbd30176e1b1125ad06fa729da9f3f4c1c61710", + "parentHash": "0x26118cf71453320edcebbc4ebb34af5b578087a32385b80108bf691fa23efc42", "feeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "stateRoot": "0xc8c8e840369eac89a610bfe2ec21fcdee4c9c43bec4876f0129fcd4b5311f6dd", + "stateRoot": "0x9b8c4a9a86cb49252075c0db2f0e72fb1e49350a0f70ea36f26f700201961e62", "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", "gasLimit": "0x1c9c380", @@ -22,27 +22,22 @@ "timestamp": "0x10", "extraData": "0x", "baseFeePerGas": "0x7", + "excessBlobGas": "0x0", + "parentBeaconBlockRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", "transactions": [], - "withdrawals": [ - { - "index": "0x0", - "validatorIndex": "0x0", - "address": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "amount": "0x3" - }, - { - "index": "0x1", - "validatorIndex": "0x1", - "address": "0xfe3b557e8fb62b89f4916b721be55ceb828dbd73", - "amount": "0x4" - } - ], - "deposits": null, + "withdrawals": [], "blockNumber": "0x1", - "blockHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13", - "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421" + "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421", + "blockHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", + "blobGasUsed": "0x0" }, - "blockValue": "0x0" + "blockValue": "0x0", + "blobsBundle": { + "commitments": [], + "proofs": [], + "blobs": [] + }, + "shouldOverrideBuilder": false } }, "statusCode": 200 diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/03_shanghai_newPayloadV2.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/03_cancun_newPayloadV3.json similarity index 58% rename from acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/03_shanghai_newPayloadV2.json rename to acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/03_cancun_newPayloadV3.json index 938362327ba..d4d76df31de 100644 --- a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/03_shanghai_newPayloadV2.json +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/03_cancun_newPayloadV3.json @@ -1,12 +1,12 @@ { "request": { "jsonrpc": "2.0", - "method": "engine_newPayloadV2", + "method": "engine_newPayloadV3", "params": [ { - "parentHash": "0xfe950635b1bd2a416ff6283b0bbd30176e1b1125ad06fa729da9f3f4c1c61710", + "parentHash": "0x26118cf71453320edcebbc4ebb34af5b578087a32385b80108bf691fa23efc42", "feeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "stateRoot": "0xc8c8e840369eac89a610bfe2ec21fcdee4c9c43bec4876f0129fcd4b5311f6dd", + "stateRoot": "0x9b8c4a9a86cb49252075c0db2f0e72fb1e49350a0f70ea36f26f700201961e62", "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", "gasLimit": "0x1c9c380", @@ -15,24 +15,15 @@ "extraData": "0x", "baseFeePerGas": "0x7", "transactions": [], - "withdrawals": [ - { - "index": "0x0", - "validatorIndex": "0x0", - "address": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "amount": "0x3" - }, - { - "index": "0x1", - "validatorIndex": "0x1", - "address": "0xfe3b557e8fb62b89f4916b721be55ceb828dbd73", - "amount": "0x4" - } - ], + "withdrawals": [], "blockNumber": "0x1", - "blockHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13", - "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421" - } + "blockHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", + "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421", + "excessBlobGas": "0x0", + "blobGasUsed": "0x0" + }, + [], + "0x0000000000000000000000000000000000000000000000000000000000000000" ], "id": 67 }, @@ -41,7 +32,7 @@ "id": 67, "result": { "status": "VALID", - "latestValidHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13", + "latestValidHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", "validationError": null } }, diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/04_cancun_forkchoiceUpdatedV3.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/04_cancun_forkchoiceUpdatedV3.json new file mode 100644 index 00000000000..80610561a0f --- /dev/null +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/04_cancun_forkchoiceUpdatedV3.json @@ -0,0 +1,28 @@ +{ + "request": { + "jsonrpc": "2.0", + "method": "engine_forkchoiceUpdatedV3", + "params": [ + { + "headBlockHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", + "safeBlockHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", + "finalizedBlockHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356" + }, + null + ], + "id": 67 + }, + "response": { + "jsonrpc": "2.0", + "id": 67, + "result": { + "payloadStatus": { + "status": "VALID", + "latestValidHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", + "validationError": null + }, + "payloadId": null + } + }, + "statusCode": 200 +} \ No newline at end of file diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/04_shanghai_forkchoiceUpdatedV2.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/04_shanghai_forkchoiceUpdatedV2.json deleted file mode 100644 index 871832b9c99..00000000000 --- a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/04_shanghai_forkchoiceUpdatedV2.json +++ /dev/null @@ -1,28 +0,0 @@ -{ - "request": { - "jsonrpc": "2.0", - "method": "engine_forkchoiceUpdatedV2", - "params": [ - { - "headBlockHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13", - "safeBlockHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13", - "finalizedBlockHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13" - }, - null - ], - "id": 67 - }, - "response": { - "jsonrpc": "2.0", - "id": 67, - "result": { - "payloadStatus": { - "status": "VALID", - "latestValidHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13", - "validationError": null - }, - "payloadId": null - } - }, - "statusCode": 200 -} \ No newline at end of file diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/05_eip6110_forkchoiceUpdatedV3.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/05_eip6110_forkchoiceUpdatedV3.json new file mode 100644 index 00000000000..4f83ac7dcee --- /dev/null +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/05_eip6110_forkchoiceUpdatedV3.json @@ -0,0 +1,34 @@ +{ + "request": { + "jsonrpc": "2.0", + "method": "engine_forkchoiceUpdatedV3", + "params": [ + { + "headBlockHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", + "safeBlockHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", + "finalizedBlockHash": "0x0000000000000000000000000000000000000000000000000000000000000000" + }, + { + "timestamp": "0x20", + "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", + "suggestedFeeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", + "withdrawals": [], + "parentBeaconBlockRoot": "0x0000000000000000000000000000000000000000000000000000000000000000" + } + ], + "id": 67 + }, + "response": { + "jsonrpc": "2.0", + "id": 67, + "result": { + "payloadStatus": { + "status": "VALID", + "latestValidHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", + "validationError": null + }, + "payloadId": "0x282643b9c2d2a4df" + } + }, + "statusCode": 200 +} \ No newline at end of file diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/05_eip6110_prepare_payload.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/05_eip6110_prepare_payload.json deleted file mode 100644 index d3d7ee37e30..00000000000 --- a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/05_eip6110_prepare_payload.json +++ /dev/null @@ -1,46 +0,0 @@ -{ - "request": { - "jsonrpc": "2.0", - "method": "engine_forkchoiceUpdatedV2", - "params": [ - { - "headBlockHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13", - "safeBlockHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13", - "finalizedBlockHash": "0x0000000000000000000000000000000000000000000000000000000000000000" - }, - { - "timestamp": "0x20", - "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", - "suggestedFeeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "withdrawals": [ - { - "index": "0x0", - "validatorIndex": "0x0", - "address": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "amount": "0x1" - }, - { - "index": "0x1", - "validatorIndex": "0x0", - "address": "0xfe3b557e8fb62b89f4916b721be55ceb828dbd73", - "amount": "0x2" - } - ] - } - ], - "id": 67 - }, - "response": { - "jsonrpc": "2.0", - "id": 67, - "result": { - "payloadStatus": { - "status": "VALID", - "latestValidHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13", - "validationError": null - }, - "payloadId": "0x0065bd63871ad606" - } - }, - "statusCode" : 200 -} \ No newline at end of file diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/06_eip6110_get_payload.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/06_eip6110_getPayloadV6110.json similarity index 58% rename from acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/06_eip6110_get_payload.json rename to acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/06_eip6110_getPayloadV6110.json index 8b311c2841c..9b1653a198d 100644 --- a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/06_eip6110_get_payload.json +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/06_eip6110_getPayloadV6110.json @@ -1,9 +1,9 @@ { "request": { "jsonrpc": "2.0", - "method": "engine_getPayloadV2", + "method": "engine_getPayloadV6110", "params": [ - "0x0065bd63871ad606" + "0x282643b9c2d2a4df" ], "id": 67 }, @@ -12,9 +12,9 @@ "id": 67, "result": { "executionPayload": { - "parentHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13", + "parentHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", "feeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "stateRoot": "0x21395692fae33414143728c9ffc0aed8dcc76eb6731dd0f5a3239977478ca969", + "stateRoot": "0x9b8c4a9a86cb49252075c0db2f0e72fb1e49350a0f70ea36f26f700201961e62", "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", "gasLimit": "0x1c9c380", @@ -22,27 +22,23 @@ "timestamp": "0x20", "extraData": "0x", "baseFeePerGas": "0x7", + "excessBlobGas": "0x0", + "parentBeaconBlockRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", "transactions": [], - "withdrawals": [ - { - "index": "0x0", - "validatorIndex": "0x0", - "address": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "amount": "0x1" - }, - { - "index": "0x1", - "validatorIndex": "0x0", - "address": "0xfe3b557e8fb62b89f4916b721be55ceb828dbd73", - "amount": "0x2" - } - ], - "deposits" : [], + "withdrawals": [], + "depositReceipts": [], "blockNumber": "0x2", - "blockHash": "0x4c4418c408aeadb4659d31d1c05108f26fabf713bb6f8cc487dba8424a725bf5", - "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421" + "blockHash": "0xf6c3f1180ba58d6ea4c69c9328c7afb1fda41df06c368741c1f8310567879de7", + "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421", + "blobGasUsed": "0x0" }, - "blockValue": "0x0" + "blockValue": "0x0", + "blobsBundle": { + "commitments": [], + "proofs": [], + "blobs": [] + }, + "shouldOverrideBuilder": false } }, "statusCode": 200 diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/07_eip6110_send_raw_transaction.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/07_eip6110_send_raw_transaction.json new file mode 100644 index 00000000000..90072c28e0e --- /dev/null +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/07_eip6110_send_raw_transaction.json @@ -0,0 +1,14 @@ +{ + "request": { + "jsonrpc": "2.0", + "method": "eth_sendRawTransaction", + "params": ["0x02f9021c8217de808459682f008459682f0e830271009442424242424242424242424242424242424242428901bc16d674ec800000b901a422895118000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000e00000000000000000000000000000000000000000000000000000000000000120749715de5d1226545c6b3790f515d551a5cc5bf1d49c87a696860554d2fc4f14000000000000000000000000000000000000000000000000000000000000003096a96086cff07df17668f35f7418ef8798079167e3f4f9b72ecde17b28226137cf454ab1dd20ef5d924786ab3483c2f9000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000020003f5102dabe0a27b1746098d1dc17a5d3fbd478759fea9287e4e419b3c3cef20000000000000000000000000000000000000000000000000000000000000060b1acdb2c4d3df3f1b8d3bfd33421660df358d84d78d16c4603551935f4b67643373e7eb63dcb16ec359be0ec41fee33b03a16e80745f2374ff1d3c352508ac5d857c6476d3c3bcf7e6ca37427c9209f17be3af5264c0e2132b3dd1156c28b4e9c080a09f597089338d7f44f5c59f8230bb38f243849228a8d4e9d2e2956e6050f5b2c7a076486996c7e62802b8f95eee114783e4b403fd11093ba96286ff42c595f24452"], + "id": 67 + }, + "response": { + "jsonrpc": "2.0", + "id": 67, + "result": "0x8ff1a50169f52f14cc1cf0300ec037c054a9b99df462e6372c7ca655bf1f00cd" + }, + "statusCode": 200 +} \ No newline at end of file diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/08_eip6110_execute_payload.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/08_eip6110_execute_payload.json deleted file mode 100644 index 6fe5f5a5640..00000000000 --- a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/08_eip6110_execute_payload.json +++ /dev/null @@ -1,50 +0,0 @@ -{ - "request": { - "jsonrpc": "2.0", - "method": "engine_newPayloadV2", - "params": [ - { - "parentHash": "0xfdd94e3620a88f08927bffb318981a36b663a26e6fd62ab273eb800b90723c13", - "feeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "stateRoot": "0x21395692fae33414143728c9ffc0aed8dcc76eb6731dd0f5a3239977478ca969", - "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", - "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", - "gasLimit": "0x1c9c380", - "gasUsed": "0x0", - "timestamp": "0x20", - "extraData": "0x", - "baseFeePerGas": "0x7", - "transactions": [], - "withdrawals": [ - { - "index": "0x0", - "validatorIndex": "0x0", - "address": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "amount": "0x1" - }, - { - "index": "0x1", - "validatorIndex": "0x0", - "address": "0xfe3b557e8fb62b89f4916b721be55ceb828dbd73", - "amount": "0x2" - } - ], - "deposits": [], - "blockNumber": "0x2", - "blockHash": "0xdfdf57a09e352c38bb2873c5fd7d0d199481c6e13661c4a004d116417377b2e5", - "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421" - } - ], - "id": 67 - }, - "response": { - "jsonrpc": "2.0", - "id": 67, - "result": { - "status": "VALID", - "latestValidHash": "0xdfdf57a09e352c38bb2873c5fd7d0d199481c6e13661c4a004d116417377b2e5", - "validationError": null - } - }, - "statusCode": 200 -} \ No newline at end of file diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/07_eip6110_invalid_null_deposits_execute_payload.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/08_eip6110_invalid_null_deposits_execute_payload.json similarity index 57% rename from acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/07_eip6110_invalid_null_deposits_execute_payload.json rename to acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/08_eip6110_invalid_null_deposits_execute_payload.json index ebdd0ad81c2..ebf5de77500 100644 --- a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/07_eip6110_invalid_null_deposits_execute_payload.json +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/08_eip6110_invalid_null_deposits_execute_payload.json @@ -1,12 +1,12 @@ { "request": { "jsonrpc": "2.0", - "method": "engine_newPayloadV2", + "method": "engine_newPayloadV6110", "params": [ { - "parentHash": "0x4f88d512a0045bc6d447ba74a18eac0ed2ebb8d9faca325f5f55b2ca84be0705", + "parentHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", "feeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "stateRoot": "0x1a10dba514dc4faff7ec13edd9b5ef653c1cd14eb26608bfc2b37717730a55a4", + "stateRoot": "0x9b8c4a9a86cb49252075c0db2f0e72fb1e49350a0f70ea36f26f700201961e62", "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", "gasLimit": "0x1c9c380", @@ -14,26 +14,17 @@ "timestamp": "0x20", "extraData": "0x", "baseFeePerGas": "0x7", + "excessBlobGas": "0x0", "transactions": [], - "withdrawals": [ - { - "index": "0x0", - "validatorIndex": "0x0", - "address": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "amount": "0x1" - }, - { - "index": "0x1", - "validatorIndex": "0x0", - "address": "0xfe3b557e8fb62b89f4916b721be55ceb828dbd73", - "amount": "0x2" - } - ], - "deposits": null, - "blockNumber": "0x3", - "blockHash": "0x1475ca311179652e44b10b7e2d7b72f3708f3201f8d729880a83f3eb397910e8", - "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421" - } + "withdrawals": [], + "depositReceipts" : null, + "blockNumber": "0x2", + "blockHash": "0xf6c3f1180ba58d6ea4c69c9328c7afb1fda41df06c368741c1f8310567879de7", + "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421", + "blobGasUsed": "0x0" + }, + [], + "0x0000000000000000000000000000000000000000000000000000000000000000" ], "id": 67 }, @@ -42,7 +33,8 @@ "id": 67, "error": { "code": -32602, - "message": "Invalid params" + "message": "Invalid params", + "data" : "Missing deposit field" } }, "statusCode": 200 diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/09_eip6110_newPayloadV6110.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/09_eip6110_newPayloadV6110.json new file mode 100644 index 00000000000..0964114e62c --- /dev/null +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/09_eip6110_newPayloadV6110.json @@ -0,0 +1,45 @@ +{ + "request": { + "jsonrpc": "2.0", + "method": "engine_newPayloadV6110", + "params": [ + { + "parentHash": "0x45811fa27a100ce9035e5e086b9669275041a4ec0ebbd920be028fd7b0aa2356", + "feeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", + "stateRoot": "0x14208ac0e218167936e220b72d5d5887a963cb858ea2f2d268518f014a3da3fa", + "logsBloom": "0x10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000400000000000000000000000020000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000", + "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", + "gasLimit": "0x1c9c380", + "gasUsed": "0x14B6E", + "timestamp": "0x20", + "extraData": "0x", + "baseFeePerGas": "0x7", + "excessBlobGas": "0x0", + "transactions": [ + "0x02f9021c8217de808459682f008459682f0e830271009442424242424242424242424242424242424242428901bc16d674ec800000b901a422895118000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000e00000000000000000000000000000000000000000000000000000000000000120749715de5d1226545c6b3790f515d551a5cc5bf1d49c87a696860554d2fc4f14000000000000000000000000000000000000000000000000000000000000003096a96086cff07df17668f35f7418ef8798079167e3f4f9b72ecde17b28226137cf454ab1dd20ef5d924786ab3483c2f9000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000020003f5102dabe0a27b1746098d1dc17a5d3fbd478759fea9287e4e419b3c3cef20000000000000000000000000000000000000000000000000000000000000060b1acdb2c4d3df3f1b8d3bfd33421660df358d84d78d16c4603551935f4b67643373e7eb63dcb16ec359be0ec41fee33b03a16e80745f2374ff1d3c352508ac5d857c6476d3c3bcf7e6ca37427c9209f17be3af5264c0e2132b3dd1156c28b4e9c080a09f597089338d7f44f5c59f8230bb38f243849228a8d4e9d2e2956e6050f5b2c7a076486996c7e62802b8f95eee114783e4b403fd11093ba96286ff42c595f24452" + ], + "withdrawals": [], + "depositReceipts" : [ + {"amount":"0x773594000","index":"0x0","pubkey":"0x96a96086cff07df17668f35f7418ef8798079167e3f4f9b72ecde17b28226137cf454ab1dd20ef5d924786ab3483c2f9","signature":"0xb1acdb2c4d3df3f1b8d3bfd33421660df358d84d78d16c4603551935f4b67643373e7eb63dcb16ec359be0ec41fee33b03a16e80745f2374ff1d3c352508ac5d857c6476d3c3bcf7e6ca37427c9209f17be3af5264c0e2132b3dd1156c28b4e9","withdrawalCredentials":"0x003f5102dabe0a27b1746098d1dc17a5d3fbd478759fea9287e4e419b3c3cef2"} + ], + "blockNumber": "0x2", + "blockHash": "0xb3b483867217b83b1e4a2f95c84d2da30cbff12eb8636f2becbcc05f4507fa7a", + "receiptsRoot": "0x79ee3424eb720a3ad4b1c5a372bb8160580cbe4d893778660f34213c685627a9", + "blobGasUsed": "0x0" + }, + [], + "0x0000000000000000000000000000000000000000000000000000000000000000" + ], + "id": 67 + }, + "response": { + "jsonrpc": "2.0", + "id": 67, + "result": { + "status": "VALID", + "latestValidHash": "0xb3b483867217b83b1e4a2f95c84d2da30cbff12eb8636f2becbcc05f4507fa7a", + "validationError": null + } + }, + "statusCode": 200 +} \ No newline at end of file diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/10_eip6110_forkchoiceUpdatedV3.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/10_eip6110_forkchoiceUpdatedV3.json new file mode 100644 index 00000000000..5a32569d611 --- /dev/null +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/10_eip6110_forkchoiceUpdatedV3.json @@ -0,0 +1,34 @@ +{ + "request": { + "jsonrpc": "2.0", + "method": "engine_forkchoiceUpdatedV3", + "params": [ + { + "headBlockHash": "0xb3b483867217b83b1e4a2f95c84d2da30cbff12eb8636f2becbcc05f4507fa7a", + "safeBlockHash": "0xb3b483867217b83b1e4a2f95c84d2da30cbff12eb8636f2becbcc05f4507fa7a", + "finalizedBlockHash": "0x0000000000000000000000000000000000000000000000000000000000000000" + }, + { + "timestamp": "0x30", + "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", + "suggestedFeeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", + "withdrawals": [], + "parentBeaconBlockRoot": "0x0000000000000000000000000000000000000000000000000000000000000000" + } + ], + "id": 67 + }, + "response": { + "jsonrpc": "2.0", + "id": 67, + "result": { + "payloadStatus": { + "status": "VALID", + "latestValidHash": "0xb3b483867217b83b1e4a2f95c84d2da30cbff12eb8636f2becbcc05f4507fa7a", + "validationError": null + }, + "payloadId": "0x282643daa04b7631" + } + }, + "statusCode" : 200 +} \ No newline at end of file diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/11_eip6110_getPayloadV6110.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/11_eip6110_getPayloadV6110.json new file mode 100644 index 00000000000..6c546cd267d --- /dev/null +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/eip6110/test-cases/11_eip6110_getPayloadV6110.json @@ -0,0 +1,45 @@ +{ + "request": { + "jsonrpc": "2.0", + "method": "engine_getPayloadV6110", + "params": [ + "0x282643daa04b7631" + ], + "id": 67 + }, + "response": { + "jsonrpc": "2.0", + "id": 67, + "result": { + "executionPayload": { + "parentHash": "0xb3b483867217b83b1e4a2f95c84d2da30cbff12eb8636f2becbcc05f4507fa7a", + "feeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", + "stateRoot": "0x14208ac0e218167936e220b72d5d5887a963cb858ea2f2d268518f014a3da3fa", + "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", + "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", + "gasLimit": "0x1c9c380", + "gasUsed": "0x0", + "timestamp": "0x30", + "extraData": "0x", + "baseFeePerGas": "0x7", + "excessBlobGas": "0x0", + "parentBeaconBlockRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", + "transactions": [], + "withdrawals": [], + "depositReceipts": [], + "blockNumber": "0x3", + "blockHash": "0xa28bf4db3363ce5b67848eb2ad52dbfead62ddb2287ae7eed36daa002528d1af", + "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421", + "blobGasUsed": "0x0" + }, + "blockValue": "0x0", + "blobsBundle": { + "commitments": [], + "proofs": [], + "blobs": [] + }, + "shouldOverrideBuilder": false + } + }, + "statusCode": 200 +} \ No newline at end of file diff --git a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/shanghai/test-cases/08_shanghai_getPayloadV2.json b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/shanghai/test-cases/08_shanghai_getPayloadV2.json index 2da9be48f4e..072e5146d0a 100644 --- a/acceptance-tests/tests/src/test/resources/jsonrpc/engine/shanghai/test-cases/08_shanghai_getPayloadV2.json +++ b/acceptance-tests/tests/src/test/resources/jsonrpc/engine/shanghai/test-cases/08_shanghai_getPayloadV2.json @@ -37,7 +37,6 @@ "amount": "0x2" } ], - "deposits": null, "blockNumber": "0x2", "blockHash": "0x612abd8615f544759d4aeb3dbab32f5f198a8b818e9c5436e9f7a674ef3b0f20", "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421" diff --git a/besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java b/besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java index 75c44198877..d7ce4dfa539 100644 --- a/besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java +++ b/besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java @@ -177,7 +177,7 @@ import org.hyperledger.besu.plugin.services.securitymodule.SecurityModule; import org.hyperledger.besu.plugin.services.storage.PrivacyKeyValueStorageFactory; import org.hyperledger.besu.plugin.services.storage.rocksdb.RocksDBPlugin; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory; import org.hyperledger.besu.plugin.services.txvalidator.PluginTransactionValidatorFactory; import org.hyperledger.besu.services.BesuEventsImpl; import org.hyperledger.besu.services.BesuPluginContextImpl; @@ -1294,6 +1294,11 @@ static class PermissionsOptionGroup { "Specifies the maximum number of blocks to retrieve logs from via RPC. Must be >=0. 0 specifies no limit (default: ${DEFAULT-VALUE})") private final Long rpcMaxLogsRange = 5000L; + @CommandLine.Option( + names = {"--cache-last-blocks"}, + description = "Specifies the number of last blocks to cache (default: ${DEFAULT-VALUE})") + private final Integer numberOfblocksToCache = 0; + @Mixin private P2PTLSConfigOptions p2pTLSConfigOptions; @Mixin private PkiBlockCreationOptions pkiBlockCreationOptions; @@ -2290,11 +2295,12 @@ public BesuControllerBuilder getControllerBuilder() { .lowerBoundPeers(peersLowerBound) .maxRemotelyInitiatedPeers(maxRemoteInitiatedPeers) .randomPeerPriority(p2PDiscoveryOptionGroup.randomPeerPriority) - .chainPruningConfiguration(unstableChainPruningOptions.toDomainObject()); + .chainPruningConfiguration(unstableChainPruningOptions.toDomainObject()) + .cacheLastBlocks(numberOfblocksToCache); } @NotNull - private Optional getTransactionSelectorFactory() { + private Optional getTransactionSelectorFactory() { final Optional txSelectionService = besuPluginContext.getService(TransactionSelectionService.class); return txSelectionService.isPresent() ? txSelectionService.get().get() : Optional.empty(); @@ -3549,6 +3555,8 @@ private String generateConfigurationOverview() { builder.setTxPoolImplementation(buildTransactionPoolConfiguration().getTxPoolImplementation()); + builder.setPluginContext(besuComponent.getBesuPluginContext()); + return builder.build(); } } diff --git a/besu/src/main/java/org/hyperledger/besu/cli/ConfigurationOverviewBuilder.java b/besu/src/main/java/org/hyperledger/besu/cli/ConfigurationOverviewBuilder.java index 5bd1d84a298..508b195a8eb 100644 --- a/besu/src/main/java/org/hyperledger/besu/cli/ConfigurationOverviewBuilder.java +++ b/besu/src/main/java/org/hyperledger/besu/cli/ConfigurationOverviewBuilder.java @@ -16,6 +16,7 @@ import org.hyperledger.besu.BesuInfo; import org.hyperledger.besu.ethereum.eth.transactions.TransactionPoolConfiguration; +import org.hyperledger.besu.services.BesuPluginContextImpl; import org.hyperledger.besu.util.log.FramedLogMessage; import org.hyperledger.besu.util.platform.PlatformDetector; @@ -50,6 +51,7 @@ public class ConfigurationOverviewBuilder { private boolean isHighSpec = false; private TransactionPoolConfiguration.Implementation txPoolImplementation; private Map environment; + private BesuPluginContextImpl besuPluginContext; /** * @param logger the logger @@ -193,7 +195,7 @@ public ConfigurationOverviewBuilder setEngineJwtFile(final String engineJwtFileP /** * Sets the environment variables. * - * @param environment the enveironment variables + * @param environment the environment variables * @return the builder */ public ConfigurationOverviewBuilder setEnvironment(final Map environment) { @@ -277,6 +279,12 @@ public String build() { lines.add("Total memory: " + normalizeSize(hardwareInfo.getMemory().getTotal())); lines.add("CPU cores: " + hardwareInfo.getProcessor().getLogicalProcessorCount()); + lines.add(""); + + if (besuPluginContext != null) { + lines.addAll(besuPluginContext.getPluginsSummaryLog()); + } + return FramedLogMessage.generate(lines); } @@ -308,4 +316,13 @@ private void detectJemalloc(final List lines) { private String normalizeSize(final long size) { return String.format("%.02f", (double) (size) / 1024 / 1024 / 1024) + " GB"; } + + /** + * set the plugin context + * + * @param besuPluginContext the plugin context + */ + public void setPluginContext(final BesuPluginContextImpl besuPluginContext) { + this.besuPluginContext = besuPluginContext; + } } diff --git a/besu/src/main/java/org/hyperledger/besu/cli/options/stable/TransactionPoolOptions.java b/besu/src/main/java/org/hyperledger/besu/cli/options/stable/TransactionPoolOptions.java index 6353618d24d..d481e0453a0 100644 --- a/besu/src/main/java/org/hyperledger/besu/cli/options/stable/TransactionPoolOptions.java +++ b/besu/src/main/java/org/hyperledger/besu/cli/options/stable/TransactionPoolOptions.java @@ -24,6 +24,7 @@ import org.hyperledger.besu.cli.converter.PercentageConverter; import org.hyperledger.besu.cli.options.CLIOptions; import org.hyperledger.besu.cli.util.CommandLineUtils; +import org.hyperledger.besu.datatypes.Address; import org.hyperledger.besu.datatypes.Wei; import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration; import org.hyperledger.besu.ethereum.eth.transactions.TransactionPoolConfiguration; @@ -32,19 +33,25 @@ import java.io.File; import java.util.List; +import java.util.Set; import picocli.CommandLine; /** The Transaction pool Cli stable options. */ public class TransactionPoolOptions implements CLIOptions { private static final String TX_POOL_IMPLEMENTATION = "--tx-pool"; + /** Use TX_POOL_NO_LOCAL_PRIORITY instead */ + @Deprecated(forRemoval = true) private static final String TX_POOL_DISABLE_LOCALS = "--tx-pool-disable-locals"; + + private static final String TX_POOL_NO_LOCAL_PRIORITY = "--tx-pool-no-local-priority"; private static final String TX_POOL_ENABLE_SAVE_RESTORE = "--tx-pool-enable-save-restore"; private static final String TX_POOL_SAVE_FILE = "--tx-pool-save-file"; private static final String TX_POOL_PRICE_BUMP = "--tx-pool-price-bump"; private static final String RPC_TX_FEECAP = "--rpc-tx-feecap"; private static final String STRICT_TX_REPLAY_PROTECTION_ENABLED_FLAG = "--strict-tx-replay-protection-enabled"; + private static final String TX_POOL_PRIORITY_SENDERS = "--tx-pool-priority-senders"; @CommandLine.Option( names = {TX_POOL_IMPLEMENTATION}, @@ -54,13 +61,13 @@ public class TransactionPoolOptions implements CLIOptions prioritySenders = TransactionPoolConfiguration.DEFAULT_PRIORITY_SENDERS; + @CommandLine.ArgGroup( validate = false, heading = "@|bold Tx Pool Layered Implementation Options|@%n") @@ -200,11 +216,12 @@ public static TransactionPoolOptions fromConfig(final TransactionPoolConfigurati final TransactionPoolOptions options = TransactionPoolOptions.create(); options.txPoolImplementation = config.getTxPoolImplementation(); options.saveRestoreEnabled = config.getEnableSaveRestore(); - options.disableLocalTxs = config.getDisableLocalTransactions(); + options.noLocalPriority = config.getNoLocalPriority(); options.priceBump = config.getPriceBump(); options.txFeeCap = config.getTxFeeCap(); options.saveFile = config.getSaveFile(); options.strictTxReplayProtectionEnabled = config.getStrictTransactionReplayProtectionEnabled(); + options.prioritySenders = config.getPrioritySenders(); options.layeredOptions.txPoolLayerMaxCapacity = config.getPendingTransactionsLayerMaxCapacityBytes(); options.layeredOptions.txPoolMaxPrioritized = config.getMaxPrioritizedTransactions(); @@ -242,11 +259,12 @@ public TransactionPoolConfiguration toDomainObject() { return ImmutableTransactionPoolConfiguration.builder() .txPoolImplementation(txPoolImplementation) .enableSaveRestore(saveRestoreEnabled) - .disableLocalTransactions(disableLocalTxs) + .noLocalPriority(noLocalPriority) .priceBump(priceBump) .txFeeCap(txFeeCap) .saveFile(saveFile) .strictTransactionReplayProtectionEnabled(strictTxReplayProtectionEnabled) + .prioritySenders(prioritySenders) .pendingTransactionsLayerMaxCapacityBytes(layeredOptions.txPoolLayerMaxCapacity) .maxPrioritizedTransactions(layeredOptions.txPoolMaxPrioritized) .maxFutureBySender(layeredOptions.txPoolMaxFutureBySender) diff --git a/besu/src/main/java/org/hyperledger/besu/cli/util/CommandLineUtils.java b/besu/src/main/java/org/hyperledger/besu/cli/util/CommandLineUtils.java index 0115420005e..a44e616bd3d 100644 --- a/besu/src/main/java/org/hyperledger/besu/cli/util/CommandLineUtils.java +++ b/besu/src/main/java/org/hyperledger/besu/cli/util/CommandLineUtils.java @@ -110,7 +110,7 @@ public static void checkMultiOptionDependencies( } /** - * Fail if option doesnt meet requirement. + * Fail if option doesn't meet requirement. * * @param commandLine the command line * @param errorMessage the error message @@ -126,7 +126,8 @@ public static void failIfOptionDoesntMeetRequirement( final String affectedOptions = getAffectedOptions(commandLine, dependentOptionsNames); if (!affectedOptions.isEmpty()) { - throw new CommandLine.ParameterException(commandLine, errorMessage); + throw new CommandLine.ParameterException( + commandLine, errorMessage + " [" + affectedOptions + "]"); } } } diff --git a/besu/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java b/besu/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java index d023cdd76da..802be1215b5 100644 --- a/besu/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java +++ b/besu/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java @@ -95,7 +95,7 @@ import org.hyperledger.besu.metrics.ObservableMetricsSystem; import org.hyperledger.besu.plugin.services.MetricsSystem; import org.hyperledger.besu.plugin.services.permissioning.NodeMessagePermissioningProvider; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory; import org.hyperledger.besu.plugin.services.txvalidator.PluginTransactionValidatorFactory; import java.io.Closeable; @@ -182,12 +182,14 @@ public abstract class BesuControllerBuilder implements MiningParameterOverrides private NetworkingConfiguration networkingConfiguration; private Boolean randomPeerPriority; - private Optional transactionSelectorFactory = Optional.empty(); + private Optional transactionSelectorFactory = Optional.empty(); /** the Dagger configured context that can provide dependencies */ protected Optional besuComponent = Optional.empty(); private PluginTransactionValidatorFactory pluginTransactionValidatorFactory; + private int numberOfBlocksToCache = 0; + /** * Provide a BesuComponent which can be used to get other dependencies * @@ -505,6 +507,17 @@ public BesuControllerBuilder chainPruningConfiguration( return this; } + /** + * Chain pruning configuration besu controller builder. + * + * @param numberOfBlocksToCache the number of blocks to cache + * @return the besu controller builder + */ + public BesuControllerBuilder cacheLastBlocks(final Integer numberOfBlocksToCache) { + this.numberOfBlocksToCache = numberOfBlocksToCache; + return this; + } + /** * sets the networkConfiguration in the builder * @@ -535,7 +548,7 @@ public BesuControllerBuilder randomPeerPriority(final Boolean randomPeerPriority * @return the besu controller builder */ public BesuControllerBuilder transactionSelectorFactory( - final Optional transactionSelectorFactory) { + final Optional transactionSelectorFactory) { this.transactionSelectorFactory = transactionSelectorFactory; return this; } @@ -592,7 +605,8 @@ public BesuController build() { blockchainStorage, metricsSystem, reorgLoggingThreshold, - dataDirectory.toString()); + dataDirectory.toString(), + numberOfBlocksToCache); final CachedMerkleTrieLoader cachedMerkleTrieLoader = besuComponent @@ -1035,7 +1049,7 @@ protected ProtocolContext createProtocolContext( final WorldStateArchive worldStateArchive, final ProtocolSchedule protocolSchedule, final ConsensusContextFactory consensusContextFactory, - final Optional transactionSelectorFactory) { + final Optional transactionSelectorFactory) { return ProtocolContext.init( blockchain, worldStateArchive, diff --git a/besu/src/main/java/org/hyperledger/besu/controller/ConsensusScheduleBesuControllerBuilder.java b/besu/src/main/java/org/hyperledger/besu/controller/ConsensusScheduleBesuControllerBuilder.java index 6d0d02b22f0..921289c66e5 100644 --- a/besu/src/main/java/org/hyperledger/besu/controller/ConsensusScheduleBesuControllerBuilder.java +++ b/besu/src/main/java/org/hyperledger/besu/controller/ConsensusScheduleBesuControllerBuilder.java @@ -62,7 +62,7 @@ import org.hyperledger.besu.evm.internal.EvmConfiguration; import org.hyperledger.besu.metrics.ObservableMetricsSystem; import org.hyperledger.besu.plugin.services.permissioning.NodeMessagePermissioningProvider; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory; import java.math.BigInteger; import java.nio.file.Path; @@ -176,7 +176,7 @@ protected ProtocolContext createProtocolContext( final WorldStateArchive worldStateArchive, final ProtocolSchedule protocolSchedule, final ConsensusContextFactory consensusContextFactory, - final Optional transactionSelectorFactory) { + final Optional transactionSelectorFactory) { return MigratingProtocolContext.init( blockchain, worldStateArchive, diff --git a/besu/src/main/java/org/hyperledger/besu/controller/TransitionBesuControllerBuilder.java b/besu/src/main/java/org/hyperledger/besu/controller/TransitionBesuControllerBuilder.java index 9323f3b81d6..6db71ef81ca 100644 --- a/besu/src/main/java/org/hyperledger/besu/controller/TransitionBesuControllerBuilder.java +++ b/besu/src/main/java/org/hyperledger/besu/controller/TransitionBesuControllerBuilder.java @@ -60,7 +60,7 @@ import org.hyperledger.besu.evm.internal.EvmConfiguration; import org.hyperledger.besu.metrics.ObservableMetricsSystem; import org.hyperledger.besu.plugin.services.permissioning.NodeMessagePermissioningProvider; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory; import java.math.BigInteger; import java.nio.file.Path; @@ -190,7 +190,7 @@ protected ProtocolContext createProtocolContext( final WorldStateArchive worldStateArchive, final ProtocolSchedule protocolSchedule, final ConsensusContextFactory consensusContextFactory, - final Optional transactionSelectorFactory) { + final Optional transactionSelectorFactory) { final ProtocolContext protocolContext = super.createProtocolContext( blockchain, diff --git a/besu/src/main/java/org/hyperledger/besu/services/BesuPluginContextImpl.java b/besu/src/main/java/org/hyperledger/besu/services/BesuPluginContextImpl.java index 04bf5447a44..184c85f45de 100644 --- a/besu/src/main/java/org/hyperledger/besu/services/BesuPluginContextImpl.java +++ b/besu/src/main/java/org/hyperledger/besu/services/BesuPluginContextImpl.java @@ -75,6 +75,7 @@ private enum Lifecycle { private final Map, ? super BesuService> serviceRegistry = new HashMap<>(); private final List plugins = new ArrayList<>(); private final List pluginVersions = new ArrayList<>(); + final List lines = new ArrayList<>(); /** * Add service. @@ -104,6 +105,7 @@ public Optional getService(final Class serviceType * @param pluginsDir the plugins dir */ public void registerPlugins(final Path pluginsDir) { + lines.add("Plugins:"); checkState( state == Lifecycle.UNINITIALIZED, "Besu plugins have already been registered. Cannot register additional plugins."); @@ -116,10 +118,13 @@ public void registerPlugins(final Path pluginsDir) { final ServiceLoader serviceLoader = ServiceLoader.load(BesuPlugin.class, pluginLoader); + int pluginsCount = 0; for (final BesuPlugin plugin : serviceLoader) { + pluginsCount++; try { plugin.register(this); LOG.info("Registered plugin of type {}.", plugin.getClass().getName()); + lines.add(String.format(plugin.getClass().getSimpleName())); addPluginVersion(plugin); } catch (final Exception e) { LOG.error( @@ -127,16 +132,30 @@ public void registerPlugins(final Path pluginsDir) { + plugin.getClass().getName() + ", start and stop will not be called.", e); + lines.add(String.format("ERROR %s", plugin.getClass().getSimpleName())); continue; } plugins.add(plugin); } LOG.debug("Plugin registration complete."); + lines.add( + String.format( + "TOTAL = %d of %d plugins successfully loaded", plugins.size(), pluginsCount)); + lines.add(String.format("from %s", pluginsDir.toAbsolutePath())); state = Lifecycle.REGISTERED; } + /** + * get the summary log, as a list of string lines + * + * @return the summary + */ + public List getPluginsSummaryLog() { + return lines; + } + private void addPluginVersion(final BesuPlugin plugin) { final Package pluginPackage = plugin.getClass().getPackage(); final String implTitle = @@ -259,7 +278,7 @@ List getPlugins() { private Optional pluginDirectoryLoader(final Path pluginsDir) { if (pluginsDir != null && pluginsDir.toFile().isDirectory()) { - LOG.debug("Searching for plugins in {}", pluginsDir.toAbsolutePath().toString()); + LOG.debug("Searching for plugins in {}", pluginsDir.toAbsolutePath()); try (final Stream pluginFilesList = Files.list(pluginsDir)) { final URL[] pluginJarURLs = diff --git a/besu/src/main/java/org/hyperledger/besu/services/RpcEndpointServiceImpl.java b/besu/src/main/java/org/hyperledger/besu/services/RpcEndpointServiceImpl.java index 4513bf0bf39..94bb5560839 100644 --- a/besu/src/main/java/org/hyperledger/besu/services/RpcEndpointServiceImpl.java +++ b/besu/src/main/java/org/hyperledger/besu/services/RpcEndpointServiceImpl.java @@ -28,7 +28,7 @@ import java.util.function.Function; import java.util.stream.Collectors; -/** The Rpc endpoint service implementation. */ +/** The RPC endpoint service implementation. */ public class RpcEndpointServiceImpl implements RpcEndpointService { private final Map> rpcMethods = new HashMap<>(); diff --git a/besu/src/main/java/org/hyperledger/besu/services/TransactionSelectionServiceImpl.java b/besu/src/main/java/org/hyperledger/besu/services/TransactionSelectionServiceImpl.java index 42b1ae88a76..3175fe731f9 100644 --- a/besu/src/main/java/org/hyperledger/besu/services/TransactionSelectionServiceImpl.java +++ b/besu/src/main/java/org/hyperledger/besu/services/TransactionSelectionServiceImpl.java @@ -15,23 +15,23 @@ package org.hyperledger.besu.services; import org.hyperledger.besu.plugin.services.TransactionSelectionService; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory; import java.util.Optional; /** The Transaction Selection service implementation. */ public class TransactionSelectionServiceImpl implements TransactionSelectionService { - private Optional factory = Optional.empty(); + private Optional factory = Optional.empty(); @Override - public Optional get() { + public Optional get() { return factory; } @Override public void registerTransactionSelectorFactory( - final TransactionSelectorFactory transactionSelectorFactory) { + final PluginTransactionSelectorFactory transactionSelectorFactory) { factory = Optional.ofNullable(transactionSelectorFactory); } } diff --git a/besu/src/test/java/org/hyperledger/besu/cli/BesuCommandTest.java b/besu/src/test/java/org/hyperledger/besu/cli/BesuCommandTest.java index 553220c9431..af28e7ba7c2 100644 --- a/besu/src/test/java/org/hyperledger/besu/cli/BesuCommandTest.java +++ b/besu/src/test/java/org/hyperledger/besu/cli/BesuCommandTest.java @@ -5585,4 +5585,16 @@ public void snapsyncForHealingFeaturesShouldFailWhenHealingIsNotEnabled() { .contains( "--Xsnapsync-synchronizer-flat option can only be used when -Xsnapsync-synchronizer-flat-db-healing-enabled is true"); } + + @Test + public void cacheLastBlocksOptionShouldWork() { + int numberOfBlocksToCache = 512; + parseCommand("--cache-last-blocks", String.valueOf(numberOfBlocksToCache)); + verify(mockControllerBuilder).cacheLastBlocks(intArgumentCaptor.capture()); + verify(mockControllerBuilder).build(); + + assertThat(intArgumentCaptor.getValue()).isEqualTo(numberOfBlocksToCache); + assertThat(commandOutput.toString(UTF_8)).isEmpty(); + assertThat(commandErrorOutput.toString(UTF_8)).isEmpty(); + } } diff --git a/besu/src/test/java/org/hyperledger/besu/cli/CommandTestAbstract.java b/besu/src/test/java/org/hyperledger/besu/cli/CommandTestAbstract.java index 126052df0b9..1c991d64935 100644 --- a/besu/src/test/java/org/hyperledger/besu/cli/CommandTestAbstract.java +++ b/besu/src/test/java/org/hyperledger/besu/cli/CommandTestAbstract.java @@ -245,6 +245,8 @@ public void initMocks() throws Exception { .thenReturn(mockControllerBuilder); when(mockControllerBuilder.besuComponent(any(BesuComponent.class))) .thenReturn(mockControllerBuilder); + when(mockControllerBuilder.cacheLastBlocks(any())).thenReturn(mockControllerBuilder); + // doReturn used because of generic BesuController doReturn(mockController).when(mockControllerBuilder).build(); lenient().when(mockController.getProtocolManager()).thenReturn(mockEthProtocolManager); diff --git a/besu/src/test/java/org/hyperledger/besu/cli/options/stable/TransactionPoolOptionsTest.java b/besu/src/test/java/org/hyperledger/besu/cli/options/stable/TransactionPoolOptionsTest.java index 1f0705e9948..e7977a921e1 100644 --- a/besu/src/test/java/org/hyperledger/besu/cli/options/stable/TransactionPoolOptionsTest.java +++ b/besu/src/test/java/org/hyperledger/besu/cli/options/stable/TransactionPoolOptionsTest.java @@ -20,6 +20,7 @@ import org.hyperledger.besu.cli.options.AbstractCLIOptionsTest; import org.hyperledger.besu.cli.options.OptionParser; +import org.hyperledger.besu.datatypes.Address; import org.hyperledger.besu.datatypes.Wei; import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration; import org.hyperledger.besu.ethereum.eth.transactions.TransactionPoolConfiguration; @@ -73,20 +74,20 @@ public void pendingTransactionRetentionPeriod() { @Test public void disableLocalsDefault() { - internalTestSuccess(config -> assertThat(config.getDisableLocalTransactions()).isFalse()); + internalTestSuccess(config -> assertThat(config.getNoLocalPriority()).isFalse()); } @Test public void disableLocalsOn() { internalTestSuccess( - config -> assertThat(config.getDisableLocalTransactions()).isTrue(), + config -> assertThat(config.getNoLocalPriority()).isTrue(), "--tx-pool-disable-locals=true"); } @Test public void disableLocalsOff() { internalTestSuccess( - config -> assertThat(config.getDisableLocalTransactions()).isFalse(), + config -> assertThat(config.getNoLocalPriority()).isFalse(), "--tx-pool-disable-locals=false"); } @@ -214,6 +215,61 @@ public void failIfLayeredOptionsWhenLegacySelectedByArg() { "--tx-pool-max-prioritized=1000"); } + @Test + public void byDefaultNoPrioritySenders() { + internalTestSuccess(config -> assertThat(config.getPrioritySenders()).isEmpty()); + } + + @Test + public void onePrioritySenderWorks() { + final Address prioritySender = Address.fromHexString("0xABC123"); + internalTestSuccess( + config -> assertThat(config.getPrioritySenders()).containsExactly(prioritySender), + "--tx-pool-priority-senders", + prioritySender.toHexString()); + } + + @Test + public void morePrioritySendersWorks() { + final Address prioritySender1 = Address.fromHexString("0xABC123"); + final Address prioritySender2 = Address.fromHexString("0xDEF456"); + final Address prioritySender3 = Address.fromHexString("0x789000"); + internalTestSuccess( + config -> + assertThat(config.getPrioritySenders()) + .containsExactly(prioritySender1, prioritySender2, prioritySender3), + "--tx-pool-priority-senders", + prioritySender1.toHexString() + + "," + + prioritySender2.toHexString() + + "," + + prioritySender3.toHexString()); + } + + @Test + public void atLeastOnePrioritySenders() { + internalTestFailure( + "Missing required parameter for option '--tx-pool-priority-senders' at index 0 (Comma separated list of addresses)", + "--tx-pool-priority-senders"); + } + + @Test + public void malformedListOfPrioritySenders() { + final Address prioritySender1 = Address.fromHexString("0xABC123"); + final Address prioritySender2 = Address.fromHexString("0xDEF456"); + final Address prioritySender3 = Address.fromHexString("0x789000"); + internalTestFailure( + "Invalid value for option '--tx-pool-priority-senders' at index 0 (Comma separated list of addresses): " + + "cannot convert '0x0000000000000000000000000000000000abc123;0x0000000000000000000000000000000000def456' " + + "to Address (java.lang.IllegalArgumentException: Invalid odd-length hex binary representation)", + "--tx-pool-priority-senders", + prioritySender1.toHexString() + + ";" + + prioritySender2.toHexString() + + "," + + prioritySender3.toHexString()); + } + @Override protected TransactionPoolConfiguration createDefaultDomainObject() { return TransactionPoolConfiguration.DEFAULT; diff --git a/besu/src/test/resources/everything_config.toml b/besu/src/test/resources/everything_config.toml index e88acec6dc9..f4c1d517d91 100644 --- a/besu/src/test/resources/everything_config.toml +++ b/besu/src/test/resources/everything_config.toml @@ -88,6 +88,7 @@ rpc-http-max-batch-size=1 rpc-http-max-request-content-length = 5242880 rpc-max-logs-range=100 json-pretty-print-enabled=false +cache-last-blocks=512 # PRIVACY TLS privacy-tls-enabled=false @@ -174,7 +175,8 @@ tx-pool="layered" tx-pool-price-bump=13 rpc-tx-feecap=2000000000000000000 strict-tx-replay-protection-enabled=true -tx-pool-disable-locals=false +tx-pool-no-local-priority=false +tx-pool-priority-senders=["0xABC0000000000000000000000000000000001234","0xDEF0000000000000000000000000000000001234"] tx-pool-enable-save-restore=true tx-pool-save-file="txpool.dump" ## Layered @@ -226,4 +228,4 @@ Xp2p-tls-crl-file="none.file" Xp2p-tls-clienthello-sni=false #contracts -Xevm-jumpdest-cache-weight-kb=32000 +Xevm-jumpdest-cache-weight-kb=32000 \ No newline at end of file diff --git a/build.gradle b/build.gradle index 8ce41eb46f1..a1950eb65db 100644 --- a/build.gradle +++ b/build.gradle @@ -14,6 +14,7 @@ */ +import groovy.transform.CompileStatic import groovy.transform.Memoized import net.ltgt.gradle.errorprone.CheckSeverity @@ -147,7 +148,20 @@ allprojects { } } - dependencies { errorprone 'com.google.errorprone:error_prone_core' } + dependencies { + components.all(BouncyCastleCapability) + errorprone 'com.google.errorprone:error_prone_core' + } + + configurations.all { + resolutionStrategy.capabilitiesResolution.withCapability('org.bouncycastle:bcprov-jdk18on') { + selectHighestVersion() + } + resolutionStrategy.capabilitiesResolution.withCapability('org.bouncycastle:bcpkix-jdk18on') { + selectHighestVersion() + } + } + apply plugin: 'com.diffplug.spotless' spotless { @@ -1044,6 +1058,29 @@ dependencies { errorprone 'com.google.errorprone:error_prone_core' } +@CompileStatic +class BouncyCastleCapability implements ComponentMetadataRule { + void execute(ComponentMetadataContext context) { + context.details.with { + if (id.group == "org.bouncycastle") { + if(id.name == "bcprov-jdk15on") { + allVariants { + it.withCapabilities { + it.addCapability("org.bouncycastle", "bcprov-jdk18on", "0") + } + } + } else if(id.name == "bcpkix-jdk15on") { + allVariants { + it.withCapabilities { + it.addCapability("org.bouncycastle", "bcpkix-jdk18on", "0") + } + } + } + } + } + } +} + distributions { main { contents { diff --git a/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/MigratingProtocolContext.java b/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/MigratingProtocolContext.java index e4304a3dbd0..32099e23fdf 100644 --- a/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/MigratingProtocolContext.java +++ b/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/MigratingProtocolContext.java @@ -20,7 +20,7 @@ import org.hyperledger.besu.ethereum.chain.MutableBlockchain; import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule; import org.hyperledger.besu.ethereum.worldstate.WorldStateArchive; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory; import java.util.Optional; @@ -41,7 +41,7 @@ public MigratingProtocolContext( final MutableBlockchain blockchain, final WorldStateArchive worldStateArchive, final ForksSchedule consensusContextSchedule, - final Optional transactionSelectorFactory) { + final Optional transactionSelectorFactory) { super(blockchain, worldStateArchive, null, transactionSelectorFactory); this.consensusContextSchedule = consensusContextSchedule; } @@ -61,7 +61,7 @@ public static ProtocolContext init( final WorldStateArchive worldStateArchive, final ProtocolSchedule protocolSchedule, final ConsensusContextFactory consensusContextFactory, - final Optional transactionSelectorFactory) { + final Optional transactionSelectorFactory) { final ConsensusContext consensusContext = consensusContextFactory.create(blockchain, worldStateArchive, protocolSchedule); final MigratingContext migratingContext = consensusContext.as(MigratingContext.class); diff --git a/consensus/merge/src/main/java/org/hyperledger/besu/consensus/merge/blockcreation/MergeCoordinator.java b/consensus/merge/src/main/java/org/hyperledger/besu/consensus/merge/blockcreation/MergeCoordinator.java index d7e2856941f..ffd57c4b27e 100644 --- a/consensus/merge/src/main/java/org/hyperledger/besu/consensus/merge/blockcreation/MergeCoordinator.java +++ b/consensus/merge/src/main/java/org/hyperledger/besu/consensus/merge/blockcreation/MergeCoordinator.java @@ -594,7 +594,8 @@ && isDescendantOf(newHead, blockchain.getChainHeadHeader())) { Optional parentOfNewHead = blockchain.getBlockHeader(newHead.getParentHash()); if (parentOfNewHead.isPresent() - && parentOfNewHead.get().getTimestamp() >= newHead.getTimestamp()) { + && Long.compareUnsigned(newHead.getTimestamp(), parentOfNewHead.get().getTimestamp()) + <= 0) { return ForkchoiceResult.withFailure( INVALID, "new head timestamp not greater than parent", latestValid); } diff --git a/consensus/merge/src/test/java/org/hyperledger/besu/consensus/merge/blockcreation/MergeCoordinatorTest.java b/consensus/merge/src/test/java/org/hyperledger/besu/consensus/merge/blockcreation/MergeCoordinatorTest.java index 232140fac57..bc7e424133d 100644 --- a/consensus/merge/src/test/java/org/hyperledger/besu/consensus/merge/blockcreation/MergeCoordinatorTest.java +++ b/consensus/merge/src/test/java/org/hyperledger/besu/consensus/merge/blockcreation/MergeCoordinatorTest.java @@ -58,11 +58,11 @@ import org.hyperledger.besu.ethereum.core.BlockHeaderTestFixture; import org.hyperledger.besu.ethereum.core.Difficulty; import org.hyperledger.besu.ethereum.core.MiningParameters; -import org.hyperledger.besu.ethereum.core.Transaction; import org.hyperledger.besu.ethereum.core.TransactionTestFixture; import org.hyperledger.besu.ethereum.eth.manager.EthContext; import org.hyperledger.besu.ethereum.eth.sync.backwardsync.BackwardSyncContext; import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration; +import org.hyperledger.besu.ethereum.eth.transactions.PendingTransaction; import org.hyperledger.besu.ethereum.eth.transactions.TransactionBroadcaster; import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool; import org.hyperledger.besu.ethereum.eth.transactions.TransactionPoolConfiguration; @@ -314,8 +314,8 @@ public void exceptionDuringBuildingBlockShouldNotBeInvalid() invocation -> { if (retries.getAndIncrement() < txPerBlock) { // a new transaction every time a block is built - transactions.addLocalTransaction( - createTransaction(retries.get() - 1), Optional.empty()); + transactions.addTransaction( + createLocalTransaction(retries.get() - 1), Optional.empty()); } else { // when we have 5 transactions finalize block creation willThrow.finalizeProposalById( @@ -387,8 +387,8 @@ public void shouldContinueBuildingBlocksUntilFinalizeIsCalled() invocation -> { if (retries.getAndIncrement() < 5) { // a new transaction every time a block is built - transactions.addLocalTransaction( - createTransaction(retries.get() - 1), Optional.empty()); + transactions.addTransaction( + createLocalTransaction(retries.get() - 1), Optional.empty()); } else { // when we have 5 transactions finalize block creation coordinator.finalizeProposalById( @@ -506,7 +506,7 @@ public void shouldRetryBlockCreationOnRecoverableError() .when(mergeContext) .putPayloadById(any()); - transactions.addLocalTransaction(createTransaction(0), Optional.empty()); + transactions.addTransaction(createLocalTransaction(0), Optional.empty()); var payloadId = coordinator.preparePayload( @@ -643,8 +643,8 @@ public void shouldNotStartAnotherBlockCreationJobIfCalledAgainWithTheSamePayload invocation -> { if (retries.getAndIncrement() < 5) { // a new transaction every time a block is built - transactions.addLocalTransaction( - createTransaction(retries.get() - 1), Optional.empty()); + transactions.addTransaction( + createLocalTransaction(retries.get() - 1), Optional.empty()); } else { // when we have 5 transactions finalize block creation coordinator.finalizeProposalById( @@ -1022,20 +1022,24 @@ private BlockHeader nextBlockHeader( .buildHeader(); } - private Transaction createTransaction(final long transactionNumber) { - return new TransactionTestFixture() - .value(Wei.of(transactionNumber + 1)) - .to(Optional.of(Address.ZERO)) - .gasLimit(53000L) - .gasPrice( - Wei.fromHexString("0x00000000000000000000000000000000000000000000000000000013b9aca00")) - .maxFeePerGas( - Optional.of( + private PendingTransaction createLocalTransaction(final long transactionNumber) { + return PendingTransaction.newPendingTransaction( + new TransactionTestFixture() + .value(Wei.of(transactionNumber + 1)) + .to(Optional.of(Address.ZERO)) + .gasLimit(53000L) + .gasPrice( Wei.fromHexString( - "0x00000000000000000000000000000000000000000000000000000013b9aca00"))) - .maxPriorityFeePerGas(Optional.of(Wei.of(100_000))) - .nonce(transactionNumber) - .createTransaction(KEYS1); + "0x00000000000000000000000000000000000000000000000000000013b9aca00")) + .maxFeePerGas( + Optional.of( + Wei.fromHexString( + "0x00000000000000000000000000000000000000000000000000000013b9aca00"))) + .maxPriorityFeePerGas(Optional.of(Wei.of(100_000))) + .nonce(transactionNumber) + .createTransaction(KEYS1), + true, + true); } private static BlockHeader mockBlockHeader() { diff --git a/datatypes/src/main/java/org/hyperledger/besu/datatypes/KZGCommitment.java b/datatypes/src/main/java/org/hyperledger/besu/datatypes/KZGCommitment.java index f64ebf5aead..de2195887bb 100644 --- a/datatypes/src/main/java/org/hyperledger/besu/datatypes/KZGCommitment.java +++ b/datatypes/src/main/java/org/hyperledger/besu/datatypes/KZGCommitment.java @@ -17,18 +17,18 @@ import org.hyperledger.besu.ethereum.rlp.RLPInput; import org.hyperledger.besu.ethereum.rlp.RLPOutput; -import org.apache.tuweni.bytes.Bytes; +import org.apache.tuweni.bytes.Bytes48; /** This class contains the data for a KZG commitment. */ public class KZGCommitment { - final Bytes data; + final Bytes48 data; /** * Constructor for a KZG commitment. * * @param data The data for the KZG commitment. */ - public KZGCommitment(final Bytes data) { + public KZGCommitment(final Bytes48 data) { this.data = data; } @@ -39,7 +39,7 @@ public KZGCommitment(final Bytes data) { * @return The KZG commitment. */ public static KZGCommitment readFrom(final RLPInput input) { - final Bytes bytes = input.readBytes(); + final Bytes48 bytes = input.readBytes48(); return new KZGCommitment(bytes); } @@ -57,7 +57,7 @@ public void writeTo(final RLPOutput out) { * * @return The data for the KZG commitment. */ - public Bytes getData() { + public Bytes48 getData() { return data; } } diff --git a/datatypes/src/main/java/org/hyperledger/besu/datatypes/KZGProof.java b/datatypes/src/main/java/org/hyperledger/besu/datatypes/KZGProof.java index 3a7a2d0686a..6410609e238 100644 --- a/datatypes/src/main/java/org/hyperledger/besu/datatypes/KZGProof.java +++ b/datatypes/src/main/java/org/hyperledger/besu/datatypes/KZGProof.java @@ -17,18 +17,18 @@ import org.hyperledger.besu.ethereum.rlp.RLPInput; import org.hyperledger.besu.ethereum.rlp.RLPOutput; -import org.apache.tuweni.bytes.Bytes; +import org.apache.tuweni.bytes.Bytes48; /** This class contains the data for a KZG proof for a KZG commitment. */ public class KZGProof { - final Bytes data; + final Bytes48 data; /** * Constructor for a KZG proof. * * @param data The data for the KZG proof. */ - public KZGProof(final Bytes data) { + public KZGProof(final Bytes48 data) { this.data = data; } @@ -39,7 +39,7 @@ public KZGProof(final Bytes data) { * @return The KZG proof. */ public static KZGProof readFrom(final RLPInput input) { - final Bytes bytes = input.readBytes(); + final Bytes48 bytes = input.readBytes48(); return new KZGProof(bytes); } @@ -57,7 +57,7 @@ public void writeTo(final RLPOutput out) { * * @return The data for the KZG proof. */ - public Bytes getData() { + public Bytes48 getData() { return data; } } diff --git a/datatypes/src/main/java/org/hyperledger/besu/datatypes/PendingTransaction.java b/datatypes/src/main/java/org/hyperledger/besu/datatypes/PendingTransaction.java index 4a8f5795ff8..e19fb13f1d1 100644 --- a/datatypes/src/main/java/org/hyperledger/besu/datatypes/PendingTransaction.java +++ b/datatypes/src/main/java/org/hyperledger/besu/datatypes/PendingTransaction.java @@ -30,6 +30,13 @@ public interface PendingTransaction { */ boolean isReceivedFromLocalSource(); + /** + * Should this transaction be prioritized? + * + * @return true if it is a transaction with priority + */ + boolean hasPriority(); + /** * Timestamp in millisecond when this transaction has been added to the pool * diff --git a/datatypes/src/test/java/org/hyperledger/besu/datatypes/BlobsWithCommitmentsTest.java b/datatypes/src/test/java/org/hyperledger/besu/datatypes/BlobsWithCommitmentsTest.java index 9816288e82a..a3eb7bbe0f6 100644 --- a/datatypes/src/test/java/org/hyperledger/besu/datatypes/BlobsWithCommitmentsTest.java +++ b/datatypes/src/test/java/org/hyperledger/besu/datatypes/BlobsWithCommitmentsTest.java @@ -23,6 +23,7 @@ import org.apache.tuweni.bytes.Bytes; import org.apache.tuweni.bytes.Bytes32; +import org.apache.tuweni.bytes.Bytes48; import org.junit.jupiter.api.Test; public class BlobsWithCommitmentsTest { @@ -45,9 +46,9 @@ public void blobsWithCommitmentsMustHaveSameNumberOfElementsVersionedHashes() { InvalidParameterException.class, () -> new BlobsWithCommitments( - List.of(new KZGCommitment(Bytes.of(1))), + List.of(new KZGCommitment(Bytes48.fromHexStringLenient("1"))), List.of(new Blob(Bytes.EMPTY)), - List.of(new KZGProof(Bytes.EMPTY)), + List.of(new KZGProof(Bytes48.ZERO)), List.of())) .getMessage(); final String expectedMessage = @@ -64,7 +65,7 @@ public void blobsWithCommitmentsMustHaveSameNumberOfElementsKZGCommitment() { new BlobsWithCommitments( List.of(), List.of(new Blob(Bytes.EMPTY)), - List.of(new KZGProof(Bytes.EMPTY)), + List.of(new KZGProof(Bytes48.ZERO)), List.of(new VersionedHash(Bytes32.rightPad(Bytes.fromHexString("0x01")))))) .getMessage(); final String expectedMessage = @@ -79,7 +80,7 @@ public void blobsWithCommitmentsMustHaveSameNumberOfElementsKZGProof() { InvalidParameterException.class, () -> new BlobsWithCommitments( - List.of(new KZGCommitment(Bytes.of(1))), + List.of(new KZGCommitment(Bytes48.fromHexStringLenient("1"))), List.of(new Blob(Bytes.EMPTY)), List.of(), List.of(new VersionedHash(Bytes32.rightPad(Bytes.fromHexString("0x01")))))) diff --git a/ethereum/api/build.gradle b/ethereum/api/build.gradle index d6c230ae018..ed395df8c5d 100644 --- a/ethereum/api/build.gradle +++ b/ethereum/api/build.gradle @@ -78,6 +78,7 @@ dependencies { implementation 'io.tmio:tuweni-bytes' implementation 'io.tmio:tuweni-units' implementation 'org.web3j:abi' + implementation 'com.github.ben-manes.caffeine:caffeine' annotationProcessor "org.immutables:value" implementation "org.immutables:value-annotations" diff --git a/ethereum/api/src/integration-test/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/fork/frontier/EthGetFilterChangesIntegrationTest.java b/ethereum/api/src/integration-test/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/fork/frontier/EthGetFilterChangesIntegrationTest.java index bdb94af6e3d..555146ca7cb 100644 --- a/ethereum/api/src/integration-test/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/fork/frontier/EthGetFilterChangesIntegrationTest.java +++ b/ethereum/api/src/integration-test/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/fork/frontier/EthGetFilterChangesIntegrationTest.java @@ -50,6 +50,7 @@ import org.hyperledger.besu.ethereum.eth.manager.EthContext; import org.hyperledger.besu.ethereum.eth.manager.EthPeers; import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration; +import org.hyperledger.besu.ethereum.eth.transactions.PendingTransaction; import org.hyperledger.besu.ethereum.eth.transactions.PendingTransactions; import org.hyperledger.besu.ethereum.eth.transactions.TransactionBroadcaster; import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool; @@ -90,7 +91,8 @@ public class EthGetFilterChangesIntegrationTest { private static final int MAX_TRANSACTIONS = 5; private static final KeyPair keyPair = SignatureAlgorithmFactory.getInstance().generateKeyPair(); - private final Transaction transaction = createTransaction(1); + private final PendingTransaction pendingTransaction = + new PendingTransaction.Local((createTransaction(1))); private FilterManager filterManager; private EthGetFilterChanges method; @@ -193,7 +195,7 @@ public void shouldReturnHashesIfNewBlocks() { JsonRpcResponse actual = method.response(request); assertThat(actual).usingRecursiveComparison().isEqualTo(expected); - final Block block = appendBlock(transaction); + final Block block = appendBlock(pendingTransaction.getTransaction()); // We've added one block, so there should be one new hash. expected = new JsonRpcSuccessResponse(null, Lists.newArrayList(block.getHash().toString())); @@ -223,11 +225,12 @@ public void shouldReturnHashesIfNewPendingTransactions() { JsonRpcResponse actual = method.response(request); assertThat(actual).usingRecursiveComparison().isEqualTo(expected); - transactions.addRemoteTransaction(transaction, Optional.empty()); + transactions.addTransaction(pendingTransaction, Optional.empty()); // We've added one transaction, so there should be one new hash. expected = - new JsonRpcSuccessResponse(null, Lists.newArrayList(String.valueOf(transaction.getHash()))); + new JsonRpcSuccessResponse( + null, Lists.newArrayList(String.valueOf(pendingTransaction.getHash()))); actual = method.response(request); assertThat(actual).usingRecursiveComparison().isEqualTo(expected); diff --git a/ethereum/api/src/integration-test/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/fork/london/EthGetFilterChangesIntegrationTest.java b/ethereum/api/src/integration-test/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/fork/london/EthGetFilterChangesIntegrationTest.java index 3ea2ee83633..873956d34b1 100644 --- a/ethereum/api/src/integration-test/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/fork/london/EthGetFilterChangesIntegrationTest.java +++ b/ethereum/api/src/integration-test/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/fork/london/EthGetFilterChangesIntegrationTest.java @@ -50,6 +50,7 @@ import org.hyperledger.besu.ethereum.eth.manager.EthContext; import org.hyperledger.besu.ethereum.eth.manager.EthPeers; import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration; +import org.hyperledger.besu.ethereum.eth.transactions.PendingTransaction; import org.hyperledger.besu.ethereum.eth.transactions.PendingTransactions; import org.hyperledger.besu.ethereum.eth.transactions.TransactionBroadcaster; import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool; @@ -90,7 +91,8 @@ public class EthGetFilterChangesIntegrationTest { private static final int MAX_TRANSACTIONS = 5; private static final KeyPair keyPair = SignatureAlgorithmFactory.getInstance().generateKeyPair(); - private final Transaction transaction = createTransaction(1); + private final PendingTransaction pendingTransaction = + new PendingTransaction.Local(createTransaction(1)); private FilterManager filterManager; private EthGetFilterChanges method; @@ -193,7 +195,7 @@ public void shouldReturnHashesIfNewBlocks() { JsonRpcResponse actual = method.response(request); assertThat(actual).usingRecursiveComparison().isEqualTo(expected); - final Block block = appendBlock(transaction); + final Block block = appendBlock(pendingTransaction.getTransaction()); // We've added one block, so there should be one new hash. expected = new JsonRpcSuccessResponse(null, Lists.newArrayList(block.getHash().toString())); @@ -223,11 +225,12 @@ public void shouldReturnHashesIfNewPendingTransactions() { JsonRpcResponse actual = method.response(request); assertThat(actual).usingRecursiveComparison().isEqualTo(expected); - transactions.addRemoteTransaction(transaction, Optional.empty()); + transactions.addTransaction(pendingTransaction, Optional.empty()); // We've added one transaction, so there should be one new hash. expected = - new JsonRpcSuccessResponse(null, Lists.newArrayList(String.valueOf(transaction.getHash()))); + new JsonRpcSuccessResponse( + null, Lists.newArrayList(String.valueOf(pendingTransaction.getHash()))); actual = method.response(request); assertThat(actual).usingRecursiveComparison().isEqualTo(expected); diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcErrorConverter.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcErrorConverter.java index b27c84bf04e..48ea3f9c069 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcErrorConverter.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/JsonRpcErrorConverter.java @@ -77,6 +77,8 @@ public static RpcErrorType convertTransactionInvalidReason( return RpcErrorType.TOTAL_BLOB_GAS_TOO_HIGH; case TX_POOL_DISABLED: return RpcErrorType.TX_POOL_DISABLED; + case PLUGIN_TX_VALIDATOR: + return RpcErrorType.PLUGIN_TX_VALIDATOR; default: return RpcErrorType.INTERNAL_ERROR; } diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/RpcMethod.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/RpcMethod.java index 979ff433688..fe7e432081b 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/RpcMethod.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/RpcMethod.java @@ -53,9 +53,11 @@ public enum RpcMethod { ENGINE_GET_PAYLOAD_V1("engine_getPayloadV1"), ENGINE_GET_PAYLOAD_V2("engine_getPayloadV2"), ENGINE_GET_PAYLOAD_V3("engine_getPayloadV3"), + ENGINE_GET_PAYLOAD_V6110("engine_getPayloadV6110"), ENGINE_NEW_PAYLOAD_V1("engine_newPayloadV1"), ENGINE_NEW_PAYLOAD_V2("engine_newPayloadV2"), ENGINE_NEW_PAYLOAD_V3("engine_newPayloadV3"), + ENGINE_NEW_PAYLOAD_V6110("engine_newPayloadV6110"), ENGINE_FORKCHOICE_UPDATED_V1("engine_forkchoiceUpdatedV1"), ENGINE_FORKCHOICE_UPDATED_V2("engine_forkchoiceUpdatedV2"), ENGINE_FORKCHOICE_UPDATED_V3("engine_forkchoiceUpdatedV3"), diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthFeeHistory.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthFeeHistory.java index d505568bd4e..ffe3b96b05a 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthFeeHistory.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthFeeHistory.java @@ -16,6 +16,7 @@ import static java.util.stream.Collectors.toUnmodifiableList; +import org.hyperledger.besu.datatypes.Hash; import org.hyperledger.besu.datatypes.Wei; import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext; @@ -36,25 +37,30 @@ import org.hyperledger.besu.ethereum.mainnet.feemarket.BaseFeeMarket; import org.hyperledger.besu.ethereum.mainnet.feemarket.FeeMarket; -import java.util.AbstractMap; import java.util.ArrayList; import java.util.Arrays; import java.util.Comparator; import java.util.List; -import java.util.Map; import java.util.Optional; import java.util.stream.LongStream; import java.util.stream.Stream; +import com.github.benmanes.caffeine.cache.Cache; +import com.github.benmanes.caffeine.cache.Caffeine; import com.google.common.collect.Streams; public class EthFeeHistory implements JsonRpcMethod { private final ProtocolSchedule protocolSchedule; private final Blockchain blockchain; + private final Cache> cache; + private static final int MAXIMUM_CACHE_SIZE = 100_000; + + record RewardCacheKey(Hash blockHash, List rewardPercentiles) {} public EthFeeHistory(final ProtocolSchedule protocolSchedule, final Blockchain blockchain) { this.protocolSchedule = protocolSchedule; this.blockchain = blockchain; + this.cache = Caffeine.newBuilder().maximumSize(MAXIMUM_CACHE_SIZE).build(); } @Override @@ -96,6 +102,7 @@ public JsonRpcResponse response(final JsonRpcRequestContext request) { final List blockHeaders = LongStream.range(oldestBlock, lastBlock) + .parallel() .mapToObj(blockchain::getBlockHeader) .flatMap(Optional::stream) .collect(toUnmodifiableList()); @@ -142,16 +149,30 @@ public JsonRpcResponse response(final JsonRpcRequestContext request) { final Optional>> maybeRewards = maybeRewardPercentiles.map( - rewardPercentiles -> - LongStream.range(oldestBlock, lastBlock) - .mapToObj(blockchain::getBlockByNumber) - .flatMap(Optional::stream) - .map( - block -> - computeRewards( - rewardPercentiles.stream().sorted().collect(toUnmodifiableList()), - block)) - .collect(toUnmodifiableList())); + rewardPercentiles -> { + var sortedPercentiles = rewardPercentiles.stream().sorted().toList(); + return blockHeaders.stream() + .parallel() + .map( + blockHeader -> { + final RewardCacheKey key = + new RewardCacheKey(blockHeader.getBlockHash(), rewardPercentiles); + return Optional.ofNullable(cache.getIfPresent(key)) + .or( + () -> { + Optional block = + blockchain.getBlockByHash(blockHeader.getBlockHash()); + return block.map( + b -> { + List rewards = computeRewards(sortedPercentiles, b); + cache.put(key, rewards); + return rewards; + }); + }); + }) + .flatMap(Optional::stream) + .toList(); + }); return new JsonRpcSuccessResponse( requestId, @@ -188,13 +209,21 @@ private List computeRewards(final List rewardPercentiles, final Blo : transactionReceipt.getCumulativeGasUsed() - transactionsGasUsed.get(transactionsGasUsed.size() - 1)); } - final List> transactionsAndGasUsedAscendingEffectiveGasFee = + + record TransactionInfo(Transaction transaction, Long gasUsed, Wei effectivePriorityFeePerGas) {} + + final List transactionsInfo = Streams.zip( - transactions.stream(), transactionsGasUsed.stream(), AbstractMap.SimpleEntry::new) - .sorted( - Comparator.comparing( - transactionAndGasUsed -> - transactionAndGasUsed.getKey().getEffectivePriorityFeePerGas(baseFee))) + transactions.stream(), + transactionsGasUsed.stream(), + (transaction, gasUsed) -> + new TransactionInfo( + transaction, gasUsed, transaction.getEffectivePriorityFeePerGas(baseFee))) + .collect(toUnmodifiableList()); + + final List transactionsAndGasUsedAscendingEffectiveGasFee = + transactionsInfo.stream() + .sorted(Comparator.comparing(TransactionInfo::effectivePriorityFeePerGas)) .collect(toUnmodifiableList()); // We need to weight the percentile of rewards by the gas used in the transaction. @@ -203,18 +232,21 @@ private List computeRewards(final List rewardPercentiles, final Blo final ArrayList rewards = new ArrayList<>(); int rewardPercentileIndex = 0; long gasUsed = 0; - for (final Map.Entry transactionAndGasUsed : + for (final TransactionInfo transactionAndGasUsed : transactionsAndGasUsedAscendingEffectiveGasFee) { - gasUsed += transactionAndGasUsed.getValue(); + gasUsed += transactionAndGasUsed.gasUsed(); while (rewardPercentileIndex < rewardPercentiles.size() && 100.0 * gasUsed / block.getHeader().getGasUsed() >= rewardPercentiles.get(rewardPercentileIndex)) { - rewards.add(transactionAndGasUsed.getKey().getEffectivePriorityFeePerGas(baseFee)); + rewards.add(transactionAndGasUsed.effectivePriorityFeePerGas); rewardPercentileIndex++; } } + // Put the computed rewards in the cache + cache.put(new RewardCacheKey(block.getHeader().getBlockHash(), rewardPercentiles), rewards); + return rewards; } } diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthSendRawTransaction.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthSendRawTransaction.java index 82cf4410a51..6f80a372ffe 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthSendRawTransaction.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthSendRawTransaction.java @@ -19,6 +19,7 @@ import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.exception.InvalidJsonRpcRequestException; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcError; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcResponse; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcSuccessResponse; @@ -33,6 +34,7 @@ import java.util.function.Supplier; import com.google.common.base.Suppliers; +import org.jetbrains.annotations.NotNull; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -90,12 +92,29 @@ public JsonRpcResponse response(final JsonRpcRequestContext requestContext) { () -> new JsonRpcSuccessResponse( requestContext.getRequest().getId(), transaction.getHash().toString()), - errorReason -> - sendEmptyHashOnInvalidBlock - ? new JsonRpcSuccessResponse( - requestContext.getRequest().getId(), Hash.EMPTY.toString()) - : new JsonRpcErrorResponse( - requestContext.getRequest().getId(), - JsonRpcErrorConverter.convertTransactionInvalidReason(errorReason))); + errorReason -> getJsonRpcResponse(requestContext, errorReason, validationResult)); + } + + @NotNull + private JsonRpcResponse getJsonRpcResponse( + final JsonRpcRequestContext requestContext, + final TransactionInvalidReason errorReason, + final ValidationResult validationResult) { + if (sendEmptyHashOnInvalidBlock) { + return new JsonRpcSuccessResponse(requestContext.getRequest().getId(), Hash.EMPTY.toString()); + } else { + if (errorReason == TransactionInvalidReason.PLUGIN_TX_VALIDATOR) { + final RpcErrorType rpcErrorType = + JsonRpcErrorConverter.convertTransactionInvalidReason( + validationResult.getInvalidReason()); + return new JsonRpcErrorResponse( + requestContext.getRequest().getId(), + new JsonRpcError(rpcErrorType.getCode(), validationResult.getErrorMessage(), null)); + } else { + return new JsonRpcErrorResponse( + requestContext.getRequest().getId(), + JsonRpcErrorConverter.convertTransactionInvalidReason(errorReason)); + } + } } } diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/TraceCallMany.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/TraceCallMany.java index e3dfb245bb8..4ee9ff04e33 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/TraceCallMany.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/TraceCallMany.java @@ -64,7 +64,7 @@ public String getName() { @Override protected BlockParameter blockParameter(final JsonRpcRequestContext request) { final Optional maybeBlockParameter = - request.getOptionalParameter(2, BlockParameter.class); + request.getOptionalParameter(1, BlockParameter.class); if (maybeBlockParameter.isPresent()) { return maybeBlockParameter.get(); diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractEngineForkchoiceUpdated.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractEngineForkchoiceUpdated.java index 7c7cfca2c44..b5c38c1e682 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractEngineForkchoiceUpdated.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractEngineForkchoiceUpdated.java @@ -113,17 +113,21 @@ public JsonRpcResponse syncResponse(final JsonRpcRequestContext requestContext) if (maybeNewHead.isEmpty()) { return syncingResponse(requestId, forkChoice); } - Optional> withdrawals = Optional.empty(); - final BlockHeader newHead = maybeNewHead.get(); + + ForkchoiceResult forkchoiceResult = null; if (!isValidForkchoiceState( - forkChoice.getSafeBlockHash(), forkChoice.getFinalizedBlockHash(), newHead)) { + forkChoice.getSafeBlockHash(), forkChoice.getFinalizedBlockHash(), maybeNewHead.get())) { logForkchoiceUpdatedCall(INVALID, forkChoice); return new JsonRpcErrorResponse(requestId, RpcErrorType.INVALID_FORKCHOICE_STATE); + } else { + forkchoiceResult = + mergeCoordinator.updateForkChoice( + maybeNewHead.get(), + forkChoice.getFinalizedBlockHash(), + forkChoice.getSafeBlockHash()); } - ForkchoiceResult result = - mergeCoordinator.updateForkChoice( - newHead, forkChoice.getFinalizedBlockHash(), forkChoice.getSafeBlockHash()); + Optional> withdrawals = Optional.empty(); if (maybePayloadAttributes.isPresent()) { final EnginePayloadAttributesParameter payloadAttributes = maybePayloadAttributes.get(); withdrawals = @@ -136,7 +140,7 @@ public JsonRpcResponse syncResponse(final JsonRpcRequestContext requestContext) .map(WithdrawalParameter::toWithdrawal) .collect(toList()))); Optional maybeError = - isPayloadAttributesValid(requestId, payloadAttributes, withdrawals, newHead); + isPayloadAttributesValid(requestId, payloadAttributes); if (maybeError.isPresent()) { LOG.atWarn() .setMessage("RpcError {}: {}") @@ -156,6 +160,20 @@ public JsonRpcResponse syncResponse(final JsonRpcRequestContext requestContext) } } + final BlockHeader newHead = maybeNewHead.get(); + if (maybePayloadAttributes.isPresent()) { + Optional maybeError = + isPayloadAttributeRelevantToNewHead(requestId, maybePayloadAttributes.get(), newHead); + if (maybeError.isPresent()) { + return maybeError.get(); + } + if (!getWithdrawalsValidator( + protocolSchedule.get(), newHead, maybePayloadAttributes.get().getTimestamp()) + .validateWithdrawals(withdrawals)) { + return new JsonRpcErrorResponse(requestId, getInvalidPayloadError()); + } + } + ValidationResult parameterValidationResult = validateParameter(forkChoice, maybePayloadAttributes); if (!parameterValidationResult.isValid()) { @@ -169,13 +187,13 @@ public JsonRpcResponse syncResponse(final JsonRpcRequestContext requestContext) maybePayloadAttributes.ifPresentOrElse( this::logPayload, () -> LOG.debug("Payload attributes are null")); - if (result.shouldNotProceedToPayloadBuildProcess()) { - if (ForkchoiceResult.Status.IGNORE_UPDATE_TO_OLD_HEAD.equals(result.getStatus())) { + if (forkchoiceResult.shouldNotProceedToPayloadBuildProcess()) { + if (ForkchoiceResult.Status.IGNORE_UPDATE_TO_OLD_HEAD.equals(forkchoiceResult.getStatus())) { logForkchoiceUpdatedCall(VALID, forkChoice); } else { logForkchoiceUpdatedCall(INVALID, forkChoice); } - return handleNonValidForkchoiceUpdate(requestId, result); + return handleNonValidForkchoiceUpdate(requestId, forkchoiceResult); } // begin preparing a block if we have a non-empty payload attributes param @@ -205,15 +223,17 @@ public JsonRpcResponse syncResponse(final JsonRpcRequestContext requestContext) requestId, new EngineUpdateForkchoiceResult( VALID, - result.getNewHead().map(BlockHeader::getHash).orElse(null), + forkchoiceResult.getNewHead().map(BlockHeader::getHash).orElse(null), payloadId.orElse(null), Optional.empty())); } - protected Optional isPayloadAttributesValid( + protected abstract Optional isPayloadAttributesValid( + final Object requestId, final EnginePayloadAttributesParameter payloadAttribute); + + protected Optional isPayloadAttributeRelevantToNewHead( final Object requestId, final EnginePayloadAttributesParameter payloadAttributes, - final Optional> maybeWithdrawals, final BlockHeader headBlockHeader) { if (payloadAttributes.getTimestamp() <= headBlockHeader.getTimestamp()) { @@ -221,11 +241,6 @@ protected Optional isPayloadAttributesValid( "Payload attributes timestamp is smaller than timestamp of header in fork choice update"); return Optional.of(new JsonRpcErrorResponse(requestId, getInvalidPayloadError())); } - if (!getWithdrawalsValidator( - protocolSchedule.get(), headBlockHeader, payloadAttributes.getTimestamp()) - .validateWithdrawals(maybeWithdrawals)) { - return Optional.of(new JsonRpcErrorResponse(requestId, getInvalidPayloadError())); - } return Optional.empty(); } diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractEngineNewPayload.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractEngineNewPayload.java index c8263450e72..a5b8acfc056 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractEngineNewPayload.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractEngineNewPayload.java @@ -267,7 +267,8 @@ public JsonRpcResponse syncResponse(final JsonRpcRequestContext requestContext) } if (maybeParentHeader.isPresent() - && (blockParam.getTimestamp() <= maybeParentHeader.get().getTimestamp())) { + && (Long.compareUnsigned(maybeParentHeader.get().getTimestamp(), blockParam.getTimestamp()) + >= 0)) { return respondWithInvalid( reqId, blockParam, diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineExchangeCapabilities.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineExchangeCapabilities.java index f59681a3854..fcc26cad641 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineExchangeCapabilities.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineExchangeCapabilities.java @@ -63,6 +63,7 @@ public JsonRpcResponse syncResponse(final JsonRpcRequestContext requestContext) .filter(e -> e.getMethodName().startsWith("engine_")) .filter(e -> !e.equals(ENGINE_EXCHANGE_CAPABILITIES)) .filter(e -> !e.equals(ENGINE_PREPARE_PAYLOAD_DEBUG)) + .filter(e -> !e.getMethodName().endsWith("6110")) .map(RpcMethod::getMethodName) .collect(Collectors.toList()); diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV1.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV1.java index a5b9eeb9ed2..6aa5f0964b2 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV1.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV1.java @@ -17,9 +17,13 @@ import org.hyperledger.besu.consensus.merge.blockcreation.MergeMiningCoordinator; import org.hyperledger.besu.ethereum.ProtocolContext; import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.EnginePayloadAttributesParameter; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType; import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule; +import java.util.Optional; + import io.vertx.core.Vertx; public class EngineForkchoiceUpdatedV1 extends AbstractEngineForkchoiceUpdated { @@ -33,6 +37,12 @@ public EngineForkchoiceUpdatedV1( super(vertx, protocolSchedule, protocolContext, mergeCoordinator, engineCallListener); } + @Override + protected Optional isPayloadAttributesValid( + final Object requestId, final EnginePayloadAttributesParameter payloadAttributes) { + return Optional.empty(); + } + @Override public String getName() { return RpcMethod.ENGINE_FORKCHOICE_UPDATED_V1.getMethodName(); diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV2.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV2.java index 44ba15cae8d..b6406a3e662 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV2.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV2.java @@ -20,11 +20,8 @@ import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.EnginePayloadAttributesParameter; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType; -import org.hyperledger.besu.ethereum.core.BlockHeader; -import org.hyperledger.besu.ethereum.core.Withdrawal; import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule; -import java.util.List; import java.util.Optional; import io.vertx.core.Vertx; @@ -53,19 +50,20 @@ public String getName() { @Override protected Optional isPayloadAttributesValid( - final Object requestId, - final EnginePayloadAttributesParameter payloadAttributes, - final Optional> maybeWithdrawals, - final BlockHeader headBlockHeader) { + final Object requestId, final EnginePayloadAttributesParameter payloadAttributes) { if (payloadAttributes.getTimestamp() >= cancunTimestamp) { - return Optional.of(new JsonRpcErrorResponse(requestId, RpcErrorType.UNSUPPORTED_FORK)); + if (payloadAttributes.getParentBeaconBlockRoot() == null + || payloadAttributes.getParentBeaconBlockRoot().isEmpty()) { + return Optional.of(new JsonRpcErrorResponse(requestId, RpcErrorType.UNSUPPORTED_FORK)); + } else { + return Optional.of(new JsonRpcErrorResponse(requestId, RpcErrorType.INVALID_PARAMS)); + } } else if (payloadAttributes.getParentBeaconBlockRoot() != null) { LOG.error( "Parent beacon block root hash present in payload attributes before cancun hardfork"); return Optional.of(new JsonRpcErrorResponse(requestId, RpcErrorType.INVALID_PARAMS)); } else { - return super.isPayloadAttributesValid( - requestId, payloadAttributes, maybeWithdrawals, headBlockHeader); + return Optional.empty(); } } } diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV3.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV3.java index 4284fdd01dc..c070d220854 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV3.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineForkchoiceUpdatedV3.java @@ -21,13 +21,10 @@ import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.EnginePayloadAttributesParameter; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType; -import org.hyperledger.besu.ethereum.core.BlockHeader; -import org.hyperledger.besu.ethereum.core.Withdrawal; import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule; import org.hyperledger.besu.ethereum.mainnet.ScheduledProtocolSpec; import org.hyperledger.besu.ethereum.mainnet.ValidationResult; -import java.util.List; import java.util.Optional; import io.vertx.core.Vertx; @@ -92,16 +89,8 @@ protected ValidationResult validateForkSupported(final long blockT @Override protected Optional isPayloadAttributesValid( - final Object requestId, - final EnginePayloadAttributesParameter payloadAttributes, - final Optional> maybeWithdrawals, - final BlockHeader headBlockHeader) { - Optional maybeError = - super.isPayloadAttributesValid( - requestId, payloadAttributes, maybeWithdrawals, headBlockHeader); - if (maybeError.isPresent()) { - return maybeError; - } else if (payloadAttributes.getParentBeaconBlockRoot() == null) { + final Object requestId, final EnginePayloadAttributesParameter payloadAttributes) { + if (payloadAttributes.getParentBeaconBlockRoot() == null) { LOG.error( "Parent beacon block root hash not present in payload attributes after cancun hardfork"); return Optional.of(new JsonRpcErrorResponse(requestId, RpcErrorType.INVALID_PARAMS)); diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineGetPayloadV6110.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineGetPayloadV6110.java new file mode 100644 index 00000000000..c9a9737a90a --- /dev/null +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineGetPayloadV6110.java @@ -0,0 +1,87 @@ +/* + * Copyright Hyperledger Besu Contributors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine; + +import org.hyperledger.besu.consensus.merge.blockcreation.MergeMiningCoordinator; +import org.hyperledger.besu.consensus.merge.blockcreation.PayloadIdentifier; +import org.hyperledger.besu.ethereum.ProtocolContext; +import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcResponse; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcSuccessResponse; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.BlockResultFactory; +import org.hyperledger.besu.ethereum.core.BlockWithReceipts; +import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule; +import org.hyperledger.besu.ethereum.mainnet.ScheduledProtocolSpec; +import org.hyperledger.besu.ethereum.mainnet.ValidationResult; + +import java.util.Optional; + +import io.vertx.core.Vertx; + +public class EngineGetPayloadV6110 extends AbstractEngineGetPayload { + + private final Optional eip6110; + + public EngineGetPayloadV6110( + final Vertx vertx, + final ProtocolContext protocolContext, + final MergeMiningCoordinator mergeMiningCoordinator, + final BlockResultFactory blockResultFactory, + final EngineCallListener engineCallListener, + final ProtocolSchedule schedule) { + super( + vertx, + schedule, + protocolContext, + mergeMiningCoordinator, + blockResultFactory, + engineCallListener); + this.eip6110 = schedule.hardforkFor(s -> s.fork().name().equalsIgnoreCase("ExperimentalEips")); + } + + @Override + public String getName() { + return RpcMethod.ENGINE_GET_PAYLOAD_V6110.getMethodName(); + } + + @Override + protected JsonRpcResponse createResponse( + final JsonRpcRequestContext request, + final PayloadIdentifier payloadId, + final BlockWithReceipts blockWithReceipts) { + + return new JsonRpcSuccessResponse( + request.getRequest().getId(), + blockResultFactory.payloadTransactionCompleteV6110(blockWithReceipts)); + } + + @Override + protected ValidationResult validateForkSupported(final long blockTimestamp) { + if (protocolSchedule.isPresent()) { + if (eip6110.isPresent() && blockTimestamp >= eip6110.get().milestone()) { + return ValidationResult.valid(); + } else { + return ValidationResult.invalid( + RpcErrorType.UNSUPPORTED_FORK, + "EIP-6110 configured to start at timestamp: " + eip6110.get().milestone()); + } + } else { + return ValidationResult.invalid( + RpcErrorType.UNSUPPORTED_FORK, "Configuration error, no schedule for EIP-6110 fork set"); + } + } +} diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadV3.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadV3.java index c7d8a10a4a1..dff7174b49d 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadV3.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadV3.java @@ -71,7 +71,8 @@ protected ValidationResult validateParameters( @Override protected ValidationResult validateForkSupported(final long blockTimestamp) { if (protocolSchedule.isPresent()) { - if (cancun.isPresent() && blockTimestamp >= cancun.get().milestone()) { + if (cancun.isPresent() + && Long.compareUnsigned(blockTimestamp, cancun.get().milestone()) >= 0) { return ValidationResult.valid(); } else { return ValidationResult.invalid( diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadV6110.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadV6110.java new file mode 100644 index 00000000000..84f203c3a73 --- /dev/null +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadV6110.java @@ -0,0 +1,89 @@ +/* + * Copyright Hyperledger Besu Contributors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine; + +import org.hyperledger.besu.consensus.merge.blockcreation.MergeMiningCoordinator; +import org.hyperledger.besu.ethereum.ProtocolContext; +import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.EnginePayloadParameter; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType; +import org.hyperledger.besu.ethereum.eth.manager.EthPeers; +import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule; +import org.hyperledger.besu.ethereum.mainnet.ScheduledProtocolSpec; +import org.hyperledger.besu.ethereum.mainnet.ValidationResult; + +import java.util.List; +import java.util.Optional; + +import io.vertx.core.Vertx; + +public class EngineNewPayloadV6110 extends AbstractEngineNewPayload { + + private final Optional eip6110; + + public EngineNewPayloadV6110( + final Vertx vertx, + final ProtocolSchedule timestampSchedule, + final ProtocolContext protocolContext, + final MergeMiningCoordinator mergeCoordinator, + final EthPeers ethPeers, + final EngineCallListener engineCallListener) { + super( + vertx, timestampSchedule, protocolContext, mergeCoordinator, ethPeers, engineCallListener); + this.eip6110 = + timestampSchedule.hardforkFor(s -> s.fork().name().equalsIgnoreCase("ExperimentalEips")); + } + + @Override + public String getName() { + return RpcMethod.ENGINE_NEW_PAYLOAD_V6110.getMethodName(); + } + + @Override + protected ValidationResult validateParameters( + final EnginePayloadParameter payloadParameter, + final Optional> maybeVersionedHashParam, + final Optional maybeBeaconBlockRootParam) { + if (payloadParameter.getBlobGasUsed() == null || payloadParameter.getExcessBlobGas() == null) { + return ValidationResult.invalid(RpcErrorType.INVALID_PARAMS, "Missing blob gas fields"); + } else if (maybeVersionedHashParam == null || maybeVersionedHashParam.isEmpty()) { + return ValidationResult.invalid( + RpcErrorType.INVALID_PARAMS, "Missing versioned hashes field"); + } else if (maybeBeaconBlockRootParam.isEmpty()) { + return ValidationResult.invalid( + RpcErrorType.INVALID_PARAMS, "Missing parent beacon block root field"); + } else if (payloadParameter.getDeposits() == null) { + return ValidationResult.invalid(RpcErrorType.INVALID_PARAMS, "Missing deposit field"); + } else { + return ValidationResult.valid(); + } + } + + @Override + protected ValidationResult validateForkSupported(final long blockTimestamp) { + if (protocolSchedule.isPresent()) { + if (eip6110.isPresent() && blockTimestamp >= eip6110.get().milestone()) { + return ValidationResult.valid(); + } else { + return ValidationResult.invalid( + RpcErrorType.UNSUPPORTED_FORK, + "EIP-6110 configured to start at timestamp: " + eip6110.get().milestone()); + } + } else { + return ValidationResult.invalid( + RpcErrorType.UNSUPPORTED_FORK, "Configuration error, no schedule for EIP-6110 fork set"); + } + } +} diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/DepositParameter.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/DepositParameter.java index c13bc0435fc..288c998977d 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/DepositParameter.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/DepositParameter.java @@ -30,7 +30,7 @@ public class DepositParameter { - private final String publicKey; + private final String pubkey; private final String withdrawalCredentials; private final String amount; @@ -40,12 +40,12 @@ public class DepositParameter { @JsonCreator public DepositParameter( - @JsonProperty("pubkey") final String pubKey, + @JsonProperty("pubkey") final String pubkey, @JsonProperty("withdrawalCredentials") final String withdrawalCredentials, @JsonProperty("amount") final String amount, @JsonProperty("signature") final String signature, @JsonProperty("index") final String index) { - this.publicKey = pubKey; + this.pubkey = pubkey; this.withdrawalCredentials = withdrawalCredentials; this.amount = amount; this.signature = signature; @@ -54,7 +54,7 @@ public DepositParameter( public static DepositParameter fromDeposit(final Deposit deposit) { return new DepositParameter( - deposit.getPublicKey().toString(), + deposit.getPubkey().toString(), deposit.getWithdrawalCredentials().toString(), deposit.getAmount().toShortHexString(), deposit.getSignature().toString(), @@ -63,7 +63,7 @@ public static DepositParameter fromDeposit(final Deposit deposit) { public Deposit toDeposit() { return new Deposit( - BLSPublicKey.fromHexString(publicKey), + BLSPublicKey.fromHexString(pubkey), Bytes32.fromHexString(withdrawalCredentials), GWei.fromHexString(amount), BLSSignature.fromHexString(signature), @@ -72,7 +72,7 @@ public Deposit toDeposit() { public JsonObject asJsonObject() { return new JsonObject() - .put("pubKey", publicKey) + .put("pubkey", pubkey) .put("withdrawalCredentials", withdrawalCredentials) .put("amount", amount) .put("signature", signature) @@ -80,8 +80,8 @@ public JsonObject asJsonObject() { } @JsonGetter - public String getPublicKey() { - return publicKey; + public String getPubkey() { + return pubkey; } @JsonGetter @@ -109,7 +109,7 @@ public boolean equals(final Object o) { if (this == o) return true; if (o == null || getClass() != o.getClass()) return false; final DepositParameter that = (DepositParameter) o; - return Objects.equals(publicKey, that.publicKey) + return Objects.equals(pubkey, that.pubkey) && Objects.equals(withdrawalCredentials, that.withdrawalCredentials) && Objects.equals(amount, that.amount) && Objects.equals(signature, that.signature) @@ -118,14 +118,14 @@ public boolean equals(final Object o) { @Override public int hashCode() { - return Objects.hash(publicKey, withdrawalCredentials, amount, signature, index); + return Objects.hash(pubkey, withdrawalCredentials, amount, signature, index); } @Override public String toString() { return "DepositParameter{" + "pubKey='" - + publicKey + + pubkey + '\'' + ", withdrawalCredentials='" + withdrawalCredentials diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/EnginePayloadParameter.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/EnginePayloadParameter.java index 1834b99a5db..355f7b218c7 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/EnginePayloadParameter.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/EnginePayloadParameter.java @@ -86,7 +86,7 @@ public EnginePayloadParameter( @JsonProperty("withdrawals") final List withdrawals, @JsonProperty("blobGasUsed") final UnsignedLongParameter blobGasUsed, @JsonProperty("excessBlobGas") final String excessBlobGas, - @JsonProperty("deposits") final List deposits) { + @JsonProperty("depositReceipts") final List deposits) { this.blockHash = blockHash; this.parentHash = parentHash; this.feeRecipient = feeRecipient; diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/UnsignedLongParameter.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/UnsignedLongParameter.java index 039592d28c2..2e9ae8087ec 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/UnsignedLongParameter.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/parameters/UnsignedLongParameter.java @@ -17,25 +17,28 @@ import static com.google.common.base.Preconditions.checkArgument; import com.fasterxml.jackson.annotation.JsonCreator; +import org.checkerframework.checker.signedness.qual.Unsigned; public class UnsignedLongParameter { - private final long value; + @Unsigned private final long value; @JsonCreator public UnsignedLongParameter(final String value) { checkArgument(value != null); - this.value = Long.decode(value); - checkArgument(this.value >= 0); + if (value.startsWith("0x")) { + this.value = Long.parseUnsignedLong(value.substring(2), 16); + } else { + this.value = Long.parseUnsignedLong(value, 16); + } } @JsonCreator - public UnsignedLongParameter(final long value) { + public UnsignedLongParameter(final @Unsigned long value) { this.value = value; - checkArgument(this.value >= 0); } - public long getValue() { + public @Unsigned long getValue() { return value; } } diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/response/JsonRpcErrorResponse.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/response/JsonRpcErrorResponse.java index ed8f275c07a..3dffff4956a 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/response/JsonRpcErrorResponse.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/response/JsonRpcErrorResponse.java @@ -107,7 +107,7 @@ private RpcErrorType findErrorType(final int code, final String message) { return Arrays.stream(RpcErrorType.values()) .filter(e -> e.getCode() == code && message.startsWith(e.getMessage())) .findFirst() - .get(); + .orElse(RpcErrorType.UNKNOWN); } @SuppressWarnings({"unchecked", "rawtypes"}) diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/response/RpcErrorType.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/response/RpcErrorType.java index c3ef46016d9..268af90a0cc 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/response/RpcErrorType.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/response/RpcErrorType.java @@ -74,6 +74,7 @@ public enum RpcErrorType { LOWER_NONCE_INVALID_TRANSACTION_EXISTS( -32000, "An invalid transaction with a lower nonce exists"), TOTAL_BLOB_GAS_TOO_HIGH(-32000, "Total blob gas too high"), + PLUGIN_TX_VALIDATOR(-32000, "Plugin has marked the transaction as invalid"), // Execution engine failures UNKNOWN_PAYLOAD(-32001, "Payload does not exist / is not available"), @@ -210,7 +211,9 @@ public enum RpcErrorType { // Retesteth Errors BLOCK_RLP_IMPORT_ERROR(-32000, "Could not decode RLP for Block"), - BLOCK_IMPORT_ERROR(-32000, "Could not import Block"); + BLOCK_IMPORT_ERROR(-32000, "Could not import Block"), + + UNKNOWN(-32603, "Unknown internal error"); private final int code; private final String message; diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/BlockResultFactory.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/BlockResultFactory.java index 9ec3acd7f76..9596151bdfe 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/BlockResultFactory.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/BlockResultFactory.java @@ -120,7 +120,6 @@ public EngineGetPayloadResultV2 payloadTransactionCompleteV2( blockWithReceipts.getHeader(), txs, blockWithReceipts.getBlock().getBody().getWithdrawals(), - blockWithReceipts.getBlock().getBody().getDeposits(), Quantity.create(blockValue)); } @@ -155,6 +154,29 @@ public EngineGetPayloadResultV3 payloadTransactionCompleteV3( blobsBundleV1); } + public EngineGetPayloadResultV6110 payloadTransactionCompleteV6110( + final BlockWithReceipts blockWithReceipts) { + final List txs = + blockWithReceipts.getBlock().getBody().getTransactions().stream() + .map( + transaction -> + TransactionEncoder.encodeOpaqueBytes(transaction, EncodingContext.BLOCK_BODY)) + .map(Bytes::toHexString) + .collect(Collectors.toList()); + + final Wei blockValue = new BlockValueCalculator().calculateBlockValue(blockWithReceipts); + + final BlobsBundleV1 blobsBundleV1 = + new BlobsBundleV1(blockWithReceipts.getBlock().getBody().getTransactions()); + return new EngineGetPayloadResultV6110( + blockWithReceipts.getHeader(), + txs, + blockWithReceipts.getBlock().getBody().getWithdrawals(), + blockWithReceipts.getBlock().getBody().getDeposits(), + Quantity.create(blockValue), + blobsBundleV1); + } + public BlockResult transactionHash(final BlockWithMetadata blockWithMetadata) { return transactionHash(blockWithMetadata, false); } diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/EngineGetPayloadResultV2.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/EngineGetPayloadResultV2.java index b5e9fafabe2..e3919517f3b 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/EngineGetPayloadResultV2.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/EngineGetPayloadResultV2.java @@ -14,10 +14,8 @@ */ package org.hyperledger.besu.ethereum.api.jsonrpc.internal.results; -import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.DepositParameter; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.WithdrawalParameter; import org.hyperledger.besu.ethereum.core.BlockHeader; -import org.hyperledger.besu.ethereum.core.Deposit; import org.hyperledger.besu.ethereum.core.Withdrawal; import java.util.List; @@ -41,9 +39,8 @@ public EngineGetPayloadResultV2( final BlockHeader header, final List transactions, final Optional> withdrawals, - final Optional> deposits, final String blockValue) { - this.executionPayload = new PayloadResult(header, transactions, withdrawals, deposits); + this.executionPayload = new PayloadResult(header, transactions, withdrawals); this.blockValue = blockValue; } @@ -74,13 +71,11 @@ public static class PayloadResult { private final String baseFeePerGas; protected final List transactions; private final List withdrawals; - private final List deposits; public PayloadResult( final BlockHeader header, final List transactions, - final Optional> withdrawals, - final Optional> deposits) { + final Optional> withdrawals) { this.blockNumber = Quantity.create(header.getNumber()); this.blockHash = header.getHash().toString(); this.parentHash = header.getParentHash().toString(); @@ -103,11 +98,6 @@ public PayloadResult( .map(WithdrawalParameter::fromWithdrawal) .collect(Collectors.toList())) .orElse(null); - this.deposits = - deposits - .map( - ds -> ds.stream().map(DepositParameter::fromDeposit).collect(Collectors.toList())) - .orElse(null); } @JsonGetter(value = "blockNumber") @@ -180,11 +170,6 @@ public List getWithdrawals() { return withdrawals; } - @JsonGetter(value = "deposits") - public List getDeposits() { - return deposits; - } - @JsonGetter(value = "feeRecipient") @JsonInclude(JsonInclude.Include.NON_NULL) public String getFeeRecipient() { diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/EngineGetPayloadResultV6110.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/EngineGetPayloadResultV6110.java new file mode 100644 index 00000000000..4b47101fb64 --- /dev/null +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/EngineGetPayloadResultV6110.java @@ -0,0 +1,230 @@ +/* + * Copyright Hyperledger Besu Contributors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.api.jsonrpc.internal.results; + +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.DepositParameter; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.WithdrawalParameter; +import org.hyperledger.besu.ethereum.core.BlockHeader; +import org.hyperledger.besu.ethereum.core.Deposit; +import org.hyperledger.besu.ethereum.core.Withdrawal; + +import java.util.List; +import java.util.Optional; +import java.util.stream.Collectors; + +import com.fasterxml.jackson.annotation.JsonGetter; +import com.fasterxml.jackson.annotation.JsonInclude; +import com.fasterxml.jackson.annotation.JsonPropertyOrder; +import org.apache.tuweni.bytes.Bytes32; + +@JsonPropertyOrder({"executionPayload", "blockValue", "blobsBundle", "shouldOverrideBuilder"}) +public class EngineGetPayloadResultV6110 { + protected final PayloadResult executionPayload; + private final String blockValue; + private final BlobsBundleV1 blobsBundle; + private final boolean shouldOverrideBuilder; + + public EngineGetPayloadResultV6110( + final BlockHeader header, + final List transactions, + final Optional> withdrawals, + final Optional> deposits, + final String blockValue, + final BlobsBundleV1 blobsBundle) { + this.executionPayload = new PayloadResult(header, transactions, withdrawals, deposits); + this.blockValue = blockValue; + this.blobsBundle = blobsBundle; + this.shouldOverrideBuilder = false; + } + + @JsonGetter(value = "executionPayload") + public PayloadResult getExecutionPayload() { + return executionPayload; + } + + @JsonGetter(value = "blockValue") + public String getBlockValue() { + return blockValue; + } + + @JsonGetter(value = "blobsBundle") + public BlobsBundleV1 getBlobsBundle() { + return blobsBundle; + } + + @JsonGetter(value = "shouldOverrideBuilder") + public boolean shouldOverrideBuilder() { + return shouldOverrideBuilder; + } + + public static class PayloadResult { + + protected final String blockHash; + private final String parentHash; + private final String feeRecipient; + private final String stateRoot; + private final String receiptsRoot; + private final String logsBloom; + private final String prevRandao; + private final String blockNumber; + private final String gasLimit; + private final String gasUsed; + private final String timestamp; + private final String extraData; + private final String baseFeePerGas; + private final String excessBlobGas; + private final String blobGasUsed; + private final String parentBeaconBlockRoot; + + protected final List transactions; + private final List withdrawals; + private final List deposits; + + public PayloadResult( + final BlockHeader header, + final List transactions, + final Optional> withdrawals, + final Optional> deposits) { + this.blockNumber = Quantity.create(header.getNumber()); + this.blockHash = header.getHash().toString(); + this.parentHash = header.getParentHash().toString(); + this.logsBloom = header.getLogsBloom().toString(); + this.stateRoot = header.getStateRoot().toString(); + this.receiptsRoot = header.getReceiptsRoot().toString(); + this.extraData = header.getExtraData().toString(); + this.baseFeePerGas = header.getBaseFee().map(Quantity::create).orElse(null); + this.gasLimit = Quantity.create(header.getGasLimit()); + this.gasUsed = Quantity.create(header.getGasUsed()); + this.timestamp = Quantity.create(header.getTimestamp()); + this.transactions = transactions; + this.feeRecipient = header.getCoinbase().toString(); + this.prevRandao = header.getPrevRandao().map(Bytes32::toHexString).orElse(null); + this.withdrawals = + withdrawals + .map( + ws -> + ws.stream() + .map(WithdrawalParameter::fromWithdrawal) + .collect(Collectors.toList())) + .orElse(null); + this.deposits = + deposits + .map( + ds -> ds.stream().map(DepositParameter::fromDeposit).collect(Collectors.toList())) + .orElse(null); + this.blobGasUsed = header.getBlobGasUsed().map(Quantity::create).orElse(Quantity.HEX_ZERO); + this.excessBlobGas = + header.getExcessBlobGas().map(Quantity::create).orElse(Quantity.HEX_ZERO); + this.parentBeaconBlockRoot = + header.getParentBeaconBlockRoot().map(Bytes32::toHexString).orElse(null); + } + + @JsonGetter(value = "blockNumber") + public String getNumber() { + return blockNumber; + } + + @JsonGetter(value = "blockHash") + public String getHash() { + return blockHash; + } + + @JsonGetter(value = "parentHash") + public String getParentHash() { + return parentHash; + } + + @JsonGetter(value = "logsBloom") + public String getLogsBloom() { + return logsBloom; + } + + @JsonGetter(value = "prevRandao") + public String getPrevRandao() { + return prevRandao; + } + + @JsonGetter(value = "stateRoot") + public String getStateRoot() { + return stateRoot; + } + + @JsonGetter(value = "receiptsRoot") + public String getReceiptRoot() { + return receiptsRoot; + } + + @JsonGetter(value = "extraData") + public String getExtraData() { + return extraData; + } + + @JsonGetter(value = "baseFeePerGas") + public String getBaseFeePerGas() { + return baseFeePerGas; + } + + @JsonGetter(value = "gasLimit") + public String getGasLimit() { + return gasLimit; + } + + @JsonGetter(value = "gasUsed") + public String getGasUsed() { + return gasUsed; + } + + @JsonGetter(value = "timestamp") + public String getTimestamp() { + return timestamp; + } + + @JsonGetter(value = "transactions") + public List getTransactions() { + return transactions; + } + + @JsonGetter(value = "withdrawals") + public List getWithdrawals() { + return withdrawals; + } + + @JsonGetter(value = "depositReceipts") + public List getDeposits() { + return deposits; + } + + @JsonGetter(value = "feeRecipient") + @JsonInclude(JsonInclude.Include.NON_NULL) + public String getFeeRecipient() { + return feeRecipient; + } + + @JsonGetter(value = "excessBlobGas") + public String getExcessBlobGas() { + return excessBlobGas; + } + + @JsonGetter(value = "blobGasUsed") + public String getBlobGasUseds() { + return blobGasUsed; + } + + @JsonGetter(value = "parentBeaconBlockRoot") + public String getParentBeaconBlockRoot() { + return parentBeaconBlockRoot; + } + } +} diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/ExecutionEngineJsonRpcMethods.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/ExecutionEngineJsonRpcMethods.java index 3c5ab9a6c41..9f4480b82be 100644 --- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/ExecutionEngineJsonRpcMethods.java +++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/methods/ExecutionEngineJsonRpcMethods.java @@ -28,9 +28,11 @@ import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetPayloadV1; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetPayloadV2; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetPayloadV3; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetPayloadV6110; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineNewPayloadV1; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineNewPayloadV2; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineNewPayloadV3; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineNewPayloadV6110; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EnginePreparePayloadDebug; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineQosTimer; import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.BlockResultFactory; @@ -158,6 +160,26 @@ protected Map create() { protocolSchedule)); } + if (protocolSchedule.anyMatch(p -> p.spec().getName().equalsIgnoreCase("ExperimentalEips"))) { + executionEngineApisSupported.add( + new EngineGetPayloadV6110( + consensusEngineServer, + protocolContext, + mergeCoordinator.get(), + blockResultFactory, + engineQosTimer, + protocolSchedule)); + + executionEngineApisSupported.add( + new EngineNewPayloadV6110( + consensusEngineServer, + protocolSchedule, + protocolContext, + mergeCoordinator.get(), + ethPeers, + engineQosTimer)); + } + return mapOf(executionEngineApisSupported); } else { return mapOf( diff --git a/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/graphql/EthGraphQLHttpBySpecTest.java b/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/graphql/EthGraphQLHttpBySpecTest.java index bc1e3c84bc6..4cd31646007 100644 --- a/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/graphql/EthGraphQLHttpBySpecTest.java +++ b/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/graphql/EthGraphQLHttpBySpecTest.java @@ -14,7 +14,14 @@ */ package org.hyperledger.besu.ethereum.api.graphql; +import static com.google.common.base.Preconditions.checkState; + import java.io.IOException; +import java.net.URISyntaxException; +import java.net.URL; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; import java.util.stream.Stream; import com.google.common.base.Charsets; @@ -30,80 +37,30 @@ public class EthGraphQLHttpBySpecTest extends AbstractEthGraphQLHttpServiceTest { - public static Stream specs() { - return Stream.of( - Arguments.of("eth_blockNumber"), - Arguments.of("eth_call_Block8"), - Arguments.of("eth_call_Block8_invalidHexBytesData"), - Arguments.of("eth_call_BlockLatest"), - Arguments.of("eth_call_from_contract"), - Arguments.of("eth_estimateGas_transfer"), - Arguments.of("eth_estimateGas_noParams"), - Arguments.of("eth_estimateGas_contractDeploy"), - Arguments.of("eth_estimateGas_from_contract"), - Arguments.of("eth_gasPrice"), - Arguments.of("eth_getBalance_0x19"), - Arguments.of("eth_getBalance_invalidAccountBlockNumber"), - Arguments.of("eth_getBalance_invalidAccountLatest"), - Arguments.of("eth_getBalance_latest"), - Arguments.of("eth_getBalance_toobig_bn"), - Arguments.of("eth_getBalance_without_addr"), - Arguments.of("eth_getBlock_byHash"), - Arguments.of("eth_getBlock_byHash_InvalidHexBytes32Hash"), - Arguments.of("eth_getBlock_byHashInvalid"), - Arguments.of("eth_getBlock_byNumber"), - Arguments.of("eth_getBlock_byNumberInvalid"), - Arguments.of("eth_getBlock_wrongParams"), - Arguments.of("eth_getBlockTransactionCount_byHash"), - Arguments.of("eth_getBlockTransactionCount_byNumber"), - Arguments.of("eth_getCode"), - Arguments.of("eth_getCode_noCode"), - Arguments.of("eth_getLogs_emptyListParam"), - Arguments.of("eth_getLogs_matchTopic"), - Arguments.of("eth_getLogs_matchAnyTopic"), - Arguments.of("eth_getLogs_range"), - Arguments.of("eth_getStorageAt"), - Arguments.of("eth_getStorageAt_illegalRangeGreaterThan"), - Arguments.of("eth_getTransaction_byBlockHashAndIndex"), - Arguments.of("eth_getTransaction_byBlockNumberAndIndex"), - Arguments.of("eth_getTransaction_byBlockNumberAndInvalidIndex"), - Arguments.of("eth_getTransaction_byHash"), - Arguments.of("eth_getTransaction_byHashNull"), - Arguments.of("eth_getTransactionCount"), - Arguments.of("eth_getTransactionReceipt"), - Arguments.of("eth_sendRawTransaction_contractCreation"), - Arguments.of("eth_sendRawTransaction_messageCall"), - Arguments.of("eth_sendRawTransaction_nonceTooLow"), - Arguments.of("eth_sendRawTransaction_transferEther"), - Arguments.of("eth_sendRawTransaction_unsignedTransaction"), - Arguments.of("eth_syncing"), - Arguments.of("graphql_blocks_byFrom"), - Arguments.of("graphql_blocks_byRange"), - Arguments.of("graphql_blocks_byWrongRange"), - Arguments.of("graphql_pending"), - Arguments.of("graphql_tooComplex"), - Arguments.of("graphql_tooComplexSchema"), - Arguments.of("graphql_variable_address"), - Arguments.of("graphql_variable_bytes"), - Arguments.of("graphql_variable_bytes32"), - Arguments.of("graphql_variable_long"), - Arguments.of("block_withdrawals_pre_shanghai"), - Arguments.of("block_withdrawals"), - Arguments.of("eth_getTransaction_type2"), - Arguments.of("eth_getBlock_shanghai")); + @SuppressWarnings("StreamResourceLeak") + public static Stream specs() throws IOException, URISyntaxException { + final URL url = + EthGraphQLHttpBySpecTest.class.getResource( + "/org/hyperledger/besu/ethereum/api/graphql/eth_blockNumber.json"); + checkState(url != null, "Cannot find test directory org/hyperledger/besu/ethereum/api/graphql"); + final Path dir = Paths.get(url.toURI()).getParent(); + return Files.list(dir) + .map(Path::getFileName) + .map(Path::toString) + .filter(p -> p.endsWith(".json")) + .filter(p -> !p.contains("genesis")) + .map(Arguments::of); } @ParameterizedTest(name = "{index}: {0}") @MethodSource("specs") - public void graphQLCallWithSpecFile(final String specFileName) throws Exception { + void graphQLCallWithSpecFile(final String specFileName) throws Exception { graphQLCall(specFileName); } private void graphQLCall(final String name) throws IOException { - final String testSpecFile = name + ".json"; final String json = - Resources.toString( - EthGraphQLHttpBySpecTest.class.getResource(testSpecFile), Charsets.UTF_8); + Resources.toString(EthGraphQLHttpBySpecTest.class.getResource(name), Charsets.UTF_8); final JsonObject spec = new JsonObject(json); final String rawRequestBody = spec.getString("request"); final String rawVariables = spec.getString("variables"); diff --git a/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractScheduledApiTest.java b/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractScheduledApiTest.java index 84704e3f2a1..6d71932ff96 100644 --- a/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractScheduledApiTest.java +++ b/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/AbstractScheduledApiTest.java @@ -40,7 +40,7 @@ public class AbstractScheduledApiTest { protected final ScheduledProtocolSpec.Hardfork cancunHardfork = new ScheduledProtocolSpec.Hardfork("Cancun", 30); protected final ScheduledProtocolSpec.Hardfork experimentalHardfork = - new ScheduledProtocolSpec.Hardfork("Experimental", 40); + new ScheduledProtocolSpec.Hardfork("ExperimentalEips", 40); @Mock protected DefaultProtocolSchedule protocolSchedule; diff --git a/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineGetPayloadV2Test.java b/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineGetPayloadV2Test.java index 4c85b18630f..d9f7f9b4291 100644 --- a/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineGetPayloadV2Test.java +++ b/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineGetPayloadV2Test.java @@ -93,30 +93,6 @@ public void shouldReturnBlockForKnownPayloadId() { verify(engineCallListener, times(1)).executionEngineCalled(); } - @Test - public void shouldReturnBlockForKnownPayloadIdPostV6110() { - // should return deposits for a post-V6110 block - when(mergeContext.retrieveBlockById(mockPid)) - .thenReturn(Optional.of(mockBlockWithReceiptsAndDeposits)); - - final var resp = resp(RpcMethod.ENGINE_GET_PAYLOAD_V2.getMethodName(), mockPid); - assertThat(resp).isInstanceOf(JsonRpcSuccessResponse.class); - Optional.of(resp) - .map(JsonRpcSuccessResponse.class::cast) - .ifPresent( - r -> { - assertThat(r.getResult()).isInstanceOf(EngineGetPayloadResultV2.class); - final EngineGetPayloadResultV2 res = (EngineGetPayloadResultV2) r.getResult(); - assertThat(res.getExecutionPayload().getDeposits()).isNotNull(); - assertThat(res.getExecutionPayload().getHash()) - .isEqualTo(mockHeader.getHash().toString()); - assertThat(res.getBlockValue()).isEqualTo(Quantity.create(0)); - assertThat(res.getExecutionPayload().getPrevRandao()) - .isEqualTo(mockHeader.getPrevRandao().map(Bytes32::toString).orElse("")); - }); - verify(engineCallListener, times(1)).executionEngineCalled(); - } - @Test public void shouldReturnExecutionPayloadWithoutWithdrawals_PreShanghaiBlock() { final var resp = resp(RpcMethod.ENGINE_GET_PAYLOAD_V2.getMethodName(), mockPid); @@ -128,25 +104,6 @@ public void shouldReturnExecutionPayloadWithoutWithdrawals_PreShanghaiBlock() { assertThat(r.getResult()).isInstanceOf(EngineGetPayloadResultV2.class); final EngineGetPayloadResultV2 res = (EngineGetPayloadResultV2) r.getResult(); assertThat(res.getExecutionPayload().getWithdrawals()).isNull(); - assertThat(res.getExecutionPayload().getDeposits()).isNull(); - }); - verify(engineCallListener, times(1)).executionEngineCalled(); - } - - @Test - public void shouldReturnExecutionPayloadWithoutDeposits_PreV6110Block() { - when(mergeContext.retrieveBlockById(mockPid)) - .thenReturn(Optional.of(mockBlockWithReceiptsAndWithdrawals)); - - final var resp = resp(RpcMethod.ENGINE_GET_PAYLOAD_V2.getMethodName(), mockPid); - assertThat(resp).isInstanceOf(JsonRpcSuccessResponse.class); - Optional.of(resp) - .map(JsonRpcSuccessResponse.class::cast) - .ifPresent( - r -> { - assertThat(r.getResult()).isInstanceOf(EngineGetPayloadResultV2.class); - final EngineGetPayloadResultV2 res = (EngineGetPayloadResultV2) r.getResult(); - assertThat(res.getExecutionPayload().getDeposits()).isNull(); }); verify(engineCallListener, times(1)).executionEngineCalled(); } diff --git a/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineGetPayloadV6110Test.java b/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineGetPayloadV6110Test.java new file mode 100644 index 00000000000..879f11e724c --- /dev/null +++ b/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineGetPayloadV6110Test.java @@ -0,0 +1,177 @@ +/* + * Copyright Hyperledger Besu Contributors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.lenient; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import org.hyperledger.besu.consensus.merge.blockcreation.PayloadIdentifier; +import org.hyperledger.besu.datatypes.Address; +import org.hyperledger.besu.datatypes.BlobGas; +import org.hyperledger.besu.datatypes.BlobsWithCommitments; +import org.hyperledger.besu.datatypes.Hash; +import org.hyperledger.besu.datatypes.TransactionType; +import org.hyperledger.besu.datatypes.Wei; +import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcSuccessResponse; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.EngineGetPayloadResultV6110; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.Quantity; +import org.hyperledger.besu.ethereum.core.BlobTestFixture; +import org.hyperledger.besu.ethereum.core.Block; +import org.hyperledger.besu.ethereum.core.BlockBody; +import org.hyperledger.besu.ethereum.core.BlockHeader; +import org.hyperledger.besu.ethereum.core.BlockHeaderTestFixture; +import org.hyperledger.besu.ethereum.core.BlockWithReceipts; +import org.hyperledger.besu.ethereum.core.Transaction; +import org.hyperledger.besu.ethereum.core.TransactionReceipt; +import org.hyperledger.besu.ethereum.core.TransactionTestFixture; + +import java.math.BigInteger; +import java.util.Collections; +import java.util.List; +import java.util.Optional; + +import org.apache.tuweni.bytes.Bytes; +import org.apache.tuweni.bytes.Bytes32; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.Mockito; +import org.mockito.junit.jupiter.MockitoExtension; + +@ExtendWith( + MockitoExtension.class) // mocks in parent class may not be used, throwing unnecessary stubbing +public class EngineGetPayloadV6110Test extends AbstractEngineGetPayloadTest { + + public EngineGetPayloadV6110Test() { + super(); + } + + @BeforeEach + @Override + public void before() { + super.before(); + lenient() + .when(mergeContext.retrieveBlockById(mockPid)) + .thenReturn(Optional.of(mockBlockWithReceiptsAndDeposits)); + when(protocolContext.safeConsensusContext(Mockito.any())).thenReturn(Optional.of(mergeContext)); + this.method = + new EngineGetPayloadV6110( + vertx, + protocolContext, + mergeMiningCoordinator, + factory, + engineCallListener, + protocolSchedule); + } + + @Override + @Test + public void shouldReturnExpectedMethodName() { + assertThat(method.getName()).isEqualTo("engine_getPayloadV6110"); + } + + @Override + @Test + public void shouldReturnBlockForKnownPayloadId() { + + BlockHeader eip6110Header = + new BlockHeaderTestFixture() + .prevRandao(Bytes32.random()) + .timestamp(experimentalHardfork.milestone() + 1) + .excessBlobGas(BlobGas.of(10L)) + .buildHeader(); + // should return withdrawals, deposits and excessGas for a post-6110 block + PayloadIdentifier postEip6110Pid = + PayloadIdentifier.forPayloadParams( + Hash.ZERO, + experimentalHardfork.milestone(), + Bytes32.random(), + Address.fromHexString("0x42"), + Optional.empty(), + Optional.empty()); + + BlobTestFixture blobTestFixture = new BlobTestFixture(); + BlobsWithCommitments bwc = blobTestFixture.createBlobsWithCommitments(1); + Transaction blobTx = + new TransactionTestFixture() + .to(Optional.of(Address.fromHexString("0xDEADBEEFDEADBEEFDEADBEEFDEADBEEFDEADBEEF"))) + .type(TransactionType.BLOB) + .chainId(Optional.of(BigInteger.ONE)) + .maxFeePerGas(Optional.of(Wei.of(15))) + .maxFeePerBlobGas(Optional.of(Wei.of(128))) + .maxPriorityFeePerGas(Optional.of(Wei.of(1))) + .blobsWithCommitments(Optional.of(bwc)) + .versionedHashes(Optional.of(bwc.getVersionedHashes())) + .createTransaction(senderKeys); + TransactionReceipt blobReceipt = mock(TransactionReceipt.class); + when(blobReceipt.getCumulativeGasUsed()).thenReturn(100L); + BlockWithReceipts postEip6110Block = + new BlockWithReceipts( + new Block( + eip6110Header, + new BlockBody( + List.of(blobTx), + Collections.emptyList(), + Optional.of(Collections.emptyList()), + Optional.of(Collections.emptyList()))), + List.of(blobReceipt)); + + when(mergeContext.retrieveBlockById(postEip6110Pid)).thenReturn(Optional.of(postEip6110Block)); + + final var resp = resp(RpcMethod.ENGINE_GET_PAYLOAD_V6110.getMethodName(), postEip6110Pid); + assertThat(resp).isInstanceOf(JsonRpcSuccessResponse.class); + Optional.of(resp) + .map(JsonRpcSuccessResponse.class::cast) + .ifPresent( + r -> { + assertThat(r.getResult()).isInstanceOf(EngineGetPayloadResultV6110.class); + final EngineGetPayloadResultV6110 res = (EngineGetPayloadResultV6110) r.getResult(); + assertThat(res.getExecutionPayload().getWithdrawals()).isNotNull(); + assertThat(res.getExecutionPayload().getDeposits()).isNotNull(); + assertThat(res.getExecutionPayload().getHash()) + .isEqualTo(eip6110Header.getHash().toString()); + assertThat(res.getBlockValue()).isEqualTo(Quantity.create(0)); + assertThat(res.getExecutionPayload().getPrevRandao()) + .isEqualTo(eip6110Header.getPrevRandao().map(Bytes32::toString).orElse("")); + // excessBlobGas: QUANTITY, 256 bits + String expectedQuantityOf10 = Bytes32.leftPad(Bytes.of(10)).toQuantityHexString(); + assertThat(res.getExecutionPayload().getExcessBlobGas()).isNotEmpty(); + assertThat(res.getExecutionPayload().getExcessBlobGas()) + .isEqualTo(expectedQuantityOf10); + }); + verify(engineCallListener, times(1)).executionEngineCalled(); + } + + @Test + public void shouldReturnUnsupportedFork() { + final var resp = resp(RpcMethod.ENGINE_GET_PAYLOAD_V6110.getMethodName(), mockPid); + + assertThat(resp).isInstanceOf(JsonRpcErrorResponse.class); + assertThat(((JsonRpcErrorResponse) resp).getErrorType()) + .isEqualTo(RpcErrorType.UNSUPPORTED_FORK); + } + + @Override + protected String getMethodName() { + return RpcMethod.ENGINE_GET_PAYLOAD_V6110.getMethodName(); + } +} diff --git a/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadEIP6110Test.java b/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadV6110Test.java similarity index 98% rename from ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadEIP6110Test.java rename to ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadV6110Test.java index 0e89e66e192..6390b8716c6 100644 --- a/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadEIP6110Test.java +++ b/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/engine/EngineNewPayloadV6110Test.java @@ -12,7 +12,6 @@ * * SPDX-License-Identifier: Apache-2.0 */ - package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine; import static org.assertj.core.api.Assertions.assertThat; @@ -55,11 +54,11 @@ import org.mockito.junit.jupiter.MockitoExtension; @ExtendWith(MockitoExtension.class) -public class EngineNewPayloadEIP6110Test extends EngineNewPayloadV3Test { +public class EngineNewPayloadV6110Test extends EngineNewPayloadV3Test { private static final Address depositContractAddress = Address.fromHexString("0x00000000219ab540356cbb839cbe05303d7705fa"); - public EngineNewPayloadEIP6110Test() {} + public EngineNewPayloadV6110Test() {} @BeforeEach @Override diff --git a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_blockNumber.json b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_blockNumber.json index 247ec0146e5..35b79c9ab7b 100644 --- a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_blockNumber.json +++ b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_blockNumber.json @@ -3,7 +3,7 @@ "response": { "data": { "block": { - "number": "0x21" + "number": "0x22" } } }, diff --git a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_call_BlockLatest.json b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_call_BlockLatest.json index 9b5bb2fd89c..bd133f0f060 100644 --- a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_call_BlockLatest.json +++ b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_call_BlockLatest.json @@ -3,7 +3,7 @@ "response": { "data": { "block": { - "number": "0x21", + "number": "0x22", "call": { "data": "0x0000000000000000000000000000000000000000000000000000000000000001", "status": "0x1" diff --git a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_call_from_contract.json b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_call_from_contract.json index 211a97be18f..dea5433a856 100644 --- a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_call_from_contract.json +++ b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_call_from_contract.json @@ -3,7 +3,7 @@ "response": { "data": { "block": { - "number": "0x21", + "number": "0x22", "call": { "data": "0x", "status": "0x1" diff --git a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_gasPrice.json b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_gasPrice.json index 744e7ea98b9..40b8bf0fa0d 100644 --- a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_gasPrice.json +++ b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_gasPrice.json @@ -3,7 +3,7 @@ "response": { "data": { "gasPrice": "0x1", - "maxPriorityFeePerGas": "0x0" + "maxPriorityFeePerGas": "0x3b9aca00" } }, "statusCode": 200 diff --git a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_getBalance_toobig_bn.json b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_getBalance_toobig_bn.json index cbc24c848e1..ef45299e915 100644 --- a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_getBalance_toobig_bn.json +++ b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_getBalance_toobig_bn.json @@ -1,9 +1,9 @@ { - "request": "{ block(number:\"0x22\") { account(address: \"0x6295ee1b4f6dd65047762f924ecd367c17eabf8f\") { balance } } }", + "request": "{ block(number:\"0x220\") { account(address: \"0x6295ee1b4f6dd65047762f924ecd367c17eabf8f\") { balance } } }", "response": { "errors": [ { - "message": "Exception while fetching data (/block) : Block number 34 was not found", + "message": "Exception while fetching data (/block) : Block number 544 was not found", "locations": [ { "line": 1, diff --git a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_getBlock_cancun.json b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_getBlock_cancun.json new file mode 100644 index 00000000000..3b9c0039152 --- /dev/null +++ b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_getBlock_cancun.json @@ -0,0 +1,25 @@ +{ + "request": "{block (number: 34) { baseFeePerGas difficulty extraData miner { address } mixHash nonce stateRoot totalDifficulty withdrawalsRoot withdrawals { address amount index validator } blobGasUsed excessBlobGas transactions { maxFeePerBlobGas blobGasUsed blobGasPrice } }} ", + "response":{ + "data": { + "block":{ + "baseFeePerGas":"0x3437004a", + "difficulty":"0x0", + "extraData":"0x", + "miner": { + "address":"0x0000000000000000000000000000000000000000" + }, + "mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000", + "nonce":"0x0000000000000000", + "stateRoot":"0x34727aff24d1c51cd63fdc14515b15ddaa156fa0671c58a96c72b1553819945d", + "totalDifficulty":"0x427c00", + "withdrawalsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421", + "withdrawals":[], + "blobGasUsed":"0x40000", + "excessBlobGas":"0x0", + "transactions":[{"maxFeePerBlobGas":"0x3b9aca00","blobGasUsed":"0x40000","blobGasPrice":"0x1"}] + } + } + }, + "statusCode": 200 +} \ No newline at end of file diff --git a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_getTransactionCount.json b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_getTransactionCount.json index a1fc9e307c2..3bacae05c80 100644 --- a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_getTransactionCount.json +++ b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/eth_getTransactionCount.json @@ -4,7 +4,7 @@ "data": { "pending": { "account": { - "transactionCount": "0x21" + "transactionCount": "0x22" } } } diff --git a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/graphql_blocks_byFrom.json b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/graphql_blocks_byFrom.json index 800542adbc0..33046a3a548 100644 --- a/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/graphql_blocks_byFrom.json +++ b/ethereum/api/src/test/resources/org/hyperledger/besu/ethereum/api/graphql/graphql_blocks_byFrom.json @@ -14,6 +14,9 @@ }, { "number": "0x21" + }, + { + "number": "0x22" } ] } diff --git a/ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/BlockTransactionSelector.java b/ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/BlockTransactionSelector.java index a5d168ae8c1..03c6ef28951 100644 --- a/ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/BlockTransactionSelector.java +++ b/ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/BlockTransactionSelector.java @@ -18,6 +18,7 @@ import org.hyperledger.besu.datatypes.Wei; import org.hyperledger.besu.ethereum.GasLimitCalculator; import org.hyperledger.besu.ethereum.blockcreation.txselection.selectors.AbstractTransactionSelector; +import org.hyperledger.besu.ethereum.blockcreation.txselection.selectors.AllAcceptingTransactionSelector; import org.hyperledger.besu.ethereum.blockcreation.txselection.selectors.BlobPriceTransactionSelector; import org.hyperledger.besu.ethereum.blockcreation.txselection.selectors.BlockSizeTransactionSelector; import org.hyperledger.besu.ethereum.blockcreation.txselection.selectors.PriceTransactionSelector; @@ -37,10 +38,11 @@ import org.hyperledger.besu.ethereum.vm.BlockHashLookup; import org.hyperledger.besu.ethereum.vm.CachingBlockHashLookup; import org.hyperledger.besu.evm.gascalculator.GasCalculator; +import org.hyperledger.besu.evm.tracing.OperationTracer; import org.hyperledger.besu.evm.worldstate.WorldUpdater; import org.hyperledger.besu.plugin.data.TransactionSelectionResult; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelector; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelector; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory; import java.util.List; import java.util.Optional; @@ -73,6 +75,7 @@ */ public class BlockTransactionSelector { private static final Logger LOG = LoggerFactory.getLogger(BlockTransactionSelector.class); + private final Supplier isCancelled; private final MainnetTransactionProcessor transactionProcessor; private final Blockchain blockchain; @@ -82,7 +85,8 @@ public class BlockTransactionSelector { private final TransactionSelectionResults transactionSelectionResults = new TransactionSelectionResults(); private final List transactionSelectors; - private final List externalTransactionSelectors; + private final PluginTransactionSelector pluginTransactionSelector; + private final OperationTracer pluginOperationTracer; public BlockTransactionSelector( final MainnetTransactionProcessor transactionProcessor, @@ -99,7 +103,7 @@ public BlockTransactionSelector( final FeeMarket feeMarket, final GasCalculator gasCalculator, final GasLimitCalculator gasLimitCalculator, - final Optional transactionSelectorFactory) { + final Optional transactionSelectorFactory) { this.transactionProcessor = transactionProcessor; this.blockchain = blockchain; this.worldState = worldState; @@ -117,8 +121,20 @@ public BlockTransactionSelector( miningBeneficiary, transactionPool); transactionSelectors = createTransactionSelectors(blockSelectionContext); - externalTransactionSelectors = - transactionSelectorFactory.map(TransactionSelectorFactory::create).orElse(List.of()); + pluginTransactionSelector = + transactionSelectorFactory + .map(PluginTransactionSelectorFactory::create) + .orElse(AllAcceptingTransactionSelector.INSTANCE); + pluginOperationTracer = pluginTransactionSelector.getOperationTracer(); + } + + private List createTransactionSelectors( + final BlockSelectionContext context) { + return List.of( + new BlockSizeTransactionSelector(context), + new PriceTransactionSelector(context), + new BlobPriceTransactionSelector(context), + new ProcessingResultTransactionSelector(context)); } /** @@ -135,17 +151,7 @@ public TransactionSelectionResults buildTransactionListForBlock() { .setMessage("Transaction pool stats {}") .addArgument(blockSelectionContext.transactionPool().logStats()) .log(); - blockSelectionContext - .transactionPool() - .selectTransactions( - pendingTransaction -> { - final var res = evaluateTransaction(pendingTransaction); - if (!res.selected()) { - transactionSelectionResults.updateNotSelected( - pendingTransaction.getTransaction(), res); - } - return res; - }); + blockSelectionContext.transactionPool().selectTransactions(this::evaluateTransaction); LOG.atTrace() .setMessage("Transaction selection result {}") .addArgument(transactionSelectionResults::toTraceLog) @@ -164,81 +170,40 @@ public TransactionSelectionResults buildTransactionListForBlock() { */ public TransactionSelectionResults evaluateTransactions(final List transactions) { transactions.forEach( - transaction -> { - final var res = evaluateTransaction(new PendingTransaction.Local(transaction)); - if (!res.selected()) { - transactionSelectionResults.updateNotSelected(transaction, res); - } - }); + transaction -> evaluateTransaction(new PendingTransaction.Local.Priority(transaction))); return transactionSelectionResults; } - /* + /** * Passed into the PendingTransactions, and is called on each transaction until sufficient - * transactions are found which fill a block worth of gas. - * - * This function will continue to be called until the block under construction is suitably - * full (in terms of gasLimit) and the provided transaction's gasLimit does not fit within - * the space remaining in the block. + * transactions are found which fill a block worth of gas. This function will continue to be + * called until the block under construction is suitably full (in terms of gasLimit) and the + * provided transaction's gasLimit does not fit within the space remaining in the block. * + * @param pendingTransaction The transaction to be evaluated. + * @return The result of the transaction evaluation process. + * @throws CancellationException if the transaction selection process is cancelled. */ private TransactionSelectionResult evaluateTransaction( final PendingTransaction pendingTransaction) { - if (isCancelled.get()) { - throw new CancellationException("Cancelled during transaction selection."); - } + checkCancellation(); - final Transaction transaction = pendingTransaction.getTransaction(); - - TransactionSelectionResult selectionResult = - evaluateTransactionPreProcessing(pendingTransaction); + TransactionSelectionResult selectionResult = evaluatePreProcessing(pendingTransaction); if (!selectionResult.selected()) { - return selectionResult; + return handleTransactionNotSelected(pendingTransaction, selectionResult); } final WorldUpdater worldStateUpdater = worldState.updater(); - final BlockHashLookup blockHashLookup = - new CachingBlockHashLookup(blockSelectionContext.processableBlockHeader(), blockchain); + final TransactionProcessingResult processingResult = + processTransaction(pendingTransaction, worldStateUpdater); - final TransactionProcessingResult effectiveResult = - transactionProcessor.processTransaction( - blockchain, - worldStateUpdater, - blockSelectionContext.processableBlockHeader(), - transaction, - blockSelectionContext.miningBeneficiary(), - blockHashLookup, - false, - TransactionValidationParams.mining(), - blockSelectionContext.blobGasPrice()); - - var transactionWithProcessingContextResult = - evaluateTransactionPostProcessing(pendingTransaction, effectiveResult); - if (!transactionWithProcessingContextResult.selected()) { - return transactionWithProcessingContextResult; + var postProcessingSelectionResult = + evaluatePostProcessing(pendingTransaction, processingResult); + if (!postProcessingSelectionResult.selected()) { + return handleTransactionNotSelected(pendingTransaction, postProcessingSelectionResult); } - final long gasUsedByTransaction = transaction.getGasLimit() - effectiveResult.getGasRemaining(); - final long cumulativeGasUsed = - transactionSelectionResults.getCumulativeGasUsed() + gasUsedByTransaction; - - worldStateUpdater.commit(); - final TransactionReceipt receipt = - transactionReceiptFactory.create( - transaction.getType(), effectiveResult, worldState, cumulativeGasUsed); - - final long blobGasUsed = - blockSelectionContext.gasCalculator().blobGasCost(transaction.getBlobCount()); - - transactionSelectionResults.updateSelected( - transaction, receipt, gasUsedByTransaction, blobGasUsed); - - LOG.atTrace() - .setMessage("Selected {} for block creation") - .addArgument(transaction::toTraceLog) - .log(); - - return TransactionSelectionResult.SELECTED; + return handleTransactionSelected(pendingTransaction, processingResult, worldStateUpdater); } /** @@ -250,31 +215,18 @@ private TransactionSelectionResult evaluateTransaction( * @param pendingTransaction The transaction to be evaluated. * @return The result of the transaction selection process. */ - private TransactionSelectionResult evaluateTransactionPreProcessing( + private TransactionSelectionResult evaluatePreProcessing( final PendingTransaction pendingTransaction) { - // Process the transaction through internal selectors for (var selector : transactionSelectors) { TransactionSelectionResult result = selector.evaluateTransactionPreProcessing( pendingTransaction, transactionSelectionResults); - // If the transaction is not selected by any internal selector, return the result if (!result.equals(TransactionSelectionResult.SELECTED)) { return result; } } - - // Process the transaction through external selectors - for (var selector : externalTransactionSelectors) { - TransactionSelectionResult result = - selector.evaluateTransactionPreProcessing(pendingTransaction); - // If the transaction is not selected by any external selector, return the result - if (!result.equals(TransactionSelectionResult.SELECTED)) { - return result; - } - } - // If the transaction is selected by all selectors, return SELECTED - return TransactionSelectionResult.SELECTED; + return pluginTransactionSelector.evaluateTransactionPreProcessing(pendingTransaction); } /** @@ -287,41 +239,111 @@ private TransactionSelectionResult evaluateTransactionPreProcessing( * @param processingResult The result of the transaction processing. * @return The result of the transaction selection process. */ - private TransactionSelectionResult evaluateTransactionPostProcessing( + private TransactionSelectionResult evaluatePostProcessing( final PendingTransaction pendingTransaction, final TransactionProcessingResult processingResult) { - // Process the transaction through internal selectors for (var selector : transactionSelectors) { TransactionSelectionResult result = selector.evaluateTransactionPostProcessing( pendingTransaction, transactionSelectionResults, processingResult); - // If the transaction is not selected by any selector, return the result if (!result.equals(TransactionSelectionResult.SELECTED)) { return result; } } + return pluginTransactionSelector.evaluateTransactionPostProcessing( + pendingTransaction, processingResult); + } - // Process the transaction through external selectors - for (var selector : externalTransactionSelectors) { - TransactionSelectionResult result = - selector.evaluateTransactionPostProcessing(pendingTransaction, processingResult); - // If the transaction is not selected by any external selector, return the result - if (!result.equals(TransactionSelectionResult.SELECTED)) { - return result; - } - } + /** + * Processes a transaction + * + * @param pendingTransaction The transaction to be processed. + * @param worldStateUpdater The world state updater. + * @return The result of the transaction processing. + */ + private TransactionProcessingResult processTransaction( + final PendingTransaction pendingTransaction, final WorldUpdater worldStateUpdater) { + final BlockHashLookup blockHashLookup = + new CachingBlockHashLookup(blockSelectionContext.processableBlockHeader(), blockchain); + return transactionProcessor.processTransaction( + blockchain, + worldStateUpdater, + blockSelectionContext.processableBlockHeader(), + pendingTransaction.getTransaction(), + blockSelectionContext.miningBeneficiary(), + pluginOperationTracer, + blockHashLookup, + false, + TransactionValidationParams.mining(), + blockSelectionContext.blobGasPrice()); + } + + /** + * Handles a selected transaction by committing the world state updates, creating a transaction + * receipt, updating the TransactionSelectionResults with the selected transaction, and notifying + * the external transaction selector. + * + * @param pendingTransaction The pending transaction. + * @param processingResult The result of the transaction processing. + * @param worldStateUpdater The world state updater. + * @return The result of the transaction selection process. + */ + private TransactionSelectionResult handleTransactionSelected( + final PendingTransaction pendingTransaction, + final TransactionProcessingResult processingResult, + final WorldUpdater worldStateUpdater) { + worldStateUpdater.commit(); + final Transaction transaction = pendingTransaction.getTransaction(); + + final long gasUsedByTransaction = + transaction.getGasLimit() - processingResult.getGasRemaining(); + final long cumulativeGasUsed = + transactionSelectionResults.getCumulativeGasUsed() + gasUsedByTransaction; + final long blobGasUsed = + blockSelectionContext.gasCalculator().blobGasCost(transaction.getBlobCount()); + + final TransactionReceipt receipt = + transactionReceiptFactory.create( + transaction.getType(), processingResult, worldState, cumulativeGasUsed); + + logTransactionSelection(pendingTransaction.getTransaction()); + + transactionSelectionResults.updateSelected( + pendingTransaction.getTransaction(), receipt, gasUsedByTransaction, blobGasUsed); + pluginTransactionSelector.onTransactionSelected(pendingTransaction, processingResult); - // If the transaction is selected by all selectors, return SELECTED return TransactionSelectionResult.SELECTED; } - private List createTransactionSelectors( - final BlockSelectionContext context) { - return List.of( - new BlockSizeTransactionSelector(context), - new PriceTransactionSelector(context), - new BlobPriceTransactionSelector(context), - new ProcessingResultTransactionSelector(context)); + /** + * Handles the scenario when a transaction is not selected. It updates the + * TransactionSelectionResults with the unselected transaction, and notifies the external + * transaction selector. + * + * @param pendingTransaction The unselected pending transaction. + * @param selectionResult The result of the transaction selection process. + * @return The result of the transaction selection process. + */ + private TransactionSelectionResult handleTransactionNotSelected( + final PendingTransaction pendingTransaction, + final TransactionSelectionResult selectionResult) { + transactionSelectionResults.updateNotSelected( + pendingTransaction.getTransaction(), selectionResult); + pluginTransactionSelector.onTransactionNotSelected(pendingTransaction, selectionResult); + return selectionResult; + } + + private void checkCancellation() { + if (isCancelled.get()) { + throw new CancellationException("Cancelled during transaction selection."); + } + } + + private void logTransactionSelection(final Transaction transaction) { + LOG.atTrace() + .setMessage("Selected {} for block creation") + .addArgument(transaction::toTraceLog) + .log(); } } diff --git a/ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/selectors/AllAcceptingTransactionSelector.java b/ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/selectors/AllAcceptingTransactionSelector.java new file mode 100644 index 00000000000..9032ddca575 --- /dev/null +++ b/ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/selectors/AllAcceptingTransactionSelector.java @@ -0,0 +1,54 @@ +/* + * Copyright Hyperledger Besu Contributors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.blockcreation.txselection.selectors; + +import org.hyperledger.besu.datatypes.PendingTransaction; +import org.hyperledger.besu.plugin.data.TransactionProcessingResult; +import org.hyperledger.besu.plugin.data.TransactionSelectionResult; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelector; + +/** A TransactionSelector that unconditionally selects all transactions. */ +public class AllAcceptingTransactionSelector implements PluginTransactionSelector { + public static final AllAcceptingTransactionSelector INSTANCE = + new AllAcceptingTransactionSelector(); + + private AllAcceptingTransactionSelector() {} + + /** + * Always selects the transaction in the pre-processing stage. + * + * @param pendingTransaction The transaction to be evaluated. + * @return Always SELECTED. + */ + @Override + public TransactionSelectionResult evaluateTransactionPreProcessing( + final PendingTransaction pendingTransaction) { + return TransactionSelectionResult.SELECTED; + } + + /** + * Always selects the transaction in the post-processing stage. + * + * @param pendingTransaction The transaction to be evaluated. + * @param processingResult The result of the transaction processing. + * @return Always SELECTED. + */ + @Override + public TransactionSelectionResult evaluateTransactionPostProcessing( + final PendingTransaction pendingTransaction, + final TransactionProcessingResult processingResult) { + return TransactionSelectionResult.SELECTED; + } +} diff --git a/ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/selectors/PriceTransactionSelector.java b/ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/selectors/PriceTransactionSelector.java index a6c1889e497..9828650b41b 100644 --- a/ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/selectors/PriceTransactionSelector.java +++ b/ethereum/blockcreation/src/main/java/org/hyperledger/besu/ethereum/blockcreation/txselection/selectors/PriceTransactionSelector.java @@ -73,8 +73,7 @@ private boolean transactionCurrentPriceBelowMin(final PendingTransaction pending final Transaction transaction = pendingTransaction.getTransaction(); // Here we only care about EIP1159 since for Frontier and local transactions the checks // that we do when accepting them in the pool are enough - if (transaction.getType().supports1559FeeMarket() - && !pendingTransaction.isReceivedFromLocalSource()) { + if (transaction.getType().supports1559FeeMarket() && !pendingTransaction.hasPriority()) { // For EIP1559 transactions, the price is dynamic and depends on network conditions, so we can // only calculate at this time the current minimum price the transaction is willing to pay @@ -90,7 +89,7 @@ private boolean transactionCurrentPriceBelowMin(final PendingTransaction pending if (context.minTransactionGasPrice().compareTo(currentMinTransactionGasPriceInBlock) > 0) { LOG.trace( "Current gas fee of {} is lower than configured minimum {}, skipping", - transaction, + pendingTransaction, context.minTransactionGasPrice()); return true; } diff --git a/ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/AbstractBlockTransactionSelectorTest.java b/ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/AbstractBlockTransactionSelectorTest.java index fd5ed17f59e..863d164df43 100644 --- a/ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/AbstractBlockTransactionSelectorTest.java +++ b/ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/AbstractBlockTransactionSelectorTest.java @@ -19,6 +19,9 @@ import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.anyBoolean; import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.spy; +import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; import org.hyperledger.besu.config.GenesisConfigFile; @@ -33,6 +36,7 @@ import org.hyperledger.besu.ethereum.ProtocolContext; import org.hyperledger.besu.ethereum.blockcreation.txselection.BlockTransactionSelector; import org.hyperledger.besu.ethereum.blockcreation.txselection.TransactionSelectionResults; +import org.hyperledger.besu.ethereum.blockcreation.txselection.selectors.AllAcceptingTransactionSelector; import org.hyperledger.besu.ethereum.chain.DefaultBlockchain; import org.hyperledger.besu.ethereum.chain.GenesisState; import org.hyperledger.besu.ethereum.chain.MutableBlockchain; @@ -67,8 +71,8 @@ import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem; import org.hyperledger.besu.plugin.data.TransactionSelectionResult; import org.hyperledger.besu.plugin.services.MetricsSystem; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelector; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelector; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory; import org.hyperledger.besu.services.kvstore.InMemoryKeyValueStorage; import java.math.BigInteger; @@ -84,6 +88,7 @@ import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.mockito.Answers; +import org.mockito.ArgumentCaptor; import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; import org.mockito.junit.jupiter.MockitoSettings; @@ -552,28 +557,27 @@ public void transactionSelectionPluginShouldWork_PreProcessing() { final Transaction notSelectedInvalid = createTransaction(2, Wei.of(10), 21_000); ensureTransactionIsValid(notSelectedInvalid, 21_000, 0); - final TransactionSelectorFactory transactionSelectorFactory = + final PluginTransactionSelectorFactory transactionSelectorFactory = () -> - List.of( - new TransactionSelector() { - @Override - public TransactionSelectionResult evaluateTransactionPreProcessing( - final PendingTransaction pendingTransaction) { - if (pendingTransaction.getTransaction().equals(notSelectedTransient)) - return TransactionSelectionResult.invalidTransient("transient"); - if (pendingTransaction.getTransaction().equals(notSelectedInvalid)) - return TransactionSelectionResult.invalid("invalid"); - return TransactionSelectionResult.SELECTED; - } - - @Override - public TransactionSelectionResult evaluateTransactionPostProcessing( - final PendingTransaction pendingTransaction, - final org.hyperledger.besu.plugin.data.TransactionProcessingResult - processingResult) { - return TransactionSelectionResult.SELECTED; - } - }); + new PluginTransactionSelector() { + @Override + public TransactionSelectionResult evaluateTransactionPreProcessing( + final PendingTransaction pendingTransaction) { + if (pendingTransaction.getTransaction().equals(notSelectedTransient)) + return TransactionSelectionResult.invalidTransient("transient"); + if (pendingTransaction.getTransaction().equals(notSelectedInvalid)) + return TransactionSelectionResult.invalid("invalid"); + return TransactionSelectionResult.SELECTED; + } + + @Override + public TransactionSelectionResult evaluateTransactionPostProcessing( + final PendingTransaction pendingTransaction, + final org.hyperledger.besu.plugin.data.TransactionProcessingResult + processingResult) { + return TransactionSelectionResult.SELECTED; + } + }; final Address miningBeneficiary = AddressHelpers.ofValue(1); final BlockTransactionSelector selector = @@ -617,29 +621,27 @@ public void transactionSelectionPluginShouldWork_PostProcessing() { final Transaction selected3 = createTransaction(3, Wei.of(10), 21_000); ensureTransactionIsValid(selected3, maxGasUsedByTransaction, 0); - final TransactionSelectorFactory transactionSelectorFactory = + final PluginTransactionSelectorFactory transactionSelectorFactory = () -> - List.of( - new TransactionSelector() { - @Override - public TransactionSelectionResult evaluateTransactionPreProcessing( - final PendingTransaction pendingTransaction) { - return TransactionSelectionResult.SELECTED; - } - - @Override - public TransactionSelectionResult evaluateTransactionPostProcessing( - final PendingTransaction pendingTransaction, - final org.hyperledger.besu.plugin.data.TransactionProcessingResult - processingResult) { - // the transaction with max gas +1 should fail - if (processingResult.getEstimateGasUsedByTransaction() - > maxGasUsedByTransaction) { - return TransactionSelectionResult.invalidTransient("Invalid"); - } - return TransactionSelectionResult.SELECTED; - } - }); + new PluginTransactionSelector() { + @Override + public TransactionSelectionResult evaluateTransactionPreProcessing( + final PendingTransaction pendingTransaction) { + return TransactionSelectionResult.SELECTED; + } + + @Override + public TransactionSelectionResult evaluateTransactionPostProcessing( + final PendingTransaction pendingTransaction, + final org.hyperledger.besu.plugin.data.TransactionProcessingResult + processingResult) { + // the transaction with max gas +1 should fail + if (processingResult.getEstimateGasUsedByTransaction() > maxGasUsedByTransaction) { + return TransactionSelectionResult.invalidTransient("Invalid"); + } + return TransactionSelectionResult.SELECTED; + } + }; final Address miningBeneficiary = AddressHelpers.ofValue(1); final BlockTransactionSelector selector = @@ -662,6 +664,49 @@ public TransactionSelectionResult evaluateTransactionPostProcessing( .containsOnly(entry(notSelected, TransactionSelectionResult.invalidTransient("Invalid"))); } + @Test + public void transactionSelectionPluginShouldBeNotifiedWhenTransactionSelectionCompletes() { + final PluginTransactionSelectorFactory transactionSelectorFactory = + mock(PluginTransactionSelectorFactory.class); + PluginTransactionSelector transactionSelector = spy(AllAcceptingTransactionSelector.INSTANCE); + when(transactionSelectorFactory.create()).thenReturn(transactionSelector); + + final Transaction transaction = createTransaction(0, Wei.of(10), 21_000); + ensureTransactionIsValid(transaction, 21_000, 0); + + final TransactionInvalidReason invalidReason = TransactionInvalidReason.PLUGIN_TX_VALIDATOR; + final Transaction invalidTransaction = createTransaction(1, Wei.of(10), 21_000); + ensureTransactionIsInvalid(invalidTransaction, TransactionInvalidReason.PLUGIN_TX_VALIDATOR); + transactionPool.addRemoteTransactions(List.of(transaction, invalidTransaction)); + + createBlockSelectorWithTxSelPlugin( + transactionProcessor, + createBlock(300_000), + Wei.ZERO, + AddressHelpers.ofValue(1), + Wei.ZERO, + MIN_OCCUPANCY_80_PERCENT, + transactionSelectorFactory) + .buildTransactionListForBlock(); + + ArgumentCaptor argumentCaptor = + ArgumentCaptor.forClass(PendingTransaction.class); + + // selected transaction must be notified to the selector + verify(transactionSelector) + .onTransactionSelected(argumentCaptor.capture(), any(TransactionProcessingResult.class)); + PendingTransaction selected = argumentCaptor.getValue(); + assertThat(selected.getTransaction()).isEqualTo(transaction); + + // unselected transaction must be notified to the selector with correct reason + verify(transactionSelector) + .onTransactionNotSelected( + argumentCaptor.capture(), + eq(TransactionSelectionResult.invalid(invalidReason.toString()))); + PendingTransaction rejectedTransaction = argumentCaptor.getValue(); + assertThat(rejectedTransaction.getTransaction()).isEqualTo(invalidTransaction); + } + @Test public void transactionWithIncorrectNonceRemainsInPoolAndNotSelected() { final ProcessableBlockHeader blockHeader = createBlock(5_000_000); @@ -728,7 +773,7 @@ protected BlockTransactionSelector createBlockSelectorWithTxSelPlugin( final Address miningBeneficiary, final Wei blobGasPrice, final double minBlockOccupancyRatio, - final TransactionSelectorFactory transactionSelectorFactory) { + final PluginTransactionSelectorFactory transactionSelectorFactory) { final BlockTransactionSelector selector = new BlockTransactionSelector( transactionProcessor, @@ -809,7 +854,7 @@ protected void ensureTransactionIsValid(final Transaction tx) { protected void ensureTransactionIsValid( final Transaction tx, final long gasUsedByTransaction, final long gasRemaining) { when(transactionProcessor.processTransaction( - any(), any(), any(), eq(tx), any(), any(), anyBoolean(), any(), any())) + any(), any(), any(), eq(tx), any(), any(), any(), anyBoolean(), any(), any())) .thenReturn( TransactionProcessingResult.successful( new ArrayList<>(), @@ -822,7 +867,7 @@ protected void ensureTransactionIsValid( protected void ensureTransactionIsInvalid( final Transaction tx, final TransactionInvalidReason invalidReason) { when(transactionProcessor.processTransaction( - any(), any(), any(), eq(tx), any(), any(), anyBoolean(), any(), any())) + any(), any(), any(), eq(tx), any(), any(), any(), anyBoolean(), any(), any())) .thenReturn(TransactionProcessingResult.invalid(ValidationResult.invalid(invalidReason))); } diff --git a/ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/LondonFeeMarketBlockTransactionSelectorTest.java b/ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/LondonFeeMarketBlockTransactionSelectorTest.java index 29860d555a6..57b89f8a1d7 100644 --- a/ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/LondonFeeMarketBlockTransactionSelectorTest.java +++ b/ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/LondonFeeMarketBlockTransactionSelectorTest.java @@ -155,7 +155,7 @@ public void eip1559TransactionCurrentGasPriceGreaterThanMinimumIsSelected() { } @Test - public void eip1559LocalTransactionCurrentGasPriceLessThanMinimumIsSelected() { + public void eip1559PriorityTransactionCurrentGasPriceLessThanMinimumIsSelected() { final ProcessableBlockHeader blockHeader = createBlock(301_000, Wei.ONE); final Address miningBeneficiary = AddressHelpers.ofValue(1); diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/ProtocolContext.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/ProtocolContext.java index 8c0fc460dc2..745c5640689 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/ProtocolContext.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/ProtocolContext.java @@ -18,7 +18,7 @@ import org.hyperledger.besu.ethereum.core.Synchronizer; import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule; import org.hyperledger.besu.ethereum.worldstate.WorldStateArchive; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory; import java.util.Optional; @@ -31,7 +31,7 @@ public class ProtocolContext { private final MutableBlockchain blockchain; private final WorldStateArchive worldStateArchive; private final ConsensusContext consensusContext; - private final Optional transactionSelectorFactory; + private final Optional transactionSelectorFactory; private Optional synchronizer; @@ -46,7 +46,7 @@ public ProtocolContext( final MutableBlockchain blockchain, final WorldStateArchive worldStateArchive, final ConsensusContext consensusContext, - final Optional transactionSelectorFactory) { + final Optional transactionSelectorFactory) { this.blockchain = blockchain; this.worldStateArchive = worldStateArchive; this.consensusContext = consensusContext; @@ -59,7 +59,7 @@ public static ProtocolContext init( final WorldStateArchive worldStateArchive, final ProtocolSchedule protocolSchedule, final ConsensusContextFactory consensusContextFactory, - final Optional transactionSelectorFactory) { + final Optional transactionSelectorFactory) { return new ProtocolContext( blockchain, worldStateArchive, @@ -93,7 +93,7 @@ public Optional safeConsensusContext(final Class .map(klass::cast); } - public Optional getTransactionSelectorFactory() { + public Optional getTransactionSelectorFactory() { return transactionSelectorFactory; } } diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchain.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchain.java index 22c60862450..cbde737684b 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchain.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchain.java @@ -31,6 +31,7 @@ import org.hyperledger.besu.ethereum.core.Transaction; import org.hyperledger.besu.ethereum.core.TransactionReceipt; import org.hyperledger.besu.metrics.BesuMetricCategory; +import org.hyperledger.besu.metrics.prometheus.PrometheusMetricsSystem; import org.hyperledger.besu.plugin.services.MetricsSystem; import org.hyperledger.besu.util.InvalidConfigurationException; import org.hyperledger.besu.util.Subscribers; @@ -49,8 +50,11 @@ import java.util.stream.Stream; import com.google.common.annotations.VisibleForTesting; +import com.google.common.cache.Cache; +import com.google.common.cache.CacheBuilder; import com.google.common.collect.Lists; import com.google.common.collect.Streams; +import io.prometheus.client.guava.cache.CacheMetricsCollector; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -73,12 +77,18 @@ public class DefaultBlockchain implements MutableBlockchain { private Comparator blockChoiceRule; + private final int numberOfBlocksToCache; + private final Optional> blockHeadersCache; + private final Optional> blockBodiesCache; + private final Optional>> transactionReceiptsCache; + private final Optional> totalDifficultyCache; + private DefaultBlockchain( final Optional genesisBlock, final BlockchainStorage blockchainStorage, final MetricsSystem metricsSystem, final long reorgLoggingThreshold) { - this(genesisBlock, blockchainStorage, metricsSystem, reorgLoggingThreshold, null); + this(genesisBlock, blockchainStorage, metricsSystem, reorgLoggingThreshold, null, 0); } private DefaultBlockchain( @@ -86,7 +96,8 @@ private DefaultBlockchain( final BlockchainStorage blockchainStorage, final MetricsSystem metricsSystem, final long reorgLoggingThreshold, - final String dataDirectory) { + final String dataDirectory, + final int numberOfBlocksToCache) { checkNotNull(genesisBlock); checkNotNull(blockchainStorage); checkNotNull(metricsSystem); @@ -144,6 +155,34 @@ private DefaultBlockchain( this.reorgLoggingThreshold = reorgLoggingThreshold; this.blockChoiceRule = heaviestChainBlockChoiceRule; + this.numberOfBlocksToCache = numberOfBlocksToCache; + + if (numberOfBlocksToCache != 0) { + blockHeadersCache = + Optional.of( + CacheBuilder.newBuilder().recordStats().maximumSize(numberOfBlocksToCache).build()); + blockBodiesCache = + Optional.of( + CacheBuilder.newBuilder().recordStats().maximumSize(numberOfBlocksToCache).build()); + transactionReceiptsCache = + Optional.of( + CacheBuilder.newBuilder().recordStats().maximumSize(numberOfBlocksToCache).build()); + totalDifficultyCache = + Optional.of( + CacheBuilder.newBuilder().recordStats().maximumSize(numberOfBlocksToCache).build()); + CacheMetricsCollector cacheMetrics = new CacheMetricsCollector(); + cacheMetrics.addCache("blockHeaders", blockHeadersCache.get()); + cacheMetrics.addCache("blockBodies", blockBodiesCache.get()); + cacheMetrics.addCache("transactionReceipts", transactionReceiptsCache.get()); + cacheMetrics.addCache("totalDifficulty", totalDifficultyCache.get()); + if (metricsSystem instanceof PrometheusMetricsSystem prometheusMetricsSystem) + prometheusMetricsSystem.addCollector(BesuMetricCategory.BLOCKCHAIN, () -> cacheMetrics); + } else { + blockHeadersCache = Optional.empty(); + blockBodiesCache = Optional.empty(); + transactionReceiptsCache = Optional.empty(); + totalDifficultyCache = Optional.empty(); + } } public static MutableBlockchain createMutable( @@ -153,7 +192,12 @@ public static MutableBlockchain createMutable( final long reorgLoggingThreshold) { checkNotNull(genesisBlock); return new DefaultBlockchain( - Optional.of(genesisBlock), blockchainStorage, metricsSystem, reorgLoggingThreshold); + Optional.of(genesisBlock), + blockchainStorage, + metricsSystem, + reorgLoggingThreshold, + null, + 0); } public static MutableBlockchain createMutable( @@ -168,7 +212,25 @@ public static MutableBlockchain createMutable( blockchainStorage, metricsSystem, reorgLoggingThreshold, - dataDirectory); + dataDirectory, + 0); + } + + public static MutableBlockchain createMutable( + final Block genesisBlock, + final BlockchainStorage blockchainStorage, + final MetricsSystem metricsSystem, + final long reorgLoggingThreshold, + final String dataDirectory, + final int numberOfBlocksToCache) { + checkNotNull(genesisBlock); + return new DefaultBlockchain( + Optional.of(genesisBlock), + blockchainStorage, + metricsSystem, + reorgLoggingThreshold, + dataDirectory, + numberOfBlocksToCache); } public static Blockchain create( @@ -227,22 +289,37 @@ public Block getChainHeadBlock() { @Override public Optional getBlockHeader(final long blockNumber) { - return blockchainStorage.getBlockHash(blockNumber).flatMap(blockchainStorage::getBlockHeader); + return blockchainStorage.getBlockHash(blockNumber).flatMap(this::getBlockHeader); } @Override public Optional getBlockHeader(final Hash blockHeaderHash) { - return blockchainStorage.getBlockHeader(blockHeaderHash); + return blockHeadersCache + .map( + cache -> + Optional.ofNullable(cache.getIfPresent(blockHeaderHash)) + .or(() -> blockchainStorage.getBlockHeader(blockHeaderHash))) + .orElseGet(() -> blockchainStorage.getBlockHeader(blockHeaderHash)); } @Override public Optional getBlockBody(final Hash blockHeaderHash) { - return blockchainStorage.getBlockBody(blockHeaderHash); + return blockBodiesCache + .map( + cache -> + Optional.ofNullable(cache.getIfPresent(blockHeaderHash)) + .or(() -> blockchainStorage.getBlockBody(blockHeaderHash))) + .orElseGet(() -> blockchainStorage.getBlockBody(blockHeaderHash)); } @Override public Optional> getTxReceipts(final Hash blockHeaderHash) { - return blockchainStorage.getTransactionReceipts(blockHeaderHash); + return transactionReceiptsCache + .map( + cache -> + Optional.ofNullable(cache.getIfPresent(blockHeaderHash)) + .or(() -> blockchainStorage.getTransactionReceipts(blockHeaderHash))) + .orElseGet(() -> blockchainStorage.getTransactionReceipts(blockHeaderHash)); } @Override @@ -252,7 +329,12 @@ public Optional getBlockHashByNumber(final long number) { @Override public Optional getTotalDifficultyByHash(final Hash blockHeaderHash) { - return blockchainStorage.getTotalDifficulty(blockHeaderHash); + return totalDifficultyCache + .map( + cache -> + Optional.ofNullable(cache.getIfPresent(blockHeaderHash)) + .or(() -> blockchainStorage.getTotalDifficulty(blockHeaderHash))) + .orElseGet(() -> blockchainStorage.getTotalDifficulty(blockHeaderHash)); } @Override @@ -283,14 +365,24 @@ public void setBlockChoiceRule(final Comparator blockChoiceRule) { @Override public synchronized void appendBlock(final Block block, final List receipts) { + if (numberOfBlocksToCache != 0) cacheBlockData(block, receipts); appendBlockHelper(new BlockWithReceipts(block, receipts), false); } @Override public synchronized void storeBlock(final Block block, final List receipts) { + if (numberOfBlocksToCache != 0) cacheBlockData(block, receipts); appendBlockHelper(new BlockWithReceipts(block, receipts), true); } + private void cacheBlockData(final Block block, final List receipts) { + blockHeadersCache.ifPresent(cache -> cache.put(block.getHash(), block.getHeader())); + blockBodiesCache.ifPresent(cache -> cache.put(block.getHash(), block.getBody())); + transactionReceiptsCache.ifPresent(cache -> cache.put(block.getHash(), receipts)); + totalDifficultyCache.ifPresent( + cache -> cache.put(block.getHash(), block.getHeader().getDifficulty())); + } + private boolean blockShouldBeProcessed( final Block block, final List receipts) { checkArgument( @@ -768,4 +860,20 @@ int observerCount() { private void notifyChainReorgBlockAdded(final BlockWithReceipts blockWithReceipts) { blockReorgObservers.forEach(observer -> observer.onBlockAdded(blockWithReceipts, this)); } + + public Optional> getBlockHeadersCache() { + return blockHeadersCache; + } + + public Optional> getBlockBodiesCache() { + return blockBodiesCache; + } + + public Optional>> getTransactionReceiptsCache() { + return transactionReceiptsCache; + } + + public Optional> getTotalDifficultyCache() { + return totalDifficultyCache; + } } diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/BlockHeaderBuilder.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/BlockHeaderBuilder.java index e218bf1d25b..fc38b663ea9 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/BlockHeaderBuilder.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/BlockHeaderBuilder.java @@ -238,7 +238,6 @@ private void validateProcessableBlockHeader() { checkState(this.difficulty != null, "Missing block difficulty"); checkState(this.number > -1L, "Missing block number"); checkState(this.gasLimit > -1L, "Missing gas limit"); - checkState(this.timestamp > -1L, "Missing timestamp"); } private void validateSealableBlockHeader() { @@ -360,7 +359,6 @@ public BlockHeaderBuilder gasUsed(final long gasUsed) { } public BlockHeaderBuilder timestamp(final long timestamp) { - checkArgument(timestamp >= 0); this.timestamp = timestamp; return this; } diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Deposit.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Deposit.java index b01d6deee32..ed362199b23 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Deposit.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Deposit.java @@ -32,19 +32,19 @@ public class Deposit implements org.hyperledger.besu.plugin.data.Deposit { - private final BLSPublicKey pubKey; + private final BLSPublicKey pubkey; private final Bytes32 depositWithdrawalCredentials; private final GWei amount; private final BLSSignature signature; private final UInt64 index; public Deposit( - final BLSPublicKey pubKey, + final BLSPublicKey pubkey, final Bytes32 depositWithdrawalCredentials, final GWei amount, final BLSSignature signature, final UInt64 index) { - this.pubKey = pubKey; + this.pubkey = pubkey; this.depositWithdrawalCredentials = depositWithdrawalCredentials; this.amount = amount; this.signature = signature; @@ -64,8 +64,8 @@ public void writeTo(final RLPOutput out) { } @Override - public PublicKey getPublicKey() { - return pubKey; + public PublicKey getPubkey() { + return pubkey; } @Override @@ -92,7 +92,7 @@ public UInt64 getIndex() { public String toString() { return "Deposit{" + "pubKey=" - + pubKey + + pubkey + ", withdrawalCredentials=" + depositWithdrawalCredentials + ", amount=" @@ -109,7 +109,7 @@ public boolean equals(final Object o) { if (this == o) return true; if (o == null || getClass() != o.getClass()) return false; final Deposit that = (Deposit) o; - return Objects.equals(pubKey, that.pubKey) + return Objects.equals(pubkey, that.pubkey) && Objects.equals(depositWithdrawalCredentials, that.depositWithdrawalCredentials) && Objects.equals(amount, that.amount) && Objects.equals(signature, that.signature) @@ -118,6 +118,6 @@ public boolean equals(final Object o) { @Override public int hashCode() { - return Objects.hash(pubKey, depositWithdrawalCredentials, amount, signature, index); + return Objects.hash(pubkey, depositWithdrawalCredentials, amount, signature, index); } } diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Transaction.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Transaction.java index aeead13c362..cc54415ff5c 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Transaction.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Transaction.java @@ -134,6 +134,8 @@ public static Transaction readFrom(final RLPInput rlpInput) { /** * Instantiates a transaction instance. * + * @param forCopy true when using to create a copy of an already validated transaction avoid to + * redo the validation * @param transactionType the transaction type * @param nonce the nonce * @param gasPrice the gas price @@ -154,7 +156,8 @@ public static Transaction readFrom(final RLPInput rlpInput) { *

The {@code chainId} must be greater than 0 to be applied to a specific chain; otherwise * it will default to any chain. */ - public Transaction( + private Transaction( + final boolean forCopy, final TransactionType transactionType, final long nonce, final Optional gasPrice, @@ -172,36 +175,40 @@ public Transaction( final Optional> versionedHashes, final Optional blobsWithCommitments) { - if (transactionType.requiresChainId()) { - checkArgument( - chainId.isPresent(), "Chain id must be present for transaction type %s", transactionType); - } + if (!forCopy) { + if (transactionType.requiresChainId()) { + checkArgument( + chainId.isPresent(), + "Chain id must be present for transaction type %s", + transactionType); + } - if (maybeAccessList.isPresent()) { - checkArgument( - transactionType.supportsAccessList(), - "Must not specify access list for transaction not supporting it"); - } + if (maybeAccessList.isPresent()) { + checkArgument( + transactionType.supportsAccessList(), + "Must not specify access list for transaction not supporting it"); + } - if (Objects.equals(transactionType, TransactionType.ACCESS_LIST)) { - checkArgument( - maybeAccessList.isPresent(), "Must specify access list for access list transaction"); - } + if (Objects.equals(transactionType, TransactionType.ACCESS_LIST)) { + checkArgument( + maybeAccessList.isPresent(), "Must specify access list for access list transaction"); + } - if (versionedHashes.isPresent() || maxFeePerBlobGas.isPresent()) { - checkArgument( - transactionType.supportsBlob(), - "Must not specify blob versioned hashes or max fee per blob gas for transaction not supporting it"); - } + if (versionedHashes.isPresent() || maxFeePerBlobGas.isPresent()) { + checkArgument( + transactionType.supportsBlob(), + "Must not specify blob versioned hashes or max fee per blob gas for transaction not supporting it"); + } - if (transactionType.supportsBlob()) { - checkArgument( - versionedHashes.isPresent(), "Must specify blob versioned hashes for blob transaction"); - checkArgument( - !versionedHashes.get().isEmpty(), - "Blob transaction must have at least one versioned hash"); - checkArgument( - maxFeePerBlobGas.isPresent(), "Must specify max fee per blob gas for blob transaction"); + if (transactionType.supportsBlob()) { + checkArgument( + versionedHashes.isPresent(), "Must specify blob versioned hashes for blob transaction"); + checkArgument( + !versionedHashes.get().isEmpty(), + "Blob transaction must have at least one versioned hash"); + checkArgument( + maxFeePerBlobGas.isPresent(), "Must specify max fee per blob gas for blob transaction"); + } } this.transactionType = transactionType; @@ -221,7 +228,7 @@ public Transaction( this.versionedHashes = versionedHashes; this.blobsWithCommitments = blobsWithCommitments; - if (isUpfrontGasCostTooHigh()) { + if (!forCopy && isUpfrontGasCostTooHigh()) { throw new IllegalArgumentException("Upfront gas cost exceeds UInt256"); } } @@ -920,17 +927,19 @@ public String toString() { sb.append("nonce=").append(getNonce()).append(", "); getGasPrice() .ifPresent( - gasPrice -> sb.append("gasPrice=").append(gasPrice.toShortHexString()).append(", ")); + gasPrice -> + sb.append("gasPrice=").append(gasPrice.toHumanReadableString()).append(", ")); if (getMaxPriorityFeePerGas().isPresent() && getMaxFeePerGas().isPresent()) { sb.append("maxPriorityFeePerGas=") - .append(getMaxPriorityFeePerGas().map(Wei::toShortHexString).get()) + .append(getMaxPriorityFeePerGas().map(Wei::toHumanReadableString).get()) .append(", "); sb.append("maxFeePerGas=") - .append(getMaxFeePerGas().map(Wei::toShortHexString).get()) + .append(getMaxFeePerGas().map(Wei::toHumanReadableString).get()) .append(", "); getMaxFeePerBlobGas() .ifPresent( - wei -> sb.append("maxFeePerBlobGas=").append(wei.toShortHexString()).append(", ")); + wei -> + sb.append("maxFeePerBlobGas=").append(wei.toHumanReadableString()).append(", ")); } sb.append("gasLimit=").append(getGasLimit()).append(", "); if (getTo().isPresent()) sb.append("to=").append(getTo().get()).append(", "); @@ -986,7 +995,7 @@ public String toTraceLog() { } sb.append("gl: ").append(getGasLimit()).append(", "); sb.append("v: ").append(getValue().toHumanReadableString()).append(", "); - getTo().ifPresent(to -> sb.append(to)); + getTo().ifPresent(to -> sb.append("to: ").append(to)); return sb.append("}").toString(); } @@ -998,6 +1007,84 @@ public Optional

contractAddress() { return Optional.empty(); } + /** + * Creates a copy of this transaction that does not share any underlying byte array. + * + *

This is useful in case the transaction is built from a block body and fields, like to or + * payload, are wrapping (and so keeping references) sections of the large RPL encoded block body, + * and we plan to keep the transaction around for some time, like in the txpool in case of a + * reorg, and do not want to keep all the block body in memory for a long time, but only the + * actual transaction. + * + * @return a copy of the transaction + */ + public Transaction detachedCopy() { + final Optional

detachedTo = + to.isEmpty() ? to : Optional.of(Address.wrap(to.get().copy())); + final Optional> detachedAccessList = + maybeAccessList.isEmpty() + ? maybeAccessList + : Optional.of( + maybeAccessList.get().stream().map(this::accessListDetachedCopy).toList()); + final Optional> detachedVersionedHashes = + versionedHashes.isEmpty() + ? versionedHashes + : Optional.of( + versionedHashes.get().stream() + .map(vh -> new VersionedHash(vh.toBytes().copy())) + .toList()); + final Optional detachedBlobsWithCommitments = + blobsWithCommitments.isEmpty() + ? blobsWithCommitments + : Optional.of( + blobsWithCommitmentsDetachedCopy( + blobsWithCommitments.get(), detachedVersionedHashes.get())); + + return new Transaction( + true, + transactionType, + nonce, + gasPrice, + maxPriorityFeePerGas, + maxFeePerGas, + maxFeePerBlobGas, + gasLimit, + detachedTo, + value, + signature, + payload.copy(), + detachedAccessList, + sender, + chainId, + detachedVersionedHashes, + detachedBlobsWithCommitments); + } + + private AccessListEntry accessListDetachedCopy(final AccessListEntry accessListEntry) { + final Address detachedAddress = Address.wrap(accessListEntry.address().copy()); + final var detachedStorage = accessListEntry.storageKeys().stream().map(Bytes32::copy).toList(); + return new AccessListEntry(detachedAddress, detachedStorage); + } + + private BlobsWithCommitments blobsWithCommitmentsDetachedCopy( + final BlobsWithCommitments blobsWithCommitments, final List versionedHashes) { + final var detachedCommitments = + blobsWithCommitments.getKzgCommitments().stream() + .map(kc -> new KZGCommitment(kc.getData().copy())) + .toList(); + final var detachedBlobs = + blobsWithCommitments.getBlobs().stream() + .map(blob -> new Blob(blob.getData().copy())) + .toList(); + final var detachedProofs = + blobsWithCommitments.getKzgProofs().stream() + .map(proof -> new KZGProof(proof.getData().copy())) + .toList(); + + return new BlobsWithCommitments( + detachedCommitments, detachedBlobs, detachedProofs, versionedHashes); + } + public static class Builder { private static final Optional> EMPTY_ACCESS_LIST = Optional.of(List.of()); @@ -1134,6 +1221,7 @@ public TransactionType getTransactionType() { public Transaction build() { if (transactionType == null) guessType(); return new Transaction( + false, transactionType, nonce, Optional.ofNullable(gasPrice), diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/encoding/BlobPooledTransactionDecoder.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/encoding/BlobPooledTransactionDecoder.java index 8f2efde53f8..1ccd5c4ad42 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/encoding/BlobPooledTransactionDecoder.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/encoding/BlobPooledTransactionDecoder.java @@ -24,9 +24,9 @@ /** * Class responsible for decoding blob transactions from the transaction pool. Blob transactions - * have two network representations. During transaction gossip responses (PooledTransactions), the - * EIP-2718 TransactionPayload of the blob transaction is wrapped to become: rlp([tx_payload_body, - * blobs, commitments, proofs]). + * have two representations. The network representation is used during transaction gossip responses + * (PooledTransactions), the EIP-2718 TransactionPayload of the blob transaction is wrapped to + * become: rlp([tx_payload_body, blobs, commitments, proofs]). */ public class BlobPooledTransactionDecoder { diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/encoding/DepositEncoder.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/encoding/DepositEncoder.java index d5171d18f79..15f825e0757 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/encoding/DepositEncoder.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/encoding/DepositEncoder.java @@ -24,7 +24,7 @@ public class DepositEncoder { public static void encode(final Deposit deposit, final RLPOutput rlpOutput) { rlpOutput.startList(); - rlpOutput.writeBytes(deposit.getPublicKey()); + rlpOutput.writeBytes(deposit.getPubkey()); rlpOutput.writeBytes(deposit.getWithdrawalCredentials()); rlpOutput.writeUInt64Scalar(deposit.getAmount()); rlpOutput.writeBytes(deposit.getSignature()); diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetProtocolSpecs.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetProtocolSpecs.java index 02f757483df..e477bf6f75e 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetProtocolSpecs.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetProtocolSpecs.java @@ -640,8 +640,6 @@ static ProtocolSpecBuilder cancunDefinition( final EvmConfiguration evmConfiguration) { final int stackSizeLimit = configStackSizeLimit.orElse(MessageFrame.DEFAULT_MAX_STACK_SIZE); - final int contractSizeLimit = - configContractSizeLimit.orElse(SPURIOUS_DRAGON_CONTRACT_SIZE_LIMIT); final long londonForkBlockNumber = genesisConfigOptions.getLondonBlockNumber().orElse(0L); final BaseFeeMarket cancunFeeMarket = genesisConfigOptions.isZeroBaseFee() @@ -668,17 +666,6 @@ static ProtocolSpecBuilder cancunDefinition( (gasCalculator, jdCacheConfig) -> MainnetEVMs.cancun( gasCalculator, chainId.orElse(BigInteger.ZERO), evmConfiguration)) - // change contract call creator to accept EOF code - .contractCreationProcessorBuilder( - (gasCalculator, evm) -> - new ContractCreationProcessor( - gasCalculator, - evm, - true, - List.of( - MaxCodeSizeRule.of(contractSizeLimit), EOFValidationCodeRule.of(1, false)), - 1, - SPURIOUS_DRAGON_FORCE_DELETE_WHEN_EMPTY_ADDRESSES)) // use Cancun fee market .transactionProcessorBuilder( (gasCalculator, @@ -723,7 +710,8 @@ static ProtocolSpecBuilder futureEipsDefinition( final boolean enableRevertReason, final GenesisConfigOptions genesisConfigOptions, final EvmConfiguration evmConfiguration) { - + final int contractSizeLimit = + configContractSizeLimit.orElse(SPURIOUS_DRAGON_CONTRACT_SIZE_LIMIT); return cancunDefinition( chainId, configContractSizeLimit, @@ -731,10 +719,23 @@ static ProtocolSpecBuilder futureEipsDefinition( enableRevertReason, genesisConfigOptions, evmConfiguration) + // Use Future EIP configured EVM .evmBuilder( (gasCalculator, jdCacheConfig) -> MainnetEVMs.futureEips( gasCalculator, chainId.orElse(BigInteger.ZERO), evmConfiguration)) + // change contract call creator to accept EOF code + .contractCreationProcessorBuilder( + (gasCalculator, evm) -> + new ContractCreationProcessor( + gasCalculator, + evm, + true, + List.of( + MaxCodeSizeRule.of(contractSizeLimit), EOFValidationCodeRule.of(1, false)), + 1, + SPURIOUS_DRAGON_FORCE_DELETE_WHEN_EMPTY_ADDRESSES)) + // use future configured precompiled contracts .precompileContractRegistryBuilder(MainnetPrecompiledContractRegistries::futureEips) .name("FutureEips"); } diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionProcessor.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionProcessor.java index 927e4cd8a29..cdeec19b117 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionProcessor.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionProcessor.java @@ -339,12 +339,11 @@ public TransactionProcessingResult processTransaction( gasCalculator.accessListGasCost(accessListEntries.size(), accessListStorageCount); final long gasAvailable = transaction.getGasLimit() - intrinsicGas - accessListGas; LOG.trace( - "Gas available for execution {} = {} - {} - {} - {} (limit - intrinsic - accessList - data)", + "Gas available for execution {} = {} - {} - {} (limit - intrinsic - accessList)", gasAvailable, transaction.getGasLimit(), intrinsicGas, - accessListGas, - blobGas); + accessListGas); final WorldUpdater worldUpdater = worldState.updater(); final ImmutableMap.Builder contextVariablesBuilder = diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java index cfcd411bf59..339e4c1d9c3 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java @@ -22,7 +22,6 @@ import org.hyperledger.besu.datatypes.BlobsWithCommitments; import org.hyperledger.besu.datatypes.Hash; import org.hyperledger.besu.datatypes.KZGCommitment; -import org.hyperledger.besu.datatypes.KZGProof; import org.hyperledger.besu.datatypes.TransactionType; import org.hyperledger.besu.datatypes.VersionedHash; import org.hyperledger.besu.datatypes.Wei; @@ -51,8 +50,6 @@ */ public class MainnetTransactionValidator implements TransactionValidator { - private static final byte BLOB_COMMITMENT_VERSION_KZG = 0x01; - private final GasCalculator gasCalculator; private final GasLimitCalculator gasLimitCalculator; private final FeeMarket feeMarket; @@ -173,17 +170,6 @@ private ValidationResult validateCostAndFee( } } - if (transaction.getType().supportsBlob()) { - final long txTotalBlobGas = gasCalculator.blobGasCost(transaction.getBlobCount()); - if (txTotalBlobGas > gasLimitCalculator.currentBlobGasLimit()) { - return ValidationResult.invalid( - TransactionInvalidReason.TOTAL_BLOB_GAS_TOO_HIGH, - String.format( - "total blob gas %d exceeds max blob gas per block %d", - txTotalBlobGas, gasLimitCalculator.currentBlobGasLimit())); - } - } - final long intrinsicGasCost = gasCalculator.transactionIntrinsicGasCost( transaction.getPayload(), transaction.isContractCreation()) @@ -336,11 +322,11 @@ public ValidationResult validateTransactionsBlobs( final KZGCommitment commitment = blobsWithCommitments.getKzgCommitments().get(i); final VersionedHash versionedHash = versionedHashes.get(i); - if (versionedHash.getVersionId() != BLOB_COMMITMENT_VERSION_KZG) { + if (versionedHash.getVersionId() != VersionedHash.SHA256_VERSION_ID) { return ValidationResult.invalid( TransactionInvalidReason.INVALID_BLOBS, "transaction blobs commitment version is not supported. Expected " - + BLOB_COMMITMENT_VERSION_KZG + + VersionedHash.SHA256_VERSION_ID + ", found " + versionedHash.getVersionId()); } @@ -353,30 +339,27 @@ public ValidationResult validateTransactionsBlobs( } } - final Bytes blobs = - blobsWithCommitments.getBlobs().stream() - .map(Blob::getData) - .reduce(Bytes::concatenate) - .orElseThrow(); + final byte[] blobs = + Bytes.wrap(blobsWithCommitments.getBlobs().stream().map(Blob::getData).toList()) + .toArrayUnsafe(); - final Bytes kzgCommitments = - blobsWithCommitments.getKzgCommitments().stream() - .map(KZGCommitment::getData) - .reduce(Bytes::concatenate) - .orElseThrow(); + final byte[] kzgCommitments = + Bytes.wrap( + blobsWithCommitments.getKzgCommitments().stream() + .map(kc -> (Bytes) kc.getData()) + .toList()) + .toArrayUnsafe(); - final Bytes kzgProofs = - blobsWithCommitments.getKzgProofs().stream() - .map(KZGProof::getData) - .reduce(Bytes::concatenate) - .orElseThrow(); + final byte[] kzgProofs = + Bytes.wrap( + blobsWithCommitments.getKzgProofs().stream() + .map(kp -> (Bytes) kp.getData()) + .toList()) + .toArrayUnsafe(); final boolean kzgVerification = CKZG4844JNI.verifyBlobKzgProofBatch( - blobs.toArrayUnsafe(), - kzgCommitments.toArrayUnsafe(), - kzgProofs.toArrayUnsafe(), - blobsWithCommitments.getBlobs().size()); + blobs, kzgCommitments, kzgProofs, blobsWithCommitments.getBlobs().size()); if (!kzgVerification) { return ValidationResult.invalid( @@ -387,14 +370,6 @@ public ValidationResult validateTransactionsBlobs( return ValidationResult.valid(); } - /* - private VersionedHash hashCommitment(final Bytes32 commitment) { - return new VersionedHash( - VersionedHash.SHA256_VERSION_ID, Sha256Hash.hash(commitment)); - } - - */ - private VersionedHash hashCommitment(final KZGCommitment commitment) { final SHA256Digest digest = new SHA256Digest(); digest.update(commitment.getData().toArrayUnsafe(), 0, commitment.getData().size()); @@ -403,7 +378,7 @@ private VersionedHash hashCommitment(final KZGCommitment commitment) { digest.doFinal(dig, 0); - dig[0] = BLOB_COMMITMENT_VERSION_KZG; + dig[0] = VersionedHash.SHA256_VERSION_ID; return new VersionedHash(Bytes32.wrap(dig)); } } diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/ParentBeaconBlockRootHelper.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/ParentBeaconBlockRootHelper.java index 80eb7a19e9f..5f257c3dd32 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/ParentBeaconBlockRootHelper.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/ParentBeaconBlockRootHelper.java @@ -15,9 +15,12 @@ package org.hyperledger.besu.ethereum.mainnet; import org.hyperledger.besu.datatypes.Address; +import org.hyperledger.besu.datatypes.Hash; import org.hyperledger.besu.evm.account.MutableAccount; import org.hyperledger.besu.evm.worldstate.WorldUpdater; +import com.google.common.primitives.Longs; +import org.apache.tuweni.bytes.Bytes; import org.apache.tuweni.bytes.Bytes32; import org.apache.tuweni.units.bigints.UInt256; @@ -25,8 +28,8 @@ public interface ParentBeaconBlockRootHelper { // Modulus to use for the timestamp to store the root - public static final long HISTORY_BUFFER_LENGTH = 8191; - public static final Address BEACON_ROOTS_ADDRESS = + long HISTORY_BUFFER_LENGTH = 8191; + Address BEACON_ROOTS_ADDRESS = Address.fromHexString("0x000F3df6D732807Ef1319fB7B8bB8522d0Beac02"); static void storeParentBeaconBlockRoot( @@ -34,14 +37,20 @@ static void storeParentBeaconBlockRoot( /* see EIP-4788: https://github.com/ethereum/EIPs/blob/master/EIPS/eip-4788.md */ - final long timestampReduced = timestamp % HISTORY_BUFFER_LENGTH; + // If code is not deployed don't do anything + final MutableAccount account = worldUpdater.getOrCreate(BEACON_ROOTS_ADDRESS); + if (Hash.EMPTY.equals(account.getCodeHash())) { + return; + } + + final long timestampReduced = Long.remainderUnsigned(timestamp, HISTORY_BUFFER_LENGTH); final long timestampExtended = timestampReduced + HISTORY_BUFFER_LENGTH; final UInt256 timestampIndex = UInt256.valueOf(timestampReduced); final UInt256 rootIndex = UInt256.valueOf(timestampExtended); - final MutableAccount account = worldUpdater.getOrCreate(BEACON_ROOTS_ADDRESS); - account.setStorageValue(timestampIndex, UInt256.valueOf(timestamp)); + account.setStorageValue( + timestampIndex, UInt256.fromBytes(Bytes.of(Longs.toByteArray(timestamp)))); account.setStorageValue(rootIndex, UInt256.fromBytes(root)); worldUpdater.commit(); } diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/ScheduledProtocolSpec.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/ScheduledProtocolSpec.java index 012bb469c6a..d03a3fe90ef 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/ScheduledProtocolSpec.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/ScheduledProtocolSpec.java @@ -62,7 +62,7 @@ private TimestampProtocolSpec(final long timestamp, final ProtocolSpec protocolS @Override public boolean isOnOrAfterMilestoneBoundary(final ProcessableBlockHeader header) { - return header.getTimestamp() >= timestamp; + return Long.compareUnsigned(header.getTimestamp(), timestamp) >= 0; } @Override diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/headervalidationrules/IncrementalTimestampRule.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/headervalidationrules/IncrementalTimestampRule.java index bc29e0db9f2..72b5161a775 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/headervalidationrules/IncrementalTimestampRule.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/headervalidationrules/IncrementalTimestampRule.java @@ -28,6 +28,6 @@ public boolean validate( final long blockTimestamp = header.getTimestamp(); final long parentTimestamp = parent.getTimestamp(); - return blockTimestamp > parentTimestamp; + return Long.compareUnsigned(blockTimestamp, parentTimestamp) > 0; } } diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/transaction/TransactionInvalidReason.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/transaction/TransactionInvalidReason.java index 3971b2f502b..6b9db98dea2 100644 --- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/transaction/TransactionInvalidReason.java +++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/transaction/TransactionInvalidReason.java @@ -48,6 +48,7 @@ public enum TransactionInvalidReason { INTERNAL_ERROR, TX_POOL_DISABLED, INVALID_BLOBS, + PLUGIN_TX_VALIDATOR, // Private Transaction Invalid Reasons PRIVATE_TRANSACTION_INVALID, PRIVATE_TRANSACTION_FAILED, @@ -55,6 +56,5 @@ public enum TransactionInvalidReason { OFFCHAIN_PRIVACY_GROUP_DOES_NOT_EXIST, PRIVATE_NONCE_TOO_HIGH, PRIVATE_VALUE_NOT_ZERO, - PRIVATE_UNIMPLEMENTED_TRANSACTION_TYPE, - PLUGIN_TX_VALIDATOR_INVALIDATED + PRIVATE_UNIMPLEMENTED_TRANSACTION_TYPE } diff --git a/ethereum/core/src/test-support/java/org/hyperledger/besu/ethereum/core/BlobTestFixture.java b/ethereum/core/src/test-support/java/org/hyperledger/besu/ethereum/core/BlobTestFixture.java index 55500f0603d..d22d20023aa 100644 --- a/ethereum/core/src/test-support/java/org/hyperledger/besu/ethereum/core/BlobTestFixture.java +++ b/ethereum/core/src/test-support/java/org/hyperledger/besu/ethereum/core/BlobTestFixture.java @@ -15,7 +15,6 @@ package org.hyperledger.besu.ethereum.core; -import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.fail; import org.hyperledger.besu.datatypes.Blob; @@ -33,6 +32,7 @@ import ethereum.ckzg4844.CKZG4844JNI; import org.apache.tuweni.bytes.Bytes; import org.apache.tuweni.bytes.Bytes32; +import org.apache.tuweni.bytes.Bytes48; import org.bouncycastle.crypto.digests.SHA256Digest; public class BlobTestFixture { @@ -68,10 +68,10 @@ public BlobTriplet createBlobTriplet() { fail("Failed to read blob file", e); } - Bytes commitment = Bytes.wrap(CKZG4844JNI.blobToKzgCommitment(rawMaterial)); + Bytes48 commitment = Bytes48.wrap(CKZG4844JNI.blobToKzgCommitment(rawMaterial)); - assertThat(commitment.size()).isEqualTo(48); - Bytes proof = Bytes.wrap(CKZG4844JNI.computeBlobKzgProof(rawMaterial, commitment.toArray())); + Bytes48 proof = + Bytes48.wrap(CKZG4844JNI.computeBlobKzgProof(rawMaterial, commitment.toArray())); VersionedHash versionedHash = hashCommitment(new KZGCommitment(commitment)); return new BlobTriplet( new Blob(Bytes.wrap(rawMaterial)), diff --git a/ethereum/core/src/test/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchainTest.java b/ethereum/core/src/test/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchainTest.java index 80c8eee5d37..66c9ac5593b 100644 --- a/ethereum/core/src/test/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchainTest.java +++ b/ethereum/core/src/test/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchainTest.java @@ -942,6 +942,66 @@ public void blockAddedObserver_invokedMultiple() { assertThat(observer3Invoked.get()).isTrue(); } + @Test + public void testCacheEmptyWhenNumberOfBlocksToCacheIsZero() { + final BlockDataGenerator gen = new BlockDataGenerator(); + final KeyValueStorage kvStore = new InMemoryKeyValueStorage(); + final KeyValueStorage kvStoreVariables = new InMemoryKeyValueStorage(); + final Block genesisBlock = gen.genesisBlock(); + final DefaultBlockchain blockchain = + createMutableBlockchain(kvStore, kvStoreVariables, genesisBlock); + + assertThat(blockchain.getBlockHeadersCache()).isEmpty(); + assertThat(blockchain.getBlockBodiesCache()).isEmpty(); + assertThat(blockchain.getTransactionReceiptsCache()).isEmpty(); + assertThat(blockchain.getTotalDifficultyCache()).isEmpty(); + } + + @Test + public void testCacheUsedWhenNumberOfBlocksToCacheNotZero() { + final BlockDataGenerator gen = new BlockDataGenerator(); + final KeyValueStorage kvStore = new InMemoryKeyValueStorage(); + final KeyValueStorage kvStoreVariables = new InMemoryKeyValueStorage(); + final Block genesisBlock = gen.genesisBlock(); + final DefaultBlockchain blockchain = + createMutableBlockchain(kvStore, kvStoreVariables, genesisBlock, "/data/test", 512); + + final BlockDataGenerator.BlockOptions options = + new BlockDataGenerator.BlockOptions() + .setBlockNumber(1L) + .setParentHash(genesisBlock.getHash()); + final Block newBlock = gen.block(options); + final List receipts = gen.receipts(newBlock); + + assertThat(blockchain.getBlockHeadersCache()).isNotEmpty(); + assertThat(blockchain.getBlockBodiesCache()).isNotEmpty(); + assertThat(blockchain.getTransactionReceiptsCache()).isNotEmpty(); + assertThat(blockchain.getTotalDifficultyCache()).isNotEmpty(); + + assertThat(blockchain.getBlockHeadersCache().get().size()).isEqualTo(0); + assertThat(blockchain.getBlockBodiesCache().get().size()).isEqualTo(0); + assertThat(blockchain.getTransactionReceiptsCache().get().size()).isEqualTo(0); + assertThat(blockchain.getTotalDifficultyCache().get().size()).isEqualTo(0); + + blockchain.appendBlock(newBlock, receipts); + + assertThat(blockchain.getBlockHeadersCache().get().size()).isEqualTo(1); + assertThat(blockchain.getBlockHeadersCache().get().getIfPresent(newBlock.getHash())) + .isEqualTo(newBlock.getHeader()); + + assertThat(blockchain.getBlockBodiesCache().get().size()).isEqualTo(1); + assertThat(blockchain.getBlockBodiesCache().get().getIfPresent(newBlock.getHash())) + .isEqualTo(newBlock.getBody()); + + assertThat(blockchain.getTransactionReceiptsCache().get().size()).isEqualTo(1); + assertThat(blockchain.getTransactionReceiptsCache().get().getIfPresent(newBlock.getHash())) + .isEqualTo(receipts); + + assertThat(blockchain.getTotalDifficultyCache().get().size()).isEqualTo(1); + assertThat(blockchain.getTotalDifficultyCache().get().getIfPresent(newBlock.getHash())) + .isEqualTo(newBlock.getHeader().getDifficulty()); + } + /* * Check that block header, block body, block number, transaction locations, and receipts for this * block are all stored. @@ -1026,4 +1086,20 @@ private Blockchain createBlockchain( return DefaultBlockchain.create( createStorage(kvStore, kvStorageVariables), new NoOpMetricsSystem(), 0); } + + private DefaultBlockchain createMutableBlockchain( + final KeyValueStorage kvStore, + final KeyValueStorage kvStorageVariables, + final Block genesisBlock, + final String dataDirectory, + final int numberOfBlocksToCache) { + return (DefaultBlockchain) + DefaultBlockchain.createMutable( + genesisBlock, + createStorage(kvStore, kvStorageVariables), + new NoOpMetricsSystem(), + 0, + dataDirectory, + numberOfBlocksToCache); + } } diff --git a/ethereum/core/src/test/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidatorTest.java b/ethereum/core/src/test/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidatorTest.java index 536a93957ae..5ab351c2efa 100644 --- a/ethereum/core/src/test/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidatorTest.java +++ b/ethereum/core/src/test/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidatorTest.java @@ -57,6 +57,7 @@ import com.google.common.base.Suppliers; import org.apache.tuweni.bytes.Bytes; +import org.apache.tuweni.bytes.Bytes48; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.mockito.Mock; @@ -503,9 +504,9 @@ public void shouldRejectContractCreateWithBlob() { .blobsWithCommitments( Optional.of( new BlobsWithCommitments( - List.of(new KZGCommitment(Bytes.EMPTY)), + List.of(new KZGCommitment(Bytes48.ZERO)), List.of(new Blob(Bytes.EMPTY)), - List.of(new KZGProof(Bytes.EMPTY)), + List.of(new KZGProof(Bytes48.ZERO)), List.of(VersionedHash.DEFAULT_VERSIONED_HASH)))) .versionedHashes(Optional.of(List.of(VersionedHash.DEFAULT_VERSIONED_HASH))) .createTransaction(senderKeys); diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/PeerReputation.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/PeerReputation.java index 0c2edf226ed..e152d8c91b6 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/PeerReputation.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/PeerReputation.java @@ -56,7 +56,8 @@ public PeerReputation() { } public PeerReputation(final int initialScore, final int maxScore) { - checkArgument(initialScore <= maxScore, "Inital score must be less than or equal to max score"); + checkArgument( + initialScore <= maxScore, "Initial score must be less than or equal to max score"); this.maxScore = maxScore; this.score = initialScore; } diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/messages/StatusMessage.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/messages/StatusMessage.java index 9e8af4b0e57..4e268e6b411 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/messages/StatusMessage.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/messages/StatusMessage.java @@ -111,9 +111,9 @@ public Difficulty totalDifficulty() { } /** - * Return The hash of the head of the associated node's local blockchian. + * Return The hash of the head of the associated node's local blockchain. * - * @return The hash of the head of the associated node's local blockchian. + * @return The hash of the head of the associated node's local blockchain. */ public Hash bestHash() { return status().bestHash; diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/DisabledPendingTransactions.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/DisabledPendingTransactions.java index da77f6f3196..de9d6a07e8e 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/DisabledPendingTransactions.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/DisabledPendingTransactions.java @@ -41,14 +41,13 @@ public List getLocalTransactions() { } @Override - public TransactionAddedResult addRemoteTransaction( - final Transaction transaction, final Optional maybeSenderAccount) { - return TransactionAddedResult.DISABLED; + public List getPriorityTransactions() { + return List.of(); } @Override - public TransactionAddedResult addLocalTransaction( - final Transaction transaction, final Optional maybeSenderAccount) { + public TransactionAddedResult addTransaction( + final PendingTransaction transaction, final Optional maybeSenderAccount) { return TransactionAddedResult.DISABLED; } @@ -117,9 +116,4 @@ public String toTraceLog() { public String logStats() { return "Disabled"; } - - @Override - public boolean isLocalSender(final Address sender) { - return false; - } } diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransaction.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransaction.java index a716b1675e3..dbc461ecc38 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransaction.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransaction.java @@ -31,14 +31,20 @@ public abstract class PendingTransaction implements org.hyperledger.besu.datatypes.PendingTransaction { static final int NOT_INITIALIZED = -1; - static final int FRONTIER_BASE_MEMORY_SIZE = 944; - static final int ACCESS_LIST_BASE_MEMORY_SIZE = 944; - static final int EIP1559_BASE_MEMORY_SIZE = 1056; - static final int OPTIONAL_TO_MEMORY_SIZE = 92; + static final int FRONTIER_AND_ACCESS_LIST_BASE_MEMORY_SIZE = 872; + static final int EIP1559_AND_EIP4844_BASE_MEMORY_SIZE = 984; + static final int OPTIONAL_TO_MEMORY_SIZE = 112; + static final int OPTIONAL_CHAIN_ID_MEMORY_SIZE = 80; static final int PAYLOAD_BASE_MEMORY_SIZE = 32; static final int ACCESS_LIST_STORAGE_KEY_MEMORY_SIZE = 32; - static final int ACCESS_LIST_ENTRY_BASE_MEMORY_SIZE = 128; + static final int ACCESS_LIST_ENTRY_BASE_MEMORY_SIZE = 248; static final int OPTIONAL_ACCESS_LIST_MEMORY_SIZE = 24; + static final int VERSIONED_HASH_SIZE = 96; + static final int BASE_LIST_SIZE = 48; + static final int BASE_OPTIONAL_SIZE = 16; + static final int KZG_COMMITMENT_OR_PROOF_SIZE = 112; + static final int BLOB_SIZE = 131136; + static final int BLOBS_WITH_COMMITMENTS_SIZE = 32; static final int PENDING_TRANSACTION_MEMORY_SIZE = 40; private static final AtomicLong TRANSACTIONS_ADDED = new AtomicLong(); private final Transaction transaction; @@ -47,10 +53,37 @@ public abstract class PendingTransaction private int memorySize = NOT_INITIALIZED; - protected PendingTransaction(final Transaction transaction, final long addedAt) { + private PendingTransaction( + final Transaction transaction, final long addedAt, final long sequence) { this.transaction = transaction; this.addedAt = addedAt; - this.sequence = TRANSACTIONS_ADDED.getAndIncrement(); + this.sequence = sequence; + } + + private PendingTransaction(final Transaction transaction, final long addedAt) { + this(transaction, addedAt, TRANSACTIONS_ADDED.getAndIncrement()); + } + + public static PendingTransaction newPendingTransaction( + final Transaction transaction, final boolean isLocal, final boolean hasPriority) { + return newPendingTransaction(transaction, isLocal, hasPriority, System.currentTimeMillis()); + } + + public static PendingTransaction newPendingTransaction( + final Transaction transaction, + final boolean isLocal, + final boolean hasPriority, + final long addedAt) { + if (isLocal) { + if (hasPriority) { + return new Local.Priority(transaction, addedAt); + } + return new Local(transaction, addedAt); + } + if (hasPriority) { + return new Remote.Priority(transaction, addedAt); + } + return new Remote(transaction, addedAt); } @Override @@ -90,6 +123,8 @@ public int memorySize() { return memorySize; } + public abstract PendingTransaction detachedCopy(); + private int computeMemorySize() { return switch (transaction.getType()) { case FRONTIER -> computeFrontierMemorySize(); @@ -101,30 +136,49 @@ private int computeMemorySize() { } private int computeFrontierMemorySize() { - return FRONTIER_BASE_MEMORY_SIZE + computePayloadMemorySize() + computeToMemorySize(); + return FRONTIER_AND_ACCESS_LIST_BASE_MEMORY_SIZE + + computePayloadMemorySize() + + computeToMemorySize() + + computeChainIdMemorySize(); } private int computeAccessListMemorySize() { - return ACCESS_LIST_BASE_MEMORY_SIZE + return FRONTIER_AND_ACCESS_LIST_BASE_MEMORY_SIZE + computePayloadMemorySize() + computeToMemorySize() + + computeChainIdMemorySize() + computeAccessListEntriesMemorySize(); } private int computeEIP1559MemorySize() { - return EIP1559_BASE_MEMORY_SIZE + return EIP1559_AND_EIP4844_BASE_MEMORY_SIZE + computePayloadMemorySize() + computeToMemorySize() + + computeChainIdMemorySize() + computeAccessListEntriesMemorySize(); } private int computeBlobMemorySize() { - // ToDo 4844: adapt for blobs - return computeEIP1559MemorySize(); + return computeEIP1559MemorySize() + + BASE_OPTIONAL_SIZE // for the versionedHashes field + + computeBlobWithCommitmentsMemorySize(); + } + + private int computeBlobWithCommitmentsMemorySize() { + final int blobCount = transaction.getBlobCount(); + + return BASE_OPTIONAL_SIZE + + BLOBS_WITH_COMMITMENTS_SIZE + + (BASE_LIST_SIZE * 4) + + (KZG_COMMITMENT_OR_PROOF_SIZE * blobCount * 2) + + (VERSIONED_HASH_SIZE * blobCount) + + (BLOB_SIZE * blobCount); } private int computePayloadMemorySize() { - return PAYLOAD_BASE_MEMORY_SIZE + transaction.getPayload().size(); + return transaction.getPayload().size() > 0 + ? PAYLOAD_BASE_MEMORY_SIZE + transaction.getPayload().size() + : 0; } private int computeToMemorySize() { @@ -134,6 +188,13 @@ private int computeToMemorySize() { return 0; } + private int computeChainIdMemorySize() { + if (transaction.getChainId().isPresent()) { + return OPTIONAL_CHAIN_ID_MEMORY_SIZE; + } + return 0; + } + private int computeAccessListEntriesMemorySize() { return transaction .getAccessList() @@ -187,6 +248,8 @@ public String toString() { + sequence + ", isLocal=" + isReceivedFromLocalSource() + + ", hasPriority=" + + hasPriority() + '}'; } @@ -197,6 +260,8 @@ public String toTraceLog() { + addedAt + ", isLocal=" + isReceivedFromLocalSource() + + ", hasPriority=" + + hasPriority() + ", " + transaction.toTraceLog() + "}"; @@ -212,10 +277,48 @@ public Local(final Transaction transaction) { this(transaction, System.currentTimeMillis()); } + private Local(final long sequence, final Transaction transaction) { + super(transaction, System.currentTimeMillis(), sequence); + } + + @Override + public PendingTransaction detachedCopy() { + return new Local(getSequence(), getTransaction().detachedCopy()); + } + @Override public boolean isReceivedFromLocalSource() { return true; } + + @Override + public boolean hasPriority() { + return false; + } + + public static class Priority extends Local { + public Priority(final Transaction transaction) { + this(transaction, System.currentTimeMillis()); + } + + public Priority(final Transaction transaction, final long addedAt) { + super(transaction, addedAt); + } + + public Priority(final long sequence, final Transaction transaction) { + super(sequence, transaction); + } + + @Override + public PendingTransaction detachedCopy() { + return new Priority(getSequence(), getTransaction().detachedCopy()); + } + + @Override + public boolean hasPriority() { + return true; + } + } } public static class Remote extends PendingTransaction { @@ -228,9 +331,47 @@ public Remote(final Transaction transaction) { this(transaction, System.currentTimeMillis()); } + private Remote(final long sequence, final Transaction transaction) { + super(transaction, System.currentTimeMillis(), sequence); + } + + @Override + public PendingTransaction detachedCopy() { + return new Remote(getSequence(), getTransaction().detachedCopy()); + } + @Override public boolean isReceivedFromLocalSource() { return false; } + + @Override + public boolean hasPriority() { + return false; + } + + public static class Priority extends Remote { + public Priority(final Transaction transaction) { + this(transaction, System.currentTimeMillis()); + } + + public Priority(final Transaction transaction, final long addedAt) { + super(transaction, addedAt); + } + + public Priority(final long sequence, final Transaction transaction) { + super(sequence, transaction); + } + + @Override + public PendingTransaction detachedCopy() { + return new Priority(getSequence(), getTransaction().detachedCopy()); + } + + @Override + public boolean hasPriority() { + return true; + } + } } } diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransactions.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransactions.java index 199e88675e0..66f3bc0f4b0 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransactions.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransactions.java @@ -35,11 +35,10 @@ public interface PendingTransactions { List getLocalTransactions(); - TransactionAddedResult addRemoteTransaction( - Transaction transaction, Optional maybeSenderAccount); + List getPriorityTransactions(); - TransactionAddedResult addLocalTransaction( - Transaction transaction, Optional maybeSenderAccount); + TransactionAddedResult addTransaction( + PendingTransaction transaction, Optional maybeSenderAccount); void selectTransactions(TransactionSelector selector); @@ -84,8 +83,6 @@ default void signalInvalidAndRemoveDependentTransactions(final Transaction trans // no-op } - boolean isLocalSender(Address sender); - @FunctionalInterface interface TransactionSelector { TransactionSelectionResult evaluateTransaction(PendingTransaction pendingTransaction); diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java index 9b24f6cb539..c41027deb1b 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java @@ -63,7 +63,9 @@ import java.util.Optional; import java.util.OptionalLong; import java.util.Queue; +import java.util.Set; import java.util.concurrent.CompletableFuture; +import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentLinkedQueue; import java.util.concurrent.ExecutionException; import java.util.concurrent.TimeUnit; @@ -106,6 +108,7 @@ public class TransactionPool implements BlockAddedObserver { new PendingTransactionsListenersProxy(); private volatile OptionalLong subscribeConnectId = OptionalLong.empty(); private final SaveRestoreManager saveRestoreManager = new SaveRestoreManager(); + private final Set
localSenders = ConcurrentHashMap.newKeySet(); private final Lock blockAddedLock = new ReentrantLock(); private final Queue blockAddedQueue = new ConcurrentLinkedQueue<>(); @@ -135,6 +138,7 @@ public TransactionPool( } private void initLogForReplay() { + // log the initial block header data LOG_FOR_REPLAY .atTrace() .setMessage("{},{},{},{}") @@ -148,6 +152,16 @@ private void initLogForReplay() { .addArgument(() -> getChainHeadBlockHeader().map(BlockHeader::getGasUsed).orElse(0L)) .addArgument(() -> getChainHeadBlockHeader().map(BlockHeader::getGasLimit).orElse(0L)) .log(); + // log the priority senders + LOG_FOR_REPLAY + .atTrace() + .setMessage("{}") + .addArgument( + () -> + configuration.getPrioritySenders().stream() + .map(Address::toHexString) + .collect(Collectors.joining(","))) + .log(); } @VisibleForTesting @@ -159,57 +173,12 @@ void handleConnect(final EthPeer peer) { public ValidationResult addTransactionViaApi( final Transaction transaction) { - if (configuration.getDisableLocalTransactions()) { - final var result = addRemoteTransaction(transaction); - if (result.isValid()) { - transactionBroadcaster.onTransactionsAdded(List.of(transaction)); - } - return result; - } - return addLocalTransaction(transaction); - } - - ValidationResult addLocalTransaction(final Transaction transaction) { - final ValidationResultAndAccount validationResult = validateLocalTransaction(transaction); - - if (validationResult.result.isValid()) { - - final TransactionAddedResult transactionAddedResult = - pendingTransactions.addLocalTransaction(transaction, validationResult.maybeAccount); - - if (transactionAddedResult.isRejected()) { - final var rejectReason = - transactionAddedResult - .maybeInvalidReason() - .orElseGet( - () -> { - LOG.warn("Missing invalid reason for status {}", transactionAddedResult); - return INTERNAL_ERROR; - }); - return ValidationResult.invalid(rejectReason); - } - + final var result = addTransaction(transaction, true); + if (result.isValid()) { + localSenders.add(transaction.getSender()); transactionBroadcaster.onTransactionsAdded(List.of(transaction)); - } else { - metrics.incrementRejected(true, validationResult.result.getInvalidReason(), "txpool"); } - - return validationResult.result; - } - - private Optional getMaxGasPrice(final Transaction transaction) { - return transaction.getGasPrice().map(Optional::of).orElse(transaction.getMaxFeePerGas()); - } - - private boolean isMaxGasPriceBelowConfiguredMinGasPrice(final Transaction transaction) { - return getMaxGasPrice(transaction) - .map(g -> g.lessThan(miningParameters.getMinTransactionGasPrice())) - .orElse(true); - } - - private Stream sortedBySenderAndNonce(final Collection transactions) { - return transactions.stream() - .sorted(Comparator.comparing(Transaction::getSender).thenComparing(Transaction::getNonce)); + return result; } public Map> addRemoteTransactions( @@ -225,7 +194,7 @@ public Map> addRemoteTransactio Collectors.toMap( Transaction::getHash, transaction -> { - final var result = addRemoteTransaction(transaction); + final var result = addTransaction(transaction, false); if (result.isValid()) { addedTransactions.add(transaction); } @@ -254,26 +223,33 @@ public Map> addRemoteTransactio return validationResults; } - private ValidationResult addRemoteTransaction( - final Transaction transaction) { + private ValidationResult addTransaction( + final Transaction transaction, final boolean isLocal) { + + final boolean hasPriority = isPriorityTransaction(transaction, isLocal); + if (pendingTransactions.containsTransaction(transaction)) { LOG.atTrace() .setMessage("Discard already present transaction {}") .addArgument(transaction::toTraceLog) .log(); // We already have this transaction, don't even validate it. - metrics.incrementRejected(false, TRANSACTION_ALREADY_KNOWN, "txpool"); + metrics.incrementRejected(isLocal, hasPriority, TRANSACTION_ALREADY_KNOWN, "txpool"); return ValidationResult.invalid(TRANSACTION_ALREADY_KNOWN); } - final ValidationResultAndAccount validationResult = validateRemoteTransaction(transaction); + final ValidationResultAndAccount validationResult = + validateTransaction(transaction, isLocal, hasPriority); if (validationResult.result.isValid()) { final TransactionAddedResult status = - pendingTransactions.addRemoteTransaction(transaction, validationResult.maybeAccount); + pendingTransactions.addTransaction( + PendingTransaction.newPendingTransaction(transaction, isLocal, hasPriority), + validationResult.maybeAccount); if (status.isSuccess()) { LOG.atTrace() - .setMessage("Added remote transaction {}") + .setMessage("Added {} transaction {}") + .addArgument(() -> isLocal ? "local" : "remote") .addArgument(transaction::toTraceLog) .log(); } else { @@ -290,7 +266,7 @@ private ValidationResult addRemoteTransaction( .addArgument(transaction::toTraceLog) .addArgument(rejectReason) .log(); - metrics.incrementRejected(false, rejectReason, "txpool"); + metrics.incrementRejected(isLocal, hasPriority, rejectReason, "txpool"); return ValidationResult.invalid(rejectReason); } } else { @@ -299,13 +275,40 @@ private ValidationResult addRemoteTransaction( .addArgument(transaction::toTraceLog) .addArgument(validationResult.result::getInvalidReason) .log(); - metrics.incrementRejected(false, validationResult.result.getInvalidReason(), "txpool"); - pendingTransactions.signalInvalidAndRemoveDependentTransactions(transaction); + metrics.incrementRejected( + isLocal, hasPriority, validationResult.result.getInvalidReason(), "txpool"); + if (!isLocal) { + pendingTransactions.signalInvalidAndRemoveDependentTransactions(transaction); + } } return validationResult.result; } + private Optional getMaxGasPrice(final Transaction transaction) { + return transaction.getGasPrice().map(Optional::of).orElse(transaction.getMaxFeePerGas()); + } + + private boolean isMaxGasPriceBelowConfiguredMinGasPrice(final Transaction transaction) { + return getMaxGasPrice(transaction) + .map(g -> g.lessThan(miningParameters.getMinTransactionGasPrice())) + .orElse(true); + } + + private Stream sortedBySenderAndNonce(final Collection transactions) { + return transactions.stream() + .sorted(Comparator.comparing(Transaction::getSender).thenComparing(Transaction::getNonce)); + } + + private boolean isPriorityTransaction(final Transaction transaction, final boolean isLocal) { + if (isLocal && !configuration.getNoLocalPriority()) { + // unless no-local-priority option is specified, senders of local sent txs are prioritized + return true; + } + // otherwise check if the sender belongs to the priority list + return configuration.getPrioritySenders().contains(transaction.getSender()); + } + public long subscribePendingTransactions(final PendingTransactionAddedListener listener) { return pendingTransactionsListenersProxy.onAddedListeners.subscribe(listener); } @@ -374,17 +377,12 @@ private void reAddTransactions(final List reAddTransactions) { if (!reAddTransactions.isEmpty()) { var txsByOrigin = reAddTransactions.stream() - .collect( - Collectors.partitioningBy( - tx -> - configuration.getDisableLocalTransactions() - ? false - : pendingTransactions.isLocalSender(tx.getSender()))); + .collect(Collectors.partitioningBy(tx -> isLocalSender(tx.getSender()))); var reAddLocalTxs = txsByOrigin.get(true); var reAddRemoteTxs = txsByOrigin.get(false); if (!reAddLocalTxs.isEmpty()) { logReAddedTransactions(reAddLocalTxs, "local"); - sortedBySenderAndNonce(reAddLocalTxs).forEach(this::addLocalTransaction); + sortedBySenderAndNonce(reAddLocalTxs).forEach(this::addTransactionViaApi); } if (!reAddRemoteTxs.isEmpty()) { logReAddedTransactions(reAddRemoteTxs, "remote"); @@ -412,16 +410,8 @@ private TransactionValidator getTransactionValidator() { .get(); } - private ValidationResultAndAccount validateLocalTransaction(final Transaction transaction) { - return validateTransaction(transaction, true); - } - - private ValidationResultAndAccount validateRemoteTransaction(final Transaction transaction) { - return validateTransaction(transaction, false); - } - private ValidationResultAndAccount validateTransaction( - final Transaction transaction, final boolean isLocal) { + final Transaction transaction, final boolean isLocal, final boolean hasPriority) { final BlockHeader chainHeadBlockHeader = getChainHeadBlockHeader().orElse(null); if (chainHeadBlockHeader == null) { @@ -436,7 +426,7 @@ private ValidationResultAndAccount validateTransaction( protocolSchedule.getByBlockHeader(chainHeadBlockHeader).getFeeMarket(); final TransactionInvalidReason priceInvalidReason = - validatePrice(transaction, isLocal, feeMarket); + validatePrice(transaction, isLocal, hasPriority, feeMarket); if (priceInvalidReason != null) { return ValidationResultAndAccount.invalid(priceInvalidReason); } @@ -451,7 +441,7 @@ private ValidationResultAndAccount validateTransaction( return new ValidationResultAndAccount(basicValidationResult); } - if (isLocal + if (hasPriority && strictReplayProtectionShouldBeEnforcedLocally(chainHeadBlockHeader) && transaction.getChainId().isEmpty()) { // Strict replay protection is enabled but the tx is not replay-protected @@ -476,11 +466,13 @@ && strictReplayProtectionShouldBeEnforcedLocally(chainHeadBlockHeader) } // Call the transaction validator plugin if one is available - if (pluginTransactionValidator != null - && !pluginTransactionValidator.validateTransaction(transaction)) { - return ValidationResultAndAccount.invalid( - TransactionInvalidReason.PLUGIN_TX_VALIDATOR_INVALIDATED, - "Plugin transaction vaildator returned false"); + if (pluginTransactionValidator != null) { + final Optional maybeError = + pluginTransactionValidator.validateTransaction(transaction); + if (maybeError.isPresent()) { + return ValidationResultAndAccount.invalid( + TransactionInvalidReason.PLUGIN_TX_VALIDATOR, maybeError.get()); + } } try (final var worldState = @@ -505,14 +497,19 @@ && strictReplayProtectionShouldBeEnforcedLocally(chainHeadBlockHeader) } private TransactionInvalidReason validatePrice( - final Transaction transaction, final boolean isLocal, final FeeMarket feeMarket) { + final Transaction transaction, + final boolean isLocal, + final boolean hasPriority, + final FeeMarket feeMarket) { if (isLocal) { if (!configuration.getTxFeeCap().isZero() && getMaxGasPrice(transaction).get().greaterThan(configuration.getTxFeeCap())) { return TransactionInvalidReason.TX_FEECAP_EXCEEDED; } - // allow local transactions to be below minGas as long as we are mining + } + if (hasPriority) { + // allow priority transactions to be below minGas as long as we are mining // or at least gas price is above the configured floor if ((!miningParameters.isMiningEnabled() && isMaxGasPriceBelowConfiguredMinGasPrice(transaction)) @@ -552,8 +549,8 @@ private Optional getChainHeadBlockHeader() { return blockchain.getBlockHeader(blockchain.getChainHeadHash()); } - public boolean isLocalSender(final Address sender) { - return pendingTransactions.isLocalSender(sender); + private boolean isLocalSender(final Address sender) { + return localSenders.contains(sender); } public int count() { @@ -802,12 +799,8 @@ private void executeLoadFromDisk() { final Transaction tx = Transaction.readFrom(Bytes.fromBase64String(line.substring(1))); - final ValidationResult result; - if (isLocal && !configuration.getDisableLocalTransactions()) { - result = addLocalTransaction(tx); - } else { - result = addRemoteTransaction(tx); - } + final ValidationResult result = + addTransaction(tx, isLocal); return result.isValid() ? 1 : 0; }) diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolConfiguration.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolConfiguration.java index aaf8c326c0a..874446b7671 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolConfiguration.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolConfiguration.java @@ -14,12 +14,14 @@ */ package org.hyperledger.besu.ethereum.eth.transactions; +import org.hyperledger.besu.datatypes.Address; import org.hyperledger.besu.datatypes.Wei; import org.hyperledger.besu.util.number.Fraction; import org.hyperledger.besu.util.number.Percentage; import java.io.File; import java.time.Duration; +import java.util.Set; import org.immutables.value.Value; @@ -53,21 +55,20 @@ enum Implementation { } String DEFAULT_SAVE_FILE_NAME = "txpool.dump"; - int DEFAULT_MAX_PENDING_TRANSACTIONS = 4096; Fraction DEFAULT_LIMIT_TX_POOL_BY_ACCOUNT_PERCENTAGE = Fraction.fromFloat(0.001f); // 0.1% int DEFAULT_TX_RETENTION_HOURS = 13; boolean DEFAULT_STRICT_TX_REPLAY_PROTECTION_ENABLED = false; Percentage DEFAULT_PRICE_BUMP = Percentage.fromInt(10); Wei DEFAULT_RPC_TX_FEE_CAP = Wei.fromEth(1); - boolean DEFAULT_DISABLE_LOCAL_TXS = false; + boolean DEFAULT_NO_LOCAL_PRIORITY = false; boolean DEFAULT_ENABLE_SAVE_RESTORE = false; - File DEFAULT_SAVE_FILE = new File(DEFAULT_SAVE_FILE_NAME); long DEFAULT_PENDING_TRANSACTIONS_LAYER_MAX_CAPACITY_BYTES = 12_500_000L; int DEFAULT_MAX_PRIORITIZED_TRANSACTIONS = 2000; int DEFAULT_MAX_FUTURE_BY_SENDER = 200; Implementation DEFAULT_TX_POOL_IMPLEMENTATION = Implementation.LAYERED; + Set
DEFAULT_PRIORITY_SENDERS = Set.of(); TransactionPoolConfiguration DEFAULT = ImmutableTransactionPoolConfiguration.builder().build(); @@ -107,8 +108,8 @@ default Boolean getStrictTransactionReplayProtectionEnabled() { } @Value.Default - default Boolean getDisableLocalTransactions() { - return DEFAULT_DISABLE_LOCAL_TXS; + default Boolean getNoLocalPriority() { + return DEFAULT_NO_LOCAL_PRIORITY; } @Value.Default @@ -141,6 +142,11 @@ default int getMaxFutureBySender() { return DEFAULT_MAX_FUTURE_BY_SENDER; } + @Value.Default + default Set
getPrioritySenders() { + return DEFAULT_PRIORITY_SENDERS; + } + @Value.Default default Unstable getUnstable() { return Unstable.DEFAULT; diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolMetrics.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolMetrics.java index 30ddb9db3db..9ace1819cc1 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolMetrics.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolMetrics.java @@ -56,6 +56,7 @@ public TransactionPoolMetrics(final MetricsSystem metricsSystem) { ADDED_COUNTER_NAME, "Count of transactions added to the transaction pool", "source", + "priority", "layer"); removedCounter = @@ -64,6 +65,7 @@ public TransactionPoolMetrics(final MetricsSystem metricsSystem) { REMOVED_COUNTER_NAME, "Count of transactions removed from the transaction pool", "source", + "priority", "operation", "layer"); @@ -73,6 +75,7 @@ public TransactionPoolMetrics(final MetricsSystem metricsSystem) { REJECTED_COUNTER_NAME, "Count of transactions not accepted to the transaction pool", "source", + "priority", "reason", "layer"); @@ -143,20 +146,46 @@ public void initExpiredMessagesCounter(final String message) { SKIPPED_MESSAGES_LOGGING_THRESHOLD)); } - public void incrementAdded(final boolean receivedFromLocalSource, final String layer) { - addedCounter.labels(location(receivedFromLocalSource), layer).inc(); + public void incrementAdded(final PendingTransaction pendingTransaction, final String layer) { + addedCounter + .labels( + location(pendingTransaction.isReceivedFromLocalSource()), + priority(pendingTransaction.hasPriority()), + layer) + .inc(); } public void incrementRemoved( - final boolean receivedFromLocalSource, final String operation, final String layer) { - removedCounter.labels(location(receivedFromLocalSource), operation, layer).inc(); + final PendingTransaction pendingTransaction, final String operation, final String layer) { + removedCounter + .labels( + location(pendingTransaction.isReceivedFromLocalSource()), + priority(pendingTransaction.hasPriority()), + operation, + layer) + .inc(); + } + + public void incrementRejected( + final PendingTransaction pendingTransaction, + final TransactionInvalidReason rejectReason, + final String layer) { + incrementRejected( + pendingTransaction.isReceivedFromLocalSource(), + pendingTransaction.hasPriority(), + rejectReason, + layer); } public void incrementRejected( final boolean receivedFromLocalSource, + final boolean hasPriority, final TransactionInvalidReason rejectReason, final String layer) { - rejectedCounter.labels(location(receivedFromLocalSource), rejectReason.name(), layer).inc(); + rejectedCounter + .labels( + location(receivedFromLocalSource), priority(hasPriority), rejectReason.name(), layer) + .inc(); } public void incrementExpiredMessages(final String message) { @@ -170,4 +199,8 @@ public void incrementAlreadySeenTransactions(final String message, final long co private String location(final boolean receivedFromLocalSource) { return receivedFromLocalSource ? "local" : "remote"; } + + private String priority(final boolean hasPriority) { + return hasPriority ? "yes" : "no"; + } } diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/AbstractTransactionsLayer.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/AbstractTransactionsLayer.java index 026d1bf4f72..136f80499c1 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/AbstractTransactionsLayer.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/AbstractTransactionsLayer.java @@ -150,7 +150,7 @@ public TransactionAddedResult add(final PendingTransaction pendingTransaction, f } if (addStatus.isSuccess()) { - processAdded(pendingTransaction); + processAdded(pendingTransaction.detachedCopy()); addStatus.maybeReplacedTransaction().ifPresent(this::replaced); nextLayer.notifyAdded(pendingTransaction); @@ -163,7 +163,7 @@ public TransactionAddedResult add(final PendingTransaction pendingTransaction, f notifyTransactionAdded(pendingTransaction); } else { final var rejectReason = addStatus.maybeInvalidReason().orElseThrow(); - metrics.incrementRejected(false, rejectReason, name()); + metrics.incrementRejected(pendingTransaction, rejectReason, name()); LOG.atTrace() .setMessage("Transaction {} rejected reason {}") .addArgument(pendingTransaction::toTraceLog) @@ -245,7 +245,7 @@ public PendingTransaction promoteFor(final Address sender, final long nonce) { if (senderTxs.firstKey() == expectedNonce) { final PendingTransaction promotedTx = senderTxs.pollFirstEntry().getValue(); processRemove(senderTxs, promotedTx.getTransaction(), PROMOTED); - metrics.incrementRemoved(promotedTx.isReceivedFromLocalSource(), "promoted", name()); + metrics.incrementRemoved(promotedTx, "promoted", name()); if (senderTxs.isEmpty()) { txsBySender.remove(sender); @@ -282,7 +282,7 @@ private void processAdded(final PendingTransaction addedTx) { final var senderTxs = txsBySender.computeIfAbsent(addedTx.getSender(), s -> new TreeMap<>()); senderTxs.put(addedTx.getNonce(), addedTx); increaseSpaceUsed(addedTx); - metrics.incrementAdded(addedTx.isReceivedFromLocalSource(), name()); + metrics.incrementAdded(addedTx, name()); internalAdd(senderTxs, addedTx); } @@ -328,7 +328,7 @@ private void evict(final long spaceToFree, final int txsToEvict) { protected void replaced(final PendingTransaction replacedTx) { pendingTransactions.remove(replacedTx.getHash()); decreaseSpaceUsed(replacedTx); - metrics.incrementRemoved(replacedTx.isReceivedFromLocalSource(), REPLACED.label(), name()); + metrics.incrementRemoved(replacedTx, REPLACED.label(), name()); internalReplaced(replacedTx); notifyTransactionDropped(replacedTx); } @@ -363,8 +363,7 @@ protected PendingTransaction processRemove( final PendingTransaction removedTx = pendingTransactions.remove(transaction.getHash()); if (removedTx != null) { decreaseSpaceUsed(removedTx); - metrics.incrementRemoved( - removedTx.isReceivedFromLocalSource(), removalReason.label(), name()); + metrics.incrementRemoved(removedTx, removalReason.label(), name()); internalRemove(senderTxs, removedTx, removalReason); } return removedTx; @@ -377,7 +376,7 @@ protected PendingTransaction processEvict( final PendingTransaction removedTx = pendingTransactions.remove(evictedTx.getHash()); if (removedTx != null) { decreaseSpaceUsed(evictedTx); - metrics.incrementRemoved(evictedTx.isReceivedFromLocalSource(), reason.label(), name()); + metrics.incrementRemoved(evictedTx, reason.label(), name()); internalEvict(senderTxs, removedTx); } return removedTx; @@ -431,7 +430,7 @@ private void confirmed(final Address sender, final long maxConfirmedNonce) { itConfirmedTxs.remove(); processRemove(senderTxs, confirmedTx.getTransaction(), CONFIRMED); - metrics.incrementRemoved(confirmedTx.isReceivedFromLocalSource(), "confirmed", name()); + metrics.incrementRemoved(confirmedTx, "confirmed", name()); LOG.atTrace() .setMessage("Removed confirmed pending transactions {}") .addArgument(confirmedTx::toTraceLog) @@ -482,6 +481,17 @@ public List getAllLocal() { return localTxs; } + @Override + public List getAllPriority() { + final var priorityTxs = + pendingTransactions.values().stream() + .filter(PendingTransaction::hasPriority) + .map(PendingTransaction::getTransaction) + .collect(Collectors.toCollection(ArrayList::new)); + priorityTxs.addAll(nextLayer.getAllPriority()); + return priorityTxs; + } + Stream stream(final Address sender) { return txsBySender.getOrDefault(sender, EMPTY_SENDER_TXS).values().stream(); } diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/BaseFeePrioritizedTransactions.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/BaseFeePrioritizedTransactions.java index 3dcdfd734f8..f5ee0de2ae4 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/BaseFeePrioritizedTransactions.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/BaseFeePrioritizedTransactions.java @@ -53,7 +53,8 @@ public BaseFeePrioritizedTransactions( @Override protected int compareByFee(final PendingTransaction pt1, final PendingTransaction pt2) { - return Comparator.comparing( + return Comparator.comparing(PendingTransaction::hasPriority) + .thenComparing( (PendingTransaction pendingTransaction) -> pendingTransaction.getTransaction().getEffectivePriorityFeePerGas(nextBlockBaseFee)) .thenComparing( diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/EndLayer.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/EndLayer.java index 12d3147b326..e79ce1d8a71 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/EndLayer.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/EndLayer.java @@ -77,8 +77,7 @@ public List getAll() { @Override public TransactionAddedResult add(final PendingTransaction pendingTransaction, final int gap) { notifyTransactionDropped(pendingTransaction); - metrics.incrementRemoved( - pendingTransaction.isReceivedFromLocalSource(), DROPPED.label(), name()); + metrics.incrementRemoved(pendingTransaction, DROPPED.label(), name()); ++droppedCount; return TransactionAddedResult.DROPPED; } @@ -99,6 +98,11 @@ public List getAllLocal() { return List.of(); } + @Override + public List getAllPriority() { + return List.of(); + } + @Override public int count() { return 0; diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/GasPricePrioritizedTransactions.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/GasPricePrioritizedTransactions.java index 725b5138d34..762e7469f45 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/GasPricePrioritizedTransactions.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/GasPricePrioritizedTransactions.java @@ -43,7 +43,7 @@ public GasPricePrioritizedTransactions( @Override protected int compareByFee(final PendingTransaction pt1, final PendingTransaction pt2) { - return comparing(PendingTransaction::isReceivedFromLocalSource) + return comparing(PendingTransaction::hasPriority) .thenComparing(PendingTransaction::getGasPrice) .thenComparing(PendingTransaction::getSequence) .compare(pt1, pt2); diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayeredPendingTransactions.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayeredPendingTransactions.java index 5502e9a6f1a..bd93f0d9fa0 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayeredPendingTransactions.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayeredPendingTransactions.java @@ -58,7 +58,6 @@ public class LayeredPendingTransactions implements PendingTransactions { private static final Logger LOG = LoggerFactory.getLogger(LayeredPendingTransactions.class); private static final Logger LOG_FOR_REPLAY = LoggerFactory.getLogger("LOG_FOR_REPLAY"); private final TransactionPoolConfiguration poolConfig; - private final Set
localSenders = new HashSet<>(); private final AbstractPrioritizedTransactions prioritizedTransactions; public LayeredPendingTransactions( @@ -74,25 +73,7 @@ public synchronized void reset() { } @Override - public synchronized TransactionAddedResult addRemoteTransaction( - final Transaction transaction, final Optional maybeSenderAccount) { - - return addTransaction(new PendingTransaction.Remote(transaction), maybeSenderAccount); - } - - @Override - public synchronized TransactionAddedResult addLocalTransaction( - final Transaction transaction, final Optional maybeSenderAccount) { - - final TransactionAddedResult addedResult = - addTransaction(new PendingTransaction.Local(transaction), maybeSenderAccount); - if (addedResult.isSuccess()) { - localSenders.add(transaction.getSender()); - } - return addedResult; - } - - TransactionAddedResult addTransaction( + public synchronized TransactionAddedResult addTransaction( final PendingTransaction pendingTransaction, final Optional maybeSenderAccount) { final long stateSenderNonce = maybeSenderAccount.map(AccountState::getNonce).orElse(0L); @@ -305,8 +286,8 @@ public synchronized List getLocalTransactions() { } @Override - public synchronized boolean isLocalSender(final Address sender) { - return localSenders.contains(sender); + public synchronized List getPriorityTransactions() { + return prioritizedTransactions.getAllPriority(); } @Override diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/ReadyTransactions.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/ReadyTransactions.java index 227d0e4daa3..484f131b0b9 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/ReadyTransactions.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/ReadyTransactions.java @@ -43,7 +43,8 @@ public class ReadyTransactions extends AbstractSequentialTransactionsLayer { private final NavigableSet orderByMaxFee = new TreeSet<>( - Comparator.comparing((PendingTransaction pt) -> pt.getTransaction().getMaxGasPrice()) + Comparator.comparing(PendingTransaction::hasPriority) + .thenComparing((PendingTransaction pt) -> pt.getTransaction().getMaxGasPrice()) .thenComparing(PendingTransaction::getSequence)); public ReadyTransactions( diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/SparseTransactions.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/SparseTransactions.java index 64a4142bccb..3567fb84266 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/SparseTransactions.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/SparseTransactions.java @@ -30,6 +30,7 @@ import java.util.Comparator; import java.util.HashMap; import java.util.HashSet; +import java.util.Iterator; import java.util.List; import java.util.Map; import java.util.NavigableMap; @@ -44,11 +45,15 @@ import java.util.stream.IntStream; import java.util.stream.Stream; +import com.google.common.collect.Iterables; + public class SparseTransactions extends AbstractTransactionsLayer { private final NavigableSet sparseEvictionOrder = - new TreeSet<>(Comparator.comparing(PendingTransaction::getSequence)); + new TreeSet<>( + Comparator.comparing(PendingTransaction::hasPriority) + .thenComparing(PendingTransaction::getSequence)); private final Map gapBySender = new HashMap<>(); - private final List> orderByGap; + private final List orderByGap; public SparseTransactions( final TransactionPoolConfiguration poolConfig, @@ -59,7 +64,7 @@ public SparseTransactions( super(poolConfig, nextLayer, transactionReplacementTester, metrics); orderByGap = new ArrayList<>(poolConfig.getMaxFutureBySender()); IntStream.range(0, poolConfig.getMaxFutureBySender()) - .forEach(i -> orderByGap.add(new HashSet<>())); + .forEach(i -> orderByGap.add(new SendersByPriority())); } @Override @@ -82,7 +87,7 @@ public void reset() { super.reset(); sparseEvictionOrder.clear(); gapBySender.clear(); - orderByGap.forEach(Set::clear); + orderByGap.forEach(SendersByPriority::clear); } @Override @@ -92,12 +97,12 @@ protected TransactionAddedResult canAdd( pendingTransaction.getSender(), (sender, currGap) -> { if (currGap == null) { - orderByGap.get(gap).add(sender); + orderByGap.get(gap).add(pendingTransaction); return gap; } if (pendingTransaction.getNonce() < txsBySender.get(sender).firstKey()) { orderByGap.get(currGap).remove(sender); - orderByGap.get(gap).add(sender); + orderByGap.get(gap).add(pendingTransaction); return gap; } return currGap; @@ -390,8 +395,8 @@ public String internalLogStats() { } private void updateGap(final Address sender, final int currGap, final int newGap) { - orderByGap.get(currGap).remove(sender); - orderByGap.get(newGap).add(sender); + final boolean hasPriority = orderByGap.get(currGap).remove(sender); + orderByGap.get(newGap).add(sender, hasPriority); gapBySender.put(sender, newGap); } @@ -430,4 +435,50 @@ protected void internalConsistencyCheck( } }); } + + private static class SendersByPriority implements Iterable
{ + final Set
prioritySenders = new HashSet<>(); + final Set
standardSenders = new HashSet<>(); + + void clear() { + prioritySenders.clear(); + standardSenders.clear(); + } + + public void add(final Address sender, final boolean hasPriority) { + if (hasPriority) { + if (standardSenders.contains(sender)) { + throw new IllegalStateException( + "Sender " + sender + " cannot simultaneously have and not have priority"); + } + prioritySenders.add(sender); + } else { + if (prioritySenders.contains(sender)) { + throw new IllegalStateException( + "Sender " + sender + " cannot simultaneously have and not have priority"); + } + standardSenders.add(sender); + } + } + + void add(final PendingTransaction pendingTransaction) { + add(pendingTransaction.getSender(), pendingTransaction.hasPriority()); + } + + boolean remove(final Address sender) { + if (standardSenders.remove(sender)) { + return false; + } + return prioritySenders.remove(sender); + } + + public boolean contains(final Address sender) { + return standardSenders.contains(sender) || prioritySenders.contains(sender); + } + + @Override + public Iterator
iterator() { + return Iterables.concat(prioritySenders, standardSenders).iterator(); + } + } } diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/TransactionsLayer.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/TransactionsLayer.java index c6fcc5e7cee..85227766b40 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/TransactionsLayer.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/layered/TransactionsLayer.java @@ -53,6 +53,8 @@ void blockAdded( List getAllLocal(); + List getAllPriority(); + int count(); OptionalLong getNextNonceFor(Address sender); diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/AbstractPendingTransactionsSorter.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/AbstractPendingTransactionsSorter.java index ea12b3c4f6a..ad2595c2658 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/AbstractPendingTransactionsSorter.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/AbstractPendingTransactionsSorter.java @@ -97,8 +97,6 @@ public abstract class AbstractPendingTransactionsSorter implements PendingTransa protected final TransactionPoolReplacementHandler transactionReplacementHandler; protected final Supplier chainHeadHeaderSupplier; - private final Set
localSenders; - public AbstractPendingTransactionsSorter( final TransactionPoolConfiguration poolConfig, final Clock clock, @@ -106,8 +104,6 @@ public AbstractPendingTransactionsSorter( final Supplier chainHeadHeaderSupplier) { this.poolConfig = poolConfig; this.pendingTransactions = new ConcurrentHashMap<>(poolConfig.getTxPoolMaxSize()); - this.localSenders = - poolConfig.getDisableLocalTransactions() ? Set.of() : ConcurrentHashMap.newKeySet(); this.clock = clock; this.chainHeadHeaderSupplier = chainHeadHeaderSupplier; this.transactionReplacementHandler = @@ -172,10 +168,19 @@ public List getLocalTransactions() { } @Override - public TransactionAddedResult addRemoteTransaction( - final Transaction transaction, final Optional maybeSenderAccount) { + public List getPriorityTransactions() { + return pendingTransactions.values().stream() + .filter(PendingTransaction::hasPriority) + .map(PendingTransaction::getTransaction) + .collect(Collectors.toList()); + } + + @Override + public TransactionAddedResult addTransaction( + final PendingTransaction transaction, final Optional maybeSenderAccount) { - if (lowestInvalidKnownNonceCache.hasInvalidLowerNonce(transaction)) { + if (!transaction.isReceivedFromLocalSource() + && lowestInvalidKnownNonceCache.hasInvalidLowerNonce(transaction.getTransaction())) { LOG.atDebug() .setMessage( "Dropping transaction {} since the sender has an invalid transaction with lower nonce") @@ -184,30 +189,19 @@ public TransactionAddedResult addRemoteTransaction( return LOWER_NONCE_INVALID_TRANSACTION_KNOWN; } - final PendingTransaction pendingTransaction = - new PendingTransaction.Remote(transaction, clock.millis()); final TransactionAddedResult transactionAddedStatus = - addTransaction(pendingTransaction, maybeSenderAccount); + internalAddTransaction(transaction, maybeSenderAccount); if (transactionAddedStatus.equals(ADDED)) { - lowestInvalidKnownNonceCache.registerValidTransaction(transaction); - remoteTransactionAddedCounter.inc(); + if (!transaction.isReceivedFromLocalSource()) { + lowestInvalidKnownNonceCache.registerValidTransaction(transaction.getTransaction()); + remoteTransactionAddedCounter.inc(); + } else { + localTransactionAddedCounter.inc(); + } } return transactionAddedStatus; } - @Override - public TransactionAddedResult addLocalTransaction( - final Transaction transaction, final Optional maybeSenderAccount) { - final TransactionAddedResult transactionAdded = - addTransaction( - new PendingTransaction.Local(transaction, clock.millis()), maybeSenderAccount); - if (transactionAdded.equals(ADDED)) { - localSenders.add(transaction.getSender()); - localTransactionAddedCounter.inc(); - } - return transactionAdded; - } - void removeTransaction(final Transaction transaction) { removeTransaction(transaction, false); notifyTransactionDropped(transaction); @@ -428,7 +422,7 @@ private void removeTransaction(final Transaction transaction, final boolean adde protected abstract void prioritizeTransaction(final PendingTransaction pendingTransaction); - private TransactionAddedResult addTransaction( + private TransactionAddedResult internalAddTransaction( final PendingTransaction pendingTransaction, final Optional maybeSenderAccount) { final Transaction transaction = pendingTransaction.getTransaction(); synchronized (lock) { @@ -540,9 +534,4 @@ public void signalInvalidAndRemoveDependentTransactions(final Transaction transa signalInvalidAndGetDependentTransactions(transaction).forEach(this::removeTransaction); } } - - @Override - public boolean isLocalSender(final Address sender) { - return poolConfig.getDisableLocalTransactions() ? false : localSenders.contains(sender); - } } diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/BaseFeePendingTransactionsSorter.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/BaseFeePendingTransactionsSorter.java index aeeb2b3e469..2ccb4d476ee 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/BaseFeePendingTransactionsSorter.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/BaseFeePendingTransactionsSorter.java @@ -63,7 +63,7 @@ public BaseFeePendingTransactionsSorter( */ private final NavigableSet prioritizedTransactionsStaticRange = new TreeSet<>( - comparing(PendingTransaction::isReceivedFromLocalSource) + comparing(PendingTransaction::hasPriority) .thenComparing( pendingTx -> pendingTx @@ -79,7 +79,7 @@ public BaseFeePendingTransactionsSorter( private final NavigableSet prioritizedTransactionsDynamicRange = new TreeSet<>( - comparing(PendingTransaction::isReceivedFromLocalSource) + comparing(PendingTransaction::hasPriority) .thenComparing( pendingTx -> pendingTx diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/GasPricePendingTransactionsSorter.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/GasPricePendingTransactionsSorter.java index 29b3406b059..ce8f8802592 100644 --- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/GasPricePendingTransactionsSorter.java +++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/GasPricePendingTransactionsSorter.java @@ -37,7 +37,7 @@ public class GasPricePendingTransactionsSorter extends AbstractPendingTransactio private final NavigableSet prioritizedTransactions = new TreeSet<>( - comparing(PendingTransaction::isReceivedFromLocalSource) + comparing(PendingTransaction::hasPriority) .thenComparing(PendingTransaction::getGasPrice) .thenComparing(PendingTransaction::getAddedAt) .thenComparing(PendingTransaction::getSequence) diff --git a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/AbstractTransactionPoolTest.java b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/AbstractTransactionPoolTest.java index beeb2b1e43b..d9c98f3a92b 100644 --- a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/AbstractTransactionPoolTest.java +++ b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/AbstractTransactionPoolTest.java @@ -48,6 +48,7 @@ import org.hyperledger.besu.config.StubGenesisConfigOptions; import org.hyperledger.besu.crypto.KeyPair; import org.hyperledger.besu.crypto.SignatureAlgorithmFactory; +import org.hyperledger.besu.datatypes.Address; import org.hyperledger.besu.datatypes.TransactionType; import org.hyperledger.besu.datatypes.Wei; import org.hyperledger.besu.ethereum.ProtocolContext; @@ -296,39 +297,39 @@ private TransactionPool createTransactionPool( @ParameterizedTest @ValueSource(booleans = {true, false}) - public void localTransactionHappyPath(final boolean disableLocalTxs) { - this.transactionPool = createTransactionPool(b -> b.disableLocalTransactions(disableLocalTxs)); + public void localTransactionHappyPath(final boolean noLocalPriority) { + this.transactionPool = createTransactionPool(b -> b.noLocalPriority(noLocalPriority)); final Transaction transaction = createTransaction(0); givenTransactionIsValid(transaction); - addAndAssertTransactionViaApiValid(transaction, disableLocalTxs); + addAndAssertTransactionViaApiValid(transaction, noLocalPriority); } @ParameterizedTest @ValueSource(booleans = {true, false}) - public void shouldReturnLocalTransactionsWhenAppropriate(final boolean disableLocalTxs) { - this.transactionPool = createTransactionPool(b -> b.disableLocalTransactions(disableLocalTxs)); + public void shouldReturnLocalTransactionsWhenAppropriate(final boolean noLocalPriority) { + this.transactionPool = createTransactionPool(b -> b.noLocalPriority(noLocalPriority)); final Transaction localTransaction2 = createTransaction(2); givenTransactionIsValid(localTransaction2); givenTransactionIsValid(transaction0); givenTransactionIsValid(transaction1); - addAndAssertTransactionViaApiValid(localTransaction2, disableLocalTxs); - addAndAssertRemoteTransactionValid(transaction0); - addAndAssertRemoteTransactionValid(transaction1); + addAndAssertTransactionViaApiValid(localTransaction2, noLocalPriority); + addAndAssertRemoteTransactionsValid(transaction0); + addAndAssertRemoteTransactionsValid(transaction1); assertThat(transactions.size()).isEqualTo(3); - List localTransactions = transactions.getLocalTransactions(); - assertThat(localTransactions.size()).isEqualTo(disableLocalTxs ? 0 : 1); + assertThat(transactions.getLocalTransactions()).contains(localTransaction2); + assertThat(transactions.getPriorityTransactions().size()).isEqualTo(noLocalPriority ? 0 : 1); } @Test public void shouldRemoveTransactionsFromPendingListWhenIncludedInBlockOnchain() { givenTransactionIsValid(transaction0); - addAndAssertRemoteTransactionValid(transaction0); + addAndAssertRemoteTransactionsValid(transaction0); appendBlock(transaction0); @@ -340,8 +341,8 @@ public void shouldRemoveMultipleTransactionsAddedInOneBlock() { givenTransactionIsValid(transaction0); givenTransactionIsValid(transaction1); - addAndAssertRemoteTransactionValid(transaction0); - addAndAssertRemoteTransactionValid(transaction1); + addAndAssertRemoteTransactionsValid(transaction0); + addAndAssertRemoteTransactionsValid(transaction1); appendBlock(transaction0, transaction1); @@ -354,7 +355,7 @@ public void shouldRemoveMultipleTransactionsAddedInOneBlock() { public void shouldIgnoreUnknownTransactionsThatAreAddedInABlock() { givenTransactionIsValid(transaction0); - addAndAssertRemoteTransactionValid(transaction0); + addAndAssertRemoteTransactionsValid(transaction0); appendBlock(transaction0, transaction1); @@ -367,7 +368,7 @@ public void shouldIgnoreUnknownTransactionsThatAreAddedInABlock() { public void shouldNotRemovePendingTransactionsWhenABlockAddedToAFork() { givenTransactionIsValid(transaction0); - addAndAssertRemoteTransactionValid(transaction0); + addAndAssertRemoteTransactionsValid(transaction0); final BlockHeader commonParent = getHeaderForCurrentChainHead(); final Block canonicalHead = appendBlock(Difficulty.of(1000), commonParent); @@ -383,8 +384,8 @@ public void shouldRemovePendingTransactionsFromAllBlocksOnAForkWhenItBecomesTheC givenTransactionIsValid(transaction0); givenTransactionIsValid(transaction1); - addAndAssertRemoteTransactionValid(transaction0); - addAndAssertRemoteTransactionValid(transaction1); + addAndAssertRemoteTransactionsValid(transaction0); + addAndAssertRemoteTransactionsValid(transaction1); final BlockHeader commonParent = getHeaderForCurrentChainHead(); final Block originalChainHead = appendBlock(Difficulty.of(1000), commonParent); @@ -436,8 +437,8 @@ public void shouldNotReAddTransactionsThatAreInBothForksWhenReorgHappens() { givenTransactionIsValid(transaction0); givenTransactionIsValid(transaction1); - addAndAssertRemoteTransactionValid(transaction0); - addAndAssertRemoteTransactionValid(transaction1); + addAndAssertRemoteTransactionsValid(transaction0); + addAndAssertRemoteTransactionsValid(transaction1); final BlockHeader commonParent = getHeaderForCurrentChainHead(); final Block originalFork1 = appendBlock(Difficulty.of(1000), commonParent, transaction0); @@ -462,8 +463,8 @@ public void shouldNotReAddBlobTxsWhenReorgHappens() { givenTransactionIsValid(transaction1); givenTransactionIsValid(transactionBlob); - addAndAssertRemoteTransactionValid(transaction0); - addAndAssertRemoteTransactionValid(transaction1); + addAndAssertRemoteTransactionsValid(transaction0); + addAndAssertRemoteTransactionsValid(transaction1); addAndAssertRemoteTransactionInvalid(transactionBlob); final BlockHeader commonParent = getHeaderForCurrentChainHead(); @@ -493,17 +494,15 @@ public void shouldNotReAddBlobTxsWhenReorgHappens() { @ParameterizedTest @ValueSource(booleans = {true, false}) public void addLocalTransaction_strictReplayProtectionOn_txWithChainId_chainIdIsConfigured( - final boolean disableLocalTxs) { + final boolean noLocalPriority) { protocolSupportsTxReplayProtection(1337, true); transactionPool = createTransactionPool( - b -> - b.strictTransactionReplayProtectionEnabled(true) - .disableLocalTransactions(disableLocalTxs)); + b -> b.strictTransactionReplayProtectionEnabled(true).noLocalPriority(noLocalPriority)); final Transaction tx = createTransaction(1); givenTransactionIsValid(tx); - addAndAssertTransactionViaApiValid(tx, disableLocalTxs); + addAndAssertTransactionViaApiValid(tx, noLocalPriority); } @Test @@ -513,7 +512,7 @@ public void addRemoteTransactions_strictReplayProtectionOn_txWithChainId_chainId final Transaction tx = createTransaction(1); givenTransactionIsValid(tx); - addAndAssertRemoteTransactionValid(tx); + addAndAssertRemoteTransactionsValid(tx); } @Test @@ -525,6 +524,17 @@ public void shouldNotAddRemoteTransactionsWhenGasPriceBelowMinimum() { verifyNoMoreInteractions(transactionValidatorFactory); } + @Test + public void shouldAddRemotePriorityTransactionsWhenGasPriceBelowMinimum() { + final Transaction transaction = createTransaction(1, Wei.of(7)); + transactionPool = + createTransactionPool(b -> b.prioritySenders(Set.of(transaction.getSender()))); + + givenTransactionIsValid(transaction); + + addAndAssertRemotePriorityTransactionsValid(transaction); + } + @Test public void shouldNotAddRemoteTransactionsThatAreInvalidAccordingToStateDependentChecks() { givenTransactionIsValid(transaction0); @@ -551,8 +561,8 @@ public void shouldNotAddRemoteTransactionsThatAreInvalidAccordingToStateDependen @ParameterizedTest @ValueSource(booleans = {true, false}) public void shouldAllowSequenceOfTransactionsWithIncreasingNonceFromSameSender( - final boolean disableLocalTxs) { - transactionPool = createTransactionPool(b -> b.disableLocalTransactions(disableLocalTxs)); + final boolean noLocalPriority) { + transactionPool = createTransactionPool(b -> b.noLocalPriority(noLocalPriority)); final Transaction transaction1 = createTransaction(1); final Transaction transaction2 = createTransaction(2); final Transaction transaction3 = createTransaction(3); @@ -561,9 +571,9 @@ public void shouldAllowSequenceOfTransactionsWithIncreasingNonceFromSameSender( givenTransactionIsValid(transaction2); givenTransactionIsValid(transaction3); - addAndAssertTransactionViaApiValid(transaction1, disableLocalTxs); - addAndAssertTransactionViaApiValid(transaction2, disableLocalTxs); - addAndAssertTransactionViaApiValid(transaction3, disableLocalTxs); + addAndAssertTransactionViaApiValid(transaction1, noLocalPriority); + addAndAssertTransactionViaApiValid(transaction2, noLocalPriority); + addAndAssertTransactionViaApiValid(transaction3, noLocalPriority); } @Test @@ -575,9 +585,9 @@ public void shouldAllowSequenceOfTransactionsWithIncreasingNonceFromSameSender( givenTransactionIsValid(transaction1); givenTransactionIsValid(transaction2); - addAndAssertRemoteTransactionValid(transaction2); - addAndAssertRemoteTransactionValid(transaction0); - addAndAssertRemoteTransactionValid(transaction1); + addAndAssertRemoteTransactionsValid(transaction2); + addAndAssertRemoteTransactionsValid(transaction0); + addAndAssertRemoteTransactionsValid(transaction1); } @Test @@ -597,22 +607,22 @@ public void shouldNotNotifyBatchListenerWhenRemoteTransactionDoesNotReplaceExist givenTransactionIsValid(transaction0a); givenTransactionIsValid(transaction0b); - addAndAssertRemoteTransactionValid(transaction0a); + addAndAssertRemoteTransactionsValid(transaction0a); addAndAssertRemoteTransactionInvalid(transaction0b); } @ParameterizedTest @ValueSource(booleans = {true, false}) public void shouldNotNotifyBatchListenerWhenLocalTransactionDoesNotReplaceExisting( - final boolean disableLocalTxs) { - transactionPool = createTransactionPool(b -> b.disableLocalTransactions(disableLocalTxs)); + final boolean noLocalPriority) { + transactionPool = createTransactionPool(b -> b.noLocalPriority(noLocalPriority)); final Transaction transaction0a = createTransaction(0, Wei.of(10)); final Transaction transaction0b = createTransaction(0, Wei.of(9)); givenTransactionIsValid(transaction0a); givenTransactionIsValid(transaction0b); - addAndAssertTransactionViaApiValid(transaction0a, disableLocalTxs); + addAndAssertTransactionViaApiValid(transaction0a, noLocalPriority); addAndAssertTransactionViaApiInvalid(transaction0b, TRANSACTION_REPLACEMENT_UNDERPRICED); } @@ -638,7 +648,7 @@ public void shouldRejectRemoteTransactionsWhereGasLimitExceedBlockGasLimit() { @Test public void shouldAcceptLocalTransactionsEvenIfAnInvalidTransactionWithLowerNonceExists() { - transactionPool = createTransactionPool(b -> b.disableLocalTransactions(false)); + transactionPool = createTransactionPool(b -> b.noLocalPriority(false)); final Transaction invalidTx = createBaseTransaction(0).gasLimit(blockGasLimit + 1).createTransaction(KEY_PAIR1); @@ -653,8 +663,8 @@ public void shouldAcceptLocalTransactionsEvenIfAnInvalidTransactionWithLowerNonc @ParameterizedTest @ValueSource(booleans = {true, false}) - public void shouldRejectLocalTransactionsWhenNonceTooFarInFuture(final boolean disableLocalTxs) { - transactionPool = createTransactionPool(b -> b.disableLocalTransactions(disableLocalTxs)); + public void shouldRejectLocalTransactionsWhenNonceTooFarInFuture(final boolean noLocalPriority) { + transactionPool = createTransactionPool(b -> b.noLocalPriority(noLocalPriority)); final Transaction transactionFarFuture = createTransaction(Integer.MAX_VALUE); givenTransactionIsValid(transactionFarFuture); @@ -732,22 +742,23 @@ public void shouldCallValidatorWithExpectedValidationParameters() { @ParameterizedTest @ValueSource(booleans = {true, false}) - public void shouldIgnoreFeeCapIfSetZero(final boolean disableLocalTxs) { + public void shouldIgnoreFeeCapIfSetZero(final boolean noLocalPriority) { final Wei twoEthers = Wei.fromEth(2); transactionPool = - createTransactionPool(b -> b.txFeeCap(Wei.ZERO).disableLocalTransactions(disableLocalTxs)); + createTransactionPool(b -> b.txFeeCap(Wei.ZERO).noLocalPriority(noLocalPriority)); final Transaction transaction = createTransaction(0, twoEthers.add(Wei.of(1))); givenTransactionIsValid(transaction); - addAndAssertTransactionViaApiValid(transaction, disableLocalTxs); + addAndAssertTransactionViaApiValid(transaction, noLocalPriority); } - @Test - public void shouldRejectLocalTransactionIfFeeCapExceeded() { + @ParameterizedTest + @ValueSource(booleans = {true, false}) + public void shouldRejectLocalTransactionIfFeeCapExceeded(final boolean noLocalPriority) { final Wei twoEthers = Wei.fromEth(2); transactionPool = - createTransactionPool(b -> b.txFeeCap(twoEthers).disableLocalTransactions(false)); + createTransactionPool(b -> b.txFeeCap(twoEthers).noLocalPriority(noLocalPriority)); final Transaction transactionLocal = createTransaction(0, twoEthers.add(1)); @@ -758,8 +769,23 @@ public void shouldRejectLocalTransactionIfFeeCapExceeded() { @ParameterizedTest @ValueSource(booleans = {true, false}) - public void shouldRejectZeroGasPriceLocalTransactionWhenNotMining(final boolean disableLocalTxs) { - transactionPool = createTransactionPool(b -> b.disableLocalTransactions(disableLocalTxs)); + public void shouldAcceptRemoteTransactionEvenIfFeeCapExceeded(final boolean hasPriority) { + final Wei twoEthers = Wei.fromEth(2); + final Transaction remoteTransaction = createTransaction(0, twoEthers.add(1)); + final Set
prioritySenders = + hasPriority ? Set.of(remoteTransaction.getSender()) : Set.of(); + transactionPool = + createTransactionPool(b -> b.txFeeCap(twoEthers).prioritySenders(prioritySenders)); + + givenTransactionIsValid(remoteTransaction); + + addAndAssertRemoteTransactionsValid(hasPriority, remoteTransaction); + } + + @ParameterizedTest + @ValueSource(booleans = {true, false}) + public void shouldRejectZeroGasPriceLocalTransactionWhenNotMining(final boolean noLocalPriority) { + transactionPool = createTransactionPool(b -> b.noLocalPriority(noLocalPriority)); when(miningParameters.isMiningEnabled()).thenReturn(false); final Transaction transaction = createTransaction(0, Wei.ZERO); @@ -769,39 +795,78 @@ public void shouldRejectZeroGasPriceLocalTransactionWhenNotMining(final boolean addAndAssertTransactionViaApiInvalid(transaction, GAS_PRICE_TOO_LOW); } + @Test + @DisabledIf("isBaseFeeMarket") + public void shouldAcceptZeroGasPriceFrontierLocalPriorityTransactionsWhenMining() { + transactionPool = createTransactionPool(b -> b.noLocalPriority(false)); + when(miningParameters.isMiningEnabled()).thenReturn(true); + + final Transaction transaction = createTransaction(0, Wei.ZERO); + + givenTransactionIsValid(transaction); + + addAndAssertTransactionViaApiValid(transaction, false); + } + @ParameterizedTest @ValueSource(booleans = {true, false}) - public void transactionNotRejectedByPluginShouldBeAdded(final boolean disableLocalTxs) { + public void shouldRejectZeroGasPriceRemoteTransactionWhenNotMining(final boolean hasPriority) { + final Transaction transaction = createTransaction(0, Wei.ZERO); + final Set
prioritySenders = hasPriority ? Set.of(transaction.getSender()) : Set.of(); + transactionPool = createTransactionPool(b -> b.prioritySenders(prioritySenders)); + when(miningParameters.isMiningEnabled()).thenReturn(false); + + givenTransactionIsValid(transaction); + + addAndAssertRemoteTransactionInvalid(transaction); + } + + @Test + @DisabledIf("isBaseFeeMarket") + public void shouldAcceptZeroGasPriceFrontierRemotePriorityTransactionsWhenMining() { + final Transaction transaction = createTransaction(0, Wei.ZERO); + transactionPool = + createTransactionPool(b -> b.prioritySenders(Set.of(transaction.getSender()))); + when(miningParameters.isMiningEnabled()).thenReturn(true); + + givenTransactionIsValid(transaction); + + addAndAssertRemoteTransactionsValid(true, transaction); + } + + @ParameterizedTest + @ValueSource(booleans = {true, false}) + public void transactionNotRejectedByPluginShouldBeAdded(final boolean noLocalPriority) { final PluginTransactionValidatorFactory pluginTransactionValidatorFactory = - getPluginTransactionValidatorFactoryReturning(true); + getPluginTransactionValidatorFactoryReturning(null); // null -> not rejecting !! this.transactionPool = createTransactionPool( - b -> b.disableLocalTransactions(disableLocalTxs), pluginTransactionValidatorFactory); + b -> b.noLocalPriority(noLocalPriority), pluginTransactionValidatorFactory); givenTransactionIsValid(transaction0); - addAndAssertTransactionViaApiValid(transaction0, disableLocalTxs); + addAndAssertTransactionViaApiValid(transaction0, noLocalPriority); } @ParameterizedTest @ValueSource(booleans = {true, false}) - public void transactionRejectedByPluginShouldNotBeAdded(final boolean disableLocalTxs) { + public void transactionRejectedByPluginShouldNotBeAdded(final boolean noLocalPriority) { final PluginTransactionValidatorFactory pluginTransactionValidatorFactory = - getPluginTransactionValidatorFactoryReturning(false); + getPluginTransactionValidatorFactoryReturning("false"); this.transactionPool = createTransactionPool( - b -> b.disableLocalTransactions(disableLocalTxs), pluginTransactionValidatorFactory); + b -> b.noLocalPriority(noLocalPriority), pluginTransactionValidatorFactory); givenTransactionIsValid(transaction0); addAndAssertTransactionViaApiInvalid( - transaction0, TransactionInvalidReason.PLUGIN_TX_VALIDATOR_INVALIDATED); + transaction0, TransactionInvalidReason.PLUGIN_TX_VALIDATOR); } @Test public void remoteTransactionRejectedByPluginShouldNotBeAdded() { final PluginTransactionValidatorFactory pluginTransactionValidatorFactory = - getPluginTransactionValidatorFactoryReturning(false); + getPluginTransactionValidatorFactoryReturning("false"); this.transactionPool = createTransactionPool(b -> {}, pluginTransactionValidatorFactory); givenTransactionIsValid(transaction0); @@ -814,17 +879,15 @@ public void remoteTransactionRejectedByPluginShouldNotBeAdded() { @DisabledIf("isBaseFeeMarket") public void addLocalTransaction_strictReplayProtectionOn_txWithoutChainId_chainIdIsConfigured_protectionNotSupportedAtCurrentBlock( - final boolean disableLocalTxs) { + final boolean noLocalPriority) { protocolSupportsTxReplayProtection(1337, false); transactionPool = createTransactionPool( - b -> - b.strictTransactionReplayProtectionEnabled(true) - .disableLocalTransactions(disableLocalTxs)); + b -> b.strictTransactionReplayProtectionEnabled(true).noLocalPriority(noLocalPriority)); final Transaction tx = createTransactionWithoutChainId(1); givenTransactionIsValid(tx); - addAndAssertTransactionViaApiValid(tx, disableLocalTxs); + addAndAssertTransactionViaApiValid(tx, noLocalPriority); } @Test @@ -836,24 +899,23 @@ public void remoteTransactionRejectedByPluginShouldNotBeAdded() { final Transaction tx = createTransactionWithoutChainId(1); givenTransactionIsValid(tx); - addAndAssertRemoteTransactionValid(tx); + addAndAssertRemoteTransactionsValid(tx); } @ParameterizedTest @ValueSource(booleans = {true, false}) @DisabledIf("isBaseFeeMarket") public void addLocalTransaction_strictReplayProtectionOff_txWithoutChainId_chainIdIsConfigured( - final boolean disableLocalTxs) { + final boolean noLocalPriority) { protocolSupportsTxReplayProtection(1337, true); transactionPool = createTransactionPool( b -> - b.strictTransactionReplayProtectionEnabled(false) - .disableLocalTransactions(disableLocalTxs)); + b.strictTransactionReplayProtectionEnabled(false).noLocalPriority(noLocalPriority)); final Transaction tx = createTransactionWithoutChainId(1); givenTransactionIsValid(tx); - addAndAssertTransactionViaApiValid(tx, disableLocalTxs); + addAndAssertTransactionViaApiValid(tx, noLocalPriority); } @Test @@ -876,24 +938,22 @@ public void addLocalTransaction_strictReplayProtectionOn_txWithoutChainId_chainI final Transaction tx = createTransactionWithoutChainId(1); givenTransactionIsValid(tx); - addAndAssertRemoteTransactionValid(tx); + addAndAssertRemoteTransactionsValid(tx); } @ParameterizedTest @ValueSource(booleans = {true, false}) @DisabledIf("isBaseFeeMarket") public void addLocalTransaction_strictReplayProtectionOn_txWithoutChainId_chainIdIsNotConfigured( - final boolean disableLocalTxs) { + final boolean noLocalPriority) { protocolDoesNotSupportTxReplayProtection(); transactionPool = createTransactionPool( - b -> - b.strictTransactionReplayProtectionEnabled(true) - .disableLocalTransactions(disableLocalTxs)); + b -> b.strictTransactionReplayProtectionEnabled(true).noLocalPriority(noLocalPriority)); final Transaction tx = createTransactionWithoutChainId(1); givenTransactionIsValid(tx); - addAndAssertTransactionViaApiValid(tx, disableLocalTxs); + addAndAssertTransactionViaApiValid(tx, noLocalPriority); } @Test @@ -905,7 +965,7 @@ public void addLocalTransaction_strictReplayProtectionOn_txWithoutChainId_chainI final Transaction tx = createTransactionWithoutChainId(1); givenTransactionIsValid(tx); - addAndAssertRemoteTransactionValid(tx); + addAndAssertRemoteTransactionsValid(tx); } @Test @@ -925,39 +985,26 @@ public void shouldIgnoreEIP1559TransactionWhenNotAllowed() { addAndAssertTransactionViaApiInvalid(transaction, INVALID_TRANSACTION_FORMAT); } - @Test - @DisabledIf("isBaseFeeMarket") - public void shouldAcceptZeroGasPriceFrontierLocalTransactionsWhenMining() { - transactionPool = createTransactionPool(b -> b.disableLocalTransactions(false)); - when(miningParameters.isMiningEnabled()).thenReturn(true); - - final Transaction transaction = createTransaction(0, Wei.ZERO); - - givenTransactionIsValid(transaction); - - addAndAssertTransactionViaApiValid(transaction, false); - } - @ParameterizedTest @ValueSource(booleans = {true, false}) @DisabledIf("isBaseFeeMarket") public void shouldAcceptZeroGasPriceTransactionWhenMinGasPriceIsZero( - final boolean disableLocalTxs) { - transactionPool = createTransactionPool(b -> b.disableLocalTransactions(disableLocalTxs)); + final boolean noLocalPriority) { + transactionPool = createTransactionPool(b -> b.noLocalPriority(noLocalPriority)); when(miningParameters.getMinTransactionGasPrice()).thenReturn(Wei.ZERO); final Transaction transaction = createTransaction(0, Wei.ZERO); givenTransactionIsValid(transaction); - addAndAssertTransactionViaApiValid(transaction, disableLocalTxs); + addAndAssertTransactionViaApiValid(transaction, noLocalPriority); } @ParameterizedTest @ValueSource(booleans = {true, false}) public void shouldAcceptZeroGasPriceFrontierTxsWhenMinGasPriceIsZeroAndLondonWithZeroBaseFee( - final boolean disableLocalTxs) { - transactionPool = createTransactionPool(b -> b.disableLocalTransactions(disableLocalTxs)); + final boolean noLocalPriority) { + transactionPool = createTransactionPool(b -> b.noLocalPriority(noLocalPriority)); when(miningParameters.getMinTransactionGasPrice()).thenReturn(Wei.ZERO); when(protocolSpec.getFeeMarket()).thenReturn(FeeMarket.london(0, Optional.of(Wei.ZERO))); whenBlockBaseFeeIs(Wei.ZERO); @@ -965,14 +1012,14 @@ public void shouldAcceptZeroGasPriceFrontierTxsWhenMinGasPriceIsZeroAndLondonWit final Transaction frontierTransaction = createFrontierTransaction(0, Wei.ZERO); givenTransactionIsValid(frontierTransaction); - addAndAssertTransactionViaApiValid(frontierTransaction, disableLocalTxs); + addAndAssertTransactionViaApiValid(frontierTransaction, noLocalPriority); } @ParameterizedTest @ValueSource(booleans = {true, false}) public void shouldAcceptZeroGasPrice1559TxsWhenMinGasPriceIsZeroAndLondonWithZeroBaseFee( - final boolean disableLocalTxs) { - transactionPool = createTransactionPool(b -> b.disableLocalTransactions(disableLocalTxs)); + final boolean noLocalPriority) { + transactionPool = createTransactionPool(b -> b.noLocalPriority(noLocalPriority)); when(miningParameters.getMinTransactionGasPrice()).thenReturn(Wei.ZERO); when(protocolSpec.getFeeMarket()).thenReturn(FeeMarket.london(0, Optional.of(Wei.ZERO))); whenBlockBaseFeeIs(Wei.ZERO); @@ -980,12 +1027,12 @@ public void shouldAcceptZeroGasPrice1559TxsWhenMinGasPriceIsZeroAndLondonWithZer final Transaction transaction = createTransaction(0, Wei.ZERO); givenTransactionIsValid(transaction); - addAndAssertTransactionViaApiValid(transaction, disableLocalTxs); + addAndAssertTransactionViaApiValid(transaction, noLocalPriority); } @Test - public void shouldAcceptBaseFeeFloorGasPriceFrontierLocalTransactionsWhenMining() { - transactionPool = createTransactionPool(b -> b.disableLocalTransactions(false)); + public void shouldAcceptBaseFeeFloorGasPriceFrontierLocalPriorityTransactionsWhenMining() { + transactionPool = createTransactionPool(b -> b.noLocalPriority(false)); final Transaction frontierTransaction = createFrontierTransaction(0, BASE_FEE_FLOOR); givenTransactionIsValid(frontierTransaction); @@ -994,7 +1041,19 @@ public void shouldAcceptBaseFeeFloorGasPriceFrontierLocalTransactionsWhenMining( } @Test - public void shouldRejectRemote1559TxsWhenMaxFeePerGasBelowMinGasPrice() { + public void shouldAcceptBaseFeeFloorGasPriceFrontierRemotePriorityTransactionsWhenMining() { + final Transaction frontierTransaction = createFrontierTransaction(0, BASE_FEE_FLOOR); + transactionPool = + createTransactionPool(b -> b.prioritySenders(Set.of(frontierTransaction.getSender()))); + + givenTransactionIsValid(frontierTransaction); + + addAndAssertRemoteTransactionsValid(frontierTransaction); + } + + @ParameterizedTest + @ValueSource(booleans = {true, false}) + public void shouldRejectRemote1559TxsWhenMaxFeePerGasBelowMinGasPrice(final boolean hasPriority) { final Wei genesisBaseFee = Wei.of(100L); final Wei minGasPrice = Wei.of(200L); final Wei lastBlockBaseFee = minGasPrice.add(50L); @@ -1002,12 +1061,14 @@ public void shouldRejectRemote1559TxsWhenMaxFeePerGasBelowMinGasPrice() { assertThat( add1559TxAndGetPendingTxsCount( - genesisBaseFee, minGasPrice, lastBlockBaseFee, txMaxFeePerGas, false)) + genesisBaseFee, minGasPrice, lastBlockBaseFee, txMaxFeePerGas, false, hasPriority)) .isEqualTo(0); } - @Test - public void shouldAcceptRemote1559TxsWhenMaxFeePerGasIsAtLeastEqualToMinGasPrice() { + @ParameterizedTest + @ValueSource(booleans = {true, false}) + public void shouldAcceptRemote1559TxsWhenMaxFeePerGasIsAtLeastEqualToMinGasPrice( + final boolean hasPriority) { final Wei genesisBaseFee = Wei.of(100L); final Wei minGasPrice = Wei.of(200L); final Wei lastBlockBaseFee = minGasPrice.add(50L); @@ -1015,7 +1076,7 @@ public void shouldAcceptRemote1559TxsWhenMaxFeePerGasIsAtLeastEqualToMinGasPrice assertThat( add1559TxAndGetPendingTxsCount( - genesisBaseFee, minGasPrice, lastBlockBaseFee, txMaxFeePerGas, false)) + genesisBaseFee, minGasPrice, lastBlockBaseFee, txMaxFeePerGas, false, hasPriority)) .isEqualTo(1); } @@ -1028,7 +1089,7 @@ public void shouldRejectLocal1559TxsWhenMaxFeePerGasBelowMinGasPrice() { assertThat( add1559TxAndGetPendingTxsCount( - genesisBaseFee, minGasPrice, lastBlockBaseFee, txMaxFeePerGas, true)) + genesisBaseFee, minGasPrice, lastBlockBaseFee, txMaxFeePerGas, true, true)) .isEqualTo(0); } @@ -1041,7 +1102,7 @@ public void shouldAcceptLocal1559TxsWhenMaxFeePerGasIsAtLeastEqualToMinMinGasPri assertThat( add1559TxAndGetPendingTxsCount( - genesisBaseFee, minGasPrice, lastBlockBaseFee, txMaxFeePerGas, true)) + genesisBaseFee, minGasPrice, lastBlockBaseFee, txMaxFeePerGas, true, true)) .isEqualTo(1); } @@ -1065,8 +1126,9 @@ public void addRemoteTransactionsShouldAllowDuplicates() { } private static PluginTransactionValidatorFactory getPluginTransactionValidatorFactoryReturning( - final boolean b) { - final PluginTransactionValidator pluginTransactionValidator = transaction -> b; + final String errorMessage) { + final PluginTransactionValidator pluginTransactionValidator = + transaction -> Optional.ofNullable(errorMessage); return () -> pluginTransactionValidator; } @@ -1093,7 +1155,16 @@ protected void assertTransactionPending(final Transaction t) { assertThat(transactions.getTransactionByHash(t.getHash())).contains(t); } - protected void addAndAssertRemoteTransactionValid(final Transaction... txs) { + protected void addAndAssertRemoteTransactionsValid(final Transaction... txs) { + addAndAssertRemoteTransactionsValid(false, txs); + } + + protected void addAndAssertRemotePriorityTransactionsValid(final Transaction... txs) { + addAndAssertRemoteTransactionsValid(true, txs); + } + + protected void addAndAssertRemoteTransactionsValid( + final boolean hasPriority, final Transaction... txs) { transactionPool.addRemoteTransactions(List.of(txs)); verify(transactionBroadcaster) @@ -1101,24 +1172,24 @@ protected void addAndAssertRemoteTransactionValid(final Transaction... txs) { argThat(btxs -> btxs.size() == txs.length && btxs.containsAll(List.of(txs)))); Arrays.stream(txs).forEach(this::assertTransactionPending); assertThat(transactions.getLocalTransactions()).doesNotContain(txs); - } - - protected void addAndAssertTransactionViaApiValid(final Transaction tx) { - addAndAssertTransactionViaApiValid(tx, false); + if (hasPriority) { + assertThat(transactions.getPriorityTransactions()).contains(txs); + } } protected void addAndAssertTransactionViaApiValid( - final Transaction tx, final boolean disableLocals) { + final Transaction tx, final boolean disableLocalPriority) { final ValidationResult result = transactionPool.addTransactionViaApi(tx); assertThat(result.isValid()).isTrue(); assertTransactionPending(tx); verify(transactionBroadcaster).onTransactionsAdded(singletonList(tx)); - if (disableLocals) { - assertThat(transactions.getLocalTransactions()).doesNotContain(tx); + assertThat(transactions.getLocalTransactions()).contains(tx); + if (disableLocalPriority) { + assertThat(transactions.getPriorityTransactions()).doesNotContain(tx); } else { - assertThat(transactions.getLocalTransactions()).contains(tx); + assertThat(transactions.getPriorityTransactions()).contains(tx); } } @@ -1242,13 +1313,17 @@ protected int add1559TxAndGetPendingTxsCount( final Wei minGasPrice, final Wei lastBlockBaseFee, final Wei txMaxFeePerGas, - final boolean isLocal) { + final boolean isLocal, + final boolean hasPriority) { when(miningParameters.getMinTransactionGasPrice()).thenReturn(minGasPrice); when(protocolSpec.getFeeMarket()).thenReturn(FeeMarket.london(0, Optional.of(genesisBaseFee))); whenBlockBaseFeeIs(lastBlockBaseFee); final Transaction transaction = createTransaction(0, txMaxFeePerGas); - + if (hasPriority) { + transactionPool = + createTransactionPool(b -> b.prioritySenders(Set.of(transaction.getSender()))); + } givenTransactionIsValid(transaction); if (isLocal) { diff --git a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransactionEstimatedMemorySizeTest.java b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransactionEstimatedMemorySizeTest.java index 54cbe804ab7..dde45d0a95b 100644 --- a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransactionEstimatedMemorySizeTest.java +++ b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransactionEstimatedMemorySizeTest.java @@ -19,52 +19,63 @@ import org.hyperledger.besu.crypto.SignatureAlgorithm; import org.hyperledger.besu.datatypes.AccessListEntry; import org.hyperledger.besu.datatypes.Address; +import org.hyperledger.besu.datatypes.BlobsWithCommitments; import org.hyperledger.besu.datatypes.TransactionType; import org.hyperledger.besu.datatypes.Wei; import org.hyperledger.besu.ethereum.core.Transaction; import org.hyperledger.besu.ethereum.core.TransactionTestFixture; +import org.hyperledger.besu.ethereum.core.encoding.EncodingContext; +import org.hyperledger.besu.ethereum.core.encoding.TransactionDecoder; +import org.hyperledger.besu.ethereum.core.encoding.TransactionEncoder; import org.hyperledger.besu.ethereum.eth.transactions.layered.BaseTransactionPoolTest; import org.hyperledger.besu.ethereum.rlp.BytesValueRLPInput; import org.hyperledger.besu.ethereum.rlp.BytesValueRLPOutput; +import java.math.BigInteger; +import java.util.Arrays; import java.util.HashSet; import java.util.List; import java.util.Optional; import java.util.Set; +import java.util.SortedSet; +import java.util.TreeSet; import java.util.concurrent.atomic.LongAdder; +import java.util.function.Function; +import com.google.common.collect.Sets; import org.apache.tuweni.bytes.Bytes; import org.apache.tuweni.bytes.Bytes32; -import org.junit.jupiter.api.Disabled; import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.condition.EnabledOnOs; +import org.junit.jupiter.api.condition.OS; import org.openjdk.jol.info.ClassLayout; import org.openjdk.jol.info.GraphPathRecord; import org.openjdk.jol.info.GraphVisitor; import org.openjdk.jol.info.GraphWalker; -@Disabled("Need to handle different results on different OS") +@EnabledOnOs(OS.LINUX) public class PendingTransactionEstimatedMemorySizeTest extends BaseTransactionPoolTest { private static final Set> SHARED_CLASSES = Set.of(SignatureAlgorithm.class, TransactionType.class); - private static final Set EIP1559_CONSTANT_FIELD_PATHS = Set.of(".gasPrice"); - private static final Set EIP1559_VARIABLE_SIZE_PATHS = - Set.of(".to", ".payload", ".maybeAccessList"); - + private static final Set COMMON_CONSTANT_FIELD_PATHS = + Set.of(".value.ctor", ".hashNoSignature"); + private static final Set EIP1559_EIP4844_CONSTANT_FIELD_PATHS = + Sets.union(COMMON_CONSTANT_FIELD_PATHS, Set.of(".gasPrice")); private static final Set FRONTIER_ACCESS_LIST_CONSTANT_FIELD_PATHS = - Set.of(".maxFeePerGas", ".maxPriorityFeePerGas"); - private static final Set FRONTIER_ACCESS_LIST_VARIABLE_SIZE_PATHS = - Set.of(".to", ".payload", ".maybeAccessList"); + Sets.union(COMMON_CONSTANT_FIELD_PATHS, Set.of(".maxFeePerGas", ".maxPriorityFeePerGas")); + private static final Set VARIABLE_SIZE_PATHS = + Set.of(".chainId", ".to", ".payload", ".maybeAccessList"); @Test public void toSize() { TransactionTestFixture preparedTx = - prepareTransaction(TransactionType.ACCESS_LIST, 10, Wei.of(500), 10); + prepareTransaction(TransactionType.ACCESS_LIST, 10, Wei.of(500), 10, 0); Transaction txTo = preparedTx.to(Optional.of(Address.extract(Bytes32.random()))).createTransaction(KEYS1); BytesValueRLPOutput rlpOut = new BytesValueRLPOutput(); txTo.writeTo(rlpOut); - txTo = Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)); + txTo = Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)).detachedCopy(); System.out.println(txTo.getSender()); System.out.println(txTo.getHash()); System.out.println(txTo.getSize()); @@ -78,34 +89,17 @@ public void toSize() { GraphVisitor gv = gpr -> { - // byte[] is shared so only count the specific part for each field - if (gpr.path().endsWith(".bytes")) { - if (gpr.path().contains("delegate")) { - size.add(20); - System.out.println( - "(" - + size - + ")[20 = fixed address size; overrides: " - + gpr.size() - + ", " - + gpr.path() - + ", " - + gpr.klass().toString() - + "]"); - } - } else { - size.add(gpr.size()); - System.out.println( - "(" - + size - + ")[" - + gpr.size() - + ", " - + gpr.path() - + ", " - + gpr.klass().toString() - + "]"); - } + size.add(gpr.size()); + System.out.println( + "(" + + size + + ")[" + + gpr.size() + + ", " + + gpr.path() + + ", " + + gpr.klass().toString() + + "]"); }; GraphWalker gw = new GraphWalker(gv); @@ -121,12 +115,13 @@ public void toSize() { public void payloadSize() { TransactionTestFixture preparedTx = - prepareTransaction(TransactionType.ACCESS_LIST, 10, Wei.of(500), 10); + prepareTransaction(TransactionType.ACCESS_LIST, 10, Wei.of(500), 10, 0); Transaction txPayload = preparedTx.createTransaction(KEYS1); BytesValueRLPOutput rlpOut = new BytesValueRLPOutput(); txPayload.writeTo(rlpOut); - txPayload = Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)); + txPayload = + Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)).detachedCopy(); System.out.println(txPayload.getSender()); System.out.println(txPayload.getHash()); System.out.println(txPayload.getSize()); @@ -141,16 +136,141 @@ public void payloadSize() { assertThat(size.sum()).isEqualTo(PendingTransaction.PAYLOAD_BASE_MEMORY_SIZE); } + @Test + public void chainIdSize() { + + BigInteger chainId = BigInteger.valueOf(1); + Optional maybeChainId = Optional.of(chainId); + + final ClassLayout cl = ClassLayout.parseInstance(maybeChainId); + System.out.println(cl.toPrintable()); + LongAdder size = new LongAdder(); + size.add(cl.instanceSize()); + System.out.println("Base chainId size: " + size); + + GraphVisitor gv = + gpr -> { + size.add(gpr.size()); + System.out.println( + "(" + + size + + ")[" + + gpr.size() + + ", " + + gpr.path() + + ", " + + gpr.klass().toString() + + "]"); + }; + + GraphWalker gw = new GraphWalker(gv); + + gw.walk(maybeChainId); + + assertThat(size.sum()).isEqualTo(PendingTransaction.OPTIONAL_CHAIN_ID_MEMORY_SIZE); + } + + @Test + public void kgzCommitmentsSize() { + blobsWithCommitmentsFieldSize( + t -> t.getBlobsWithCommitments().get().getKzgCommitments(), + PendingTransaction.BASE_LIST_SIZE, + PendingTransaction.KZG_COMMITMENT_OR_PROOF_SIZE); + } + + @Test + public void kgzProofsSize() { + blobsWithCommitmentsFieldSize( + t -> t.getBlobsWithCommitments().get().getKzgProofs(), + PendingTransaction.BASE_LIST_SIZE, + PendingTransaction.KZG_COMMITMENT_OR_PROOF_SIZE); + } + + @Test + public void blobsSize() { + blobsWithCommitmentsFieldSize( + t -> t.getBlobsWithCommitments().get().getBlobs(), + PendingTransaction.BASE_LIST_SIZE, + PendingTransaction.BLOB_SIZE); + } + + @Test + public void versionedHashesSize() { + blobsWithCommitmentsFieldSize( + t -> t.getBlobsWithCommitments().get().getVersionedHashes(), + PendingTransaction.BASE_LIST_SIZE, + PendingTransaction.VERSIONED_HASH_SIZE); + } + + private void blobsWithCommitmentsFieldSize( + final Function> containerExtractor, + final long containerSize, + final long itemSize) { + TransactionTestFixture preparedTx = + prepareTransaction(TransactionType.BLOB, 10, Wei.of(500), 10, 1); + Transaction txBlob = preparedTx.createTransaction(KEYS1); + BytesValueRLPOutput rlpOut = new BytesValueRLPOutput(); + TransactionEncoder.encodeRLP(txBlob, rlpOut, EncodingContext.POOLED_TRANSACTION); + + txBlob = + TransactionDecoder.decodeRLP( + new BytesValueRLPInput(rlpOut.encoded(), false), EncodingContext.POOLED_TRANSACTION) + .detachedCopy(); + System.out.println(txBlob.getSender()); + System.out.println(txBlob.getHash()); + System.out.println(txBlob.getSize()); + + final List list = containerExtractor.apply(txBlob); + + final long cSize = sizeOfField(list, ".elements["); + + System.out.println("Container size: " + cSize); + + assertThat(cSize).isEqualTo(containerSize); + + final Object item = list.get(0); + final long iSize = sizeOfField(item); + + System.out.println("Item size: " + iSize); + + assertThat(iSize).isEqualTo(itemSize); + } + + @Test + public void blobsWithCommitmentsSize() { + TransactionTestFixture preparedTx = + prepareTransaction(TransactionType.BLOB, 10, Wei.of(500), 10, 1); + Transaction txBlob = preparedTx.createTransaction(KEYS1); + BytesValueRLPOutput rlpOut = new BytesValueRLPOutput(); + TransactionEncoder.encodeRLP(txBlob, rlpOut, EncodingContext.POOLED_TRANSACTION); + + txBlob = + TransactionDecoder.decodeRLP( + new BytesValueRLPInput(rlpOut.encoded(), false), EncodingContext.POOLED_TRANSACTION) + .detachedCopy(); + System.out.println(txBlob.getSender()); + System.out.println(txBlob.getHash()); + System.out.println(txBlob.getSize()); + + final BlobsWithCommitments bwc = txBlob.getBlobsWithCommitments().get(); + final ClassLayout cl = ClassLayout.parseInstance(bwc); + System.out.println(cl.toPrintable()); + System.out.println("BlobsWithCommitments size: " + cl.instanceSize()); + + assertThat(cl.instanceSize()).isEqualTo(PendingTransaction.BLOBS_WITH_COMMITMENTS_SIZE); + } + @Test public void pendingTransactionSize() { TransactionTestFixture preparedTx = - prepareTransaction(TransactionType.ACCESS_LIST, 10, Wei.of(500), 10); + prepareTransaction(TransactionType.ACCESS_LIST, 10, Wei.of(500), 10, 0); Transaction txPayload = preparedTx.createTransaction(KEYS1); BytesValueRLPOutput rlpOut = new BytesValueRLPOutput(); txPayload.writeTo(rlpOut); - txPayload = Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)); + txPayload = + Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)).detachedCopy(); System.out.println(txPayload.getSender()); System.out.println(txPayload.getHash()); System.out.println(txPayload.getSize()); @@ -176,12 +296,13 @@ public void accessListSize() { final List ales = List.of(ale1); TransactionTestFixture preparedTx = - prepareTransaction(TransactionType.ACCESS_LIST, 0, Wei.of(500), 0); + prepareTransaction(TransactionType.ACCESS_LIST, 0, Wei.of(500), 0, 0); Transaction txAccessList = preparedTx.accessList(ales).createTransaction(KEYS1); BytesValueRLPOutput rlpOut = new BytesValueRLPOutput(); txAccessList.writeTo(rlpOut); - txAccessList = Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)); + txAccessList = + Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)).detachedCopy(); System.out.println(txAccessList.getSender()); System.out.println(txAccessList.getHash()); System.out.println(txAccessList.getSize()); @@ -200,55 +321,11 @@ public void accessListSize() { final AccessListEntry ale = optAL.get().get(0); - final ClassLayout cl3 = ClassLayout.parseInstance(ale); - System.out.println(cl3.toPrintable()); - System.out.println("AccessListEntry size: " + cl3.instanceSize()); - - LongAdder size = new LongAdder(); - size.add(cl3.instanceSize()); - - GraphVisitor gv = - gpr -> { - // byte[] is shared so only count the specific part for each field - if (gpr.path().endsWith(".bytes")) { - if (gpr.path().contains("address")) { - size.add(20); - System.out.println( - "(" - + size - + ")[20 = fixed address size; overrides: " - + gpr.size() - + ", " - + gpr.path() - + ", " - + gpr.klass().toString() - + "]"); - } - } else if (!gpr.path() - .contains( - "storageKeys.elementData[")) { // exclude elements since we want the container - // size - size.add(gpr.size()); - System.out.println( - "(" - + size - + ")[" - + gpr.size() - + ", " - + gpr.path() - + ", " - + gpr.klass().toString() - + "]"); - } - }; - - GraphWalker gw = new GraphWalker(gv); - - gw.walk(ale); + long aleSize = sizeOfField(ale, "storageKeys.elementData["); - System.out.println("AccessListEntry container size: " + size); + System.out.println("AccessListEntry container size: " + aleSize); - assertThat(size.sum()).isEqualTo(PendingTransaction.ACCESS_LIST_ENTRY_BASE_MEMORY_SIZE); + assertThat(aleSize).isEqualTo(PendingTransaction.ACCESS_LIST_ENTRY_BASE_MEMORY_SIZE); final Bytes32 storageKey = ale.storageKeys().get(0); final ClassLayout cl4 = ClassLayout.parseInstance(storageKey); @@ -260,13 +337,14 @@ public void accessListSize() { } @Test - public void baseEIP1559TransactionMemorySize() { + public void baseEIP1559AndEIP4844TransactionMemorySize() { System.setProperty("jol.magicFieldOffset", "true"); Transaction txEip1559 = createEIP1559Transaction(1, KEYS1, 10); BytesValueRLPOutput rlpOut = new BytesValueRLPOutput(); txEip1559.writeTo(rlpOut); - txEip1559 = Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)); + txEip1559 = + Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)).detachedCopy(); System.out.println(txEip1559.getSender()); System.out.println(txEip1559.getHash()); System.out.println(txEip1559.getSize()); @@ -277,138 +355,141 @@ public void baseEIP1559TransactionMemorySize() { eip1559size.add(cl.instanceSize()); System.out.println(eip1559size); - final Set skipPrefixes = new HashSet<>(); - - GraphVisitor gv = - gpr -> { - if (!skipPrefixes.stream().anyMatch(sp -> gpr.path().startsWith(sp))) { - if (SHARED_CLASSES.stream().anyMatch(scz -> scz.isAssignableFrom(gpr.klass()))) { - skipPrefixes.add(gpr.path()); - } else if (!startWithAnyOf(EIP1559_CONSTANT_FIELD_PATHS, gpr) - && !startWithAnyOf(EIP1559_VARIABLE_SIZE_PATHS, gpr)) { - eip1559size.add(gpr.size()); - System.out.println( - "(" - + eip1559size - + ")[" - + gpr.size() - + ", " - + gpr.path() - + ", " - + gpr.klass().toString() - + "]"); - } - } - }; - - GraphWalker gw = new GraphWalker(gv); + final SortedSet fieldSizes = new TreeSet<>(); + GraphWalker gw = getGraphWalker(EIP1559_EIP4844_CONSTANT_FIELD_PATHS, fieldSizes); gw.walk(txEip1559); + fieldSizes.forEach( + fieldSize -> { + eip1559size.add(fieldSize.size()); + System.out.println( + "(" + + eip1559size + + ")[" + + fieldSize.size() + + ", " + + fieldSize.path() + + ", " + + fieldSize + + "]"); + }); + System.out.println("Base EIP1559 size: " + eip1559size); - assertThat(eip1559size.sum()).isEqualTo(PendingTransaction.EIP1559_BASE_MEMORY_SIZE); + assertThat(eip1559size.sum()) + .isEqualTo(PendingTransaction.EIP1559_AND_EIP4844_BASE_MEMORY_SIZE); } @Test - public void baseAccessListTransactionMemorySize() { + public void baseFrontierAndAccessListTransactionMemorySize() { System.setProperty("jol.magicFieldOffset", "true"); - Transaction txAccessList = - createTransaction(TransactionType.ACCESS_LIST, 1, Wei.of(500), 0, KEYS1); + Transaction txFrontier = createTransaction(TransactionType.FRONTIER, 1, Wei.of(500), 0, KEYS1); BytesValueRLPOutput rlpOut = new BytesValueRLPOutput(); - txAccessList.writeTo(rlpOut); + txFrontier.writeTo(rlpOut); - txAccessList = Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)); - System.out.println(txAccessList.getSender()); - System.out.println(txAccessList.getHash()); - System.out.println(txAccessList.getSize()); + txFrontier = + Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)).detachedCopy(); + System.out.println(txFrontier.getSender()); + System.out.println(txFrontier.getHash()); + System.out.println(txFrontier.getSize()); - final ClassLayout cl = ClassLayout.parseInstance(txAccessList); + final ClassLayout cl = ClassLayout.parseInstance(txFrontier); System.out.println(cl.toPrintable()); - LongAdder accessListSize = new LongAdder(); - accessListSize.add(cl.instanceSize()); - System.out.println(accessListSize); + LongAdder frontierSize = new LongAdder(); + frontierSize.add(cl.instanceSize()); + System.out.println(frontierSize); - final Set skipPrefixes = new HashSet<>(); + final SortedSet fieldSizes = new TreeSet<>(); + + GraphWalker gw = getGraphWalker(FRONTIER_ACCESS_LIST_CONSTANT_FIELD_PATHS, fieldSizes); + gw.walk(txFrontier); + + fieldSizes.forEach( + fieldSize -> { + frontierSize.add(fieldSize.size()); + System.out.println( + "(" + + frontierSize + + ")[" + + fieldSize.size() + + ", " + + fieldSize.path() + + ", " + + fieldSize + + "]"); + }); + + System.out.println("Base Frontier size: " + frontierSize); + assertThat(frontierSize.sum()) + .isEqualTo(PendingTransaction.FRONTIER_AND_ACCESS_LIST_BASE_MEMORY_SIZE); + } + + private GraphWalker getGraphWalker( + final Set constantFieldPaths, final SortedSet fieldSizes) { + final Set skipPrefixes = new HashSet<>(); GraphVisitor gv = gpr -> { if (!skipPrefixes.stream().anyMatch(sp -> gpr.path().startsWith(sp))) { if (SHARED_CLASSES.stream().anyMatch(scz -> scz.isAssignableFrom(gpr.klass()))) { skipPrefixes.add(gpr.path()); - } else if (!startWithAnyOf(FRONTIER_ACCESS_LIST_CONSTANT_FIELD_PATHS, gpr) - && !startWithAnyOf(FRONTIER_ACCESS_LIST_VARIABLE_SIZE_PATHS, gpr)) { - accessListSize.add(gpr.size()); - System.out.println( - "(" - + accessListSize - + ")[" - + gpr.size() - + ", " - + gpr.path() - + ", " - + gpr.klass().toString() - + "]"); + } else if (!startWithAnyOf(constantFieldPaths, gpr) + && !startWithAnyOf(VARIABLE_SIZE_PATHS, gpr)) { + + fieldSizes.add(new FieldSize(gpr.path(), gpr.klass(), gpr.size())); } } }; GraphWalker gw = new GraphWalker(gv); - - gw.walk(txAccessList); - System.out.println("Base Access List size: " + accessListSize); - assertThat(accessListSize.sum()).isEqualTo(PendingTransaction.ACCESS_LIST_BASE_MEMORY_SIZE); + return gw; } - @Test - public void baseFrontierTransactionMemorySize() { - System.setProperty("jol.magicFieldOffset", "true"); - Transaction txFrontier = createTransaction(TransactionType.FRONTIER, 1, Wei.of(500), 0, KEYS1); - BytesValueRLPOutput rlpOut = new BytesValueRLPOutput(); - txFrontier.writeTo(rlpOut); - - txFrontier = Transaction.readFrom(new BytesValueRLPInput(rlpOut.encoded(), false)); - System.out.println(txFrontier.getSender()); - System.out.println(txFrontier.getHash()); - System.out.println(txFrontier.getSize()); + private boolean startWithAnyOf(final Set prefixes, final GraphPathRecord path) { + return prefixes.stream().anyMatch(prefix -> path.path().startsWith(prefix)); + } - final ClassLayout cl = ClassLayout.parseInstance(txFrontier); + private long sizeOfField(final Object container, final String... excludePaths) { + final ClassLayout cl = ClassLayout.parseInstance(container); System.out.println(cl.toPrintable()); - LongAdder frontierSize = new LongAdder(); - frontierSize.add(cl.instanceSize()); - System.out.println(frontierSize); + System.out.println("Base container size: " + cl.instanceSize()); - final Set skipPrefixes = new HashSet<>(); + LongAdder size = new LongAdder(); + size.add(cl.instanceSize()); GraphVisitor gv = gpr -> { - if (!skipPrefixes.stream().anyMatch(sp -> gpr.path().startsWith(sp))) { - if (SHARED_CLASSES.stream().anyMatch(scz -> scz.isAssignableFrom(gpr.klass()))) { - skipPrefixes.add(gpr.path()); - } else if (!startWithAnyOf(FRONTIER_ACCESS_LIST_CONSTANT_FIELD_PATHS, gpr) - && !startWithAnyOf(FRONTIER_ACCESS_LIST_VARIABLE_SIZE_PATHS, gpr)) { - frontierSize.add(gpr.size()); - System.out.println( - "(" - + frontierSize - + ")[" - + gpr.size() - + ", " - + gpr.path() - + ", " - + gpr.klass().toString() - + "]"); - } + if (Arrays.stream(excludePaths) + .anyMatch(excludePath -> gpr.path().contains(excludePath))) { + System.out.println("Excluded path " + gpr.path()); + } else { + size.add(gpr.size()); + System.out.println( + "(" + + size + + ")[" + + gpr.size() + + ", " + + gpr.path() + + ", " + + gpr.klass().toString() + + "]"); } }; GraphWalker gw = new GraphWalker(gv); - gw.walk(txFrontier); - System.out.println("Base Frontier size: " + frontierSize); - assertThat(frontierSize.sum()).isEqualTo(PendingTransaction.FRONTIER_BASE_MEMORY_SIZE); + gw.walk(container); + + System.out.println("Container size: " + size); + return size.sum(); } - private boolean startWithAnyOf(final Set prefixes, final GraphPathRecord path) { - return prefixes.stream().anyMatch(prefix -> path.path().startsWith(prefix)); + record FieldSize(String path, Class clazz, long size) implements Comparable { + + @Override + public int compareTo(final FieldSize o) { + return path.compareTo(o.path); + } } } diff --git a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/BaseTransactionPoolTest.java b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/BaseTransactionPoolTest.java index b735055445a..0d69dac4faa 100644 --- a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/BaseTransactionPoolTest.java +++ b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/BaseTransactionPoolTest.java @@ -20,7 +20,13 @@ import org.hyperledger.besu.crypto.SignatureAlgorithm; import org.hyperledger.besu.crypto.SignatureAlgorithmFactory; import org.hyperledger.besu.datatypes.Address; +import org.hyperledger.besu.datatypes.Blob; +import org.hyperledger.besu.datatypes.BlobsWithCommitments; +import org.hyperledger.besu.datatypes.Hash; +import org.hyperledger.besu.datatypes.KZGCommitment; +import org.hyperledger.besu.datatypes.KZGProof; import org.hyperledger.besu.datatypes.TransactionType; +import org.hyperledger.besu.datatypes.VersionedHash; import org.hyperledger.besu.datatypes.Wei; import org.hyperledger.besu.ethereum.core.Transaction; import org.hyperledger.besu.ethereum.core.TransactionTestFixture; @@ -33,10 +39,12 @@ import java.util.Optional; import java.util.Random; +import java.util.stream.IntStream; import com.google.common.base.Supplier; import com.google.common.base.Suppliers; import org.apache.tuweni.bytes.Bytes; +import org.apache.tuweni.bytes.Bytes48; public class BaseTransactionPoolTest { @@ -82,6 +90,12 @@ protected Transaction createEIP1559Transaction( TransactionType.EIP1559, nonce, Wei.of(5000L).multiply(gasFeeMultiplier), 0, keys); } + protected Transaction createEIP4844Transaction( + final long nonce, final KeyPair keys, final int gasFeeMultiplier, final int blobCount) { + return createTransaction( + TransactionType.BLOB, nonce, Wei.of(5000L).multiply(gasFeeMultiplier), 0, blobCount, keys); + } + protected Transaction createTransaction( final long nonce, final Wei maxGasPrice, final int payloadSize, final KeyPair keys) { @@ -97,11 +111,26 @@ protected Transaction createTransaction( final Wei maxGasPrice, final int payloadSize, final KeyPair keys) { - return prepareTransaction(type, nonce, maxGasPrice, payloadSize).createTransaction(keys); + return createTransaction(type, nonce, maxGasPrice, payloadSize, 0, keys); + } + + protected Transaction createTransaction( + final TransactionType type, + final long nonce, + final Wei maxGasPrice, + final int payloadSize, + final int blobCount, + final KeyPair keys) { + return prepareTransaction(type, nonce, maxGasPrice, payloadSize, blobCount) + .createTransaction(keys); } protected TransactionTestFixture prepareTransaction( - final TransactionType type, final long nonce, final Wei maxGasPrice, final int payloadSize) { + final TransactionType type, + final long nonce, + final Wei maxGasPrice, + final int payloadSize, + final int blobCount) { var tx = new TransactionTestFixture() @@ -116,6 +145,24 @@ protected TransactionTestFixture prepareTransaction( if (type.supports1559FeeMarket()) { tx.maxFeePerGas(Optional.of(maxGasPrice)) .maxPriorityFeePerGas(Optional.of(maxGasPrice.divide(10))); + if (type.supportsBlob() && blobCount > 0) { + final var versionHashes = + IntStream.range(0, blobCount) + .mapToObj(i -> new VersionedHash((byte) 1, Hash.ZERO)) + .toList(); + final var kgzCommitments = + IntStream.range(0, blobCount) + .mapToObj(i -> new KZGCommitment(Bytes48.random())) + .toList(); + final var kzgProofs = + IntStream.range(0, blobCount).mapToObj(i -> new KZGProof(Bytes48.random())).toList(); + final var blobs = + IntStream.range(0, blobCount).mapToObj(i -> new Blob(Bytes.random(32 * 4096))).toList(); + tx.versionedHashes(Optional.of(versionHashes)); + final var blobsWithCommitments = + new BlobsWithCommitments(kgzCommitments, blobs, kzgProofs, versionHashes); + tx.blobsWithCommitments(Optional.of(blobsWithCommitments)); + } } else { tx.gasPrice(maxGasPrice); } @@ -136,6 +183,11 @@ protected PendingTransaction createRemotePendingTransaction(final Transaction tr return new PendingTransaction.Remote(transaction); } + protected PendingTransaction createRemotePendingTransaction( + final Transaction transaction, final boolean hasPriority) { + return PendingTransaction.newPendingTransaction(transaction, false, hasPriority); + } + protected PendingTransaction createLocalPendingTransaction(final Transaction transaction) { return new PendingTransaction.Local(transaction); } @@ -163,16 +215,19 @@ protected void assertNextNonceForSender( protected void addLocalTransactions( final PendingTransactions sorter, final Account sender, final long... nonces) { for (final long nonce : nonces) { - sorter.addLocalTransaction(createTransaction(nonce), Optional.of(sender)); + sorter.addTransaction( + createLocalPendingTransaction(createTransaction(nonce)), Optional.of(sender)); } } - protected long getAddedCount(final String source, final String layer) { - return metricsSystem.getCounterValue(TransactionPoolMetrics.ADDED_COUNTER_NAME, source, layer); + protected long getAddedCount(final String source, final String priority, final String layer) { + return metricsSystem.getCounterValue( + TransactionPoolMetrics.ADDED_COUNTER_NAME, source, priority, layer); } - protected long getRemovedCount(final String source, final String operation, final String layer) { + protected long getRemovedCount( + final String source, final String priority, final String operation, final String layer) { return metricsSystem.getCounterValue( - TransactionPoolMetrics.REMOVED_COUNTER_NAME, source, operation, layer); + TransactionPoolMetrics.REMOVED_COUNTER_NAME, source, priority, operation, layer); } } diff --git a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayeredPendingTransactionsTest.java b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayeredPendingTransactionsTest.java index 7d044de859a..c878184370a 100644 --- a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayeredPendingTransactionsTest.java +++ b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayeredPendingTransactionsTest.java @@ -71,6 +71,7 @@ public class LayeredPendingTransactionsTest extends BaseTransactionPoolTest { protected static final int LIMITED_TRANSACTIONS_BY_SENDER = 4; protected static final String REMOTE = "remote"; protected static final String LOCAL = "local"; + protected static final String NO_PRIORITY = "no"; protected final PendingTransactionAddedListener listener = mock(PendingTransactionAddedListener.class); protected final PendingTransactionDroppedListener droppedListener = @@ -148,13 +149,16 @@ public void setup() { @Test public void returnExclusivelyLocalTransactionsWhenAppropriate() { final Transaction localTransaction0 = createTransaction(0, KEYS2); - pendingTransactions.addLocalTransaction(localTransaction0, Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(localTransaction0), Optional.empty()); assertThat(pendingTransactions.size()).isEqualTo(1); - pendingTransactions.addRemoteTransaction(transaction0, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction0), Optional.empty()); assertThat(pendingTransactions.size()).isEqualTo(2); - pendingTransactions.addRemoteTransaction(transaction1, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty()); assertThat(pendingTransactions.size()).isEqualTo(3); final List localTransactions = pendingTransactions.getLocalTransactions(); @@ -163,15 +167,19 @@ public void returnExclusivelyLocalTransactionsWhenAppropriate() { @Test public void addRemoteTransactions() { - pendingTransactions.addRemoteTransaction(transaction0, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction0), Optional.empty()); assertThat(pendingTransactions.size()).isEqualTo(1); - assertThat(getAddedCount(REMOTE, layers.prioritizedTransactions.name())).isEqualTo(1); + assertThat(getAddedCount(REMOTE, NO_PRIORITY, layers.prioritizedTransactions.name())) + .isEqualTo(1); - pendingTransactions.addRemoteTransaction(transaction1, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty()); assertThat(pendingTransactions.size()).isEqualTo(2); - assertThat(getAddedCount(REMOTE, layers.prioritizedTransactions.name())).isEqualTo(2); + assertThat(getAddedCount(REMOTE, NO_PRIORITY, layers.prioritizedTransactions.name())) + .isEqualTo(2); } @Test @@ -181,7 +189,8 @@ public void getNotPresentTransaction() { @Test public void getTransactionByHash() { - pendingTransactions.addRemoteTransaction(transaction0, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction0), Optional.empty()); assertTransactionPending(pendingTransactions, transaction0); } @@ -200,7 +209,7 @@ public void evictTransactionsWhenSizeLimitExceeded() { Wei.of((i + 1) * 100L), (int) poolConf.getPendingTransactionsLayerMaxCapacityBytes() + 1, SIGNATURE_ALGORITHM.get().generateKeyPair()); - pendingTransactions.addRemoteTransaction(tx, Optional.of(sender)); + pendingTransactions.addTransaction(createRemotePendingTransaction(tx), Optional.of(sender)); firstTxs.add(tx); assertTransactionPending(pendingTransactions, tx); } @@ -215,11 +224,13 @@ public void evictTransactionsWhenSizeLimitExceeded() { SIGNATURE_ALGORITHM.get().generateKeyPair()); final Account lastSender = mock(Account.class); when(lastSender.getNonce()).thenReturn(0L); - pendingTransactions.addRemoteTransaction(lastBigTx, Optional.of(lastSender)); + pendingTransactions.addTransaction( + createRemotePendingTransaction(lastBigTx), Optional.of(lastSender)); assertTransactionPending(pendingTransactions, lastBigTx); assertTransactionNotPending(pendingTransactions, firstTxs.get(0)); - assertThat(getRemovedCount(REMOTE, DROPPED.label(), layers.evictedCollector.name())) + assertThat( + getRemovedCount(REMOTE, NO_PRIORITY, DROPPED.label(), layers.evictedCollector.name())) .isEqualTo(1); assertThat(layers.evictedCollector.getEvictedTransactions()) .map(PendingTransaction::getTransaction) @@ -231,10 +242,14 @@ public void evictTransactionsWhenSizeLimitExceeded() { public void addTransactionForMultipleSenders() { final var transactionSenderA = createTransaction(0, KEYS1); final var transactionSenderB = createTransaction(0, KEYS2); - assertThat(pendingTransactions.addRemoteTransaction(transactionSenderA, Optional.empty())) + assertThat( + pendingTransactions.addTransaction( + createRemotePendingTransaction(transactionSenderA), Optional.empty())) .isEqualTo(ADDED); assertTransactionPending(pendingTransactions, transactionSenderA); - assertThat(pendingTransactions.addRemoteTransaction(transactionSenderB, Optional.empty())) + assertThat( + pendingTransactions.addTransaction( + createRemotePendingTransaction(transactionSenderB), Optional.empty())) .isEqualTo(ADDED); assertTransactionPending(pendingTransactions, transactionSenderB); } @@ -243,7 +258,9 @@ public void addTransactionForMultipleSenders() { public void dropIfTransactionTooFarInFutureForTheSender() { final var futureTransaction = createTransaction(poolConf.getTxPoolMaxFutureTransactionByAccount() + 1); - assertThat(pendingTransactions.addRemoteTransaction(futureTransaction, Optional.empty())) + assertThat( + pendingTransactions.addTransaction( + createRemotePendingTransaction(futureTransaction), Optional.empty())) .isEqualTo(NONCE_TOO_FAR_IN_FUTURE_FOR_SENDER); assertTransactionNotPending(pendingTransactions, futureTransaction); } @@ -254,7 +271,9 @@ public void dropAlreadyConfirmedTransaction() { when(sender.getNonce()).thenReturn(5L); final Transaction oldTransaction = createTransaction(2); - assertThat(pendingTransactions.addRemoteTransaction(oldTransaction, Optional.of(sender))) + assertThat( + pendingTransactions.addTransaction( + createRemotePendingTransaction(oldTransaction), Optional.of(sender))) .isEqualTo(ALREADY_KNOWN); assertThat(pendingTransactions.size()).isEqualTo(0); assertTransactionNotPending(pendingTransactions, oldTransaction); @@ -264,7 +283,8 @@ public void dropAlreadyConfirmedTransaction() { public void notifyListenerWhenRemoteTransactionAdded() { pendingTransactions.subscribePendingTransactions(listener); - pendingTransactions.addRemoteTransaction(transaction0, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction0), Optional.empty()); verify(listener).onTransactionAdded(transaction0); } @@ -273,7 +293,8 @@ public void notifyListenerWhenRemoteTransactionAdded() { public void notifyListenerWhenLocalTransactionAdded() { pendingTransactions.subscribePendingTransactions(listener); - pendingTransactions.addLocalTransaction(transaction0, Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(transaction0), Optional.empty()); verify(listener).onTransactionAdded(transaction0); } @@ -282,13 +303,15 @@ public void notifyListenerWhenLocalTransactionAdded() { public void notNotifyListenerAfterUnsubscribe() { final long id = pendingTransactions.subscribePendingTransactions(listener); - pendingTransactions.addRemoteTransaction(transaction0, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction0), Optional.empty()); verify(listener).onTransactionAdded(transaction0); pendingTransactions.unsubscribePendingTransactions(id); - pendingTransactions.addRemoteTransaction(transaction1, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty()); verifyNoMoreInteractions(listener); } @@ -297,13 +320,15 @@ public void notNotifyListenerAfterUnsubscribe() { @MethodSource public void selectTransactionsUntilSelectorRequestsNoMore( final TransactionSelectionResult selectionResult) { - pendingTransactions.addRemoteTransaction(transaction0, Optional.empty()); - pendingTransactions.addRemoteTransaction(transaction1, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction0), Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty()); final List parsedTransactions = new ArrayList<>(); pendingTransactions.selectTransactions( - pendingTX -> { - parsedTransactions.add(pendingTX.getTransaction()); + pendingTx -> { + parsedTransactions.add(pendingTx.getTransaction()); return selectionResult; }); @@ -317,8 +342,10 @@ static Stream selectTransactionsUntilSelectorRequest @Test public void selectTransactionsUntilPendingIsEmpty() { - pendingTransactions.addRemoteTransaction(transaction0, Optional.empty()); - pendingTransactions.addRemoteTransaction(transaction1, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction0), Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty()); final List parsedTransactions = new ArrayList<>(); pendingTransactions.selectTransactions( @@ -337,8 +364,10 @@ public void notSelectReplacedTransaction() { final Transaction transaction1 = createTransaction(0, KEYS1); final Transaction transaction1b = createTransactionReplacement(transaction1, KEYS1); - pendingTransactions.addRemoteTransaction(transaction1, Optional.empty()); - pendingTransactions.addRemoteTransaction(transaction1b, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction1b), Optional.empty()); final List parsedTransactions = new ArrayList<>(); pendingTransactions.selectTransactions( @@ -357,9 +386,12 @@ public void selectTransactionsFromSameSenderInNonceOrder() { final Transaction transaction2 = createTransaction(2, KEYS1); // add out of order - pendingTransactions.addLocalTransaction(transaction2, Optional.empty()); - pendingTransactions.addLocalTransaction(transaction1, Optional.empty()); - pendingTransactions.addLocalTransaction(transaction0, Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(transaction2), Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(transaction1), Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(transaction0), Optional.empty()); final List iterationOrder = new ArrayList<>(3); pendingTransactions.selectTransactions( @@ -380,10 +412,14 @@ public void ignoreSenderTransactionsAfterASkippedOne( final Transaction transaction2a = createTransaction(2, DEFAULT_BASE_FEE.add(Wei.of(20)), KEYS1); final Transaction transaction0b = createTransaction(0, DEFAULT_BASE_FEE.add(Wei.of(10)), KEYS2); - pendingTransactions.addLocalTransaction(transaction0a, Optional.empty()); - pendingTransactions.addLocalTransaction(transaction1a, Optional.empty()); - pendingTransactions.addLocalTransaction(transaction2a, Optional.empty()); - pendingTransactions.addLocalTransaction(transaction0b, Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(transaction0a), Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(transaction1a), Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(transaction2a), Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(transaction0b), Optional.empty()); final List iterationOrder = new ArrayList<>(3); pendingTransactions.selectTransactions( @@ -415,8 +451,10 @@ public void notForceNonceOrderWhenSendersDiffer() { final Transaction transactionSender1 = createTransaction(0, Wei.of(100), KEYS1); final Transaction transactionSender2 = createTransaction(1, Wei.of(200), KEYS2); - pendingTransactions.addLocalTransaction(transactionSender1, Optional.empty()); - pendingTransactions.addLocalTransaction(transactionSender2, Optional.of(sender2)); + pendingTransactions.addTransaction( + createLocalPendingTransaction(transactionSender1), Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(transactionSender2), Optional.of(sender2)); final List iterationOrder = new ArrayList<>(2); pendingTransactions.selectTransactions( @@ -430,38 +468,39 @@ public void notForceNonceOrderWhenSendersDiffer() { @Test public void invalidTransactionIsDeletedFromPendingTransactions() { - pendingTransactions.addRemoteTransaction(transaction0, Optional.empty()); - pendingTransactions.addRemoteTransaction(transaction1, Optional.empty()); + final var pendingTx0 = createRemotePendingTransaction(transaction0); + final var pendingTx1 = createRemotePendingTransaction(transaction1); + pendingTransactions.addTransaction(pendingTx0, Optional.empty()); + pendingTransactions.addTransaction(pendingTx1, Optional.empty()); - final List parsedTransactions = new ArrayList<>(1); + final List parsedTransactions = new ArrayList<>(1); pendingTransactions.selectTransactions( pendingTx -> { - parsedTransactions.add(pendingTx.getTransaction()); + parsedTransactions.add(pendingTx); return TransactionSelectionResult.invalid(UPFRONT_COST_EXCEEDS_BALANCE.name()); }); // only the first is processed since not being selected will automatically skip the processing // all the other txs from the same sender - assertThat(parsedTransactions).containsExactly(transaction0); - assertThat(pendingTransactions.getPendingTransactions()) - .map(PendingTransaction::getTransaction) - .containsExactly(transaction1); + assertThat(parsedTransactions).containsExactly(pendingTx0); + assertThat(pendingTransactions.getPendingTransactions()).containsExactly(pendingTx1); } @Test public void temporarilyInvalidTransactionIsKeptInPendingTransactions() { - pendingTransactions.addRemoteTransaction(transaction0, Optional.empty()); + final var pendingTx0 = createRemotePendingTransaction(transaction0); + pendingTransactions.addTransaction(pendingTx0, Optional.empty()); - final List parsedTransactions = new ArrayList<>(1); + final List parsedTransactions = new ArrayList<>(1); pendingTransactions.selectTransactions( pendingTx -> { - parsedTransactions.add(pendingTx.getTransaction()); + parsedTransactions.add(pendingTx); return TransactionSelectionResult.invalidTransient( GAS_PRICE_BELOW_CURRENT_BASE_FEE.name()); }); - assertThat(parsedTransactions).containsExactly(transaction0); + assertThat(parsedTransactions).containsExactly(pendingTx0); assertThat(pendingTransactions.getPendingTransactions()) .map(PendingTransaction::getTransaction) .containsExactly(transaction0); @@ -477,13 +516,17 @@ public void replaceTransactionWithSameSenderAndNonce() { final Transaction transaction1 = createTransaction(0, Wei.of(200), KEYS1); final Transaction transaction1b = createTransactionReplacement(transaction1, KEYS1); final Transaction transaction2 = createTransaction(1, Wei.of(100), KEYS1); - assertThat(pendingTransactions.addRemoteTransaction(transaction1, Optional.empty())) + assertThat( + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty())) .isEqualTo(ADDED); - assertThat(pendingTransactions.addRemoteTransaction(transaction2, Optional.empty())) + assertThat( + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction2), Optional.empty())) .isEqualTo(ADDED); assertThat( pendingTransactions - .addRemoteTransaction(transaction1b, Optional.empty()) + .addTransaction(createRemotePendingTransaction(transaction1b), Optional.empty()) .isReplacement()) .isTrue(); @@ -491,8 +534,11 @@ public void replaceTransactionWithSameSenderAndNonce() { assertTransactionPending(pendingTransactions, transaction1b); assertTransactionPending(pendingTransactions, transaction2); assertThat(pendingTransactions.size()).isEqualTo(2); - assertThat(getAddedCount(REMOTE, layers.prioritizedTransactions.name())).isEqualTo(3); - assertThat(getRemovedCount(REMOTE, REPLACED.label(), layers.prioritizedTransactions.name())) + assertThat(getAddedCount(REMOTE, NO_PRIORITY, layers.prioritizedTransactions.name())) + .isEqualTo(3); + assertThat( + getRemovedCount( + REMOTE, NO_PRIORITY, REPLACED.label(), layers.prioritizedTransactions.name())) .isEqualTo(1); } @@ -503,15 +549,20 @@ public void replaceTransactionWithSameSenderAndNonce_multipleReplacements() { Transaction duplicateTx = createTransaction(0, DEFAULT_BASE_FEE.add(Wei.of(50)), KEYS1); for (int i = 0; i < replacedTxCount; i++) { replacedTransactions.add(duplicateTx); - pendingTransactions.addRemoteTransaction(duplicateTx, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(duplicateTx), Optional.empty()); duplicateTx = createTransactionReplacement(duplicateTx, KEYS1); } final Transaction independentTx = createTransaction(1, DEFAULT_BASE_FEE.add(Wei.ONE), KEYS1); - assertThat(pendingTransactions.addRemoteTransaction(independentTx, Optional.empty())) + assertThat( + pendingTransactions.addTransaction( + createRemotePendingTransaction(independentTx), Optional.empty())) .isEqualTo(ADDED); assertThat( - pendingTransactions.addRemoteTransaction(duplicateTx, Optional.empty()).isReplacement()) + pendingTransactions + .addTransaction(createRemotePendingTransaction(duplicateTx), Optional.empty()) + .isReplacement()) .isTrue(); // All txs except the last duplicate should be removed @@ -521,9 +572,11 @@ public void replaceTransactionWithSameSenderAndNonce_multipleReplacements() { assertTransactionPending(pendingTransactions, independentTx); assertThat(pendingTransactions.size()).isEqualTo(2); - assertThat(getAddedCount(REMOTE, layers.prioritizedTransactions.name())) + assertThat(getAddedCount(REMOTE, NO_PRIORITY, layers.prioritizedTransactions.name())) .isEqualTo(replacedTxCount + 2); - assertThat(getRemovedCount(REMOTE, REPLACED.label(), layers.prioritizedTransactions.name())) + assertThat( + getRemovedCount( + REMOTE, NO_PRIORITY, REPLACED.label(), layers.prioritizedTransactions.name())) .isEqualTo(replacedTxCount); } @@ -537,19 +590,25 @@ public void replaceTransactionWithSameSenderAndNonce_multipleReplacements() { for (int i = 0; i < replacedTxCount; i++) { replacedTransactions.add(replacingTx); if (i % 2 == 0) { - pendingTransactions.addRemoteTransaction(replacingTx, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(replacingTx), Optional.empty()); remoteDuplicateCount++; } else { - pendingTransactions.addLocalTransaction(replacingTx, Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(replacingTx), Optional.empty()); } replacingTx = createTransactionReplacement(replacingTx, KEYS1); } final Transaction independentTx = createTransaction(1); assertThat( - pendingTransactions.addLocalTransaction(replacingTx, Optional.empty()).isReplacement()) + pendingTransactions + .addTransaction(createLocalPendingTransaction(replacingTx), Optional.empty()) + .isReplacement()) .isTrue(); - assertThat(pendingTransactions.addRemoteTransaction(independentTx, Optional.empty())) + assertThat( + pendingTransactions.addTransaction( + createRemotePendingTransaction(independentTx), Optional.empty())) .isEqualTo(ADDED); // All txs except the last duplicate should be removed @@ -561,13 +620,17 @@ public void replaceTransactionWithSameSenderAndNonce_multipleReplacements() { final int localDuplicateCount = replacedTxCount - remoteDuplicateCount; assertThat(pendingTransactions.size()).isEqualTo(2); - assertThat(getAddedCount(REMOTE, layers.prioritizedTransactions.name())) + assertThat(getAddedCount(REMOTE, NO_PRIORITY, layers.prioritizedTransactions.name())) .isEqualTo(remoteDuplicateCount + 1); - assertThat(getAddedCount(LOCAL, layers.prioritizedTransactions.name())) + assertThat(getAddedCount(LOCAL, NO_PRIORITY, layers.prioritizedTransactions.name())) .isEqualTo(localDuplicateCount + 1); - assertThat(getRemovedCount(REMOTE, REPLACED.label(), layers.prioritizedTransactions.name())) + assertThat( + getRemovedCount( + REMOTE, NO_PRIORITY, REPLACED.label(), layers.prioritizedTransactions.name())) .isEqualTo(remoteDuplicateCount); - assertThat(getRemovedCount(LOCAL, REPLACED.label(), layers.prioritizedTransactions.name())) + assertThat( + getRemovedCount( + LOCAL, NO_PRIORITY, REPLACED.label(), layers.prioritizedTransactions.name())) .isEqualTo(localDuplicateCount); } @@ -575,11 +638,15 @@ public void replaceTransactionWithSameSenderAndNonce_multipleReplacements() { public void notReplaceTransactionWithSameSenderAndNonceWhenGasPriceIsLower() { final Transaction transaction1 = createTransaction(0, Wei.of(2)); final Transaction transaction1b = createTransaction(0, Wei.ONE); - assertThat(pendingTransactions.addRemoteTransaction(transaction1, Optional.empty())) + assertThat( + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty())) .isEqualTo(ADDED); pendingTransactions.subscribePendingTransactions(listener); - assertThat(pendingTransactions.addRemoteTransaction(transaction1b, Optional.empty())) + assertThat( + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction1b), Optional.empty())) .isEqualTo(REJECTED_UNDERPRICED_REPLACEMENT); assertTransactionNotPending(pendingTransactions, transaction1b); @@ -595,13 +662,16 @@ public void trackNextNonceForEachSender() { when(firstSender.getNonce()).thenReturn(0L); when(firstSender.getAddress()).thenReturn(SENDER1); assertNoNextNonceForSender(pendingTransactions, SENDER1); - pendingTransactions.addRemoteTransaction(createTransaction(0, KEYS1), Optional.of(firstSender)); + pendingTransactions.addTransaction( + createRemotePendingTransaction(createTransaction(0, KEYS1)), Optional.of(firstSender)); assertNextNonceForSender(pendingTransactions, SENDER1, 1); - pendingTransactions.addRemoteTransaction(createTransaction(1, KEYS1), Optional.of(firstSender)); + pendingTransactions.addTransaction( + createRemotePendingTransaction(createTransaction(1, KEYS1)), Optional.of(firstSender)); assertNextNonceForSender(pendingTransactions, SENDER1, 2); - pendingTransactions.addRemoteTransaction(createTransaction(2, KEYS1), Optional.of(firstSender)); + pendingTransactions.addTransaction( + createRemotePendingTransaction(createTransaction(2, KEYS1)), Optional.of(firstSender)); assertNextNonceForSender(pendingTransactions, SENDER1, 3); // second sender not in orders: 3->0->2->1 @@ -609,21 +679,21 @@ public void trackNextNonceForEachSender() { when(secondSender.getNonce()).thenReturn(0L); when(secondSender.getAddress()).thenReturn(SENDER2); assertNoNextNonceForSender(pendingTransactions, SENDER2); - pendingTransactions.addRemoteTransaction( - createTransaction(3, KEYS2), Optional.of(secondSender)); + pendingTransactions.addTransaction( + createRemotePendingTransaction(createTransaction(3, KEYS2)), Optional.of(secondSender)); assertNoNextNonceForSender(pendingTransactions, SENDER2); - pendingTransactions.addRemoteTransaction( - createTransaction(0, KEYS2), Optional.of(secondSender)); + pendingTransactions.addTransaction( + createRemotePendingTransaction(createTransaction(0, KEYS2)), Optional.of(secondSender)); assertNextNonceForSender(pendingTransactions, SENDER2, 1); - pendingTransactions.addRemoteTransaction( - createTransaction(2, KEYS2), Optional.of(secondSender)); + pendingTransactions.addTransaction( + createRemotePendingTransaction(createTransaction(2, KEYS2)), Optional.of(secondSender)); assertNextNonceForSender(pendingTransactions, SENDER2, 1); // tx 1 will fill the nonce gap and all txs will be ready - pendingTransactions.addRemoteTransaction( - createTransaction(1, KEYS2), Optional.of(secondSender)); + pendingTransactions.addTransaction( + createRemotePendingTransaction(createTransaction(1, KEYS2)), Optional.of(secondSender)); assertNextNonceForSender(pendingTransactions, SENDER2, 4); } @@ -701,38 +771,49 @@ public void correctNonceIsReturnedWithRepeatedTransactions() { @Test public void shouldNotIncrementAddedCounterWhenRemoteTransactionAlreadyPresent() { - pendingTransactions.addLocalTransaction(transaction0, Optional.empty()); + pendingTransactions.addTransaction( + createLocalPendingTransaction(transaction0), Optional.empty()); assertThat(pendingTransactions.size()).isEqualTo(1); - assertThat(getAddedCount(LOCAL, layers.prioritizedTransactions.name())).isEqualTo(1); - assertThat(getAddedCount(REMOTE, layers.prioritizedTransactions.name())).isZero(); + assertThat(getAddedCount(LOCAL, NO_PRIORITY, layers.prioritizedTransactions.name())) + .isEqualTo(1); + assertThat(getAddedCount(REMOTE, NO_PRIORITY, layers.prioritizedTransactions.name())).isZero(); - assertThat(pendingTransactions.addRemoteTransaction(transaction0, Optional.empty())) + assertThat( + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction0), Optional.empty())) .isEqualTo(ALREADY_KNOWN); assertThat(pendingTransactions.size()).isEqualTo(1); - assertThat(getAddedCount(LOCAL, layers.prioritizedTransactions.name())).isEqualTo(1); - assertThat(getAddedCount(REMOTE, layers.prioritizedTransactions.name())).isZero(); + assertThat(getAddedCount(LOCAL, NO_PRIORITY, layers.prioritizedTransactions.name())) + .isEqualTo(1); + assertThat(getAddedCount(REMOTE, NO_PRIORITY, layers.prioritizedTransactions.name())).isZero(); } @Test public void shouldNotIncrementAddedCounterWhenLocalTransactionAlreadyPresent() { - pendingTransactions.addRemoteTransaction(transaction0, Optional.empty()); + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction0), Optional.empty()); assertThat(pendingTransactions.size()).isEqualTo(1); - assertThat(getAddedCount(LOCAL, layers.prioritizedTransactions.name())).isZero(); - assertThat(getAddedCount(REMOTE, layers.prioritizedTransactions.name())).isEqualTo(1); + assertThat(getAddedCount(LOCAL, NO_PRIORITY, layers.prioritizedTransactions.name())).isZero(); + assertThat(getAddedCount(REMOTE, NO_PRIORITY, layers.prioritizedTransactions.name())) + .isEqualTo(1); - assertThat(pendingTransactions.addLocalTransaction(transaction0, Optional.empty())) + assertThat( + pendingTransactions.addTransaction( + createLocalPendingTransaction(transaction0), Optional.empty())) .isEqualTo(ALREADY_KNOWN); assertThat(pendingTransactions.size()).isEqualTo(1); - assertThat(getAddedCount(LOCAL, layers.prioritizedTransactions.name())).isZero(); - assertThat(getAddedCount(REMOTE, layers.prioritizedTransactions.name())).isEqualTo(1); + assertThat(getAddedCount(LOCAL, NO_PRIORITY, layers.prioritizedTransactions.name())).isZero(); + assertThat(getAddedCount(REMOTE, NO_PRIORITY, layers.prioritizedTransactions.name())) + .isEqualTo(1); } @Test public void doNothingIfTransactionAlreadyPending() { final var addedTxs = populateCache(1, 0); assertThat( - pendingTransactions.addRemoteTransaction( - addedTxs[0].transaction, Optional.of(addedTxs[0].account))) + pendingTransactions.addTransaction( + createRemotePendingTransaction(addedTxs[0].transaction), + Optional.of(addedTxs[0].account))) .isEqualTo(ALREADY_KNOWN); assertTransactionPending(pendingTransactions, addedTxs[0].transaction); } @@ -766,7 +847,9 @@ private TransactionAndAccount[] populateCache( final var transaction = createTransaction(nonce, keys); final Account sender = mock(Account.class); when(sender.getNonce()).thenReturn(startingNonce); - final var res = pendingTransactions.addRemoteTransaction(transaction, Optional.of(sender)); + final var res = + pendingTransactions.addTransaction( + createRemotePendingTransaction(transaction), Optional.of(sender)); assertTransactionPending(pendingTransactions, transaction); assertThat(res).isEqualTo(ADDED); addedTransactions.add(new TransactionAndAccount(transaction, sender)); diff --git a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayersTest.java b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayersTest.java index 0137fa17c44..666a5d66005 100644 --- a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayersTest.java +++ b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayersTest.java @@ -19,6 +19,8 @@ import static org.hyperledger.besu.ethereum.eth.transactions.layered.LayersTest.Sender.S2; import static org.hyperledger.besu.ethereum.eth.transactions.layered.LayersTest.Sender.S3; import static org.hyperledger.besu.ethereum.eth.transactions.layered.LayersTest.Sender.S4; +import static org.hyperledger.besu.ethereum.eth.transactions.layered.LayersTest.Sender.SP1; +import static org.hyperledger.besu.ethereum.eth.transactions.layered.LayersTest.Sender.SP2; import static org.hyperledger.besu.ethereum.eth.transactions.layered.TransactionsLayer.RemovalReason.INVALIDATED; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.when; @@ -146,6 +148,12 @@ void asyncWorldStateUpdates(final Scenario scenario) { assertScenario(scenario); } + @ParameterizedTest + @MethodSource("providerPrioritySenders") + void prioritySenders(final Scenario scenario) { + assertScenario(scenario); + } + private void assertScenario(final Scenario scenario) { scenario.execute( pendingTransactions, @@ -1023,6 +1031,141 @@ static Stream providerAsyncWorldStateUpdates() { .expectedSparseForSender(S1, 8, 9, 7))); } + static Stream providerPrioritySenders() { + return Stream.of( + Arguments.of( + new Scenario("priority first same fee") + .addForSenders(S1, 0, SP1, 0) + .expectedPrioritizedForSenders(SP1, 0, S1, 0)), + Arguments.of( + new Scenario("priority first lower fee") + .addForSenders(S2, 0, SP1, 0) + .expectedPrioritizedForSenders(SP1, 0, S2, 0)), + Arguments.of( + new Scenario("priority first higher fee") + .addForSenders(S1, 0, SP2, 0) + .expectedPrioritizedForSenders(SP2, 0, S1, 0)), + Arguments.of( + new Scenario("same priority order by fee") + .addForSenders(SP1, 0, SP2, 0) + .expectedPrioritizedForSenders(SP2, 0, SP1, 0)), + Arguments.of( + new Scenario("same priority order by fee") + .addForSenders(SP2, 0, SP1, 0) + .expectedPrioritizedForSenders(SP2, 0, SP1, 0)), + Arguments.of( + new Scenario("priority first overflow to ready") + .addForSender(S2, 0, 1, 2) + .expectedPrioritizedForSender(S2, 0, 1, 2) + .addForSender(SP1, 0) + .expectedPrioritizedForSenders(SP1, 0, S2, 0, S2, 1) + .expectedReadyForSender(S2, 2)), + Arguments.of( + new Scenario("priority first overflow to ready 2") + .addForSender(S2, 0, 1, 2) + .expectedPrioritizedForSender(S2, 0, 1, 2) + .addForSender(SP1, 0, 1, 2) + .expectedPrioritizedForSender(SP1, 0, 1, 2) + .expectedReadyForSender(S2, 0, 1, 2)), + Arguments.of( + new Scenario("multiple priority senders first overflow to ready") + .addForSender(S2, 0, 1, 2) + .expectedPrioritizedForSender(S2, 0, 1, 2) + .addForSenders(SP2, 0, SP1, 0) + .expectedPrioritizedForSenders(SP2, 0, SP1, 0, S2, 0) + .expectedReadyForSender(S2, 1, 2)), + Arguments.of( + new Scenario("priority with initial gap") + .addForSender(S2, 0) + .expectedPrioritizedForSender(S2, 0) + .addForSender(SP1, 1) // initial gap + .expectedPrioritizedForSender(S2, 0) + .expectedSparseForSender(SP1, 1) + .addForSender(SP1, 0) // fill gap + .expectedPrioritizedForSenders(SP1, 0, SP1, 1, S2, 0) + .expectedSparseForSenders()), + Arguments.of( + new Scenario("priority with initial gap overflow to ready") + .addForSender(S2, 0, 1) + .expectedPrioritizedForSender(S2, 0, 1) + .addForSender(SP1, 1) // initial gap + .expectedSparseForSender(SP1, 1) + .addForSender(SP1, 0) // fill gap + .expectedPrioritizedForSenders(SP1, 0, SP1, 1, S2, 0) + .expectedReadyForSender(S2, 1) + .expectedSparseForSenders()), + Arguments.of( + new Scenario("priority with initial gap overflow to ready when prioritized is full") + .addForSender(S2, 0, 1, 2) + .expectedPrioritizedForSender(S2, 0, 1, 2) + .addForSender(SP1, 1) // initial gap + .expectedSparseForSender(SP1, 1) + .addForSender(SP1, 0) // fill gap, but there is not enough space to promote + .expectedPrioritizedForSenders(SP1, 0, S2, 0, S2, 1) + .expectedReadyForSender(S2, 2) + .expectedSparseForSender(SP1, 1) + .confirmedForSenders( + SP1, 0) // asap there is new space the priority tx is promoted first + .expectedPrioritizedForSenders(SP1, 1, S2, 0, S2, 1) + .expectedReadyForSender(S2, 2) + .expectedSparseForSenders()), + Arguments.of( + new Scenario("overflow to ready promote priority first") + .addForSender(SP2, 0, 1, 2) + .expectedPrioritizedForSender(SP2, 0, 1, 2) + .addForSender(S2, 0) + .expectedReadyForSender(S2, 0) + .addForSender(SP1, 0) + .expectedReadyForSenders(SP1, 0, S2, 0) + .confirmedForSenders( + SP2, 0) // asap there is new space the priority tx is promoted first + .expectedPrioritizedForSenders(SP2, 1, SP2, 2, SP1, 0) + .expectedReadyForSender(S2, 0)), + Arguments.of( + new Scenario("priority first overflow to sparse") + .addForSender(SP2, 0, 1, 2) + .addForSender(S3, 0) + .expectedPrioritizedForSender(SP2, 0, 1, 2) + .expectedReadyForSender(S3, 0) + .addForSender(SP1, 0, 1, 2) + .expectedPrioritizedForSender(SP2, 0, 1, 2) + .expectedReadyForSender(SP1, 0, 1, 2) + .expectedSparseForSender(S3, 0)), + Arguments.of( + new Scenario("priority first overflow to sparse 2") + .addForSender(S2, 0, 1, 2) + .addForSender(S3, 0, 1, 2) + .expectedPrioritizedForSender(S3, 0, 1, 2) + .expectedReadyForSender(S2, 0, 1, 2) + .addForSender(SP1, 0) + .expectedPrioritizedForSenders(SP1, 0, S3, 0, S3, 1) + .expectedReadyForSenders(S3, 2, S2, 0, S2, 1) + .expectedSparseForSender(S2, 2)), + Arguments.of( + new Scenario("overflow to sparse promote priority first") + .addForSender(SP2, 0, 1, 2, 3, 4, 5) + .expectedPrioritizedForSender(SP2, 0, 1, 2) + .expectedReadyForSender(SP2, 3, 4, 5) + .addForSender(S3, 0) + .expectedSparseForSender(S3, 0) + .addForSender(SP1, 0) + .expectedSparseForSenders(S3, 0, SP1, 0) + .confirmedForSenders(SP2, 0) + .expectedPrioritizedForSender(SP2, 1, 2, 3) + .expectedReadyForSenders(SP2, 4, SP2, 5, SP1, 0) + .expectedSparseForSender(S3, 0)), + Arguments.of( + new Scenario("discard priority as last") + .addForSender(SP2, 0, 1, 2, 3, 4, 5) + .expectedPrioritizedForSender(SP2, 0, 1, 2) + .expectedReadyForSender(SP2, 3, 4, 5) + .addForSender(S3, 0) + .expectedSparseForSender(S3, 0) + .addForSender(SP1, 0, 1, 2) + .expectedSparseForSender(SP1, 0, 1, 2) + .expectedDroppedForSender(S3, 0))); + } + private static BlockHeader mockBlockHeader() { final BlockHeader blockHeader = mock(BlockHeader.class); when(blockHeader.getBaseFee()).thenReturn(Optional.of(Wei.ONE)); @@ -1145,7 +1288,7 @@ private PendingTransaction get(final Sender sender, final long nonce) { private PendingTransaction createEIP1559PendingTransactions( final Sender sender, final long nonce) { return createRemotePendingTransaction( - createEIP1559Transaction(nonce, sender.key, sender.gasFeeMultiplier)); + createEIP1559Transaction(nonce, sender.key, sender.gasFeeMultiplier), sender.hasPriority); } public Scenario expectedPrioritizedForSender(final Sender sender, final long... nonce) { @@ -1338,19 +1481,24 @@ public String toString() { } enum Sender { - S1(1), - S2(2), - S3(3), - S4(4); + S1(false, 1), + S2(false, 2), + S3(false, 3), + S4(false, 4), + SP1(true, 1), + SP2(true, 2); final KeyPair key; final Address address; final int gasFeeMultiplier; - Sender(final int gasFeeMultiplier) { + final boolean hasPriority; + + Sender(final boolean hasPriority, final int gasFeeMultiplier) { this.key = SIGNATURE_ALGORITHM.get().generateKeyPair(); this.address = Util.publicKeyToAddress(key.getPublicKey()); this.gasFeeMultiplier = gasFeeMultiplier; + this.hasPriority = hasPriority; } } } diff --git a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/ReplayTest.java b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/ReplayTest.java index d68c586c8d9..589818398b5 100644 --- a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/ReplayTest.java +++ b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/ReplayTest.java @@ -20,6 +20,7 @@ import static org.mockito.Mockito.mock; import static org.mockito.Mockito.when; +import org.hyperledger.besu.crypto.SignatureAlgorithmFactory; import org.hyperledger.besu.datatypes.Address; import org.hyperledger.besu.datatypes.Wei; import org.hyperledger.besu.ethereum.core.BlockHeader; @@ -44,10 +45,12 @@ import java.math.BigInteger; import java.nio.charset.StandardCharsets; import java.time.Instant; +import java.util.Arrays; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.Optional; +import java.util.function.BiFunction; import java.util.zip.GZIPInputStream; import com.google.common.base.Splitter; @@ -60,9 +63,6 @@ public class ReplayTest { private static final Logger LOG = LoggerFactory.getLogger(ReplayTest.class); - private final TransactionPoolConfiguration poolConfig = - ImmutableTransactionPoolConfiguration.builder().build(); - private final StubMetricsSystem metricsSystem = new StubMetricsSystem(); private final TransactionPoolMetrics txPoolMetrics = new TransactionPoolMetrics(metricsSystem); @@ -103,6 +103,7 @@ public class ReplayTest { @Test @Disabled("Provide a replay file to run the test on demand") public void replay() throws IOException { + SignatureAlgorithmFactory.setDefaultInstance(); try (BufferedReader br = new BufferedReader( new InputStreamReader( @@ -111,6 +112,11 @@ public void replay() throws IOException { currBlockHeader = mockBlockHeader(br.readLine()); final BaseFeeMarket baseFeeMarket = FeeMarket.london(0L); + final TransactionPoolConfiguration poolConfig = + ImmutableTransactionPoolConfiguration.builder() + .prioritySenders(readPrioritySenders(br.readLine())) + .build(); + final AbstractPrioritizedTransactions prioritizedTransactions = createLayers(poolConfig, txPoolMetrics, baseFeeMarket); final LayeredPendingTransactions pendingTransactions = @@ -153,6 +159,10 @@ public void replay() throws IOException { } } + private List
readPrioritySenders(final String line) { + return Arrays.stream(line.split(",")).map(Address::fromHexString).toList(); + } + private BlockHeader mockBlockHeader(final String line) { final List commaSplit = Splitter.on(',').splitToList(line); final long number = Long.parseLong(commaSplit.get(0)); @@ -174,20 +184,20 @@ private BaseFeePrioritizedTransactions createLayers( final TransactionPoolMetrics txPoolMetrics, final BaseFeeMarket baseFeeMarket) { final EvictCollectorLayer evictCollector = new EvictCollectorLayer(txPoolMetrics); + final BiFunction txReplacementTester = + (tx1, tx2) -> transactionReplacementTester(poolConfig, tx1, tx2); final SparseTransactions sparseTransactions = - new SparseTransactions( - poolConfig, evictCollector, txPoolMetrics, this::transactionReplacementTester); + new SparseTransactions(poolConfig, evictCollector, txPoolMetrics, txReplacementTester); final ReadyTransactions readyTransactions = - new ReadyTransactions( - poolConfig, sparseTransactions, txPoolMetrics, this::transactionReplacementTester); + new ReadyTransactions(poolConfig, sparseTransactions, txPoolMetrics, txReplacementTester); return new BaseFeePrioritizedTransactions( poolConfig, () -> currBlockHeader, readyTransactions, txPoolMetrics, - this::transactionReplacementTester, + txReplacementTester, baseFeeMarket); } @@ -261,7 +271,10 @@ private void processTransaction( tx.getNonce(), prioritizedTransactions.logSender(senderToLog)); } - assertThat(pendingTransactions.addRemoteTransaction(tx, Optional.of(mockAccount))) + assertThat( + pendingTransactions.addTransaction( + PendingTransaction.newPendingTransaction(tx, false, false), + Optional.of(mockAccount))) .isNotEqualTo(TransactionAddedResult.INTERNAL_ERROR); if (tx.getSender().equals(senderToLog)) { LOG.warn("After {}", prioritizedTransactions.logSender(senderToLog)); @@ -281,11 +294,6 @@ private void processInvalid( } } - private boolean transactionReplacementTester( - final PendingTransaction pt1, final PendingTransaction pt2) { - return transactionReplacementTester(poolConfig, pt1, pt2); - } - private boolean transactionReplacementTester( final TransactionPoolConfiguration poolConfig, final PendingTransaction pt1, diff --git a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/AbstractPendingTransactionsTestBase.java b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/AbstractPendingTransactionsTestBase.java index f1d9ce1016d..4511dbd585e 100644 --- a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/AbstractPendingTransactionsTestBase.java +++ b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/sorter/AbstractPendingTransactionsTestBase.java @@ -36,6 +36,7 @@ import org.hyperledger.besu.ethereum.core.TransactionTestFixture; import org.hyperledger.besu.ethereum.core.Util; import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration; +import org.hyperledger.besu.ethereum.eth.transactions.PendingTransaction; import org.hyperledger.besu.ethereum.eth.transactions.PendingTransactionAddedListener; import org.hyperledger.besu.ethereum.eth.transactions.PendingTransactionDroppedListener; import org.hyperledger.besu.ethereum.eth.transactions.PendingTransactions; @@ -109,13 +110,13 @@ abstract AbstractPendingTransactionsSorter getPendingTransactions( @Test public void shouldReturnExclusivelyLocalTransactionsWhenAppropriate() { final Transaction localTransaction0 = createTransaction(0); - transactions.addLocalTransaction(localTransaction0, Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(localTransaction0), Optional.empty()); assertThat(transactions.size()).isEqualTo(1); - transactions.addRemoteTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); assertThat(transactions.size()).isEqualTo(2); - transactions.addRemoteTransaction(transaction2, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction2), Optional.empty()); assertThat(transactions.size()).isEqualTo(3); final List localTransactions = transactions.getLocalTransactions(); @@ -124,11 +125,11 @@ public void shouldReturnExclusivelyLocalTransactionsWhenAppropriate() { @Test public void shouldAddATransaction() { - transactions.addRemoteTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); assertThat(transactions.size()).isEqualTo(1); assertThat(metricsSystem.getCounterValue(ADDED_COUNTER, REMOTE)).isEqualTo(1); - transactions.addRemoteTransaction(transaction2, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction2), Optional.empty()); assertThat(transactions.size()).isEqualTo(2); assertThat(metricsSystem.getCounterValue(ADDED_COUNTER, REMOTE)).isEqualTo(2); } @@ -140,7 +141,7 @@ public void shouldReturnEmptyOptionalWhenNoTransactionWithGivenHashExists() { @Test public void shouldGetTransactionByHash() { - transactions.addRemoteTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); assertTransactionPending(transaction1); } @@ -150,13 +151,15 @@ public void shouldDropOldestTransactionWhenLimitExceeded() { transactionWithNonceSenderAndGasPrice(0, SIGNATURE_ALGORITHM.get().generateKeyPair(), 10L); final Account oldestSender = mock(Account.class); when(oldestSender.getNonce()).thenReturn(0L); - senderLimitedTransactions.addRemoteTransaction(oldestTransaction, Optional.of(oldestSender)); + senderLimitedTransactions.addTransaction( + createRemotePendingTransaction(oldestTransaction), Optional.of(oldestSender)); for (int i = 1; i < MAX_TRANSACTIONS; i++) { final Account sender = mock(Account.class); when(sender.getNonce()).thenReturn((long) i); - senderLimitedTransactions.addRemoteTransaction( - transactionWithNonceSenderAndGasPrice( - i, SIGNATURE_ALGORITHM.get().generateKeyPair(), 10L), + senderLimitedTransactions.addTransaction( + createRemotePendingTransaction( + transactionWithNonceSenderAndGasPrice( + i, SIGNATURE_ALGORITHM.get().generateKeyPair(), 10L)), Optional.of(sender)); } assertThat(senderLimitedTransactions.size()).isEqualTo(MAX_TRANSACTIONS); @@ -164,8 +167,9 @@ public void shouldDropOldestTransactionWhenLimitExceeded() { final Account lastSender = mock(Account.class); when(lastSender.getNonce()).thenReturn(6L); - senderLimitedTransactions.addRemoteTransaction( - createTransaction(MAX_TRANSACTIONS + 1), Optional.of(lastSender)); + senderLimitedTransactions.addTransaction( + createRemotePendingTransaction(createTransaction(MAX_TRANSACTIONS + 1)), + Optional.of(lastSender)); assertThat(senderLimitedTransactions.size()).isEqualTo(MAX_TRANSACTIONS); assertTransactionNotPending(oldestTransaction); assertThat(metricsSystem.getCounterValue(REMOVED_COUNTER, REMOTE, DROPPED)).isEqualTo(1); @@ -176,7 +180,8 @@ public void shouldDropTransactionWithATooFarNonce() { Transaction furthestFutureTransaction = null; for (int i = 0; i < MAX_TRANSACTIONS; i++) { furthestFutureTransaction = transactionWithNonceSenderAndGasPrice(i, KEYS1, 10L); - senderLimitedTransactions.addRemoteTransaction(furthestFutureTransaction, Optional.empty()); + senderLimitedTransactions.addTransaction( + createRemotePendingTransaction(furthestFutureTransaction), Optional.empty()); } assertThat(senderLimitedTransactions.size()) .isEqualTo(senderLimitedConfig.getTxPoolMaxFutureTransactionByAccount()); @@ -187,26 +192,32 @@ public void shouldDropTransactionWithATooFarNonce() { @Test public void shouldHandleMaximumTransactionLimitCorrectlyWhenSameTransactionAddedMultipleTimes() { - transactions.addRemoteTransaction(createTransaction(0), Optional.empty()); - transactions.addRemoteTransaction(createTransaction(0), Optional.empty()); + transactions.addTransaction( + createRemotePendingTransaction(createTransaction(0)), Optional.empty()); + transactions.addTransaction( + createRemotePendingTransaction(createTransaction(0)), Optional.empty()); for (int i = 1; i < MAX_TRANSACTIONS; i++) { - transactions.addRemoteTransaction(createTransaction(i), Optional.empty()); + transactions.addTransaction( + createRemotePendingTransaction(createTransaction(i)), Optional.empty()); } assertThat(transactions.size()).isEqualTo(MAX_TRANSACTIONS); - transactions.addRemoteTransaction(createTransaction(MAX_TRANSACTIONS + 1), Optional.empty()); - transactions.addRemoteTransaction(createTransaction(MAX_TRANSACTIONS + 2), Optional.empty()); + transactions.addTransaction( + createRemotePendingTransaction(createTransaction(MAX_TRANSACTIONS + 1)), Optional.empty()); + transactions.addTransaction( + createRemotePendingTransaction(createTransaction(MAX_TRANSACTIONS + 2)), Optional.empty()); assertThat(transactions.size()).isEqualTo(MAX_TRANSACTIONS); } @Test public void shouldPrioritizeLocalTransaction() { final Transaction localTransaction = createTransaction(0); - transactions.addLocalTransaction(localTransaction, Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(localTransaction), Optional.empty()); for (int i = 1; i <= MAX_TRANSACTIONS; i++) { - transactions.addRemoteTransaction(createTransaction(i), Optional.empty()); + transactions.addTransaction( + createRemotePendingTransaction(createTransaction(i)), Optional.empty()); } assertThat(transactions.size()).isEqualTo(MAX_TRANSACTIONS); assertTransactionPending(localTransaction); @@ -218,7 +229,8 @@ public void shouldStartDroppingLocalTransactionsWhenPoolIsFullOfLocalTransaction for (int i = 0; i <= MAX_TRANSACTIONS; i++) { lastLocalTransactionForSender = createTransaction(i); - transactions.addLocalTransaction(lastLocalTransactionForSender, Optional.empty()); + transactions.addTransaction( + createLocalPendingTransaction(lastLocalTransactionForSender), Optional.empty()); } assertThat(transactions.size()).isEqualTo(MAX_TRANSACTIONS); assertTransactionNotPending(lastLocalTransactionForSender); @@ -228,7 +240,7 @@ public void shouldStartDroppingLocalTransactionsWhenPoolIsFullOfLocalTransaction public void shouldNotifyListenerWhenRemoteTransactionAdded() { transactions.subscribePendingTransactions(listener); - transactions.addRemoteTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); verify(listener).onTransactionAdded(transaction1); } @@ -237,13 +249,13 @@ public void shouldNotifyListenerWhenRemoteTransactionAdded() { public void shouldNotNotifyListenerAfterUnsubscribe() { final long id = transactions.subscribePendingTransactions(listener); - transactions.addRemoteTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); verify(listener).onTransactionAdded(transaction1); transactions.unsubscribePendingTransactions(id); - transactions.addRemoteTransaction(transaction2, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction2), Optional.empty()); verifyNoMoreInteractions(listener); } @@ -252,14 +264,14 @@ public void shouldNotNotifyListenerAfterUnsubscribe() { public void shouldNotifyListenerWhenLocalTransactionAdded() { transactions.subscribePendingTransactions(listener); - transactions.addLocalTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction1), Optional.empty()); verify(listener).onTransactionAdded(transaction1); } @Test public void shouldNotifyDroppedListenerWhenRemoteTransactionDropped() { - transactions.addRemoteTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); transactions.subscribeDroppedTransactions(droppedListener); @@ -270,8 +282,8 @@ public void shouldNotifyDroppedListenerWhenRemoteTransactionDropped() { @Test public void shouldNotNotifyDroppedListenerAfterUnsubscribe() { - transactions.addRemoteTransaction(transaction1, Optional.empty()); - transactions.addRemoteTransaction(transaction2, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction2), Optional.empty()); final long id = transactions.subscribeDroppedTransactions(droppedListener); @@ -288,7 +300,7 @@ public void shouldNotNotifyDroppedListenerAfterUnsubscribe() { @Test public void shouldNotifyDroppedListenerWhenLocalTransactionDropped() { - transactions.addLocalTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction1), Optional.empty()); transactions.subscribeDroppedTransactions(droppedListener); @@ -299,7 +311,7 @@ public void shouldNotifyDroppedListenerWhenLocalTransactionDropped() { @Test public void shouldNotNotifyDroppedListenerWhenTransactionAddedToBlock() { - transactions.addRemoteTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); transactions.subscribeDroppedTransactions(droppedListener); @@ -310,8 +322,8 @@ public void shouldNotNotifyDroppedListenerWhenTransactionAddedToBlock() { @Test public void selectTransactionsUntilSelectorRequestsNoMore() { - transactions.addRemoteTransaction(transaction1, Optional.empty()); - transactions.addRemoteTransaction(transaction2, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction2), Optional.empty()); final List parsedTransactions = Lists.newArrayList(); transactions.selectTransactions( @@ -326,8 +338,8 @@ public void selectTransactionsUntilSelectorRequestsNoMore() { @Test public void selectTransactionsUntilPendingIsEmpty() { - transactions.addRemoteTransaction(transaction1, Optional.empty()); - transactions.addRemoteTransaction(transaction2, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction2), Optional.empty()); final List parsedTransactions = Lists.newArrayList(); transactions.selectTransactions( @@ -346,8 +358,8 @@ public void shouldNotSelectReplacedTransaction() { final Transaction transaction1 = transactionWithNonceSenderAndGasPrice(1, KEYS1, 1); final Transaction transaction2 = transactionWithNonceSenderAndGasPrice(1, KEYS1, 2); - transactions.addRemoteTransaction(transaction1, Optional.empty()); - transactions.addRemoteTransaction(transaction2, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction2), Optional.empty()); final List parsedTransactions = Lists.newArrayList(); transactions.selectTransactions( @@ -361,8 +373,8 @@ public void shouldNotSelectReplacedTransaction() { @Test public void invalidTransactionIsDeletedFromPendingTransactions() { - transactions.addRemoteTransaction(transaction1, Optional.empty()); - transactions.addRemoteTransaction(transaction2, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction2), Optional.empty()); final List parsedTransactions = Lists.newArrayList(); transactions.selectTransactions( @@ -387,7 +399,7 @@ public void shouldReturnEmptyOptionalAsMaximumNonceWhenNoTransactionsPresent() { @Test public void shouldReturnEmptyOptionalAsMaximumNonceWhenLastTransactionForSenderRemoved() { final Transaction transaction = transactionWithNonceAndSender(1, KEYS1); - transactions.addRemoteTransaction(transaction, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction), Optional.empty()); transactions.removeTransaction(transaction); assertThat(transactions.getNextNonceForSender(SENDER1)).isEmpty(); } @@ -397,9 +409,18 @@ public void shouldReplaceTransactionWithSameSenderAndNonce() { final Transaction transaction1 = transactionWithNonceSenderAndGasPrice(1, KEYS1, 1); final Transaction transaction1b = transactionWithNonceSenderAndGasPrice(1, KEYS1, 2); final Transaction transaction2 = transactionWithNonceSenderAndGasPrice(2, KEYS1, 1); - assertThat(transactions.addRemoteTransaction(transaction1, Optional.empty())).isEqualTo(ADDED); - assertThat(transactions.addRemoteTransaction(transaction2, Optional.empty())).isEqualTo(ADDED); - assertThat(transactions.addRemoteTransaction(transaction1b, Optional.empty())).isEqualTo(ADDED); + assertThat( + transactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty())) + .isEqualTo(ADDED); + assertThat( + transactions.addTransaction( + createRemotePendingTransaction(transaction2), Optional.empty())) + .isEqualTo(ADDED); + assertThat( + transactions.addTransaction( + createRemotePendingTransaction(transaction1b), Optional.empty())) + .isEqualTo(ADDED); assertTransactionNotPending(transaction1); assertTransactionPending(transaction1b); @@ -416,12 +437,17 @@ public void shouldReplaceTransactionWithSameSenderAndNonce_multipleReplacements( for (int i = 0; i < replacedTxCount; i++) { final Transaction duplicateTx = transactionWithNonceSenderAndGasPrice(1, KEYS1, i + 1); replacedTransactions.add(duplicateTx); - transactions.addRemoteTransaction(duplicateTx, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(duplicateTx), Optional.empty()); } final Transaction finalReplacingTx = transactionWithNonceSenderAndGasPrice(1, KEYS1, 100); final Transaction independentTx = transactionWithNonceSenderAndGasPrice(2, KEYS1, 1); - assertThat(transactions.addRemoteTransaction(independentTx, Optional.empty())).isEqualTo(ADDED); - assertThat(transactions.addRemoteTransaction(finalReplacingTx, Optional.empty())) + assertThat( + transactions.addTransaction( + createRemotePendingTransaction(independentTx), Optional.empty())) + .isEqualTo(ADDED); + assertThat( + transactions.addTransaction( + createRemotePendingTransaction(finalReplacingTx), Optional.empty())) .isEqualTo(ADDED); // All tx's except the last duplicate should be removed @@ -447,17 +473,22 @@ public void shouldReplaceTransactionWithSameSenderAndNonce_multipleReplacements( transactionWithNonceSenderAndGasPrice(1, KEYS1, (i * 110 / 100) + 1); replacedTransactions.add(duplicateTx); if (i % 2 == 0) { - transactions.addRemoteTransaction(duplicateTx, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(duplicateTx), Optional.empty()); remoteDuplicateCount++; } else { - transactions.addLocalTransaction(duplicateTx, Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(duplicateTx), Optional.empty()); } } final Transaction finalReplacingTx = transactionWithNonceSenderAndGasPrice(1, KEYS1, 100); final Transaction independentTx = transactionWithNonceSenderAndGasPrice(2, KEYS1, 1); - assertThat(transactions.addLocalTransaction(finalReplacingTx, Optional.empty())) + assertThat( + transactions.addTransaction( + createLocalPendingTransaction(finalReplacingTx), Optional.empty())) + .isEqualTo(ADDED); + assertThat( + transactions.addTransaction( + createRemotePendingTransaction(independentTx), Optional.empty())) .isEqualTo(ADDED); - assertThat(transactions.addRemoteTransaction(independentTx, Optional.empty())).isEqualTo(ADDED); // All tx's except the last duplicate should be removed replacedTransactions.forEach(this::assertTransactionNotPending); @@ -481,8 +512,14 @@ public void shouldReplaceTransactionWithSameSenderAndNonce_multipleReplacements( public void shouldReplaceOnlyTransactionFromSenderWhenItHasTheSameNonce() { final Transaction transaction1 = transactionWithNonceSenderAndGasPrice(1, KEYS1, 1); final Transaction transaction1b = transactionWithNonceSenderAndGasPrice(1, KEYS1, 2); - assertThat(transactions.addRemoteTransaction(transaction1, Optional.empty())).isEqualTo(ADDED); - assertThat(transactions.addRemoteTransaction(transaction1b, Optional.empty())).isEqualTo(ADDED); + assertThat( + transactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty())) + .isEqualTo(ADDED); + assertThat( + transactions.addTransaction( + createRemotePendingTransaction(transaction1b), Optional.empty())) + .isEqualTo(ADDED); assertTransactionNotPending(transaction1); assertTransactionPending(transaction1b); @@ -495,10 +532,15 @@ public void shouldReplaceOnlyTransactionFromSenderWhenItHasTheSameNonce() { public void shouldNotReplaceTransactionWithSameSenderAndNonceWhenGasPriceIsLower() { final Transaction transaction1 = transactionWithNonceSenderAndGasPrice(1, KEYS1, 2); final Transaction transaction1b = transactionWithNonceSenderAndGasPrice(1, KEYS1, 1); - assertThat(transactions.addRemoteTransaction(transaction1, Optional.empty())).isEqualTo(ADDED); + assertThat( + transactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty())) + .isEqualTo(ADDED); transactions.subscribePendingTransactions(listener); - assertThat(transactions.addRemoteTransaction(transaction1b, Optional.empty())) + assertThat( + transactions.addTransaction( + createRemotePendingTransaction(transaction1b), Optional.empty())) .isEqualTo(REJECTED_UNDERPRICED_REPLACEMENT); assertTransactionNotPending(transaction1b); @@ -509,16 +551,20 @@ public void shouldNotReplaceTransactionWithSameSenderAndNonceWhenGasPriceIsLower @Test public void shouldTrackMaximumNonceForEachSender() { - transactions.addRemoteTransaction(transactionWithNonceAndSender(0, KEYS1), Optional.empty()); + transactions.addTransaction( + createRemotePendingTransaction(transactionWithNonceAndSender(0, KEYS1)), Optional.empty()); assertMaximumNonceForSender(SENDER1, 1); - transactions.addRemoteTransaction(transactionWithNonceAndSender(1, KEYS1), Optional.empty()); + transactions.addTransaction( + createRemotePendingTransaction(transactionWithNonceAndSender(1, KEYS1)), Optional.empty()); assertMaximumNonceForSender(SENDER1, 2); - transactions.addRemoteTransaction(transactionWithNonceAndSender(2, KEYS1), Optional.empty()); + transactions.addTransaction( + createRemotePendingTransaction(transactionWithNonceAndSender(2, KEYS1)), Optional.empty()); assertMaximumNonceForSender(SENDER1, 3); - transactions.addRemoteTransaction(transactionWithNonceAndSender(4, KEYS2), Optional.empty()); + transactions.addTransaction( + createRemotePendingTransaction(transactionWithNonceAndSender(4, KEYS2)), Optional.empty()); assertMaximumNonceForSender(SENDER2, 5); assertMaximumNonceForSender(SENDER1, 3); } @@ -529,9 +575,9 @@ public void shouldIterateTransactionsFromSameSenderInNonceOrder() { final Transaction transaction2 = transactionWithNonceAndSender(1, KEYS1); final Transaction transaction3 = transactionWithNonceAndSender(2, KEYS1); - transactions.addLocalTransaction(transaction1, Optional.empty()); - transactions.addLocalTransaction(transaction2, Optional.empty()); - transactions.addLocalTransaction(transaction3, Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction1), Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction2), Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction3), Optional.empty()); final List iterationOrder = new ArrayList<>(); transactions.selectTransactions( @@ -548,8 +594,8 @@ public void shouldNotForceNonceOrderWhenSendersDiffer() { final Transaction transaction1 = transactionWithNonceAndSender(0, KEYS1); final Transaction transaction2 = transactionWithNonceAndSender(1, KEYS2); - transactions.addLocalTransaction(transaction1, Optional.empty()); - transactions.addLocalTransaction(transaction2, Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction1), Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction2), Optional.empty()); final List iterationOrder = new ArrayList<>(); transactions.selectTransactions( @@ -568,10 +614,10 @@ public void shouldNotIncreasePriorityOfTransactionsBecauseOfNonceOrder() { final Transaction transaction3 = transactionWithNonceAndSender(2, KEYS1); final Transaction transaction4 = transactionWithNonceAndSender(4, KEYS2); - transactions.addLocalTransaction(transaction1, Optional.empty()); - transactions.addLocalTransaction(transaction4, Optional.empty()); - transactions.addLocalTransaction(transaction2, Optional.empty()); - transactions.addLocalTransaction(transaction3, Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction1), Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction4), Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction2), Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction3), Optional.empty()); final List iterationOrder = new ArrayList<>(); transactions.selectTransactions( @@ -616,6 +662,19 @@ protected Transaction createTransaction(final long transactionNumber) { .createTransaction(KEYS1); } + private PendingTransaction createRemotePendingTransaction( + final Transaction transaction, final long addedAt) { + return PendingTransaction.newPendingTransaction(transaction, false, false, addedAt); + } + + private PendingTransaction createRemotePendingTransaction(final Transaction transaction) { + return PendingTransaction.newPendingTransaction(transaction, false, false); + } + + private PendingTransaction createLocalPendingTransaction(final Transaction transaction) { + return PendingTransaction.newPendingTransaction(transaction, true, true); + } + @Test public void shouldEvictMultipleOldTransactions() { final int maxTransactionRetentionHours = 1; @@ -628,9 +687,9 @@ public void shouldEvictMultipleOldTransactions() { .build(), Optional.of(clock)); - transactions.addRemoteTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); assertThat(transactions.size()).isEqualTo(1); - transactions.addRemoteTransaction(transaction2, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction2), Optional.empty()); assertThat(transactions.size()).isEqualTo(2); clock.step(2L, ChronoUnit.HOURS); @@ -649,7 +708,8 @@ public void shouldEvictSingleOldTransaction() { .txPoolLimitByAccountPercentage(Fraction.fromFloat(1.0f)) .build(), Optional.of(clock)); - evictSingleTransactions.addRemoteTransaction(transaction1, Optional.empty()); + evictSingleTransactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty()); assertThat(evictSingleTransactions.size()).isEqualTo(1); clock.step(2L, ChronoUnit.HOURS); evictSingleTransactions.evictOldTransactions(); @@ -668,10 +728,12 @@ public void shouldEvictExclusivelyOldTransactions() { .build(), Optional.of(clock)); - twoHourEvictionTransactionPool.addRemoteTransaction(transaction1, Optional.empty()); + twoHourEvictionTransactionPool.addTransaction( + createRemotePendingTransaction(transaction1, clock.millis()), Optional.empty()); assertThat(twoHourEvictionTransactionPool.size()).isEqualTo(1); clock.step(3L, ChronoUnit.HOURS); - twoHourEvictionTransactionPool.addRemoteTransaction(transaction2, Optional.empty()); + twoHourEvictionTransactionPool.addTransaction( + createRemotePendingTransaction(transaction2, clock.millis()), Optional.empty()); assertThat(twoHourEvictionTransactionPool.size()).isEqualTo(2); twoHourEvictionTransactionPool.evictOldTransactions(); assertThat(twoHourEvictionTransactionPool.size()).isEqualTo(1); @@ -680,12 +742,14 @@ public void shouldEvictExclusivelyOldTransactions() { @Test public void shouldNotIncrementAddedCounterWhenRemoteTransactionAlreadyPresent() { - transactions.addLocalTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createLocalPendingTransaction(transaction1), Optional.empty()); assertThat(transactions.size()).isEqualTo(1); assertThat(metricsSystem.getCounterValue(ADDED_COUNTER, LOCAL)).isEqualTo(1); assertThat(metricsSystem.getCounterValue(ADDED_COUNTER, REMOTE)).isEqualTo(0); - assertThat(transactions.addRemoteTransaction(transaction1, Optional.empty())) + assertThat( + transactions.addTransaction( + createRemotePendingTransaction(transaction1), Optional.empty())) .isEqualTo(ALREADY_KNOWN); assertThat(transactions.size()).isEqualTo(1); assertThat(metricsSystem.getCounterValue(ADDED_COUNTER, LOCAL)).isEqualTo(1); @@ -694,12 +758,14 @@ public void shouldNotIncrementAddedCounterWhenRemoteTransactionAlreadyPresent() @Test public void shouldNotIncrementAddedCounterWhenLocalTransactionAlreadyPresent() { - transactions.addRemoteTransaction(transaction1, Optional.empty()); + transactions.addTransaction(createRemotePendingTransaction(transaction1), Optional.empty()); assertThat(transactions.size()).isEqualTo(1); assertThat(metricsSystem.getCounterValue(ADDED_COUNTER, LOCAL)).isEqualTo(0); assertThat(metricsSystem.getCounterValue(ADDED_COUNTER, REMOTE)).isEqualTo(1); - assertThat(transactions.addLocalTransaction(transaction1, Optional.empty())) + assertThat( + transactions.addTransaction( + createLocalPendingTransaction(transaction1), Optional.empty())) .isEqualTo(ALREADY_KNOWN); assertThat(transactions.size()).isEqualTo(1); assertThat(metricsSystem.getCounterValue(ADDED_COUNTER, LOCAL)).isEqualTo(0); @@ -774,7 +840,8 @@ protected void addLocalTransactions(final PendingTransactions sorter, final long for (final long nonce : nonces) { final Account sender = mock(Account.class); when(sender.getNonce()).thenReturn(1L); - sorter.addLocalTransaction(createTransaction(nonce), Optional.of(sender)); + sorter.addTransaction( + createLocalPendingTransaction(createTransaction(nonce)), Optional.of(sender)); } } @@ -798,7 +865,8 @@ public void shouldPrioritizeGasPriceThenTimeAddedToPool() { final Transaction lowPriceTx = transactionWithNonceSenderAndGasPrice( 0, SIGNATURE_ALGORITHM.get().generateKeyPair(), 10); - transactions.addRemoteTransaction(lowPriceTx, Optional.of(randomSender)); + transactions.addTransaction( + createRemotePendingTransaction(lowPriceTx), Optional.of(randomSender)); return lowPriceTx; }) .collect(Collectors.toUnmodifiableList()); @@ -807,7 +875,8 @@ public void shouldPrioritizeGasPriceThenTimeAddedToPool() { final Account highPriceSender = mock(Account.class); final Transaction highGasPriceTransaction = transactionWithNonceSenderAndGasPrice(0, KEYS1, 100); - transactions.addRemoteTransaction(highGasPriceTransaction, Optional.of(highPriceSender)); + transactions.addTransaction( + createRemotePendingTransaction(highGasPriceTransaction), Optional.of(highPriceSender)); assertThat(transactions.size()).isEqualTo(MAX_TRANSACTIONS); assertTransactionPending(highGasPriceTransaction); diff --git a/ethereum/evmtool/src/main/java/org/hyperledger/besu/evmtool/T8nExecutor.java b/ethereum/evmtool/src/main/java/org/hyperledger/besu/evmtool/T8nExecutor.java index 388d7a9460b..a93b66e4348 100644 --- a/ethereum/evmtool/src/main/java/org/hyperledger/besu/evmtool/T8nExecutor.java +++ b/ethereum/evmtool/src/main/java/org/hyperledger/besu/evmtool/T8nExecutor.java @@ -33,7 +33,6 @@ import org.hyperledger.besu.datatypes.Wei; import org.hyperledger.besu.ethereum.core.BlockHeader; import org.hyperledger.besu.ethereum.core.BlockHeaderBuilder; -import org.hyperledger.besu.ethereum.core.MutableWorldState; import org.hyperledger.besu.ethereum.core.Transaction; import org.hyperledger.besu.ethereum.core.TransactionReceipt; import org.hyperledger.besu.ethereum.mainnet.BodyValidation; @@ -42,6 +41,7 @@ import org.hyperledger.besu.ethereum.mainnet.ProtocolSpec; import org.hyperledger.besu.ethereum.mainnet.TransactionValidationParams; import org.hyperledger.besu.ethereum.processing.TransactionProcessingResult; +import org.hyperledger.besu.ethereum.referencetests.BonsaiReferenceTestWorldState; import org.hyperledger.besu.ethereum.referencetests.ReferenceTestBlockchain; import org.hyperledger.besu.ethereum.referencetests.ReferenceTestEnv; import org.hyperledger.besu.ethereum.referencetests.ReferenceTestProtocolSchedules; @@ -238,7 +238,9 @@ static T8nResult runTest( ReferenceTestProtocolSchedules.create( new StubGenesisConfigOptions().chainId(BigInteger.valueOf(chainId))); - final MutableWorldState worldState = initialWorldState.copy(); + final BonsaiReferenceTestWorldState worldState = + (BonsaiReferenceTestWorldState) initialWorldState.copy(); + worldState.disableRootHashVerification(); final ProtocolSchedule protocolSchedule = referenceTestProtocolSchedules.getByName(fork); if (protocolSchedule == null) { diff --git a/ethereum/evmtool/src/test/resources/org/hyperledger/besu/evmtool/state-test/blockhash-zero-state-root.json b/ethereum/evmtool/src/test/resources/org/hyperledger/besu/evmtool/state-test/blockhash-zero-state-root.json deleted file mode 100644 index 444f8298379..00000000000 --- a/ethereum/evmtool/src/test/resources/org/hyperledger/besu/evmtool/state-test/blockhash-zero-state-root.json +++ /dev/null @@ -1,104 +0,0 @@ -{ - "cli": [ - "state-test", - "stdin", - "--trace", - "--trace.memory", - "--trace.stack", - "--trace.returndata", - "--notime" - ], - "stdin": { - "00000936-mixed-1": { - "env": { - "currentCoinbase": "b94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "currentDifficulty": "0x20000", - "currentRandom": "0x0000000000000000000000000000000000000000000000000000000000020000", - "currentGasLimit": "0x26e1f476fe1e22", - "currentNumber": "0x2", - "currentTimestamp": "0x3e8", - "previousHash": "0x0000000000000000000000000000000000000000000000000000000000000000", - "currentBaseFee": "0x10" - }, - "pre": { - "0x00000000000000000000000000000000000000f1": { - "code": "0x600060016001600260026101f461ffff5817907c78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b991383610389895f8a5b82723e3d6d5ff042148d326c1898713a76759ca27383404148083e7788908f409e38766cfafe083134530b5279797ef56394faa3183b3780475932387ea37a326494ff1b6b57018c79f07a9ba492a256543f0b6a813b316835b15385077d9e8d32f18c3bb159fab11734a363999e7e940a9c19699da08a82a3ff6743b13f44206194321d026490411a938b35f47cf31b91a1ff397b7c6d879f119915176f3699801bf0766b62fd855b60607d6f133f35fa40307e150a096ea35041306b9330f23441473018055f44698e928a109b02f38f515386177e819161487a5bfa7951f5317183fa743d7587f479009c8b4813089d8d3cf37e3a9b6170528b84f296f07433363a766a64529d8c62ff859c0852173e875685ff15123594f13418966f483a999990109b1694511007477171068aa03a561b8df4743642452039a43c391c3e0033f4615ab1f37cb10293387613024673187c1d5b731c3e53a19080b03e6b94755580876f7a6bf0f238f0991c775172893e7b14449e8c050968b032a29bf13265657d11386d9f8c6f360670fa1116178909f59a369c7c005f8d1464527365f0fd5b818499a2469a0b173f42478294f11c8b40567d8b9f72f43594028d67b17893668a999d8d205014687a6a9b09198bfe87a144313895a208183c8b4060a392387a5106f235a019f36d336f6034f5fe006df413a090791c3c6c52b181f047457a8a73f294f401467a79048060113b1b20fd87078a78a273368c116741128c9a715b085312949c15460b9b8613587c79365b643a67106d840b51a0009e3f893f893236527d540b14403a866d6a147df465a03a8a62489a1d7f680962036b716111f45572367e887606f282820b3d7c178af5f3846a60f2999290f3f3847882f19362113e668f19a07bfffa8aa4374793a28401fe34fd93551a7a721256130789707767a408847307067a85fe3cf1f58a64f0368d38413299f083941885867d18308e0835b054531793087b8c970490597203978d1a8a1b6ba459621b8a8d8313655711563e31073e3c3d6056337299089a94415468097df1425a91f209b13d77f493425041f4ff6718441445463c519771069e727ef59214b19b0b5a3a3b3aa01b6c963da1638117323d637c92443e97795a438593155f73123417fd7b01397b7f4587b1727a011a12450903fa62f182b168920b945a57f28a2078665088593458137d75729f8898800953933056867c1d48441839128c6238f39d3c01149b9859b1958e0b806c81966302f03407a0f5047d778c52018b20f5f13e5b7255544632708064318b31fd0b7b420b3659761da06a17584554453d4363a27c8f767941619d628b75934051", - "storage": { - "0x0000000000000000000000000000000000000000000000000000000000000000": "0x000000000000000000000000000000000000000000000000000000000000000a", - "0x0000000000000000000000000000000000000000000000000000000000000002": "0x000000000000000000000000000000000000000000000000000000000000000a", - "0x0000000000000000000000000000000000000000000000000000000000000003": "0x0000000000000000000000000000000000000000000000000000000000000002", - "0x0000000000000000000000000000000000000000000000000000000000000005": "0x0000000000000000000000000000000000000000000000000000000000000007", - "0x0000000000000000000000000000000000000000000000000000000000000008": "0x0000000000000000000000000000000000000000000000000000000000000011", - "0x0000000000000000000000000000000000000000000000000000000000000009": "0x0000000000000000000000000000000000000000000000000000000000000005", - "0x000000000000000000000000000000000000000000000000000000000000000c": "0x0000000000000000000000000000000000000000000000000000000000000008" - }, - "balance": "0x0", - "nonce": "0x0" - }, - "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b": { - "code": "0x", - "storage": {}, - "balance": "0xffffffffff", - "nonce": "0x0" - } - }, - "transaction": { - "gasPrice": "0x10", - "nonce": "0x0", - "to": "0x00000000000000000000000000000000000000f1", - "data": [ - "0xfe25cf37540066ffd30066af7690fad6d555e13067f5c3c16ced98a803e2b009db4ab89b243ac17b80386d12d621723c363e46aff1fcce9c084d" - ], - "gasLimit": [ - "0x7a1200" - ], - "value": [ - "0xdbbe" - ], - "secretKey": "0x45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8" - }, - "out": "0x", - "post": { - "Shanghai": [ - { - "hash": "0x0000000000000000000000000000000000000000000000000000000000000000", - "logs": "0x0000000000000000000000000000000000000000000000000000000000000000", - "indexes": { - "data": 0, - "gas": 0, - "value": 0 - } - } - ] - } - } - }, - "stdout": [ - {"pc":0,"op":96,"gas":"0x79bc70","gasCost":"0x3","memSize":0,"stack":[],"depth":1,"refund":0,"opName":"PUSH1"}, - {"pc":2,"op":96,"gas":"0x79bc6d","gasCost":"0x3","memSize":0,"stack":["0x0"],"depth":1,"refund":0,"opName":"PUSH1"}, - {"pc":4,"op":96,"gas":"0x79bc6a","gasCost":"0x3","memSize":0,"stack":["0x0","0x1"],"depth":1,"refund":0,"opName":"PUSH1"}, - {"pc":6,"op":96,"gas":"0x79bc67","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1"],"depth":1,"refund":0,"opName":"PUSH1"}, - {"pc":8,"op":96,"gas":"0x79bc64","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2"],"depth":1,"refund":0,"opName":"PUSH1"}, - {"pc":10,"op":97,"gas":"0x79bc61","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2"],"depth":1,"refund":0,"opName":"PUSH2"}, - {"pc":13,"op":97,"gas":"0x79bc5e","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0x1f4"],"depth":1,"refund":0,"opName":"PUSH2"}, - {"pc":16,"op":88,"gas":"0x79bc5b","gasCost":"0x2","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0x1f4","0xffff"],"depth":1,"refund":0,"opName":"PC"}, - {"pc":17,"op":23,"gas":"0x79bc59","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0x1f4","0xffff","0x10"],"depth":1,"refund":0,"opName":"OR"}, - {"pc":18,"op":144,"gas":"0x79bc56","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0x1f4","0xffff"],"depth":1,"refund":0,"opName":"SWAP1"}, - {"pc":19,"op":124,"gas":"0x79bc53","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4"],"depth":1,"refund":0,"opName":"PUSH29"}, - {"pc":49,"op":131,"gas":"0x79bc50","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913"],"depth":1,"refund":0,"opName":"DUP4"}, - {"pc":50,"op":97,"gas":"0x79bc4d","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2"],"depth":1,"refund":0,"opName":"PUSH2"}, - {"pc":53,"op":137,"gas":"0x79bc4a","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389"],"depth":1,"refund":0,"opName":"DUP10"}, - {"pc":54,"op":95,"gas":"0x79bc47","gasCost":"0x2","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0"],"depth":1,"refund":0,"opName":"PUSH0"}, - {"pc":55,"op":138,"gas":"0x79bc45","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0"],"depth":1,"refund":0,"opName":"DUP11"}, - {"pc":56,"op":91,"gas":"0x79bc42","gasCost":"0x1","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1"],"depth":1,"refund":0,"opName":"JUMPDEST"}, - {"pc":57,"op":130,"gas":"0x79bc41","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1"],"depth":1,"refund":0,"opName":"DUP3"}, - {"pc":58,"op":114,"gas":"0x79bc3e","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0"],"depth":1,"refund":0,"opName":"PUSH19"}, - {"pc":78,"op":131,"gas":"0x79bc3b","gasCost":"0x3","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273"],"depth":1,"refund":0,"opName":"DUP4"}, - {"pc":79,"op":64,"gas":"0x79bc38","gasCost":"0x14","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x0"],"depth":1,"refund":0,"opName":"BLOCKHASH"}, - {"pc":80,"op":65,"gas":"0x79bc24","gasCost":"0x2","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x44852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d"],"depth":1,"refund":0,"opName":"COINBASE"}, - {"pc":81,"op":72,"gas":"0x79bc22","gasCost":"0x2","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x44852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d","0xb94f5374fce5edbc8e2a8697c15331677e6ebf0b"],"depth":1,"refund":0,"opName":"BASEFEE"}, - {"pc":82,"op":8,"gas":"0x79bc20","gasCost":"0x8","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x44852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d","0xb94f5374fce5edbc8e2a8697c15331677e6ebf0b","0x10"],"depth":1,"refund":0,"opName":"ADDMOD"}, - {"pc":83,"op":62,"gas":"0x79bc18","gasCost":"0x0","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0xb94f5374fce5edbc8e2a8697c15331677e6ebf1b"],"depth":1,"refund":0,"opName":"RETURNDATACOPY","error":"Out of bounds"}, - {"output":"","gasUsed":"0x7a1200","test":"00000936-mixed-1","fork":"Shanghai","d":0,"g":0,"v":0,"postHash":"0xd14c10ed22a1cfb642e374be985ac581c39f3969bd59249e0405aca3beb47a47","postLogsHash":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","pass":false} - ] -} diff --git a/ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/rlpx/framing/Framer.java b/ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/rlpx/framing/Framer.java index 2e9cad6dfe5..1302b3765d4 100644 --- a/ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/rlpx/framing/Framer.java +++ b/ethereum/p2p/src/main/java/org/hyperledger/besu/ethereum/p2p/rlpx/framing/Framer.java @@ -16,7 +16,6 @@ import static io.netty.buffer.ByteBufUtil.hexDump; import static io.netty.buffer.Unpooled.wrappedBuffer; -import static org.bouncycastle.pqc.math.linearalgebra.ByteUtils.xor; import static org.hyperledger.besu.ethereum.p2p.rlpx.RlpxFrameConstants.LENGTH_FRAME_SIZE; import static org.hyperledger.besu.ethereum.p2p.rlpx.RlpxFrameConstants.LENGTH_MAX_MESSAGE_FRAME; @@ -396,6 +395,23 @@ private static int padding16(final int size) { return pad == 0 ? 0 : 16 - pad; } + /** + * Compute the bitwise XOR of two arrays of bytes. The arrays have to be of same length. No length + * checking is performed. + * + * @param x1 the first array + * @param x2 the second array + * @return x1 XOR x2 + */ + private static byte[] xor(final byte[] x1, final byte[] x2) { + byte[] out = new byte[x1.length]; + + for (int i = x1.length - 1; i >= 0; i--) { + out[i] = (byte) (x1[i] ^ x2[i]); + } + return out; + } + @FormatMethod private static FramingException error(final String s, final Object... params) { return new FramingException(String.format(s, params)); diff --git a/ethereum/referencetests/src/main/java/org/hyperledger/besu/ethereum/referencetests/BonsaiReferenceTestWorldState.java b/ethereum/referencetests/src/main/java/org/hyperledger/besu/ethereum/referencetests/BonsaiReferenceTestWorldState.java index 0e23f5406af..ff8a33814cc 100644 --- a/ethereum/referencetests/src/main/java/org/hyperledger/besu/ethereum/referencetests/BonsaiReferenceTestWorldState.java +++ b/ethereum/referencetests/src/main/java/org/hyperledger/besu/ethereum/referencetests/BonsaiReferenceTestWorldState.java @@ -49,6 +49,8 @@ public class BonsaiReferenceTestWorldState extends BonsaiWorldState private final BonsaiReferenceTestWorldStateStorage refTestStorage; private final BonsaiPreImageProxy preImageProxy; + private boolean disableRootHashVerification; + protected BonsaiReferenceTestWorldState( final BonsaiReferenceTestWorldStateStorage worldStateStorage, final CachedMerkleTrieLoader cachedMerkleTrieLoader, @@ -87,7 +89,7 @@ public ReferenceTestWorldState copy() { */ @Override protected void verifyWorldStateRoot(final Hash calculatedStateRoot, final BlockHeader header) { - if (!header.getStateRoot().equals(Hash.ZERO)) { + if (!disableRootHashVerification) { super.verifyWorldStateRoot(calculatedStateRoot, header); } } @@ -125,6 +127,10 @@ public Stream streamAccounts(final Bytes32 startKeyHash, fina return this.refTestStorage.streamAccounts(this, startKeyHash, limit); } + public void disableRootHashVerification() { + disableRootHashVerification = true; + } + static class NoOpTrieLogManager implements TrieLogManager { private final Subscribers trieLogObservers = Subscribers.create(); private final TrieLogFactory trieLogFactory = new TrieLogFactoryImpl(); diff --git a/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/AbstractRLPInput.java b/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/AbstractRLPInput.java index 43413d7995e..93a0bcebdf4 100644 --- a/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/AbstractRLPInput.java +++ b/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/AbstractRLPInput.java @@ -24,6 +24,7 @@ import org.apache.tuweni.bytes.Bytes; import org.apache.tuweni.bytes.Bytes32; +import org.apache.tuweni.bytes.Bytes48; import org.apache.tuweni.bytes.MutableBytes; import org.apache.tuweni.bytes.MutableBytes32; import org.apache.tuweni.units.bigints.UInt256; @@ -98,6 +99,8 @@ protected void init(final long inputSize, final boolean shouldFitInputSizeExactl protected abstract Bytes32 inputSlice32(long offset); + protected abstract Bytes48 inputSlice48(long offset); + protected abstract String inputHex(long offset, int length); protected abstract BigInteger getUnsignedBigInteger(long offset, int length); @@ -426,6 +429,14 @@ public Bytes32 readBytes32() { return res; } + @Override + public Bytes48 readBytes48() { + checkElt("48 bytes value", 48); + final Bytes48 res = inputSlice48(currentPayloadOffset); + setTo(nextItem()); + return res; + } + @Override public T readBytes(final Function mapper) { final Bytes res = readBytes(); diff --git a/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/BytesValueRLPInput.java b/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/BytesValueRLPInput.java index 73eee238569..ac8ac8d6f45 100644 --- a/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/BytesValueRLPInput.java +++ b/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/BytesValueRLPInput.java @@ -18,6 +18,7 @@ import org.apache.tuweni.bytes.Bytes; import org.apache.tuweni.bytes.Bytes32; +import org.apache.tuweni.bytes.Bytes48; /** An {@link RLPInput} that reads RLP encoded data from a {@link Bytes}. */ public class BytesValueRLPInput extends AbstractRLPInput { @@ -51,6 +52,11 @@ protected Bytes32 inputSlice32(final long offset) { return Bytes32.wrap(inputSlice(offset, 32)); } + @Override + protected Bytes48 inputSlice48(final long offset) { + return Bytes48.wrap(inputSlice(offset, 48)); + } + @Override protected String inputHex(final long offset, final int length) { return value.slice(Math.toIntExact(offset), length).toString().substring(2); diff --git a/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java b/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java index e8f966ed1b5..808c8f74d54 100644 --- a/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java +++ b/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java @@ -22,6 +22,7 @@ import org.apache.tuweni.bytes.Bytes; import org.apache.tuweni.bytes.Bytes32; +import org.apache.tuweni.bytes.Bytes48; import org.apache.tuweni.units.bigints.UInt256; import org.apache.tuweni.units.bigints.UInt64; @@ -300,6 +301,16 @@ default long readUnsignedInt() { */ Bytes32 readBytes32(); + /** + * Reads the next item of this input (assuming it is not a list) that must be exact 48 bytes. + * + * @return The next item read of this input. + * @throws RLPException if the next item to read is a list, the input is at the end of its current + * list (and {@link #leaveList()} hasn't been called) or the next element is not exactly 48 + * bytes. + */ + Bytes48 readBytes48(); + /** * Reads the next item of this input (assuming it is not a list) and transforms it with the * provided mapping function. diff --git a/gradle.properties b/gradle.properties index ff1e9a87604..8f513eb86d8 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,4 +1,4 @@ -version=23.10.0 +version=23.10.1-RC org.gradle.welcome=never # Set exports/opens flags required by Google Java Format and ErrorProne plugins. (JEP-396) diff --git a/gradle/verification-metadata.xml b/gradle/verification-metadata.xml index bf2f064afa6..39a5dccc217 100644 --- a/gradle/verification-metadata.xml +++ b/gradle/verification-metadata.xml @@ -3481,6 +3481,14 @@ + + + + + + + + @@ -3505,6 +3513,14 @@ + + + + + + + + @@ -3513,6 +3529,14 @@ + + + + + + + + diff --git a/gradle/versions.gradle b/gradle/versions.gradle index 1548ca5baee..af5d4fc47ce 100644 --- a/gradle/versions.gradle +++ b/gradle/versions.gradle @@ -151,7 +151,7 @@ dependencyManagement { dependency 'org.awaitility:awaitility:4.2.0' - dependencySet(group: 'org.bouncycastle', version: '1.75') { + dependencySet(group: 'org.bouncycastle', version: '1.76') { entry'bcpkix-jdk18on' entry'bcprov-jdk18on' } diff --git a/plugin-api/build.gradle b/plugin-api/build.gradle index b29b2c0e78e..f79852f2011 100644 --- a/plugin-api/build.gradle +++ b/plugin-api/build.gradle @@ -69,7 +69,7 @@ Calculated : ${currentHash} tasks.register('checkAPIChanges', FileStateChecker) { description = "Checks that the API for the Plugin-API project does not change without deliberate thought" files = sourceSets.main.allJava.files - knownHash = 'IPqcFdM1uy+ZDbcvzsKxMIrzhP9VoaSeanhBOLtbhfE=' + knownHash = 'j6NRklFHlG35Pq/t6t/oJBrT8DbYOyruGq3cJNh4ENw=' } check.dependsOn('checkAPIChanges') diff --git a/plugin-api/src/main/java/org/hyperledger/besu/plugin/data/Deposit.java b/plugin-api/src/main/java/org/hyperledger/besu/plugin/data/Deposit.java index 2fe82effe72..50e67898d75 100644 --- a/plugin-api/src/main/java/org/hyperledger/besu/plugin/data/Deposit.java +++ b/plugin-api/src/main/java/org/hyperledger/besu/plugin/data/Deposit.java @@ -34,7 +34,7 @@ public interface Deposit { * * @return public key of sender */ - PublicKey getPublicKey(); + PublicKey getPubkey(); /** * Withdrawal credential that contains info that will be used for verifying the destination of diff --git a/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/TransactionSelectionService.java b/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/TransactionSelectionService.java index 1a46e71f306..aab6e717bad 100644 --- a/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/TransactionSelectionService.java +++ b/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/TransactionSelectionService.java @@ -16,7 +16,7 @@ package org.hyperledger.besu.plugin.services; import org.hyperledger.besu.plugin.Unstable; -import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory; +import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory; import java.util.Optional; @@ -29,12 +29,13 @@ public interface TransactionSelectionService extends BesuService { * * @return the transaction selector factory */ - Optional get(); + Optional get(); /** * Registers the transaction selector factory with the service * * @param transactionSelectorFactory transaction selector factory to be used */ - void registerTransactionSelectorFactory(TransactionSelectorFactory transactionSelectorFactory); + void registerTransactionSelectorFactory( + PluginTransactionSelectorFactory transactionSelectorFactory); } diff --git a/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/TransactionSelector.java b/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/PluginTransactionSelector.java similarity index 61% rename from plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/TransactionSelector.java rename to plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/PluginTransactionSelector.java index 76d1a4b2db8..daa93279606 100644 --- a/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/TransactionSelector.java +++ b/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/PluginTransactionSelector.java @@ -16,13 +16,25 @@ package org.hyperledger.besu.plugin.services.txselection; import org.hyperledger.besu.datatypes.PendingTransaction; +import org.hyperledger.besu.evm.tracing.OperationTracer; import org.hyperledger.besu.plugin.Unstable; import org.hyperledger.besu.plugin.data.TransactionProcessingResult; import org.hyperledger.besu.plugin.data.TransactionSelectionResult; /** Interface for the transaction selector */ @Unstable -public interface TransactionSelector { +public interface PluginTransactionSelector { + + /** + * Method that returns an OperationTracer that will be used when executing transactions that are + * candidates to be added to a block. + * + * @return OperationTracer to be used to trace candidate transactions + */ + default OperationTracer getOperationTracer() { + return OperationTracer.NO_TRACING; + } + /** * Method called to decide whether a transaction is added to a block. The result can also indicate * that no further transactions can be added to the block. @@ -43,4 +55,23 @@ TransactionSelectionResult evaluateTransactionPreProcessing( */ TransactionSelectionResult evaluateTransactionPostProcessing( PendingTransaction pendingTransaction, TransactionProcessingResult processingResult); + + /** + * Method called when a transaction is selected to be added to a block. + * + * @param pendingTransaction The transaction that has been selected. + * @param processingResult The result of processing the selected transaction. + */ + default void onTransactionSelected( + final PendingTransaction pendingTransaction, + final TransactionProcessingResult processingResult) {} + /** + * Method called when a transaction is not selected to be added to a block. + * + * @param pendingTransaction The transaction that has not been selected. + * @param transactionSelectionResult The transaction selection result + */ + default void onTransactionNotSelected( + final PendingTransaction pendingTransaction, + final TransactionSelectionResult transactionSelectionResult) {} } diff --git a/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/TransactionSelectorFactory.java b/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/PluginTransactionSelectorFactory.java similarity index 81% rename from plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/TransactionSelectorFactory.java rename to plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/PluginTransactionSelectorFactory.java index 81fb2a5512b..dc6311bb557 100644 --- a/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/TransactionSelectorFactory.java +++ b/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txselection/PluginTransactionSelectorFactory.java @@ -17,16 +17,14 @@ import org.hyperledger.besu.plugin.Unstable; -import java.util.List; - /** Interface for a factory that creates transaction selectors */ @Unstable -public interface TransactionSelectorFactory { +public interface PluginTransactionSelectorFactory { /** - * Create a list of transaction selectors + * Create a transaction selector * - * @return the transaction selector list + * @return the transaction selector */ - List create(); + PluginTransactionSelector create(); } diff --git a/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txvalidator/PluginTransactionValidator.java b/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txvalidator/PluginTransactionValidator.java index 60f04497199..ab47b31f0d7 100644 --- a/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txvalidator/PluginTransactionValidator.java +++ b/plugin-api/src/main/java/org/hyperledger/besu/plugin/services/txvalidator/PluginTransactionValidator.java @@ -18,7 +18,9 @@ import org.hyperledger.besu.datatypes.Transaction; import org.hyperledger.besu.plugin.Unstable; -/** Interface for the transaction validator */ +import java.util.Optional; + +/** Interface for the transaction validator plugin */ @Unstable public interface PluginTransactionValidator { @@ -26,7 +28,8 @@ public interface PluginTransactionValidator { * Method called to decide whether a transaction can be added to the transaction pool. * * @param transaction candidate transaction - * @return true if the transaction can be added, false otherwise + * @return Optional.empty() if the transaction is valid, an Optional containing an error message, + * if not */ - boolean validateTransaction(final Transaction transaction); + Optional validateTransaction(final Transaction transaction); } diff --git a/testutil/src/main/resources/hive/testBlockchain.blocks b/testutil/src/main/resources/hive/testBlockchain.blocks index 480efc87d9f..8504fd7a8c4 100644 Binary files a/testutil/src/main/resources/hive/testBlockchain.blocks and b/testutil/src/main/resources/hive/testBlockchain.blocks differ diff --git a/testutil/src/main/resources/hive/testGenesis.json b/testutil/src/main/resources/hive/testGenesis.json index 248f0933ddd..ccec4624e62 100644 --- a/testutil/src/main/resources/hive/testGenesis.json +++ b/testutil/src/main/resources/hive/testGenesis.json @@ -4,7 +4,8 @@ "ethash": { }, "londonBlock": 33, - "shanghaiTime": 1444660030 + "shanghaiTime": 1444660030, + "cancunTime": 1444660040 }, "parentHash" : "0x0000000000000000000000000000000000000000000000000000000000000000", "coinbase" : "0x8888f1f195afa192cfee860698584c030f4c9db1",