Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allocate less memory when validating new mempool transactions #12838

Open
hai-rise opened this issue Nov 25, 2024 · 3 comments
Open

Allocate less memory when validating new mempool transactions #12838

hai-rise opened this issue Nov 25, 2024 · 3 comments
Labels
C-enhancement New feature or request M-prevent-stale Prevents old inactive issues/PRs from being closed due to inactivity S-needs-triage This issue needs to be labelled

Comments

@hai-rise
Copy link
Contributor

hai-rise commented Nov 25, 2024

Describe the feature

Not sure how accurate bytehound is but it reports that validating new mempool transactions allocates (and deallocates) ~100 GB of RAM for ~10M transactions in ~10 minutes (~15k new transactions per second).

Image

#50 [rise] <reth_transaction_pool::validate::task::TransactionValidationTaskExecutor<V> as reth_transaction_pool::validate::TransactionValidator>::validate_transaction::{{closure}}::{{closure}} [task.rs:186]
#51 [rise] <rise_mempool::validator::RiseTransactionValidator<Client,Tx> as reth_transaction_pool::validate::TransactionValidator>::validate_transaction::{{closure}} [validator.rs:81]
#52 [rise] <reth_optimism_node::txpool::OpTransactionValidator<Client,Tx> as reth_transaction_pool::validate::TransactionValidator>::validate_transaction::{{closure}} [txpool.rs:205]
#53 [rise] reth_optimism_node::txpool::OpTransactionValidator<Client,Tx>::validate_one [txpool.rs:125]
#54 [rise] reth_transaction_pool::validate::eth::EthTransactionValidator<Client,Tx>::validate_one [eth.rs:66]
#55 [rise] reth_transaction_pool::validate::eth::EthTransactionValidatorInner<Client,Tx>::validate_one [eth.rs:329]
#56 [rise] <reth_provider::providers::blockchain_provider::BlockchainProvider2<N> as reth_storage_api::state::StateProviderFactory>::latest [blockchain_provider.rs:532]
#57 [rise] reth_provider::providers::blockchain_provider::BlockchainProvider2<N>::block_state_provider [blockchain_provider.rs:136]
#58 [rise] reth_provider::providers::database::ProviderFactory<N>::history_by_block_hash [mod.rs:184]

This matches samply's complaint in #12629 about copying the 32-byte tx-cost being a big overhead in the same code path.

This issue traces the whole transaction validation process and removes unneeded allocations. For instance, one of the earliest checks, ensure_max_init_code_size, copies the 21-byte TxKind (kind()) to check if the new transaction is a Create one.

fn ensure_max_init_code_size(
&self,
max_init_code_size: usize,
) -> Result<(), InvalidPoolTransactionError> {
if self.kind().is_create() && self.input().len() > max_init_code_size {

This check alone would allocate 200MB to validate 10M transactions. Intuitively, this should be a single reference byte check of the &TxKind enum.

More to come 🙏.

Additional context

No response

@mattsse
Copy link
Collaborator

mattsse commented Nov 25, 2024

ty

Copy link
Contributor

This issue is stale because it has been open for 21 days with no activity.

@github-actions github-actions bot added the S-stale This issue/PR is stale and will close with no further activity label Dec 17, 2024
@hai-rise
Copy link
Contributor Author

@github-actions Here is an activity, sir.

@mattsse mattsse added M-prevent-stale Prevents old inactive issues/PRs from being closed due to inactivity and removed S-stale This issue/PR is stale and will close with no further activity labels Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement New feature or request M-prevent-stale Prevents old inactive issues/PRs from being closed due to inactivity S-needs-triage This issue needs to be labelled
Projects
Status: Todo
Development

No branches or pull requests

2 participants