Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce the block size limit #2195

Merged
merged 53 commits into from
Sep 27, 2024
Merged
Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
fa1288b
Add support for `ConsensusParametersVersion::V2`
rafal-ch Sep 11, 2024
0c65bbe
Update `CHANGELOG.md`
rafal-ch Sep 11, 2024
25432d2
Remove the temporary comment
rafal-ch Sep 12, 2024
0a0b6c2
Fix imports in tests
rafal-ch Sep 12, 2024
3068d2f
`#[allow(clippy::large_enum_variant)]` for `enum FakeProducedBlock` a…
rafal-ch Sep 12, 2024
0656491
Update expected GraphQL schema
rafal-ch Sep 12, 2024
a1ea188
Update `state_transition_bytecode.wasm`
rafal-ch Sep 12, 2024
345a03f
Thread the `block_transaction_size_limit` parameter across the system
rafal-ch Sep 12, 2024
256dfb7
Add `used_size` to `ExecutionData`
rafal-ch Sep 13, 2024
960745c
Remove stray comment
rafal-ch Sep 13, 2024
ecb57d8
Transaction selector respects block size limit
rafal-ch Sep 13, 2024
a75d865
`select_transactions()` does not need to `collect()`
rafal-ch Sep 13, 2024
6e2bf19
Split test cases using `rstest`
rafal-ch Sep 13, 2024
7c14352
Merge remote-tracking branch 'upstream/master' into 2133_block_size_c…
rafal-ch Sep 13, 2024
c002c0c
Applying several breaking changes to the WASM interface from backlog
xgreenx Sep 15, 2024
2cfcaa5
Make CI happy
xgreenx Sep 15, 2024
af91864
Merge branch 'master' into feature/wasm-interface-breaking-changes
xgreenx Sep 15, 2024
8032b7a
Remove timeout error because bytecode is not compiled
xgreenx Sep 15, 2024
94851b8
Use always compiled WASM bytecode for local testnet
xgreenx Sep 15, 2024
b7d8da6
Merge branch 'feature/wasm-interface-breaking-changes' into 2133_bloc…
rafal-ch Sep 16, 2024
96b4c7c
Merge branch '2133_block_size_consensus_parameter' into 2133_enforce_…
rafal-ch Sep 16, 2024
c85879b
Update patch for `fuel-vm` to `master` branch
rafal-ch Sep 16, 2024
309d1af
Merge branch '2133_block_size_consensus_parameter' into 2133_enforce_…
rafal-ch Sep 16, 2024
03004ac
Add integration test verifying the block size limit enforcement
rafal-ch Sep 17, 2024
7851fb1
Add more integration tests for block transaction size limit
rafal-ch Sep 17, 2024
445a51c
Merge remote-tracking branch 'upstream/master' into 2133_enforce_the_…
rafal-ch Sep 18, 2024
7e27fd6
Update CHANGELOG.md
rafal-ch Sep 18, 2024
b408669
Updates after the master merge
rafal-ch Sep 18, 2024
a8263fd
Simplify creation of transactions in tests
rafal-ch Sep 18, 2024
d01dcff
Revert changes to `state_transition_bytecode.wasm`
rafal-ch Sep 18, 2024
0b7633b
Merge branch 'master' into 2133_enforce_the_block_size_limit
xgreenx Sep 18, 2024
99099e6
Merge remote-tracking branch 'upstream/master' into 2133_enforce_the_…
rafal-ch Sep 20, 2024
70dbe06
Remove patch to `fuel-vm` from `Cargo.toml`
rafal-ch Sep 20, 2024
e7e3102
Merge remote-tracking branch 'upstream/2133_enforce_the_block_size_li…
rafal-ch Sep 20, 2024
f62fa43
Fix formatting
rafal-ch Sep 20, 2024
af8164c
Add missing comment
rafal-ch Sep 20, 2024
82949df
Set block transaction size limit in E2E tests to large value
rafal-ch Sep 20, 2024
474a81d
Update `too_many_transactions_are_split_in_blocks()` test to not fail…
rafal-ch Sep 20, 2024
9fdf1f7
Do not re-introduce `V2` in the `ConsensusParametersVersion` schema
rafal-ch Sep 20, 2024
f9b95c1
Update comment
rafal-ch Sep 20, 2024
010baf9
Merge branch 'master' into 2133_enforce_the_block_size_limit
xgreenx Sep 20, 2024
9bdea6d
Revert changes to `Cargo.lock`
rafal-ch Sep 21, 2024
36ab508
Simplify code of `select_transactions()` by using `saturating_add()`
rafal-ch Sep 21, 2024
15a2cf7
Remove superfluous calls to `tokio::time::sleep` in tests
rafal-ch Sep 21, 2024
26fd531
Merge remote-tracking branch 'upstream/2133_enforce_the_block_size_li…
rafal-ch Sep 21, 2024
251c01d
Merge branch 'master' into 2133_enforce_the_block_size_limit
xgreenx Sep 24, 2024
be29818
Merge branch 'master' into 2133_enforce_the_block_size_limit
xgreenx Sep 25, 2024
92da3b0
Merge remote-tracking branch 'upstream/master' into 2133_enforce_the_…
rafal-ch Sep 25, 2024
70e9178
Merge remote-tracking branch 'upstream/2133_enforce_the_block_size_li…
rafal-ch Sep 25, 2024
ef22a3b
Merge branch 'master' into 2133_enforce_the_block_size_limit
xgreenx Sep 25, 2024
e748837
Change `block_transaction_size_limit` to be `u32` instead of `u64`
rafal-ch Sep 26, 2024
0872751
Merge remote-tracking branch 'upstream/2133_enforce_the_block_size_li…
rafal-ch Sep 26, 2024
c226aee
Merge branch 'master' into 2133_enforce_the_block_size_limit
rafal-ch Sep 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
## [Unreleased]

### Added
- [2195](https://github.com/FuelLabs/fuel-core/pull/2195): Added enforcement of the limit on the size of the L2 transactions per block according to the `block_transaction_size_limit` parameter.
- [2131](https://github.com/FuelLabs/fuel-core/pull/2131): Add flow in TxPool in order to ask to newly connected peers to share their transaction pool
- [2182](https://github.com/FuelLabs/fuel-core/pull/2151): Limit number of transactions that can be fetched via TxSource::next

Expand Down
8 changes: 4 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions bin/e2e-test-client/tests/integration_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,12 @@ fn dev_config() -> Config {
.consensus_parameters
.set_tx_params(tx_parameters);
chain_config.consensus_parameters.set_fee_params(fee_params);
if let Err(_e) = chain_config
.consensus_parameters
.set_block_transaction_size_limit(u64::MAX)
{
eprintln!("failed to set block transaction size limit");
}
chain_config.state_transition_bytecode =
fuel_core::upgradable_executor::WASM_BYTECODE.to_vec();
let reader = reader.with_chain_config(chain_config);
Expand Down
10 changes: 6 additions & 4 deletions crates/fuel-core/src/service/adapters/executor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,18 @@ use fuel_core_types::{
};

impl fuel_core_executor::ports::TransactionsSource for TransactionsSource {
// TODO: Use `size_limit` https://github.com/FuelLabs/fuel-core/issues/2133
fn next(
&self,
gas_limit: u64,
transactions_limit: u16,
_: u32,
block_transaction_size_limit: u64,
) -> Vec<MaybeCheckedTransaction> {
self.txpool
.select_transactions(gas_limit, transactions_limit)
.select_transactions(
gas_limit,
transactions_limit,
block_transaction_size_limit,
)
.into_iter()
.map(|tx| {
MaybeCheckedTransaction::CheckedTransaction(
Expand All @@ -28,7 +31,6 @@ impl fuel_core_executor::ports::TransactionsSource for TransactionsSource {
.collect()
}
}

impl fuel_core_executor::ports::RelayerPort for ReyalerIterableKeyValueView {
fn enabled(&self) -> bool {
#[cfg(feature = "relayer")]
Expand Down
40 changes: 31 additions & 9 deletions crates/services/executor/src/executor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ impl TransactionsSource for OnceTransactionsSource {
&self,
_: u64,
transactions_limit: u16,
_: u32,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed that, but the block size should be u32.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in e748837

_: u64,
) -> Vec<MaybeCheckedTransaction> {
let mut lock = self.transactions.lock();
let transactions: &mut Vec<MaybeCheckedTransaction> = lock.as_mut();
Expand All @@ -209,6 +209,7 @@ impl TransactionsSource for OnceTransactionsSource {
pub struct ExecutionData {
coinbase: u64,
used_gas: u64,
used_size: u64,
tx_count: u16,
found_mint: bool,
message_ids: Vec<MessageId>,
Expand All @@ -224,6 +225,7 @@ impl ExecutionData {
ExecutionData {
coinbase: 0,
used_gas: 0,
used_size: 0,
tx_count: 0,
found_mint: false,
message_ids: Vec::new(),
Expand Down Expand Up @@ -296,6 +298,7 @@ where
skipped_transactions,
coinbase,
used_gas,
used_size,
..
} = execution_data;

Expand All @@ -306,8 +309,8 @@ where
let finalized_block_id = block.id();

debug!(
"Block {:#x} fees: {} gas: {}",
finalized_block_id, coinbase, used_gas
"Block {:#x} fees: {} gas: {} tx_size: {}",
finalized_block_id, coinbase, used_gas, used_size
);

let result = ExecutionResult {
Expand All @@ -331,6 +334,7 @@ where
let ExecutionData {
coinbase,
used_gas,
used_size,
tx_status,
events,
changes,
Expand All @@ -340,8 +344,8 @@ where
let finalized_block_id = block.id();

debug!(
"Block {:#x} fees: {} gas: {}",
finalized_block_id, coinbase, used_gas
"Block {:#x} fees: {} gas: {} tx_size: {}",
finalized_block_id, coinbase, used_gas, used_size
);

let result = ValidationResult { tx_status, events };
Expand Down Expand Up @@ -575,10 +579,12 @@ where
..
} = components;
let block_gas_limit = self.consensus_params.block_gas_limit();
let block_transaction_size_limit =
self.consensus_params.block_transaction_size_limit();

let mut remaining_gas_limit = block_gas_limit.saturating_sub(data.used_gas);
// TODO: Handle `remaining_size` https://github.com/FuelLabs/fuel-core/issues/2133
let remaining_size = u32::MAX;
let mut remaining_block_transaction_size_limit =
block_transaction_size_limit.saturating_sub(data.used_size);

// We allow at most u16::MAX transactions in a block, including the mint transaction.
// When processing l2 transactions, we must take into account transactions from the l1
Expand All @@ -587,7 +593,11 @@ where
let mut remaining_tx_count = MAX_TX_COUNT.saturating_sub(data.tx_count);

let mut regular_tx_iter = l2_tx_source
.next(remaining_gas_limit, remaining_tx_count, remaining_size)
.next(
remaining_gas_limit,
remaining_tx_count,
remaining_block_transaction_size_limit,
)
.into_iter()
.take(remaining_tx_count as usize)
.peekable();
Expand All @@ -614,11 +624,17 @@ where
}
}
remaining_gas_limit = block_gas_limit.saturating_sub(data.used_gas);
remaining_block_transaction_size_limit =
block_transaction_size_limit.saturating_sub(data.used_size);
remaining_tx_count = MAX_TX_COUNT.saturating_sub(data.tx_count);
}

regular_tx_iter = l2_tx_source
.next(remaining_gas_limit, remaining_tx_count, remaining_size)
.next(
remaining_gas_limit,
remaining_tx_count,
remaining_block_transaction_size_limit,
)
.into_iter()
.take(remaining_tx_count as usize)
.peekable();
Expand Down Expand Up @@ -1413,6 +1429,8 @@ where
tx_id: TxId,
) -> ExecutorResult<()> {
let (used_gas, tx_fee) = self.total_fee_paid(tx, &receipts, gas_price)?;
let used_size = tx.metered_bytes_size() as u64;

execution_data.coinbase = execution_data
.coinbase
.checked_add(tx_fee)
Expand All @@ -1421,6 +1439,10 @@ where
.used_gas
.checked_add(used_gas)
.ok_or(ExecutorError::GasOverflow)?;
execution_data.used_size = execution_data
.used_size
.checked_add(used_size)
.ok_or(ExecutorError::TxSizeOverflow)?;

if !reverted {
execution_data
Expand Down
4 changes: 2 additions & 2 deletions crates/services/executor/src/ports.rs
Original file line number Diff line number Diff line change
Expand Up @@ -115,14 +115,14 @@ impl TransactionExt for MaybeCheckedTransaction {
}

pub trait TransactionsSource {
/// Returns the next batch of transactions to satisfy the `gas_limit`.
/// Returns the next batch of transactions to satisfy the `gas_limit` and `block_transaction_size_limit`.
/// The returned batch has at most `tx_count_limit` transactions, none
/// of which has a size in bytes greater than `size_limit`.
fn next(
&self,
gas_limit: u64,
tx_count_limit: u16,
size_limit: u32,
block_transaction_size_limit: u64,
) -> Vec<MaybeCheckedTransaction>;
}

Expand Down
10 changes: 5 additions & 5 deletions crates/services/txpool/src/service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ use fuel_core_types::{
txpool::{
ArcPoolTx,
InsertionResult,
PoolTransaction,
TransactionStatus,
},
},
Expand Down Expand Up @@ -396,13 +395,14 @@ impl<P2P, ViewProvider, WasmChecker, GasPriceProvider, ConsensusProvider, MP>
&self,
max_gas: u64,
transactions_limit: u16,
block_transaction_size_limit: u64,
) -> Vec<ArcPoolTx> {
let mut guard = self.txpool.lock();
let txs = guard.includable();
let sorted_txs: Vec<Arc<PoolTransaction>> = select_transactions(txs, max_gas)
.into_iter()
.take(transactions_limit as usize)
.collect();
let sorted_txs: Vec<_> =
select_transactions(txs, max_gas, block_transaction_size_limit)
.take(transactions_limit as usize)
.collect();

for tx in sorted_txs.iter() {
guard.remove_committed_tx(&tx.id());
Expand Down
Loading
Loading