-
Notifications
You must be signed in to change notification settings - Fork 677
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: call_main
utility for integration test
#7217
Conversation
We have been copying the same code multiple times, this refactor pulls out the functionality the `TestEnv`.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly 👍 but curious if you think we can do better with the interface to specify transactions?
})]; | ||
|
||
self.execute_sir_tx(actions, &signer) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One thing I don't like here is that the API is non-compistional. If at some point we introduce some kind of transaction builder which returns a Tx, we'd have to mirror the API here, as there isn't a function which takes a Tx.
This is a non-trivial problem: tx has a nonce, and a block hash, and has to be signed, and we don't want the caller to get the nonce.
So perhaps we need the following, slightly hacky API?
pub fn tx(&mut self, mut tx: Transaction) {
tx.nonce = tip.height + 1;
tx.block_hash = tip.hash;
let signer = Signer::from_seed(tx.signer_id.as_str())
let tx = tx.sign(&signer)
}
The caller would then basically specify garbage nonce & hash
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, I was really hoping to get rid of the caller-site giving garbage nonce and hash.
I have a different suggestion how to make it more compositional, basically split the "create tx" part from the "execute tx" into two functions.
fn tx_from_actions(
&mut self,
actions: Vec<Action>,
signer: &InMemorySigner,
receiver: AccountId,
) -> SignedTransaction;
pub fn execute_tx(&mut self, tx: SignedTransaction) -> FinalExecutionOutcomeView;
This also opened up more removal of duplicated code in test_deploy_cost_increased
. The new interface and this additional refactor are included in the latest two commits.
@matklad Do you think this is better now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I definitelly like the new API better, in particular, execute_tx(&mut self, tx: SignedTransaction)
is very much the right boundary.
I think I'd still prefer garbage nonce approach, but that's debatable. In particular, I feel like that's more of a problem of the split between Tx and SignedTx -- ideally, "what the TX logically is" (list of actions, sender, receiver) should be separate from stuff we need to make cryptograhpy work (nonce, block hash, signature), at least in the testing API. Ideally, we'd have some kind of TxBuilder, but a Tx with Default
nonce&block hash can stand in its place. But either of these is a bigger change than warranted in this PR, even if they are good ideas.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the logical piece of work that we generally care about is just a list of actions, not a transaction.
What you describe as
"what the TX logically is" (list of actions, sender, receiver)
in my opinion is not a Transaction
. The crypto stuff to define valid placement in the chain very much belongs to a valid transaction definition. The difference between Transaction
and SignedTransaction
is just how we represent it in memory but logically they describe the same thing.
But I support the idea of a TX builder that prepares actions/signer/receiver and builds into a full TX. But fn tx_from_actions
is already quite simple to build a TX. I think to make a builder valuable, it would need to do more than just that. For example, track accounts included in genesis, or help somehow with preparing the receiver with a deployed contract first etc.
chain/client/src/test_utils.rs
Outdated
}; | ||
let signed_tx = tx.sign(signer); | ||
let tx_hash = signed_tx.get_hash().clone(); | ||
self.clients[0].process_tx(signed_tx, false, false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Magically defaulting to 0
client is somewhat odd, but I also don't feel like we should add client_id param everywhere. I'd say this OK, maybe at some point in the future we'll learn how a better API would look like.
@@ -228,6 +228,23 @@ fn prepare_env_with_congestion( | |||
(env, tx_hashes) | |||
} | |||
|
|||
/// Create a `TestEnv` with an account and a contract deployed to that account. | |||
fn prepare_env_with_contract( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be great to make it just a method of the builder object! But, with the way real and kv runtimes are setup, that's not going to be trivial, so lgtm.
})]; | ||
|
||
self.execute_sir_tx(actions, &signer) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I definitelly like the new API better, in particular, execute_tx(&mut self, tx: SignedTransaction)
is very much the right boundary.
I think I'd still prefer garbage nonce approach, but that's debatable. In particular, I feel like that's more of a problem of the split between Tx and SignedTx -- ideally, "what the TX logically is" (list of actions, sender, receiver) should be separate from stuff we need to make cryptograhpy work (nonce, block hash, signature), at least in the testing API. Ideally, we'd have some kind of TxBuilder, but a Tx with Default
nonce&block hash can stand in its place. But either of these is a bigger change than warranted in this PR, even if they are good ideas.
Simplified testing code thanks to near#7217
* Display block and chunk producers of an epoch. * Epoch length * test: simplify mocking in setup_mock_all_validators (#7194) There are many lovable aspects to `setup_mock_all_validators`! What this PR deals with is "tying the knot" infra. `setup_mock_validator` is based on creating `n` validators and setting up the mock network between them. To create a mock network, you need all `n` validators, and to create a validator, you need a mock network. The cycle is broken at runtime, in three different ways: * inside `setup_mock_all_validators`, we have `locked_connectors` * at some call-sites, we overwrite the whole `peer_manager_mock` * yet at other call-sites, we essentially create a copy of `locked_connectors` This PR unifies that to a single infra: * there's a single `Arc<OnceCell<Vec<Addr<Client>>>>` inside `setup_mock_all_validators` * `OnceCell::wait` is used as a barrier * The `&[Addr<CLient>]` are passed in as an argument to `peer_manager_mock`, so that it doesn't have to re-invent a way to get a hold of those This changes the signature of the function, so the diff is pretty large. It is designed to be reviewed per-commit. * Fix m1 build issue (#7199) * renamed Validator to AccountData (#7206) https://nearinc.atlassian.net/browse/CPR-94 * feat: new estimator variance check (#7197) The condition to check for high variance of measurements so far has been that no single sample is further than 15% from the mean away. We often missed that when running with 5 iterations and one warm-up iteration. The counter-intuitive side of this check is that is becomes less likely that variance is considered low enough, if more iterations are added. The new condition computes actual variance and standard deviation. Now we expect that standard deviation is lower than 10%. Also add an outer loop to block overhead computation. Usually there is a loop + a number of actions inside a block. But this estimation has empty blocks and therefore each measurement so far was just a single apply block call. Grouping 10 empty blocks together for each measurement and adding an outer loop for `--iters` reduces the overall variance. * test: more useful assert (#7211) Got this one spuriously(?) firing in some test with nightly features. Let's make the failure a bit more helpful here! * trie_key: don’t store single bytes as slices (#7200) trie_key module stores separators and keys as slices even though they are all just one-byte long. There are parts of the code which assume the length to be one and won’t work if the length ever changes. Simplify the code by keeping the values as plain bytes rather than slices so that it’s explicit in their type that they are always just one byte. * test: simplify cops support in KeyValue runtime (#7213) * eagarly compute assignemt of validators to blocks & chunks, which I find a bit easier to wrap my head around (you can print `EpochValidatorSet`s and just see the result) * compress the code by centralizing cfg handling * Refactor: remove some dead code (#7216) Closes #7203 There are a few structs that do not need to derive `Serialize` and `Deserialize`. Remove the derives highlights that there are some structs that we do not need at all and that we do not need to define custom deserializer: `bytes_as_string`. Tests ------- - `cargo check --tests --benches` does not complain. - Other existing tests * refactor: remove metric field in `GasCost` (#7215) Explicitly tracking the metric inside `GasCost` is awkward for zero values and it requires the estimator configuration to be passed around more than strictly necessary. This change makes the metric implicit, by virtue of having or not having the measurement values stored inside. For conversion to gas, and when printing, `ICount` measurements are given priority over `Time` measurements when for both values exist. This could be changed, if both need to be tracked simultaneously, for now only one of them is of interest at the time. * export rocksdb cf size as prometheus metric (#6635) Creating a function to get int rocks db properties for every column and adding a way to export it as a prometheus metric. Reorganising parse_statistics to write stats into provided result param. * refactor: `call_main` utility for integration test (#7217) We have been copying the same code multiple times, this refactor pulls out the functionality the `TestEnv`. * Replace a map with a for-loop Co-authored-by: Aleksey Kladov <aleksey.kladov@gmail.com> Co-authored-by: Phuong Nguyen <ChaoticTempest@users.noreply.github.com> Co-authored-by: pompon0 <pompon.pompon@gmail.com> Co-authored-by: Jakob Meier <mail@jakobmeier.ch> Co-authored-by: Michal Nazarewicz <mina86@mina86.com> Co-authored-by: Akhilesh Singhania <akhi3030@gmail.com> Co-authored-by: posvyatokum <posvyatokum@edu.hse.ru>
We have been copying the same code multiple times, this refactor
pulls out the functionality the
TestEnv
.