refactor: Strict/Non-strict mode improvements #164
Conversation
src/main.rs
Outdated
| "Stopping after according to `--stop-after-number-of-blocks {}`", | ||
| index_to_stop, | ||
| ); | ||
| std::process::exit(0); |
There was a problem hiding this comment.
CRITICAL: This might corrupt the database. We should shut nearcore down gracefully (ask @miraclx if you need help)
src/main.rs
Outdated
| if let Some(index_to_stop) = stop_after_number_of_blocks { | ||
| if (index_to_stop) as usize == index { |
There was a problem hiding this comment.
Can we just use .take(stop_after_number_of_blocks) on the iterator? This will gracefully handle all the messages in the while loop below and just exit from Indexing (we will only need to gracefully shut down nearcore after we complete here).
Co-authored-by: Vlad Frolov <frolvlad@gmail.com>
| use clap::Clap; | ||
| use std::convert::{TryFrom, TryInto}; |
There was a problem hiding this comment.
Why? std imports are more generic, so should be above
src/main.rs
Outdated
| }); | ||
| let mut handle_messages = if let Some(stop_after_n_blocks) = stop_after_number_of_blocks { | ||
| handle_messages | ||
| .take(stop_after_n_blocks as usize) |
There was a problem hiding this comment.
I would either make stop_after_number_of_blocks to be usize in the first place, or use usize::try_from().expect() since as usize might cause unexpected issues on non-64-bit platforms down the road.
src/configs.rs
Outdated
| pub non_strict_mode: bool, | ||
| /// Stops indexer completely after indexing the provided number of blocks | ||
| #[clap(long, short)] | ||
| pub stop_after_number_of_blocks: Option<u64>, |
src/main.rs
Outdated
|
|
||
| while let Some(_handled_message) = handle_messages.next().await {} | ||
| // Graceful shutdown | ||
| drop(handle_messages); | ||
| tokio::time::sleep(std::time::Duration::from_secs(5)).await; |
There was a problem hiding this comment.
Let's have a log here that the graceful shutdown will happen in 5 seconds, so whoever uses it, don't think it got stuck.
src/main.rs
Outdated
| join!( | ||
| execution_outcomes_future, | ||
| accounts_future, | ||
| access_keys_future, |
There was a problem hiding this comment.
Storing access keys may also fail in non-strict mode, so we should skip them.
P.S. Update the Clap help to indicate this.
Fixes #163
I've bumped the version to
0.10.0because I've dropped the--allow-missing-relations-in-first-blocksflag completely. It doesn't mean this PR is going to be released once merged.--allow-missing-relations-in-first-blocksflag completely--non-strict-modewhich:account_changescompletely as it is impossible to save in case of missing of related Receipts--stop-after-number-of-blocks Nwhich namely stops the indexer after processing N blocksAll those flags and features are useful for debugging or emergency purposes. They are not supposed to be used in normal circumstances.