Skip to content
This repository has been archived by the owner on Feb 6, 2025. It is now read-only.

fix: bsc panic in fullnode mode #183

Merged
merged 1 commit into from
Nov 13, 2024

Conversation

j75689
Copy link
Collaborator

@j75689 j75689 commented Nov 12, 2024

Description

fix: bsc panic in fullnode mode

Rationale

{"timestamp":"2024-11-08T16:34:53.355266Z","level":"DEBUG","fields":{"message":"Inserted block","block_number":"43834321","actions":"[(InsertCanonicalHeaders, 75.108µs), (InsertHeaders, 28.324µs), (InsertHeaderNumbers, 45.012µs), (GetParentTD, 533ns), (InsertHeaderTerminalDifficulties, 18.363µs), (GetNextTxNum, 6.687µs), (InsertTransactionSenders, 0ns), (InsertTransactions, 1.780118ms), (InsertTransactionHashNumbers, 161.529585ms), (InsertBlockSidecars, 143.74297ms), (InsertBlockBodyIndices, 41.23µs), (InsertTransactionBlocks, 29.447µs)]"},"target":"providers::db"}
{"timestamp":"2024-11-08T16:34:53.355302Z","level":"DEBUG","fields":{"message":"Writing headers and transactions."},"target":"provider::storage_writer"}
{"timestamp":"2024-11-08T16:34:53.355581Z","level":"ERROR","fields":{"message":"Persistence service failed","err":"ProviderError(UnexpectedStaticFileBlockNumber(Headers, 43834321, 43834318))"},"target":"engine::persistence"}
{"timestamp":"2024-11-08T16:34:53.545614Z","level":"DEBUG","fields":{"message":"received no engine message for some time, while waiting for persistence task to complete"},"target":"engine::tree"}
{"timestamp":"2024-11-08T16:34:53.545643Z","level":"ERROR","fields":{"message":"Advancing persistence failed","err":"channel closed"},"target":"engine::tree"}
{"timestamp":"2024-11-08T16:34:53.547768Z","level":"ERROR","fields":{"message":"Fatal error"},"target":"engine::tree","span":{"name":"ChainOrchestrator::poll"},"spans":[{"name":"ChainOrchestrator::poll"}]}
{"timestamp":"2024-11-08T16:34:53.547797Z","level":"DEBUG","fields":{"message":"Event: FatalError"},"target":"reth::cli"}
{"timestamp":"2024-11-08T16:34:53.547816Z","level":"ERROR","fields":{"message":"Fatal error in consensus engine"},"target":"reth::cli"}
{"timestamp":"2024-11-08T16:34:53.549789Z","level":"ERROR","fields":{"message":"shutting down due to error"},"target":"reth::cli"}
{"timestamp":"2024-11-08T16:34:58.550003Z","level":"DEBUG","fields":{"message":"tokio runtime shutdown timed out","err":"timed out waiting on channel"},"target":"reth::cli"}

In the original implementation, a forkchoice event was emitted to the sync engine for every update from latest_head to local_finality_head.

However, the local_finality_head in the parlia engine task is read by a separate coroutine (the parlia snapshot reader) and then re-emitted via an fcu event. If this coroutine lags, it can lead to duplicate forkchoice events, which in turn causes repeated import of the same block.

This ultimately results in attempting to import blocks onto a state that has already been pruned (in fullnode mode), causing an abnormal exit.

Example

n/a

Changes

Notable changes:

  • parlia engine task

Potential Impacts

  • no

@j75689 j75689 marked this pull request as ready for review November 12, 2024 02:15
@j75689 j75689 changed the title fix: reduce fcu sending in parlia task fix: bsc panic in fullnode mode Nov 12, 2024
@unclezoro unclezoro merged commit 10ccf2b into bnb-chain:develop Nov 13, 2024
47 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants