Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Implement checkpoint sync #2244

Closed
wants to merge 49 commits into from
Closed
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
719a934
Implement starting from a weak subectivity state
michaelsproul Feb 18, 2021
4dcf2e7
Support for historical blocks
michaelsproul Feb 23, 2021
3aa7c7b
Fix iterators and StatusMessage processing
michaelsproul Mar 4, 2021
366ae60
Merge remote-tracking branch 'origin/unstable' into weak-subj-sync
michaelsproul Mar 4, 2021
d8e8d97
Implement schema migration
michaelsproul Mar 4, 2021
6cdac53
Remove some TODOs/FIXMEs
michaelsproul Mar 5, 2021
22867d7
Clean up APIs, fix several TODOs
michaelsproul Mar 8, 2021
dd55207
Merge branch 'unstable' into weak-subj-sync
divagant-martian May 3, 2021
f86032f
Merge remote-tracking branch 'origin/unstable' into weak-subj-sync
michaelsproul Aug 4, 2021
0066231
Fixes for merge issues, block separation
michaelsproul Aug 4, 2021
25306fc
Add missed comma
michaelsproul Aug 4, 2021
6cdb165
Improve lcli pretty-ssz & logging
michaelsproul Aug 5, 2021
20a97a4
Add database HTTP APIs
michaelsproul Aug 5, 2021
8d812ce
Fix clippy
michaelsproul Aug 6, 2021
d966be6
Fix block_root_at_slot for historic blocks
michaelsproul Aug 6, 2021
5221ac6
Add consistency checks
michaelsproul Aug 16, 2021
9b290f8
Fix Altair test by setting spec
michaelsproul Aug 16, 2021
6b79baf
Add --checkpoint-sync-url
michaelsproul Aug 16, 2021
5e92a7a
Merge remote-tracking branch 'origin/unstable' into weak-subj-sync
michaelsproul Aug 16, 2021
8865e90
Update Cargo.lock
michaelsproul Aug 16, 2021
8392cac
Avoid looking up non-existent restore points
michaelsproul Aug 17, 2021
4283c1e
Remove dbg! statements
michaelsproul Aug 17, 2021
d292e3d
Implement historic state reconstruction
michaelsproul Aug 18, 2021
6d2a152
Merge remote-tracking branch 'origin/unstable' into weak-subj-sync
michaelsproul Aug 30, 2021
0ce85c4
Check initial state genesis vals root
michaelsproul Aug 30, 2021
1719ae8
Merge remote-tracking branch 'origin/unstable' into weak-subj-sync
michaelsproul Aug 30, 2021
d3738f3
Fix unused var warning
michaelsproul Aug 31, 2021
215941b
Allow one extra block in historic batch
michaelsproul Aug 31, 2021
149ae88
Allow >1 extra historic blocks
michaelsproul Sep 2, 2021
db96456
Return num blocks imported
michaelsproul Sep 3, 2021
ed18349
Mark backfill complete if genesis block reached
michaelsproul Sep 6, 2021
8dc823f
Fix overrun in load_blocks_to_replay
michaelsproul Sep 6, 2021
1c147a7
Verify backfill proposer signatures
michaelsproul Sep 8, 2021
11feeb8
Merge remote-tracking branch 'origin/unstable' into weak-subj-sync
michaelsproul Sep 10, 2021
c96a027
Background state reconstruction
michaelsproul Sep 9, 2021
5eee72f
API clean up and docs
michaelsproul Sep 10, 2021
6b7c140
Add block_backfill_complete from Age's branch
michaelsproul Sep 10, 2021
edbc3ce
Appease clippy
michaelsproul Sep 10, 2021
ff6700d
Test clean up
michaelsproul Sep 10, 2021
599956f
Remove redundant migration_mutex
michaelsproul Sep 10, 2021
8f4bf6e
Updates from self-review
michaelsproul Sep 10, 2021
f06bf2a
Network Update for Weak Subjectivity Sync (#2561)
AgeManning Sep 14, 2021
81ed51a
Additional checkpoint sync logs (#2599)
paulhauner Sep 15, 2021
8e32a1e
Log about historical backfill less frequently (#2600)
paulhauner Sep 16, 2021
b404f4d
Don't downscore peers in batch sync when disconnecting
AgeManning Sep 20, 2021
30e2219
Address review comments
michaelsproul Sep 20, 2021
6dbedfe
Tweak CLI flag help text
michaelsproul Sep 20, 2021
fc28aeb
Improved chain resuming
AgeManning Sep 20, 2021
c85c8ab
Fix clippy
michaelsproul Sep 20, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 4 additions & 25 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 26 additions & 1 deletion beacon_node/beacon_chain/src/beacon_chain.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ use crate::errors::{BeaconChainError as Error, BlockProductionError};
use crate::eth1_chain::{Eth1Chain, Eth1ChainBackend};
use crate::events::ServerSentEventHandler;
use crate::head_tracker::HeadTracker;
use crate::historical_blocks::HistoricalBlockError;
use crate::migrate::BackgroundMigrator;
use crate::naive_aggregation_pool::{
AggregatedAttestationMap, Error as NaiveAggregationError, NaiveAggregationPool,
Expand Down Expand Up @@ -431,10 +432,23 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
/// - Skipped slots contain the root of the closest prior
/// non-skipped slot (identical to the way they are stored in `state.block_roots`).
/// - Iterator returns `(Hash256, Slot)`.
///
/// Will return a `BlockOutOfRange` error if the requested start slot is before the period of
/// history for which we have blocks stored. See `get_oldest_block_slot`.
pub fn forwards_iter_block_roots(
&self,
start_slot: Slot,
) -> Result<impl Iterator<Item = Result<(Hash256, Slot), Error>>, Error> {
let oldest_block_slot = self.store.get_oldest_block_slot();
if start_slot < oldest_block_slot {
return Err(Error::HistoricalBlockError(
HistoricalBlockError::BlockOutOfRange {
slot: start_slot,
oldest_block_slot,
},
));
}

let local_head = self.head()?;

let iter = HotColdDB::forwards_block_roots_iterator(
Expand Down Expand Up @@ -620,6 +634,12 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
return Ok(Some(self.genesis_state_root));
}

// Check limits w.r.t historic state bounds.
let (historic_lower_limit, historic_upper_limit) = self.store.get_historic_state_limits();
if request_slot > historic_lower_limit && request_slot < historic_upper_limit {
return Ok(None);
}

// Try an optimized path of reading the root directly from the head state.
let fast_lookup: Option<Hash256> = self.with_head(|head| {
if head.beacon_block.slot() <= request_slot {
Expand Down Expand Up @@ -657,7 +677,8 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
/// ## Notes
///
/// - Use the `skips` parameter to define the behaviour when `request_slot` is a skipped slot.
/// - Returns `Ok(None)` for any slot higher than the current wall-clock slot.
/// - Returns `Ok(None)` for any slot higher than the current wall-clock slot, or less than
/// the oldest known block slot.
pub fn block_root_at_slot(
&self,
request_slot: Slot,
Expand All @@ -667,6 +688,10 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
WhenSlotSkipped::None => self.block_root_at_slot_skips_none(request_slot),
WhenSlotSkipped::Prev => self.block_root_at_slot_skips_prev(request_slot),
}
.or_else(|e| match e {
Error::HistoricalBlockError(_) => Ok(None),
e => Err(e),
})
}

/// Returns the block root at the given slot, if any. Only returns roots in the canonical chain.
Expand Down
19 changes: 19 additions & 0 deletions beacon_node/beacon_chain/src/block_verification.rs
Original file line number Diff line number Diff line change
Expand Up @@ -501,6 +501,9 @@ impl<T: BeaconChainTypes> GossipVerifiedBlock<T> {

let block_root = get_block_root(&block);

// Disallow blocks that conflict with the anchor (weak subjectivity checkpoint), if any.
check_block_against_anchor_slot(block.message(), chain)?;

// Do not gossip a block from a finalized slot.
check_block_against_finalized_slot(block.message(), chain)?;

Expand Down Expand Up @@ -708,6 +711,9 @@ impl<T: BeaconChainTypes> SignatureVerifiedBlock<T> {
.fork_name(&chain.spec)
.map_err(BlockError::InconsistentFork)?;

// Check the anchor slot before loading the parent, to avoid spurious lookups.
check_block_against_anchor_slot(block.message(), chain)?;

let (mut parent, block) = load_parent(block, chain)?;

// Reject any block that exceeds our limit on skipped slots.
Expand Down Expand Up @@ -1115,6 +1121,19 @@ fn check_block_skip_slots<T: BeaconChainTypes>(
Ok(())
}

/// Returns `Ok(())` if the block's slot is greater than the anchor block's slot (if any).
fn check_block_against_anchor_slot<T: BeaconChainTypes>(
block: BeaconBlockRef<'_, T::EthSpec>,
chain: &BeaconChain<T>,
) -> Result<(), BlockError<T::EthSpec>> {
if let Some(anchor_slot) = chain.store.get_anchor_slot() {
if block.slot() <= anchor_slot {
return Err(BlockError::WeakSubjectivityConflict);
}
}
Ok(())
}

/// Returns `Ok(())` if the block is later than the finalized slot on `chain`.
///
/// Returns an error if the block is earlier or equal to the finalized slot, or there was an error
Expand Down
Loading