Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kylin sync stuck #773

Closed
heifner opened this issue Sep 13, 2024 · 1 comment · Fixed by #777 or #805
Closed

Kylin sync stuck #773

heifner opened this issue Sep 13, 2024 · 1 comment · Fixed by #777 or #805
Assignees
@heifner
Labels
bug The product is not working as was intended. 👍 lgtm OCI Work exclusive to OCI team
Milestone
Spring v1.0.2

Comments

@heifner
Copy link
Member

heifner commented Sep 13, 2024

Trying to sync kylin from genesis with spring-1.0.1 release. It is stuck at the point LIB does not advance for a bunch of blocks.

image
@heifner heifner added this to the Spring v1.0.2 milestone Sep 13, 2024
@heifner heifner self-assigned this Sep 13, 2024
@enf-ci-bot enf-ci-bot moved this to Todo in Team Backlog Sep 13, 2024
@heifner heifner added bug The product is not working as was intended. OCI Work exclusive to OCI team and removed triage labels Sep 13, 2024
@heifner heifner moved this from Todo to In Progress in Team Backlog Sep 13, 2024
@heifner heifner added the triage label Sep 13, 2024
@heifner
Copy link
Member Author

heifner commented Sep 13, 2024

Syncing in irreversible mode which means chain_head == LIB. We restrict syncing to not get too far ahead of chain_head. That causes Kylin to get stuck when it reaches a span where LIB does not move for greater than sync-fetch-span.

heifner added a commit that referenced this issue Sep 13, 2024
@heifner
GH-773 Add test cases for irreversible mode where sync-fetch-span les…
a60227f 
…s than head-LIB
heifner added a commit that referenced this issue Sep 13, 2024
@heifner
GH-773 Allow irreversible mode to keep syncing if LIB not moving
88a1626
heifner added a commit that referenced this issue Sep 13, 2024
@heifner
GH-773 Log when syncing ahead even though LIB is not advancing
d9a224b
@heifner heifner mentioned this issue Sep 13, 2024
Merged
@heifner heifner linked a pull request Sep 13, 2024 that will close this issue
[1.0.2] P2P: Allow irreversible mode syncing to continue when LIB is paused #777
Merged
heifner added a commit that referenced this issue Sep 13, 2024
@heifner
GH-773 Reword confusing sync timeout log messages
bc0bc8a
heifner added a commit that referenced this issue Sep 13, 2024
@heifner
GH-773 Make it obvious that controller.get_read_mode() is thread-safe.
4bceee0
heifner added a commit that referenced this issue Sep 13, 2024
@heifner
GH-773 Only check lib not moving in irreversible mode since non-irrev…
e82613a 
…ersible mode the chain_head will still be moving and not cause net_plugin to stop syncing.

Only consider if LIB not moving when the span is greater than sync_fetch_span. Otherwise normal operation will cause net_plugin to sync ahead when fork db is just up to date.
heifner added a commit that referenced this issue Sep 13, 2024
@heifner
GH-773 Fix should_sync_from being way to picky. It would not allow a …
1d2ceb1 
…sync when the peer reported even slightly less than sync_known_lib_num which is a very normal condition. For example sync_known_lib_num: 373330666, while peer_fhead: 373330647 when trying to sync block 129728.
heifner added a commit that referenced this issue Sep 13, 2024
@heifner
GH-773 Add additional debug logging
8737c4e
heifner added a commit that referenced this issue Sep 14, 2024
@heifner
GH-773 Guard against underflow
9a72b28
@arhag arhag added 👍 lgtm and removed triage labels Sep 16, 2024
heifner added a commit that referenced this issue Sep 16, 2024
@heifner
GH-773 Add assert
cb66a69
heifner added a commit that referenced this issue Sep 17, 2024
@heifner
GH-773 Update assert to allow for block 2
2cd0188
heifner added a commit that referenced this issue Sep 18, 2024
@heifner
GH-773 Remove unneeded check
1d5d977
heifner added a commit that referenced this issue Sep 18, 2024
@heifner
GH-773 Refactor to make conditions clearer
655d0ab
heifner added a commit that referenced this issue Sep 18, 2024
@heifner
GH-773 Use non-unique handler id to allow de-dup of posted functions
a81dd0f
heifner added a commit that referenced this issue Sep 18, 2024
@heifner
Merge branch 'release/1.0' into GH-773-sync-stuck
bdde4d8
heifner added a commit that referenced this issue Sep 18, 2024
@heifner
GH-773 Simplify logic
8a2fcf3
heifner added a commit that referenced this issue Sep 19, 2024
@heifner
Merge pull request #777 from AntelopeIO/GH-773-sync-stuck
07e3e79 
[1.0.2] P2P: Allow irreversible mode syncing to continue when LIB is paused
heifner added a commit that referenced this issue Sep 19, 2024
@heifner
Merge remote-tracking branch 'spring/release/1.0' into GH-773-sync-st…
fb2d31b 
…uck-main
@heifner heifner mentioned this issue Sep 19, 2024
Merged
@heifner heifner linked a pull request Sep 19, 2024 that will close this issue
[1.0.2 -> main] P2P: Allow irreversible mode syncing to continue when LIB is paused #805
Merged
heifner added a commit that referenced this issue Sep 19, 2024
@heifner
Merge pull request #805 from AntelopeIO/GH-773-sync-stuck-main
9ed5795 
[1.0.2 -> main] P2P: Allow irreversible mode syncing to continue when LIB is paused
@github-project-automation github-project-automation bot moved this from In Progress to Done in Team Backlog Sep 19, 2024
heifner added a commit that referenced this issue Sep 19, 2024
@heifner
GH-773 Use named variables for branches pair
ad1faf5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@heifner heifner

Labels
bug The product is not working as was intended. 👍 lgtm OCI Work exclusive to OCI team
Projects
Team Backlog
Archived in project
Milestone
Spring v1.0.2
3 participants
@heifner @arhag @enf-ci-bot