Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip recursive discovery query if no useful ENRs #6207

Merged
merged 1 commit into from
Aug 6, 2024

Conversation

dapplion
Copy link
Collaborator

Issue Addressed

Lighthouse's basic simulator logs are completely un-usable because of the spam from the sequence of logs:

2024-07-30T08:50:55.2102984Z Jul 30 08:50:55.160 DEBG Starting a new peer discovery query     wanted: 6, outbound: 1, target: 6, connected: 1, service: libp2p, service: node_1
2024-07-30T08:50:55.2104783Z Jul 30 08:50:55.160 DEBG Starting a peer discovery request       target_peers: 6, service: libp2p, service: node_1
2024-07-30T08:50:55.2106299Z Jul 30 08:50:55.160 DEBG Discovery query completed               peers_found: 1, service: libp2p, service: node_1
2024-07-30T08:50:55.2108037Z Jul 30 08:50:55.160 DEBG Starting a new peer discovery query     wanted: 6, outbound: 1, target: 6, connected: 1, service: libp2p, service: node_1
2024-07-30T08:50:55.2109828Z Jul 30 08:50:55.160 DEBG Starting a peer discovery request       target_peers: 6, service: libp2p, service: node_1
2024-07-30T08:50:55.2111755Z Jul 30 08:50:55.160 DEBG Discovery query completed               peers_found: 1, service: libp2p, service: node_1

The root cause is that there are not enough peers in the network ever to satisfy the needs of the peer manager.

The peer manager enters an infinite loop of discovery queries that never yield new useful peers.

@AgeManning Tried to fix the issue with this PR but it doesn't work. The first condition of current_count < target is never met, so the spam still happens even with #6162

Note that this infinite loop can theoretically happen in Mainnet too, if somehow the pool of ENRs shrinks to less than our target, or someone tries to run lighthouse with --target-peers 10000.

Proposed Changes

IMO the recursiveness is an optimization that we can safely skip under certain circumstances to prevent the infinite loop. This PR implements the heuristic that if:

  • A query includes > 0 results
  • None of those results is diable

Then we have likely exhausted the discv5 table and should stop the recursive query. Note that if there are false positives the heartbeat will trigger queries latter anyway.

I have tested this PR locally with cargo run --release --bin simulator basic-sim and there is no more spam of the logs above and the node still reaches the expected 5 target peers quickly.

@dapplion dapplion added the ready-for-review The code is ready for review label Jul 30, 2024
Copy link
Member

@jxs jxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Lion

Copy link
Member

@AgeManning AgeManning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah agreed. I think this is a good soln. Nice!

@AgeManning
Copy link
Member

My changes to the DAS branch are superseded by this I think. So we should revert those when merging down.

@jimmygchen jimmygchen added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Aug 6, 2024
@jimmygchen
Copy link
Member

@mergify queue

Copy link

mergify bot commented Aug 6, 2024

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at f7f0bfc

mergify bot added a commit that referenced this pull request Aug 6, 2024
@mergify mergify bot merged commit f7f0bfc into sigp:unstable Aug 6, 2024
28 checks passed
chong-he pushed a commit to chong-he/lighthouse that referenced this pull request Nov 26, 2024
* Skip recursive discovery query if no useful ENRs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-merge This PR is ready to merge.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants