Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate each edge only once #4098

Closed
wants to merge 12 commits into from
Closed

Validate each edge only once #4098

wants to merge 12 commits into from

Conversation

pmnoxx
Copy link
Contributor

@pmnoxx pmnoxx commented Mar 14, 2021

Currently we may validate each edge a few times after a node connects to a network. This PR will guarantee that each node edge will be only verified once on node joining the network.

@pmnoxx pmnoxx self-assigned this Mar 14, 2021
@pmnoxx
Copy link
Contributor Author

pmnoxx commented Mar 14, 2021

#3621

Copy link
Collaborator

@bowenwang1996 bowenwang1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mfornet can you review this PR?

Copy link
Collaborator

@bowenwang1996 bowenwang1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does QueueSender do and why do we need it?

@pmnoxx
Copy link
Contributor Author

pmnoxx commented Mar 15, 2021

What does QueueSender do and why do we need it?

I noticed that on my machine we would spend 20s in PeerManager, processing response from EdgeVerifier. That would probably be over 1m on Google Cloud.
Btw, my performance stats code is not tracking time spend in futures, which where spawned with .spawn(ctx). That's why we didn't notice that.

Therefore I decided to add MPSC lock-free unbounded queue for sending edges from EdgeVerifier to PeerManager. My intention is to limit the amount of time we block PeerManager.

I'll change it so we spend at most 50ms at a time processing verified edges in PeerManager. Then we yield, and then we continue again 50ms later.

@pmnoxx pmnoxx closed this Mar 15, 2021
@pmnoxx pmnoxx deleted the piotr-duplicated-edges branch March 16, 2021 03:01
@pmnoxx
Copy link
Contributor Author

pmnoxx commented Mar 16, 2021

#4111

near-bulldozer bot pushed a commit that referenced this pull request Mar 17, 2021
Currently we may validate each edge a few times after a node connects to a network. This PR will guarantee that each node edge will be only verified once on node joining the network.

Reposting #4098 to fir CI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants