Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2. fix(state): Make FindHeaders and FindHashes run concurrently with state updates #4826

Merged
merged 1 commit into from
Jul 26, 2022

Conversation

teor2345
Copy link
Contributor

Motivation

Zebra reads 160 headers for each FindHeaders request.
The FindHeaders requests used to take 5-10 seconds to answer, they might still be a bit slow.

So we want to run them concurrently with state updates.

Depends-On: #4825

Solution

  • Make FindHeaders and FindHashes run concurrently with state updates
    • Use the new methods in existing state methods
  • Make some state queries more efficient

Review

This is an urgent PR to fix Zebra sync performance.

Reviewer Checklist

  • Code implements Specs and Designs
  • Tests for Expected Behaviour
  • Tests for Errors

@teor2345 teor2345 added C-bug Category: This is a bug P-High 🔥 I-slow Problems with performance or responsiveness I-integration-fail Continuous integration fails, including build and test failures A-state Area: State / database changes labels Jul 26, 2022
@teor2345 teor2345 requested a review from a team as a code owner July 26, 2022 00:57
@teor2345 teor2345 self-assigned this Jul 26, 2022
@teor2345 teor2345 requested review from oxarbitrage and removed request for a team July 26, 2022 00:57
@teor2345 teor2345 changed the title Make FindHeaders and FindHashes run concurrently with state updates 2. fix(state): Make FindHeaders and FindHashes run concurrently with state updates Jul 26, 2022
@codecov
Copy link

codecov bot commented Jul 26, 2022

Codecov Report

Merging #4826 (032c6a4) into main (ed553a9) will decrease coverage by 0.09%.
The diff coverage is 79.64%.

@@            Coverage Diff             @@
##             main    #4826      +/-   ##
==========================================
- Coverage   78.88%   78.78%   -0.10%     
==========================================
  Files         305      305              
  Lines       38559    38733     +174     
==========================================
+ Hits        30418    30517      +99     
- Misses       8141     8216      +75     

@teor2345
Copy link
Contributor Author

In my local tests, PRs #4825 and #4826 make all FindBlockHeaders requests complete in under 5 seconds.
(Our current performance logging limit.)

Base automatically changed from slow-block-headers to main July 26, 2022 20:21
oxarbitrage
oxarbitrage previously approved these changes Jul 26, 2022
Copy link
Contributor

@oxarbitrage oxarbitrage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

And speed up the contains_block_hash state query.
@teor2345
Copy link
Contributor Author

I rebased this branch on the squashed commit.

All the tests passed except lightwalletd (PR #4828) and mempool_transaction_expiration (ticket #4819), so I'm going to admin-merge this PR.

@teor2345 teor2345 merged commit 9b2185a into main Jul 26, 2022
@teor2345 teor2345 deleted the concurrent-get-headers branch July 26, 2022 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-state Area: State / database changes C-bug Category: This is a bug I-integration-fail Continuous integration fails, including build and test failures I-slow Problems with performance or responsiveness
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants