Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: limit reification sizes #8149

Merged
merged 3 commits into from
Feb 20, 2022
Merged

fix: limit reification sizes #8149

merged 3 commits into from
Feb 20, 2022

Conversation

vyzo
Copy link
Contributor

@vyzo vyzo commented Feb 18, 2022

It was observed that during sync from snapshot some very large reification walks would trigger, consuming excessive memory.

This introduces a limit to reification mass, and aborts if the visited object count is exceeded.

@vyzo vyzo requested a review from magik6k February 18, 2022 10:34
@vyzo vyzo requested a review from a team as a code owner February 18, 2022 10:34
@vyzo
Copy link
Contributor Author

vyzo commented Feb 18, 2022

Note that this also makes the in-memory visitors smarter so as to not waste memory on unitary objects that are not walked anyway.

@codecov
Copy link

codecov bot commented Feb 18, 2022

Codecov Report

Merging #8149 (a7b1d86) into master (cf7a041) will decrease coverage by 0.02%.
The diff coverage is 63.63%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #8149      +/-   ##
==========================================
- Coverage   39.51%   39.48%   -0.03%     
==========================================
  Files         666      666              
  Lines       72498    72507       +9     
==========================================
- Hits        28646    28632      -14     
- Misses      38902    38913      +11     
- Partials     4950     4962      +12     
Impacted Files Coverage Δ
blockstore/splitstore/visitor.go 75.00% <0.00%> (-25.00%) ⬇️
blockstore/splitstore/splitstore_reify.go 55.37% <100.00%> (+4.50%) ⬆️
cli/util.go 41.66% <0.00%> (-8.34%) ⬇️
storage/wdpost_sched.go 77.45% <0.00%> (-7.85%) ⬇️
chain/exchange/peer_tracker.go 66.66% <0.00%> (-4.31%) ⬇️
chain/stmgr/call.go 71.51% <0.00%> (-3.64%) ⬇️
extern/storage-sealing/fsm.go 56.44% <0.00%> (-2.44%) ⬇️
extern/sector-storage/sched.go 83.12% <0.00%> (-1.65%) ⬇️
chain/stmgr/searchwait.go 67.30% <0.00%> (-1.29%) ⬇️
storage/wdpost_changehandler.go 97.64% <0.00%> (-0.95%) ⬇️
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf7a041...a7b1d86. Read the comment docs.

@jennijuju
Copy link
Member

please update the pr title according to the instructions in PR template.

@vyzo vyzo changed the title limit reification sizes fix: limit reification sizes Feb 18, 2022
@magik6k magik6k merged commit 051ff5d into master Feb 20, 2022
@magik6k magik6k deleted the fix/reify-limit branch February 20, 2022 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants