Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Chunk Data Pack Pruner] Add Block Iterator #6858

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

zhangchiqing
Copy link
Member

This PR adds a height based block iterator that iterates blocks by height, without iterating siblings of finalized blocks, which will be done later by implementing view based block iterator.

}

// BlockIterator is an interface for iterating over blocks
type BlockIterator interface {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The BlockIterator interface can be implemented into height based iterator and view based iterator.

The block iterator not long can be used by chunk data pack pruner, but alsoin future to implement protocol state pruner.

The height based iterator is easy to implement, however, it can't guarantee to prune all data, since it doesn't iterate unfinalized blocks. The view based iterator can guarantee all blocks are pruned, but it's more complicated to implement.

In this PR, I first implement the height based iterator, for chunk data pack, it's OK that we only prune by height, however, for protocol state, it's better that we can prune by view and ensure a more throughout pruning.

jobCreator IteratorJobCreator
}

func NewIteratorFactory(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the interfaces in the arguments are implemented, then the logic to create the BlockIterator can be reused. That's why, I put this function here along with the interface definitions, so that it's clear to see how the interfaces will be used for creating the block iterator.

@zhangchiqing zhangchiqing changed the base branch from leo/db-ops-dbstore to master January 9, 2025 17:25
@zhangchiqing zhangchiqing changed the base branch from master to leo/db-ops-dbstore January 10, 2025 17:18
@zhangchiqing zhangchiqing marked this pull request as draft January 10, 2025 18:39
Base automatically changed from leo/db-ops-dbstore to master January 13, 2025 19:55
@zhangchiqing zhangchiqing force-pushed the leo/cdp-prune-block-iterator branch from b688ac2 to 0af09d6 Compare January 15, 2025 16:21
@zhangchiqing zhangchiqing marked this pull request as ready for review January 15, 2025 16:23
@zhangchiqing zhangchiqing requested a review from a team as a code owner January 15, 2025 16:23
@codecov-commenter
Copy link

codecov-commenter commented Jan 15, 2025

Codecov Report

Attention: Patch coverage is 43.13725% with 29 lines in your changes missing coverage. Please review.

Project coverage is 41.12%. Comparing base (b740fc0) to head (0af09d6).

Files with missing lines Patch % Lines
module/block_iterator.go 0.00% 21 Missing ⚠️
module/block_iterator/height_based/iterator.go 73.33% 6 Missing and 2 partials ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##           master    #6858   +/-   ##
=======================================
  Coverage   41.11%   41.12%           
=======================================
  Files        2116     2118    +2     
  Lines      185749   185800   +51     
=======================================
+ Hits        76378    76411   +33     
- Misses     102954   102975   +21     
+ Partials     6417     6414    -3     
Flag Coverage Δ
unittests 41.12% <43.13%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants