Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IF: pack size of bitset #57

Merged
merged 6 commits into from
Apr 23, 2024
Merged

IF: pack size of bitset #57

merged 6 commits into from
Apr 23, 2024

Conversation

heifner
Copy link
Member

@heifner heifner commented Apr 22, 2024

Change the pack/unpack scheme for boost::dynamic_bitset used by quorum_certificate.

Before this PR the number of blocks of the underlying storage was packed. On unpack the size of the dynamic bitset was the size of the number of bits in the underlying blocks.

With this PR, the size of the bitset is pack/unpack so that the bitset can be restored to its original size.

Included in this PR is the introduction of fc::dynamic_bitset defined to be boost::dynamic_bitset<uint8_t>. Using uint8_t minimizes space used for packed format and provides a more expected packed format where only the bytes that are needed are packed/unpacked.

fc::dynamic_bitset is used for votes in quorum_certificate which is included in most blocks.

@heifner heifner requested review from linh2931 and greg7mdp April 22, 2024 12:10
@heifner heifner added OCI Work exclusive to OCI team consensus-protocol Change to the consensus protocol. Impacts light client validation. labels Apr 22, 2024
@heifner heifner requested a review from arhag April 22, 2024 12:13
Comment on lines 593 to 598
blocks.reserve(num_blocks);
for( size_t i = 0; i < size.value; ++i ) {
T tmp;
fc::raw::unpack( s, tmp );
blocks.emplace_back( std::move(tmp) );
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change? It is arguably less efficient since it does an extra copy.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not exactly sure what you mean by this. The vector unpack expects the size of the vector which is not there in this case. Are you talking about something else?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I mean using this code instead:

      std::vector<T> blocks(num_blocks);
      for( size_t i = 0; i < num_blocks; ++i ) {
        fc::raw::unpack( s, blocks[i] );

Comment on lines 594 to 598
auto add_extra = [&]() {
bool extra_needed = (size % (sizeof(T) * CHAR_BIT)) != 0;
return static_cast<size_t>(extra_needed); // bool => 0,1
};
size_t num_blocks = size / (sizeof(T) * CHAR_BIT) + add_extra();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or could be the more usual:

Suggested change
auto add_extra = [&]() {
bool extra_needed = (size % (sizeof(T) * CHAR_BIT)) != 0;
return static_cast<size_t>(extra_needed); // bool => 0,1
};
size_t num_blocks = size / (sizeof(T) * CHAR_BIT) + add_extra();
constexpr size_t word_size = sizeof(T) * CHAR_BIT;
size_t num_blocks = (size + word_size - 1) / word_size;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Much nicer.

@heifner heifner requested a review from linh2931 April 23, 2024 14:34
@heifner heifner merged commit d7fd8f3 into savanna Apr 23, 2024
36 checks passed
@heifner heifner deleted the bitset-pack-size branch April 23, 2024 16:57
@ericpassmore
Copy link
Contributor

Note:start
group: IF
category: INTERNALS
summary: Improve pack/unpack function which is used for votes in quorum_certificate.
Note:end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
consensus-protocol Change to the consensus protocol. Impacts light client validation. OCI Work exclusive to OCI team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants