[update] reduce chainbase memory usage #19

greg7mdp · 2023-08-17T22:06:52Z

[alternate from PR 14 by @taokayan, description copied]. This builds upon @taokayan's work.

Reduce offset_node_base memory usage inside the RBtree of chainbase

This PR will reduce the chainbase memory usage

old implementation:

_parent, _left, _right, _color consume 32 bytes together (8 bytes each).

new implementation

_parent, _left, _right, _color consume 16 bytes (42 + 42 + 42 + 2 bits) together.
use -2 (instead of 2) as erased_flag since _color is a 2 bit signed int.

limitations:

offset pointers _parent, _left & _right only have signed 42-bit each (so 41-bit absolute value), shifted by two bits thanks to alignment, implying a maximum of chainbase size of 2^43 = 8TB.
offset_node_base objects must always be allocated within the same segment due to the 42-bit offset limit. Using a stack variable of offset_node_base is no longer valid (as stack address starts from 0x7fffxxxxxxxx which is too far away from heap addresses).
possible minor performance impact due to misalignment and extra bit operation required. Or performance could possibly be better thanks to higher memory locality.

tested results:

using mainnet snapshot snapshot-308122667.bin.tar.gz:

old chainbase memory usage: 28799134736
memory usage in this PR: 23136280976 (around 20% reduction)

[greg 8/21/2023] retested with the current PR version and confirmed the memory savings as reported by curl -X POST http://127.0.0.1:8888/v1/db_size/get after loading snapshot.

When using 42 bit offsets as pointers in `undo_index`, the existing test using `std::allocator` was failing (as 64 bit pointers were truncated to 42 bits). This uses `chainbase_node_allocator` (which uses `bip::offset_ptr<T>` as pointers), which when truncated to 42 bits allow to address 2TB of memory.

…ce_memory

taokayan · 2023-08-18T00:00:00Z

since the offset values are shifted by 3 bits, is it possible to collide with the special value 1 which is re-purposed for representing nullptr?

greg7mdp · 2023-08-18T01:08:59Z

since the offset values are shifted by 3 bits, is it possible to collide with the special value 1 which is re-purposed for representing nullptr?

No, I don't think so. I explained why in the comment:

   // A concern could be that `1` is a special value meaning `nullptr`. However we could not
   // get an offset of 8, since `sizeof(offset_node_base) == 16`, so we could not have
   // difference between two `node_ptr` less than 16.

Do you think I missed a possible issue?

Now I realize I didn't completely think it through. The chainbase_node_allocator allocates nodes in batches of 64, so the offsets between two node_ptr in a single batch would be zero modulo 16.
However I did not look at how these batches are allocated by the segment manager (bip::managed_mapped_file::segment_manager). If they also are addresses aligned to a 16 byte boundary, we should be good I think. I'll verify that tomorrow and update the comment.

Striking the comment above, because we store offsets between node_ptr in _parent, _left, _right and these offsets (when >> 3) can never be 1.

taokayan · 2023-08-18T02:15:40Z

since the offset values are shifted by 3 bits, is it possible to collide with the special value 1 which is re-purposed for representing nullptr?

No, I don't think so. I explained why in the comment:
   // A concern could be that `1` is a special value meaning `nullptr`. However we could not
   // get an offset of 8, since `sizeof(offset_node_base) == 16`, so we could not have
   // difference between two `node_ptr` less than 16.
Do you think I missed a possible issue?

Now I realize I didn't completely think it through. The chainbase_node_allocator allocates nodes in batches of 64, so the offsets between two node_ptr in a single batch would be zero modulo 16. However I did not look at how these batches are allocated by the segment manager (bip::managed_mapped_file::segment_manager). If they also are addresses aligned to a 16 byte boundary, we should be good I think. I'll verify that tomorrow and update the comment.

Striking the comment above, because we store offsets between node_ptr in _parent, _left, _right and these offsets (when >> 3) can never be 1.

ah i see. that's a good catch. thanks!

greg7mdp · 2023-08-21T14:12:18Z

@taokayan how did you measure memory usage showing the 20% reduction?

taokayan · 2023-08-21T14:18:51Z

@taokayan how did you measure memory usage showing the 20% reduction?

you can use the db_size plugin API: http://127.0.0.1:8888/v1/db_size/get

… a 4 byte alignment.

heifner · 2023-08-22T13:45:35Z

include/chainbase/undo_index.hpp

         return static_cast<hook<index0_type, Allocator>&>(to_node(obj))._color;
      }
+      static void set_removed_field(const value_type& obj, int val) {
+         static_cast<hook<index0_type, Allocator>&>(to_node(obj))._color = val;
+      }


I like this change.

heifner · 2023-08-22T13:46:24Z

include/chainbase/undo_index.hpp

+   // A concern could be that `1` is a special value meaning `nullptr`. However we could not
+   // get an offset of 4, since `sizeof(offset_node_base) == 16`, so we could not have
+   // difference between two different `node_ptr` be less than 16.
+   // --------------------------------------------------------------------------------------


Did you run any performance comparisons with these changes?

No, I didn't. Do you know of a good way to make a performance comparison? Maybe it would require to write a small benchmarking test.

You could write one I suppose, but you could also just do a replay comparison of some range of blocks.

I think I'll write a small test.

taokayan and others added 8 commits May 5, 2023 16:25

reduce state memory usage

b2c5e43

Small fixes in undo_index.hpp.

1703fda

Increase maximum memory support from 2TB to 8TB.

427d9f8

Shift by 3 bits (16TB support), add comment and asserts.

3839b1f

Cleanup include statement in undo_index.cpp.

8c84f4f

Merge branch 'main' of github.com:AntelopeIO/chainbase into greg_redu…

3880bc4

…ce_memory

Use std::filesystem instead of boost::filesystem.

fa12def

Reset the shift to be by two bits, as the pointers are allofated with…

e273264

… a 4 byte alignment.

greg7mdp requested review from heifner and taokayan August 21, 2023 20:27

taokayan approved these changes Aug 21, 2023

View reviewed changes

heifner approved these changes Aug 22, 2023

View reviewed changes

greg7mdp merged commit b81a6dc into main Aug 29, 2023

greg7mdp deleted the greg_reduce_memory branch August 29, 2023 18:02

greg7mdp mentioned this pull request Aug 29, 2023

Update chainbase to tip (reduce memory usage) AntelopeIO/leap#1575

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[update] reduce chainbase memory usage #19

[update] reduce chainbase memory usage #19

greg7mdp commented Aug 17, 2023 •

edited

Loading

taokayan commented Aug 18, 2023 •

edited

Loading

greg7mdp commented Aug 18, 2023 •

edited

Loading

taokayan commented Aug 18, 2023

greg7mdp commented Aug 21, 2023

taokayan commented Aug 21, 2023

heifner Aug 22, 2023

heifner Aug 22, 2023

greg7mdp Aug 22, 2023

heifner Aug 22, 2023

greg7mdp Aug 22, 2023

[update] reduce chainbase memory usage #19

[update] reduce chainbase memory usage #19

Conversation

greg7mdp commented Aug 17, 2023 • edited Loading

Reduce offset_node_base memory usage inside the RBtree of chainbase

old implementation:

new implementation

limitations:

tested results:

taokayan commented Aug 18, 2023 • edited Loading

greg7mdp commented Aug 18, 2023 • edited Loading

taokayan commented Aug 18, 2023

greg7mdp commented Aug 21, 2023

taokayan commented Aug 21, 2023

heifner Aug 22, 2023

Choose a reason for hiding this comment

heifner Aug 22, 2023

Choose a reason for hiding this comment

greg7mdp Aug 22, 2023

Choose a reason for hiding this comment

heifner Aug 22, 2023

Choose a reason for hiding this comment

greg7mdp Aug 22, 2023

Choose a reason for hiding this comment

greg7mdp commented Aug 17, 2023 •

edited

Loading

taokayan commented Aug 18, 2023 •

edited

Loading

greg7mdp commented Aug 18, 2023 •

edited

Loading