Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shm: Introduce first support for pools #484

Merged
merged 7 commits into from
Sep 29, 2023
Merged

Conversation

rbx
Copy link
Member

@rbx rbx commented Sep 27, 2023

This uses boost::interprocess::adaptive_pool to store reference counts for unmanaged region buffers when these are "softly" copied by FairMQ.

One additional small managed segment per unmanaged region is created to store the reference count pool. Default segment size is 10,000,000 bytes, which can be changed through regionConfig in code. RefCount object size is 2 bytes, so this should be good for up to 5 million counts.

This replaces the old functionality, which would store the ref counts as separate message objects in the main managed segment. Thus, this should reduce fragmentation of the main segment.

I also add an example that uses this, but tests are also already in place, which tested the old implementation.

@rbx rbx requested a review from dennisklein September 27, 2023 12:27
Copy link
Member

@dennisklein dennisklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you can squash the third and last commits?

fairmq/shmem/Common.h Outdated Show resolved Hide resolved
fairmq/shmem/Manager.h Outdated Show resolved Hide resolved
fairmq/shmem/Manager.h Outdated Show resolved Hide resolved
fairmq/shmem/Common.h Show resolved Hide resolved
@rbx rbx force-pushed the shm-pools branch 4 times, most recently from 28beea5 to af9890b Compare September 28, 2023 10:38
fairmq/shmem/Message.h Outdated Show resolved Hide resolved
fairmq/shmem/Message.h Outdated Show resolved Hide resolved
fairmq/shmem/Common.h Outdated Show resolved Hide resolved
fairmq/shmem/Message.h Outdated Show resolved Hide resolved
@rbx rbx merged commit 8d28824 into FairRootGroup:dev Sep 29, 2023
@rbx rbx deleted the shm-pools branch February 16, 2024 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants