Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use foonathan memory manager for reducing allocations in SharedMemManager.hpp #3833

Merged
merged 10 commits into from
Sep 28, 2023
Merged

Use foonathan memory manager for reducing allocations in SharedMemManager.hpp #3833

merged 10 commits into from
Sep 28, 2023

Conversation

ma30002000
Copy link
Contributor

@ma30002000 ma30002000 commented Sep 8, 2023

Description

When receiving subscribed messages via shared memory, the only periodic heap allocations originate in the free_buffers_ and allocated_buffers_ lists in the Segment class (ShmMemoryManager.hpp). This PR replaces the std::lists' default allocator with a foonathan provided one as indicated by the FIXME left in the code.

It would be nice to clarify if an initial size of 1 buffer is a decent starting value.

@Mergifyio backport 2.11.x 2.10.x 2.6.x

Contributor Checklist

  • Commit messages follow the project guidelines.
  • The code follows the style guidelines of this project.
  • Tests that thoroughly check the new feature have been added/Regression tests checking the bug and its fix have been added; the added tests pass locally
  • N/A Any new/modified methods have been properly documented using Doxygen.
  • Changes are ABI compatible.
  • Changes are API compatible.
  • N/A New feature has been added to the versions.md file (if applicable).
  • N/A New feature has been documented/Current behavior is correctly described in the documentation.
  • Applicable backports have been included in the description.

Reviewer Checklist

  • The PR has a milestone assigned.
  • Check contributor checklist is correct.
  • Check CI results: changes do not issue any warning.
  • Check CI results: failing tests are unrelated with the changes.

@ma30002000 ma30002000 marked this pull request as ready for review September 8, 2023 11:57
@JLBuenoLopez JLBuenoLopez added this to the v2.12.0 milestone Sep 13, 2023
@MiguelCompany MiguelCompany modified the milestones: v2.12.0, v2.12.1 Sep 19, 2023
@MiguelCompany
Copy link
Member

@ma30002000 Thank you for your contribution.

I would use the input max_allocations as the starting value, since the loop in line 408 will insert that many items in free_buffers_.

@MiguelCompany MiguelCompany modified the milestones: v2.12.1, v2.12.0 Sep 20, 2023
MiguelCompany
MiguelCompany previously approved these changes Sep 20, 2023
Copy link
Member

@MiguelCompany MiguelCompany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ma30002000 Thank you for such a well-crafted contribution.

I took the liberty of rebasing and adding some commits with some minor changes, so we could get this into the 2.12.0 release.

LGTM with green CI.

@MiguelCompany MiguelCompany added the ci-pending PR which CI is running label Sep 20, 2023
@MiguelCompany
Copy link
Member

@richiprosima Please test this

@MiguelCompany
Copy link
Member

@richiprosima Please test windows

EduPonz
EduPonz previously approved these changes Sep 27, 2023
@MiguelCompany
Copy link
Member

@richiprosima Please test this

@MiguelCompany
Copy link
Member

@richiprosima Please test this

@MiguelCompany
Copy link
Member

@richiprosima Please test linux and test windows

@MiguelCompany
Copy link
Member

@richiprosima Please test discovery-server

@EduPonz
Copy link

EduPonz commented Sep 28, 2023

@Mergifyio rebase

ma30002000 and others added 10 commits September 28, 2023 05:14
Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
…ger in order to reduce dynamic heap allocations.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
@mergify
Copy link
Contributor

mergify bot commented Sep 28, 2023

rebase

✅ Branch has been successfully rebased

@EduPonz
Copy link

EduPonz commented Sep 28, 2023

@richiprosima please test this

@MiguelCompany MiguelCompany merged commit df18056 into eProsima:master Sep 28, 2023
@MiguelCompany
Copy link
Member

@Mergifyio backport 2.11.x 2.10.x 2.6.x

@mergify
Copy link
Contributor

mergify bot commented Sep 28, 2023

backport 2.11.x 2.10.x 2.6.x

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Sep 28, 2023
…ager.hpp (#3833)

* Add helpers for std::list node sizes.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Use foonathan memory pool for storing buffer nodes in shared mem manager in order to reduce dynamic heap allocations.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Uncrustify.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Add foonathan dependency to SharedMemTests.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Initialize allocator with more realistic allocation assumption.

* Fix include order

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Removed unnecessary constexpr.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix copyright year.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix EOL.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix windows build of unit test.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

---------

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Co-authored-by: Miguel Company <MiguelCompany@eprosima.com>
(cherry picked from commit df18056)

# Conflicts:
#	test/unittest/transport/CMakeLists.txt
mergify bot pushed a commit that referenced this pull request Sep 28, 2023
…ager.hpp (#3833)

* Add helpers for std::list node sizes.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Use foonathan memory pool for storing buffer nodes in shared mem manager in order to reduce dynamic heap allocations.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Uncrustify.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Add foonathan dependency to SharedMemTests.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Initialize allocator with more realistic allocation assumption.

* Fix include order

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Removed unnecessary constexpr.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix copyright year.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix EOL.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix windows build of unit test.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

---------

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Co-authored-by: Miguel Company <MiguelCompany@eprosima.com>
(cherry picked from commit df18056)

# Conflicts:
#	test/unittest/transport/CMakeLists.txt
mergify bot pushed a commit that referenced this pull request Sep 28, 2023
…ager.hpp (#3833)

* Add helpers for std::list node sizes.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Use foonathan memory pool for storing buffer nodes in shared mem manager in order to reduce dynamic heap allocations.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Uncrustify.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Add foonathan dependency to SharedMemTests.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Initialize allocator with more realistic allocation assumption.

* Fix include order

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Removed unnecessary constexpr.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix copyright year.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix EOL.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix windows build of unit test.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

---------

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Co-authored-by: Miguel Company <MiguelCompany@eprosima.com>
(cherry picked from commit df18056)

# Conflicts:
#	test/unittest/transport/CMakeLists.txt
@ma30002000
Copy link
Contributor Author

Thanks a lot for all the additional work and bringing my changes to the 2.12 release!

juanlofer-eprosima pushed a commit that referenced this pull request Nov 10, 2023
…ager.hpp (#3833)

* Add helpers for std::list node sizes.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Use foonathan memory pool for storing buffer nodes in shared mem manager in order to reduce dynamic heap allocations.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Uncrustify.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Add foonathan dependency to SharedMemTests.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Initialize allocator with more realistic allocation assumption.

* Fix include order

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Removed unnecessary constexpr.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix copyright year.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix EOL.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix windows build of unit test.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

---------

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Co-authored-by: Miguel Company <MiguelCompany@eprosima.com>
MiguelCompany added a commit that referenced this pull request Nov 30, 2023
…ager.hpp (#3890)

* Use foonathan memory manager for reducing allocations in SharedMemManager.hpp (#3833)

* Add helpers for std::list node sizes.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Use foonathan memory pool for storing buffer nodes in shared mem manager in order to reduce dynamic heap allocations.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Uncrustify.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Add foonathan dependency to SharedMemTests.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Initialize allocator with more realistic allocation assumption.

* Fix include order

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Removed unnecessary constexpr.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix copyright year.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix EOL.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix windows build of unit test.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

---------

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Co-authored-by: Miguel Company <MiguelCompany@eprosima.com>
(cherry picked from commit df18056)

# Conflicts:
#	test/unittest/transport/CMakeLists.txt

* Fix conflicts

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

---------

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Co-authored-by: Matthias Schneider <ma30002000@users.noreply.github.com>
Co-authored-by: Miguel Company <miguelcompany@eprosima.com>
EduPonz pushed a commit that referenced this pull request Dec 1, 2023
…ager.hpp (#3833)

* Add helpers for std::list node sizes.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Use foonathan memory pool for storing buffer nodes in shared mem manager in order to reduce dynamic heap allocations.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Uncrustify.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Add foonathan dependency to SharedMemTests.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Initialize allocator with more realistic allocation assumption.

* Fix include order

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Removed unnecessary constexpr.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix copyright year.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix EOL.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix windows build of unit test.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

---------

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Co-authored-by: Miguel Company <MiguelCompany@eprosima.com>
(cherry picked from commit df18056)

# Conflicts:
#	test/unittest/transport/CMakeLists.txt
Signed-off-by: EduPonz <eduardoponz@eprosima.com>
EduPonz pushed a commit that referenced this pull request Dec 2, 2023
…ager.hpp (#3833) (#3889)

* Use foonathan memory manager for reducing allocations in SharedMemManager.hpp (#3833)

* Add helpers for std::list node sizes.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Use foonathan memory pool for storing buffer nodes in shared mem manager in order to reduce dynamic heap allocations.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Uncrustify.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Add foonathan dependency to SharedMemTests.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Initialize allocator with more realistic allocation assumption.

* Fix include order

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Removed unnecessary constexpr.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix copyright year.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix EOL.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix windows build of unit test.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

---------

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Co-authored-by: Miguel Company <MiguelCompany@eprosima.com>
(cherry picked from commit df18056)

# Conflicts:
#	test/unittest/transport/CMakeLists.txt
Signed-off-by: EduPonz <eduardoponz@eprosima.com>

* Fix conflicts

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Signed-off-by: EduPonz <eduardoponz@eprosima.com>

---------

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Signed-off-by: EduPonz <eduardoponz@eprosima.com>
Co-authored-by: Matthias Schneider <ma30002000@users.noreply.github.com>
Co-authored-by: Miguel Company <MiguelCompany@eprosima.com>
cferreiragonz pushed a commit that referenced this pull request Dec 5, 2023
…ager.hpp (#3833)

* Add helpers for std::list node sizes.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Use foonathan memory pool for storing buffer nodes in shared mem manager in order to reduce dynamic heap allocations.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Uncrustify.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Add foonathan dependency to SharedMemTests.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Initialize allocator with more realistic allocation assumption.

* Fix include order

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Removed unnecessary constexpr.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix copyright year.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix EOL.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix windows build of unit test.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

---------

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Co-authored-by: Miguel Company <MiguelCompany@eprosima.com>
(cherry picked from commit df18056)

# Conflicts:
#	test/unittest/transport/CMakeLists.txt
EduPonz pushed a commit that referenced this pull request Dec 11, 2023
…ager.hpp (#3833) (#3888)

* Use foonathan memory manager for reducing allocations in SharedMemManager.hpp (#3833)

* Add helpers for std::list node sizes.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Use foonathan memory pool for storing buffer nodes in shared mem manager in order to reduce dynamic heap allocations.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Uncrustify.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Add foonathan dependency to SharedMemTests.

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>

* Initialize allocator with more realistic allocation assumption.

* Fix include order

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Removed unnecessary constexpr.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix copyright year.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix EOL.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

* Fix windows build of unit test.

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

---------

Signed-off-by: Matthias Schneider <ma30002000@yahoo.de>
Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Co-authored-by: Miguel Company <MiguelCompany@eprosima.com>
(cherry picked from commit df18056)

# Conflicts:
#	test/unittest/transport/CMakeLists.txt

* Fix conflicts

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>

---------

Signed-off-by: Miguel Company <MiguelCompany@eprosima.com>
Co-authored-by: Matthias Schneider <ma30002000@users.noreply.github.com>
Co-authored-by: Miguel Company <miguelcompany@eprosima.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-pending PR which CI is running
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants