-
Notifications
You must be signed in to change notification settings - Fork 371
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix WrapWords excessive memory utilization #205
Conversation
@olekukonko any chance to get this PR looked at? |
@horkhe sorry for the late response, this is wonderful, will take my time to look at this next week so that it can be merged |
@horkhe can you kindly update your PR and resolve merge conflicts, I Would really like to merge this performance improvement. thank you |
@olekukonko sorry for the late response. I have rebased this PR, please take a look. |
@olekukonko any chance to get this merged? |
1 similar comment
@olekukonko any chance to get this merged? |
@horkhe will look at this this weekend and revert. Thank you for being consistent. |
@olekukonko a friendly reminder. |
Thanks a lot! Could you please also tag the master with v0.0.6, so it can be picked up by |
When feeding production data to this library we discovered that function
WordWrap
had O(n*n) memory complexity, that resulted in huge amounts of memory allocated when processing even relatively small strings. We are talking Gigabytes that made our production services killed with OOM. This PR implements 2 optimisations:splitWords
was introduced to efficiently break a line into words. Note that unlike original implementation it does not consider empty spaces to be words.WrapWords
was rewritten to be O(n) on memory complexity. A benchmark that comes with this PR shows great improvement: