-
-
Notifications
You must be signed in to change notification settings - Fork 727
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Copy for mutable frames can introduce a slowdown #3994
Comments
That is a pretty sizable performance hit. Do you have a proposed solution? I suppose the user could pick whether frames are mutable or not. Is there a smart way for us to determine that for them so that we can provide the best default experience? |
Yeah I have a WIP branch where I tried to push this more into specific object or protocol serialization. It has some test failures though, which suggest it's not quite sufficient. The idea of picking (or perhaps even identifying) which frames are mutable is an interesting one. Maybe that's a better way to go. Thanks for the suggestion 🙂 |
That wound up being much easier to do as well. Submitted as PR ( #4004 ) 😄 |
This is in |
Recently we made a fix to ensure all frames are mutable in PR ( #3967 ). This solves some issues where immutable frames were used to back Python objects.
One downside of that change is it effectively requires a copy of the frames be taken for all objects. This can be unnecessary in cases where the data may already get copied as is the case with Python builtin objects that like to own their memory or GPU objects that require data be moved from host to device (effectively a copy). As a result this copy can introduce a slowdown.
Included below is an example with and without that change using cuDF:
Without mutable frames:
With mutable frames:
The text was updated successfully, but these errors were encountered: