Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark host frames as not needing to be writeable #5824

Merged

Conversation

jakirkham
Copy link
Member

@jakirkham jakirkham commented Aug 1, 2020

Partially addresses issue ( dask/distributed#3994 )

Leverages a feature in Dask to bypass copying of frames on host. ( dask/distributed#4004 ) This can offer a notable speed up when working with "dask" serialization.

cc @quasiben

@jakirkham jakirkham requested a review from a team as a code owner August 1, 2020 00:53
@jakirkham jakirkham force-pushed the mark_host_frames_readonly branch from 8902743 to b26a07c Compare August 1, 2020 00:56
Leverages a feature in Dask to bypass copying of frames on host. This
can offer a notable speed up when working with `"dask"` serialization.
@jakirkham jakirkham force-pushed the mark_host_frames_readonly branch from b26a07c to 03000e9 Compare August 1, 2020 01:08
@jakirkham jakirkham marked this pull request as draft August 1, 2020 01:50
@jakirkham jakirkham changed the title Mark host frames as not needing to be writeable [WIP] Mark host frames as not needing to be writeable Aug 1, 2020
@jakirkham jakirkham added 2 - In Progress Currently a work in progress Python Affects Python cuDF API. dask Dask issue labels Aug 1, 2020
@codecov
Copy link

codecov bot commented Aug 1, 2020

Codecov Report

Merging #5824 into branch-0.15 will increase coverage by 0.32%.
The diff coverage is 100.00%.

Impacted file tree graph

@@               Coverage Diff               @@
##           branch-0.15    #5824      +/-   ##
===============================================
+ Coverage        84.06%   84.38%   +0.32%     
===============================================
  Files               80       80              
  Lines            13066    13387     +321     
===============================================
+ Hits             10984    11297     +313     
- Misses            2082     2090       +8     
Impacted Files Coverage Δ
python/cudf/cudf/core/abc.py 91.48% <100.00%> (+3.39%) ⬆️
python/cudf/cudf/io/feather.py 100.00% <0.00%> (ø)
python/cudf/cudf/comm/serialize.py 0.00% <0.00%> (ø)
python/custreamz/custreamz/_version.py 0.00% <0.00%> (ø)
python/dask_cudf/dask_cudf/_version.py 0.00% <0.00%> (ø)
python/dask_cudf/dask_cudf/io/tests/test_orc.py 100.00% <0.00%> (ø)
python/dask_cudf/dask_cudf/io/tests/test_json.py 100.00% <0.00%> (ø)
...ython/dask_cudf/dask_cudf/io/tests/test_parquet.py 100.00% <0.00%> (ø)
python/cudf/cudf/utils/applyutils.py 98.75% <0.00%> (+0.02%) ⬆️
python/cudf/cudf/core/join/join.py 92.41% <0.00%> (+0.03%) ⬆️
... and 30 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 91bc50c...03000e9. Read the comment docs.

@jakirkham jakirkham changed the title [WIP] Mark host frames as not needing to be writeable Mark host frames as not needing to be writeable Aug 4, 2020
@jakirkham jakirkham marked this pull request as ready for review August 4, 2020 00:13
@jakirkham
Copy link
Member Author

PR ( dask/distributed#4004 ) is now in. This should be good to go 😄

@kkraus14 kkraus14 added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 2 - In Progress Currently a work in progress labels Aug 4, 2020
@jakirkham jakirkham merged commit a4802bc into rapidsai:branch-0.15 Aug 4, 2020
@jakirkham jakirkham deleted the mark_host_frames_readonly branch August 4, 2020 00:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge dask Dask issue Python Affects Python cuDF API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants