Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] MG (Batch) Betweenness Centrality #940

Merged
merged 69 commits into from
Aug 13, 2020

Conversation

xcadet
Copy link
Contributor

@xcadet xcadet commented Jun 9, 2020

This work adds Multi-GPU Batch Betweenness Centrality for both Vertices and Edges:

  • Data Replication is operated once for the graph using dask and RAFT.
  • Task Distribution is operated at the Cython level

This version allows to use the same Graph for both Single GPU algorithms and algorithms supporting Multi GPUs Batch
processing.
Graphs can be loaded the same way they are with cudf for single GPU, and can leverage Multi GPUs Batch Processing by using G.enable_mg_batch() where G is a Graph or DiGraph. By enabling mg_batch edge list and / or adjacency lists are replicated across device using dask and RAFT's communicator. When enabled and when adding representations to the graph, replications of the representations are operated. Replication of the transposed adjacency list is not yet implemented.

This also adds functions for replciation of cudf.DataFrame and cudf.Series over multiple GPUs, and a notebook to show how to use Multi GPUs Batch Processing for Betweenness Centrality

A boolean parameter is added at the C++ level to BFS to disable MG BFS path to be taken when called from MG Batch BC.

@GPUtester
Copy link
Contributor

Please update the changelog in order to start CI tests.

View the gpuCI docs here.

@xcadet xcadet changed the title [WIP][skip-ci] OPG - Batch [WIP] OPG - Batch Betweenness Centrality Jul 1, 2020
@xcadet xcadet changed the title [WIP] OPG - Batch Betweenness Centrality [REVIEW] OPG - Batch Betweenness Centrality Jul 10, 2020
Copy link
Member

@afender afender left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Added ome minor questions/comments.

Please add a note on Batched processing in the .rst file under doc like after the Consolidation section.

@xcadet
Copy link
Contributor Author

xcadet commented Aug 4, 2020

rerun tests

Copy link
Member

@afender afender left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Need to investigate graph replication time compared to peak BW expected perf

Copy link
Contributor

@seunghwak seunghwak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just add a FIXME comment, and thanks for the contribution!

@BradReesWork
Copy link
Member

rerun tests

1 similar comment
@xcadet
Copy link
Contributor Author

xcadet commented Aug 12, 2020

rerun tests

@BradReesWork
Copy link
Member

rerun tests

@BradReesWork BradReesWork merged commit 6cbb603 into rapidsai:branch-0.15 Aug 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants