Merge fast-forwarded files from cugraph into nx-cugraph #13
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We decided to rename `cugraph-nx` to `nx-cugraph` to follow (and help establish) conventions for names of networkx backends. See: networkx/networkx#6883 This PR was created from the following commands: ```sh mv notebooks/ ../notebooks-bak find * -type f -print0 | xargs -0 sed -i 's/cugraph_nx/nx_cugraph/g' find * -type f -print0 | xargs -0 sed -i 's/cugraph-nx/nx-cugraph/g' git mv ./conda/recipes/cugraph-nx ./conda/recipes/nx-cugraph git mv ./python/cugraph-nx ./python/nx-cugraph git mv ./python/nx-cugraph/cugraph_nx ./python/nx-cugraph/nx_cugraph mv ../notebooks-bak/ notebooks ``` (a more reliable bash script would ensure the destination of `git mv` does not exist yet, b/c if the destination is a directory, it will happily--and incorrectly--move the target _into_ the directory) ```sh # Make sure everything got renamed correctly git grep -i 'cugraph.nx' find . -iname '*cugraph*nx*' -print ``` Should we remove `cugraph-nx` nightlies once this is merged? Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) - Ray Douglass (https://github.com/raydouglass) URL: rapidsai/cugraph#3840
…t. (#3848)
This allows NetworkX docstrings to be updated (among other things).
This will have a companion PR in NetworkX. We still need to determine (and agree) on the dict returned by this entry point, and NetworkX doesn't need to use everything I have here. We should probably add a string for `"description"` that gives a very short description of the backend, and maybe `"url"` or `"homepage"` or whatever so online docs can have links.
Here's how to use the entry point (Python >= 3.10) after installing it:
```python
In [1]: from importlib.metadata import entry_points
In [2]: items = entry_points(group="networkx.plugin_info")
In [3]: [plugin] = items
In [4]: plugin.load()()
Out[4]:
{'backend_name': 'cugraph',
'project': 'nx-cugraph',
'package': 'nx_cugraph',
'functions': {'betweenness_centrality',
'edge_betweenness_centrality',
'louvain_communities'},
'extra_docstrings': {'betweenness_centrality': '`weight` parameter is not yet supported.',
'edge_betweenness_centrality': '`weight` parameter is not yet supported.',
'louvain_communities': '`threshold` and `seed` parameters are currently ignored.'},
'extra_parameters': {'louvain_communities': {'max_level': 'Upper limit of the number of macro-iterations.'}}}
```
CC @rlratzel @betochimas
Authors:
- Erik Welch (https://github.com/eriknw)
Approvers:
- Rick Ratzel (https://github.com/rlratzel)
- Ray Douglass (https://github.com/raydouglass)
URL: rapidsai/cugraph#3848
Merge branch-23.10 into branch-23.12
This handles isolated nodes in `louvain_communities` similar to what is done in #3886. This is expected to be a temporary fix until pylibcugraph can handle isolated nodes. As a bonus, I added `isolates` algorithm 🎉 CC @naimnv @rlratzel Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#3897
closes rapidsai/graph_dl#299 This PR adds new benchmarks for `nx-cugraph` which can be used to compare performance for NetworkX with and without the `cugraph` backend. These benchmarks depend on `pytest`, `pytest-benchmark`, `networkx>=3.0`, `cugraph` (only for the Dataset APIs), and `nx-cugraph` Some results can be seen here:   ### Other changes: * `black` is now run on all python sources under `benchmarks`, which resulted in several format-only changes. * Several other updates to versions, etc. were applied to various .yaml files in order to resolve CI errors. * Merged in changes from [jnke2016/branch-23.10_increase-timeout](rapidsai/cugraph@cbdbd8a) to get changes needed to fix dask CI failure. Authors: - Rick Ratzel (https://github.com/rlratzel) - Joseph Nke (https://github.com/jnke2016) - AJ Schmidt (https://github.com/ajschmidt8) - Naim (https://github.com/naimnv) - Seunghwa Kang (https://github.com/seunghwak) - https://github.com/lmeyerov - Erik Welch (https://github.com/eriknw) - Alex Barghi (https://github.com/alexbarghi-nv) - Vyas Ramasubramani (https://github.com/vyasr) - Divye Gala (https://github.com/divyegala) - Tingyu Wang (https://github.com/tingyu66) - Chuck Hastings (https://github.com/ChuckHastings) - ralph (https://github.com/nv-rliu) - Paul Taylor (https://github.com/trxcllnt) Approvers: - Erik Welch (https://github.com/eriknw) - Ray Douglass (https://github.com/raydouglass) URL: rapidsai/cugraph#3854
Forward-merge branch-23.10 to branch-23.12
A couple PRs were merged after `branch-23.12` was created and contained RAPIDS versions that need to be updated in `branch-23.12`. Ref: - rapidsai/cugraph#3838 - rapidsai/cugraph#3852 Authors: - Ray Douglass (https://github.com/raydouglass) - Rick Ratzel (https://github.com/rlratzel) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Brad Rees (https://github.com/BradReesWork) URL: rapidsai/cugraph#3905
New algorithms: - `degree_centrality` - `in_degree_centrality` - `k_truss` - `number_of_selfloops` - `out_degree_centrality` Also, rename `row_indices, col_indices` to `src_indices, dst_indices` Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#3945
Also, better handle dtypes for edge values passed to pylibcugraph, which only takes float32 and float64 atm. I also defined `index_dtype` (currently int32) to globally control the dtype of indices. Authors: - Erik Welch (https://github.com/eriknw) - Ralph Liu (https://github.com/nv-rliu) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#3954
Fixes #3963 and add `connected_components`, `is_connected`, `node_connected_component`, and `number_connected_components`. Also updated `_groupby` to handle groups that are not consecutive integers starting with 0. Also, `plc.weakly_connected_components` does not handle isolated nodes well, and I needed to handle this at the Python layer as was done in #3897 Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#3965
…-cugraph to main README (#3984) * Updates the nx-cugraph README file to include nx-cugraph user documentation * Adds nx-cugraph and new News section to main README * Updates software strack diagram to remove DGL and PyG since they are not currently using PropertyGraph or cugraph-service Authors: - Rick Ratzel (https://github.com/rlratzel) - Ralph Liu (https://github.com/nv-rliu) Approvers: - Brad Rees (https://github.com/BradReesWork)
* Fixes typos (repeated word, extra periods) * Updates link to the current docs on NetworkX backends Authors: - Rick Ratzel (https://github.com/rlratzel) Approvers: - Ray Douglass (https://github.com/raydouglass)
…ffix pacakges and nx-cugraph meta-data files. (#3994) Adds `update-version.sh` support for: * nx-cugraph meta-data files * `cugraph-pyg`, `cugraph-dgl` env yaml files * `*-cu11`, `*-cu12` wheel dependencies in dependencies.yaml Authors: - Rick Ratzel (https://github.com/rlratzel) Approvers: - Alex Barghi (https://github.com/alexbarghi-nv) - Brad Rees (https://github.com/BradReesWork) - Ray Douglass (https://github.com/raydouglass)
There are many more traversal algorithms to implement, but these get us started! Authors: - Erik Welch (https://github.com/eriknw) - Brad Rees (https://github.com/BradReesWork) - Rick Ratzel (https://github.com/rlratzel) Approvers: - Brad Rees (https://github.com/BradReesWork) - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#3976
Forward-merge branch-23.12 to branch-24.02
…`pagerank` (#3968) Add `eigenvector_centrality`, `katz_centrality`, and `hits`. I may add pagerank next. Authors: - Erik Welch (https://github.com/eriknw) - Ralph Liu (https://github.com/nv-rliu) - Naim (https://github.com/naimnv) - Rick Ratzel (https://github.com/rlratzel) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#3968
Resolves conflicts from forward-merging branch-23.12 into branch-24.02
Authors: - Erik Welch (https://github.com/eriknw) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4029
…(#4092) Hooray for removing and cleaning code! Tests also added (we already tested isolated nodes for Louvain). nx-cugraph was updated to handle isolated nodes by passing the node set to PLC in #4077 Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4092
This doesn't currently work, because `plc.weakly_connected_components` only works on symmetric graphs (so it's not actually performing wcc now is it?): > RuntimeError: non-success value returned from cugraph_weakly_connected_components: CUGRAPH_UNKNOWN_ERROR cuGraph failure at file=[...]/cugraph/cpp/src/components/weakly_connected_components_impl.cuh line=283: Invalid input argument: input graph should be symmetric for weakly connected components. _These are high-priority algorithms for `nx-cugraph`, because they are widely used by networkx dependents._ Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4071
…(#4063) Getting `G.to_undirected` to work was more involved than I expected, but at least we got two algorithms "for free" out of the effort! We raise `NotImplementedError` for `multidigraph.to_undirected()` for now. I would say that understanding the reciprocity algorithms is the first step to understanding `to_undirected`. Authors: - Erik Welch (https://github.com/eriknw) - Rick Ratzel (https://github.com/rlratzel) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4063
NetworkX tests are somewhat underspecified regarding how to handle self-loops for these algorithms. Also, I'm not sure if transitivity is supposed to work on directed graphs. Once #4071 is merged, it should be easy to add `is_bipartite` function (and maybe others?). Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4093
Forward-merge branch-24.02 to branch-24.04
…graph (#4240) A [recent PR](rapidsai/cugraph#4217) was merged that added `packaging` as a runtime dependency to nx-cugraph. This PR removes that dependency and manually extracts the `major` and `minor` version numbers instead. Authors: - Rick Ratzel (https://github.com/rlratzel) - Erik Welch (https://github.com/eriknw) Approvers: - Brad Rees (https://github.com/BradReesWork) - Erik Welch (https://github.com/eriknw) URL: rapidsai/cugraph#4240
Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4242
This PR updates the nx-cugraph README.md with the latest upstream URLs by running the `update_readme.py` script on a current `objects.inv` file downloaded from NetworkX. The latest `objects.inv` file has updated URLs, which results in links in the README being updated too. This should resolve style check errors recently seen in [other PRs](rapidsai/cugraph#4320) due to the upstream change in `objects.inv` in NetworkX. Since CI runs pre-commit on all files, this caused the nx-cugraph checks to run on PRs that didn't change nx-cugraph files.
This case is tested by the networkx tests. Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4383
Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4395
Forward Merge branch-24.06 into branch-24.08
Tests were added to improve coverage. Perhaps we could/should upstream tests to networkx. Authors: - Erik Welch (https://github.com/eriknw) - Ralph Liu (https://github.com/nv-rliu) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4537
This does add new functionality (so it's not just sugar!), since dijstra methods have `cutoff=` arguments. Note that we are missing multi-source dijstra methods, since `plc.sssp` only accepts a single source. We also do not have `bidirectional_dijkstra`, since PLC does not implement this algorithm. Authors: - Erik Welch (https://github.com/eriknw) - Rick Ratzel (https://github.com/rlratzel) - Ralph Liu (https://github.com/nv-rliu) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4538
… (#4531) This was pretty tricky in places! This begins with `Graph` and `DiGraph`, which are probably tricker, because of the way edge data get merged when nodes are combined (and I wouldn't be surprised if there are other ways to do this operation). This is one of the most heavily used functions in networkx and also networkx dependents. Up next: multigraphs Authors: - Erik Welch (https://github.com/eriknw) - Ralph Liu (https://github.com/nv-rliu) Approvers: - Rick Ratzel (https://github.com/rlratzel) - Bradley Dice (https://github.com/bdice) URL: rapidsai/cugraph#4531
This is to address the feedback from @bdice in rapidsai/cugraph#4531 (comment), which I was unable to do in that PR. I know checking version strings is playing it fast-and-loose, but I believe we are abundantly safe to do so here, as networkx releases have been, and are expected to be, slow and predictable. Also, we do not have `packaging` as a runtime dependency (it _is_ a test dependency for finer control). So, even better than using `packaging.version.parse`, this PR performs a sanity check on the networkx version. I put it in both `nx_cugraph.__init__` and `_nx_cugraph.__init__` to give us obvious breadcrumbs to discover and follow. Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4571
Part of rapidsai/graph_dl#579 This PR allows `nx-cugraph` to operate with release candidate (under development) versions of `networkx` What This Pattern Allows: - Just a single digit: `3.4`, `3.9`, etc - Release Candidate format: `3.4rc0`. `3.7rc2` This is needed to complete adding nightly test coverage for development branches of `nx` Authors: - Ralph Liu (https://github.com/nv-rliu) Approvers: - Erik Welch (https://github.com/eriknw) - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4613
Contributes to rapidsai/build-planning#88 Finishes the work of dropping Python 3.9 support. This project stopped building / testing against Python 3.9 as of rapidsai/shared-workflows#235. This PR updates configuration and docs to reflect that. ## Notes for Reviewers ### How I tested this Checked that there were no remaining uses like this: ```shell git grep -E '3\.9' git grep '39' git grep 'py39' ``` And similar for variations on Python 3.8 (to catch things that were missed the last time this was done). Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Kyle Edwards (https://github.com/KyleFromNVIDIA) - Bradley Dice (https://github.com/bdice) - Brad Rees (https://github.com/BradReesWork) URL: rapidsai/cugraph#4625
Contributes to rapidsai/build-planning#40 This PR adds support for Python 3.12. ## Notes for Reviewers This is part of ongoing work to add Python 3.12 support across RAPIDS. It temporarily introduces a build/test matrix including Python 3.12, from rapidsai/shared-workflows#213. A follow-up PR will revert back to pointing at the `branch-24.10` branch of `shared-workflows` once all RAPIDS repos have added Python 3.12 support. ### This will fail until all dependencies have been updates to Python 3.12 CI here is expected to fail until all of this project's upstream dependencies support Python 3.12. This can be merged whenever all CI jobs are passing. Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Bradley Dice (https://github.com/bdice) - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4647
…workX Graph classes, needed for zero code change graph generators (#4629) This is an alternative approach to #4558 for enabling GPU-accelerated NetworkX to "just work". It has similarities to #4558. I opted to make separate classes such as `ZeroGraph`, which I think makes for cleaner separation and gives us and users more control. There are a few lingering TODOs and code comments to tidy up, but I don't think there are any show-stoppers. I have not updated methods (such as `number_of_nodes`) to optimistically try to use GPU if possible, b/c this is not strictly necessary, but we should update these soon. I have run NetworkX tests with these classes using networkx/networkx#7585 and networkx/networkx#7600. We need the behavior in 7585, and 7600 is only useful for testing. There are only 3 new failing tests, and 3 tests that hang (I'll run them overnight to see if they finish). Here's a test summary: ``` 5548 passed, 24 skipped, 16 xfailed, 25 xpassed ``` Note that 25 tests that were failing now pass. I have not investigated test failures, xfails, or xpasses yet. I would like to add tests too. We rely heavily on the networkx cache. I think this is preferred. It is late for me. I will describe and show how and why this works later. I opted for `zero=` and `ZeroGraph`, because I find them delightful! Renaming is trivial if other terms are preferred. CC @quasiben Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Bradley Dice (https://github.com/bdice) - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#4629
Closes rapidsai/graph_dl#606 and [another issue] ## Proposed Changes In preparation for GA release, this PR adds a landing page for `nx-cugraph` in the cugraph API documentation site. The new pages can be viewed by clicking `nx-cugraph` in the navigation bar at the top of the page. ### New pages nx-cugraph └─ How it works └─ Supported Algorithms └─ Getting Started └─ Benchmarks └─ FAQ ## Notes for Reviewers - In order to build and test these docs, I modified the `build.sh` file to use `sphinx-autobuild`. ```bash 122 123 cd ${REPODIR}/docs/cugraph-docs 124 #make html 125 sphinx-autobuild source build/html 126 fi 127 ``` - For now, I believe the best way to view these changes is to clone the PR branch, then run `build.sh` in order to host the webserver locally.. --------- Co-authored-by: Don Acosta <97529984+acostadon@users.noreply.github.com> Co-authored-by: rlratzel <rratzel@nvidia.com> Co-authored-by: Erik Welch <erik.n.welch@gmail.com>
… zero-code change (#4685) This is for convenience and sets or updates `NETWORKX` environment variables. Do we like `NX_CUGRAPH` as the env var name? What should we consider a true value: "True" (case-insensitive) or a non-empty value? This works with the latest dev version of NetworkX. I have not yet tried it with older NetworkX versions. Authors: - Erik Welch (https://github.com/eriknw) - https://github.com/jakirkham Approvers: - Rick Ratzel (https://github.com/rlratzel) - Ralph Liu (https://github.com/nv-rliu) URL: rapidsai/cugraph#4685
This run `make` in `python/nx-cugraph` directory and update the pre-commit hook. Also fixes `ego_graph` tests to check `nx.config.fallback_to_nx` configuration. Authors: - Erik Welch (https://github.com/eriknw) Approvers: - Rick Ratzel (https://github.com/rlratzel) - Jake Awe (https://github.com/AyodeAwe) URL: rapidsai/cugraph#4717
This PR updates the nx-cugraph README with the new `NX_CUGRAPH_AUTOCONFIG` variable. --------- Co-authored-by: Rick Ratzel <3039903+rlratzel@users.noreply.github.com>
|
/ok to test |
eriknw
approved these changes
Nov 6, 2024
|
/merge |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of https://github.com/rapidsai/graph_dl/issues/578
This PR updates branch-24.12 with any updates that were made since the last fast-forward (PR on Oct 3, 2024)
This was done by running
git log --since="2024-10-03" --name-only --pretty=format: -- python/nx-cugraph/ | sort | uniqinside the cugraph repo to see which files were modified since the last fast-forward.These files were then copied over while preserving their history
git filter-repo \ --path python/nx-cugraph/_nx_cugraph/__init__.py \ --path python/nx-cugraph/nx_cugraph/tests/test_ego_graph.py \ --path python/nx-cugraph/README.md \ --path-rename python/nx-cugraph/_nx_cugraph/__init__.py:_nx_cugraph/__init__.py \ --path-rename python/nx-cugraph/nx_cugraph/tests/test_ego_graph.py:nx_cugraph/tests/test_ego_graph.py \ --path-rename python/nx-cugraph/README.md:README.md