🧗 Add GRPO Trainer support for third-party accelerators #2836
Conversation
48739bc to
86c5569
Compare
|
Since the vllm device patch is growing larger. It might be wise to move them into a utility module instead. Wdyt. |
|
This PR helps a lot, I hope it can speed up the integration |
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
|
I think this PR is ready to be merged 🤗 @qgallouedec |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
Can you make sure sure to run |
|
|
|
Hi @ji-huazhong, Thank you for your excellent work! This PR has been incredibly helpful in enabling me to train models using GRPO on the NPU smoothly. I want to ask if this PR is ready to be merged and I'd be extremely grateful if it could be done promptly. cc @qgallouedec |
| # Check that the requested device is available | ||
| if vllm_device.split(":")[0] == "cuda" and int(vllm_device.split(":")[1]) >= torch.cuda.device_count(): | ||
| if ( | ||
| vllm_device.split(":")[0] == f"{device_type}" |
There was a problem hiding this comment.
this should always be the case, no?
There was a problem hiding this comment.
Hi @qgallouedec
Thanks for your review. In line 387，I maintained the same logic as orignal conditional statement，only repalcing the 'cuda' type with more general type.
I believe the check for device availability here is necessary. However, perhaps we could split this conditional statement into two parts.
First, we check if the device type matches, and only after this condition is met do we check if the device index is within the range of available devices. wdyt?
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
|
|
|
Hi @kashif, the failing test case seems unrelated to this PR. Could you take a look? Thanks! |
|
The failing use case is due to an error when accessing huggingface.co and has nothing to do with this PR. cc @qgallouedec |
…2836) * Add GRPO Trainer support for Ascend NPU * 更新 grpo_trainer.py Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com> * code format * 更新 grpo_trainer.py Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com> * patch mem_get_info * stylre --------- Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com> Co-authored-by: Quentin Gallouédec <gallouedec.quentin@gmail.com>
What does this PR do?
This PR makes GRPO Trainer out of the box on Ascend NPUs.
cc @qgallouedec @lewtun
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.