[ray] launch multiple GPU with ray #396
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
475668a to
61101c3
Compare
feifeibear
reviewed
Dec 18, 2024
|
PR实现了通过ray方式启动多进程。参考vllm使用RayGPUExecutor来管理多个worker，每个worker执行diffusers pipefline的逻辑。 目前这种方式和torchrun启动程序（example.py)用法差别太大。 我建议设计一个DiffusionPipeline的Ray分布式版本，RayDiffusionPipeline，然后这个类提供from_pretrained，forward等接口。 PR中hardcode了一些地方，比如对模型初始化text_encoder处理，因为目前text_encoder是没有多卡切分的，可以让每个worker都重复载入text_encoder，希望尽量保持和torchrun接口一致性。 |
feifeibear
reviewed
Dec 19, 2024
feifeibear
requested changes
Dec 19, 2024
feifeibear
reviewed
Dec 19, 2024
feifeibear
requested changes
Dec 20, 2024
examples/ray/ray_flux_example.py
Outdated
| # output is a list of results from each worker, we take the last one | ||
| for i, image in enumerate(output[-1].images): | ||
| image.save( | ||
| f"/data/results/{model_name}_result_{i}.png" |
There was a problem hiding this comment.
save to a relative path ./results/xxx
| @@ -188,6 +192,9 @@ class ParallelConfig: | |||
| sp_config: SequenceParallelConfig | |||
| pp_config: PipeFusionParallelConfig | |||
| tp_config: TensorParallelConfig | |||
| distributed_executor_backend: Optional[str] = None | |||
| world_size: int = 1 # FIXME: remove this | |||
| worker_cls: str = "xfuser.ray.worker.worker.Worker" | |||
There was a problem hiding this comment.
do we need distributed_executor_backend and worker_cls?
There was a problem hiding this comment.
We don't need distributed_executor_backend, but we need worker_cls for ray to initial worker by its class name
def init_worker(self, *args, **kwargs):
worker_class = resolve_obj_by_qualname(
self.worker_cls)
self.worker = worker_class(*args, **kwargs)
assert self.worker is not None
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Support Ray to start the pipeline