Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature Request: RPC Cuda Build to link with cudart dlls #8912

Closed
4 tasks done
jkfnc opened this issue Aug 7, 2024 · 2 comments · Fixed by #9006
Closed
4 tasks done

Feature Request: RPC Cuda Build to link with cudart dlls #8912

jkfnc opened this issue Aug 7, 2024 · 2 comments · Fixed by #9006
Assignees
@rgerganov
Labels
enhancement New feature or request

Comments

@jkfnc
Copy link

jkfnc commented Aug 7, 2024

Prerequisites

  • I am running the latest code. Mention the version if possible as well.
  • I carefully followed the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Is it possible to get RPC Builds that work with Cuda , so we dont have to compile from scratch.

Motivation

Would be easy to get started with RPC for distributed inference if we had readymade build available. Current RPC build only works with CPU.

Possible Implementation

No response
@jkfnc jkfnc added the enhancement New feature or request label Aug 7, 2024
@jkfnc jkfnc changed the title Feature Request: Req for RPC Cuda Build to link with cudart dlls Feature Request: RPC Cuda Build to link with cudart dlls Aug 7, 2024
@slaren
Copy link
Collaborator

slaren commented Aug 8, 2024

I think it would be ok to enable the RPC backend on all the builds, and remove the RPC-specific build. It should be a simple change in the .github/workflows/build.yml file.

@rgerganov
Copy link
Collaborator

I will submit a patch for this in the next few days

@rgerganov rgerganov self-assigned this Aug 11, 2024
rgerganov added a commit to rgerganov/llama.cpp that referenced this issue Aug 12, 2024
@rgerganov
ci : enable RPC in all of the released builds
7140070 
ref: ggerganov#8912
@rgerganov rgerganov mentioned this issue Aug 12, 2024
Merged
4 tasks
@slaren slaren linked a pull request Aug 12, 2024 that will close this issue
ci : enable RPC in all of the released builds #9006
Merged
4 tasks
rgerganov added a commit that referenced this issue Aug 12, 2024
@rgerganov
ci : enable RPC in all of the released builds (#9006)
1f67436 
ref: #8912
arthw pushed a commit to arthw/llama.cpp that referenced this issue Nov 15, 2024
@rgerganov @arthw
ci : enable RPC in all of the released builds (ggerganov#9006)
19b9d2a 
ref: ggerganov#8912
arthw pushed a commit to arthw/llama.cpp that referenced this issue Nov 18, 2024
@rgerganov @arthw
ci : enable RPC in all of the released builds (ggerganov#9006)
28747ba 
ref: ggerganov#8912
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rgerganov rgerganov

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ci : enable RPC in all of the released builds rgerganov/llama.cpp
3 participants
@rgerganov @slaren @jkfnc