Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rpc : get available mem for the CPU backend #7319

Merged
merged 1 commit into from
May 16, 2024
Merged

rpc : get available mem for the CPU backend #7319

merged 1 commit into from
May 16, 2024

Conversation

rgerganov
Copy link
Collaborator

This can be overridden with the -m command line option

ref: #7293

@rgerganov
rpc : get available mem for the CPU backend
24b3881 
This can be overridden with the -m command line option

ref: ggerganov#7293
@rgerganov
Copy link
Collaborator Author

I tested this on linux and win32. Not sure if it works on macos.

@mofosyne mofosyne added examples Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix server labels May 16, 2024
ggerganov
ggerganov approved these changes May 16, 2024
View reviewed changes
Copy link
Owner

@ggerganov ggerganov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works on macos:

make -j && ./bin/rpc-server 0.0.0.0 --port 5543

create_backend: using CPU backend
Starting RPC server on 0.0.0.0:5543, backend memory: 32768 MB

@rgerganov rgerganov merged commit 9afdffe into ggerganov:master May 16, 2024
61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ggerganov ggerganov ggerganov approved these changes

Assignees
No one assigned
Labels
examples Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix server
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rgerganov @ggerganov @mofosyne