Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix gpulayers autodetection for cublas & clblast backends - (Pull request koboldcpp-rocm#69 from matoro/main) #1165

Merged

Conversation

YellowRoseCx
Copy link

This fixes --gpulayers -1 paired with either --usecublas or --useclblast with no additional arguments.

If you use e.g. --usecublas --gpulayers -1, then args.usecublas = [], not None, so it errors with No GPU backend found, or could not automatically determine GPU layers. Please set it manually., even though a GPU is present. Same for useclblast. It works only if you pass an argument, e.g. --usecublas lowvram --gpulayers -1.

On the GUI, when selecting CuBLAS, it passes "normal" to the arguments list, but with the CLI it makes the list empty, and because:
[] (empty list): returns Falsy
["normal"]: returns Truthy,

The GPU layer code would work in GUI but not via CLI; checking if it's not None rather than truthiness of the argument would make it work in both situations.

YellowRoseCx#69

Fix gpulayers autodetection for cublas & clblast backends
@LostRuins
Copy link
Owner

Should be correct, thanks

@LostRuins LostRuins merged commit f029de6 into LostRuins:concedo_experimental Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants