Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix gpulayers autodetection for cublas & clblast backends #69

Merged
merged 1 commit into from
Oct 13, 2024

Conversation

matoro
Copy link

@matoro matoro commented Oct 9, 2024

This fixes --gpulayers -1 paired with either --usecublas or --useclblast with no additional arguments.

This fixes `--gpulayers -1` paired with either `--usecublas` or
`--useclblast` with no additional arguments.
@YellowRoseCx
Copy link
Owner

What issue were you experiencing with this? Auto detect layers works fine for me on Linux and Windows

@matoro
Copy link
Author

matoro commented Oct 13, 2024

What issue were you experiencing with this? Auto detect layers works fine for me on Linux and Windows

If you use e.g. --usecublas --gpulayers -1, then args.usecublas = [], not None, so it errors with No GPU backend found, or could not automatically determine GPU layers. Please set it manually., even though a GPU is present. Same for useclblast. It works only if you pass an argument, e.g. --usecublas lowvram --gpulayers -1.

@YellowRoseCx
Copy link
Owner

Ahh okay. On the GUI, when selecting CuBLAS, it passes "normal" to the arguments list, but with the CLI it makes the list empty, and because
[] (empty list): returns Falsy
["normal"]: returns Truthy,

The code would work in GUI but not via CLI; checking if it's not None rather than truthiness of the argument would make it work in both situations.
Thanks for catching that!

@YellowRoseCx YellowRoseCx merged commit 4c08a44 into YellowRoseCx:main Oct 13, 2024
YellowRoseCx added a commit that referenced this pull request Oct 14, 2024
Fix gpulayers autodetection for cublas & clblast backends
YellowRoseCx added a commit that referenced this pull request Nov 2, 2024
Fix gpulayers autodetection for cublas & clblast backends
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants