AI: -nvidia "all"
is not obvious what it does
#3370
Labels
status: triage
this issue has not been evaluated yet
-nvidia "all"
is not obvious what it does
#3370
Describe the bug
When the flag
-nvidia "all"
is used for AI inference, theaiModels.json
must match number of GPUs installed, else it will try run all GPUs with the first item in the list.Not sure if this is the expected behavior.
To Reproduce
-nvidia "all"
aiModels.json
to include only the LLM modelExpected behavior
Obviously if we specify the GPUs ie
-nvidia 0,1
then we assume theaiModels.json
will have two models loaded, it works.i.e.
I think the documentation needs to be updated, but I am unsure if this is actually the logic for the
"all"
value.Set Up
Slot 0 - 3090 (24GB Ram)
Slot 1 - 2080 ti (11GB Ram)
The text was updated successfully, but these errors were encountered: