cuda_setup is not defined

[abhishekkrthakur]

recently started getting this error on new version:

2023-02-05T10:29:54Z [job]     import bitsandbytes as bnb
2023-02-05T10:29:54Z [job]   File "/app/env/lib/python3.8/site-packages/bitsandbytes/__init__.py", line 7, in <module>
2023-02-05T10:29:54Z [job]     from .autograd._functions import (
2023-02-05T10:29:54Z [job]   File "/app/env/lib/python3.8/site-packages/bitsandbytes/autograd/__init__.py", line 1, in <module>
2023-02-05T10:29:54Z [job]     from ._functions import undo_layout, get_inverse_transform_indices
2023-02-05T10:29:54Z [job]   File "/app/env/lib/python3.8/site-packages/bitsandbytes/autograd/_functions.py", line 9, in <module>
2023-02-05T10:29:54Z [job]     import bitsandbytes.functional as F
2023-02-05T10:29:54Z [job]   File "/app/env/lib/python3.8/site-packages/bitsandbytes/functional.py", line 17, in <module>
2023-02-05T10:29:54Z [job]     from .cextension import COMPILED_WITH_CUDA, lib
2023-02-05T10:29:54Z [job]   File "/app/env/lib/python3.8/site-packages/bitsandbytes/cextension.py", line 13, in <module>
2023-02-05T10:29:54Z [job]     setup.run_cuda_setup()
2023-02-05T10:29:54Z [job]   File "/app/env/lib/python3.8/site-packages/bitsandbytes/cuda_setup/main.py", line 92, in run_cuda_setup
2023-02-05T10:29:54Z [job]     binary_name, cudart_path, cuda, cc, cuda_version_string = evaluate_cuda_setup()
2023-02-05T10:29:54Z [job]   File "/app/env/lib/python3.8/site-packages/bitsandbytes/cuda_setup/main.py", line 395, in evaluate_cuda_setup
2023-02-05T10:29:54Z [job]     has_cublaslt = is_cublasLt_compatible(cc)
2023-02-05T10:29:54Z [job]   File "/app/env/lib/python3.8/site-packages/bitsandbytes/cuda_setup/main.py", line 153, in is_cublasLt_compatible
2023-02-05T10:29:54Z [job]     cuda_setup.add_log_entry("WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!", is_warning=True)
2023-02-05T10:29:54Z [job] NameError: name 'cuda_setup' is not defined

[lolxdmainkaisemaanlu]

EDIT: Found the solution! go to env/lib/python3.x/site-packages/bitsandbytes/cuda_setup/main.py", line 153 and change it from cuda_setup.add_log_entry("WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!", is_warning=True) to CUDASetup.get_instance().add_log_entry("WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!", is_warning=True)

Getting the same error!

import bitsandbytes

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

Traceback (most recent call last):
File "", line 1, in
File "/home/siddhesh/anaconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/init.py", line 7, in
from .autograd._functions import (
File "/home/siddhesh/anaconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/autograd/init.py", line 1, in
from ._functions import undo_layout, get_inverse_transform_indices
File "/home/siddhesh/anaconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 9, in
import bitsandbytes.functional as F
File "/home/siddhesh/anaconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/functional.py", line 17, in
from .cextension import COMPILED_WITH_CUDA, lib
File "/home/siddhesh/anaconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 13, in
setup.run_cuda_setup()
File "/home/siddhesh/anaconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 92, in run_cuda_setup
binary_name, cudart_path, cuda, cc, cuda_version_string = evaluate_cuda_setup()
File "/home/siddhesh/anaconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 395, in evaluate_cuda_setup
has_cublaslt = is_cublasLt_compatible(cc)
File "/home/siddhesh/anaconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 153, in is_cublasLt_compatible
cuda_setup.add_log_entry("WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!", is_warning=True)
NameError: name 'cuda_setup' is not defined. Did you mean: 'CUDASetup'?

[Ph0rk0z]

This will get you through the error but it still won't generate text.

def is_cublasLt_compatible(cc):
    has_cublaslt = False
    if cc is not None:
        cc_major, cc_minor = cc.split('.')
        #if int(cc_major) < 7 or (int(cc_major) == 7 and int(cc_minor) < 5):
            #CUDASetup.add_log_entry("WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!", is_warning=True)
        #else:
         #   has_cublaslt = True
    return has_cublaslt

[JohnnyRacer]

@Ph0rk0z
This comment works for Compute capability < 7.5 (Pascal, Maxwell GPUs)

I just tested on a 1080ti, bitsandbytes will load correctly if you use this fix.

[shon-otmazgin]

@JohnnyRacer

It will help that - huggingface/peft#79 as well?

[JohnnyRacer]

@JohnnyRacer

It will help that - huggingface/peft#79 as well?

@shon-otmazgin

Yes. You will need to change that line everytime you reinstall the bitsandbytes package for the error to go away however.

[Ph0rk0z]

The PR's are more elegant.
70a36dc

1238d18

[onefish51]

when I used bitsandbytes==0.37.0

File "/opt/conda/lib/python3.8/site-packages/bitsandbytes/cuda_setup/main.py", line 153, in is_cublasLt_compatible
    cuda_setup.add_log_entry("WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!", is_warning=True)
NameError: name 'cuda_setup' is not defined

I fixed it by pip install bitsandbytes==0.35.0

[abhishekkrthakur]

i use the same set of packages for different architectures. its not an option for me to keep changing code in installation or not use a package. so, would be nice if this can be handled by the library. cc: @TimDettmers

[djaym7]

+1, same error
cuda_setup.add_log_entry("WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!", is_warning=True)
NameError: name 'cuda_setup' is not defined

[dgallitelli]

when I used bitsandbytes==0.37.0

File "/opt/conda/lib/python3.8/site-packages/bitsandbytes/cuda_setup/main.py", line 153, in is_cublasLt_compatible
    cuda_setup.add_log_entry("WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!", is_warning=True)
NameError: name 'cuda_setup' is not defined

I fixed it by pip install bitsandbytes==0.35.0

+1 on this - got the same issue when trying to train a Stable Diffusion 2.1 base model with Dreambooth and bitsandbytes==0.36.0 but fixed it with bitsandbytes==0.35.0.

[Ph0rk0z]

AFAIK this error is now fixed in newer BnB

[swumagic]

Bitsandbytes was not supported windows before, but my method can support windows.（yuhuang）
1 open folder J:\StableDiffusion\sdwebui，Click the address bar of the folder and enter CMD
or WIN+R, CMD 。enter，cd /d J:\StableDiffusion\sdwebui
2 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes

3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows

4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl

Replace your SD venv directory file（python.exe Folder） here（J:\StableDiffusion\sdwebui\py310）

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.