Fixing name 'torch' is not defined in bitsandbytes integration

[jamesbraza]

With transformers==4.41.2 and this:

from transformers import AutoModelForCausalLM, QuantoConfig

AutoModelForCausalLM.from_pretrained(
    "microsoft/phi-2", low_cpu_mem_usage=True, quantization_config=QuantoConfig(weights="int8")
)

I hit this traceback:

  File "/Users/user/code/repo/venv/lib/python3.12/site-packages/transformers/integrations/bitsandbytes.py", line 331, in <module>
    def dequantize_bnb_weight(weight: torch.nn.Parameter, state=None):
                                      ^^^^^
NameError: name 'torch' is not defined

---

An alternate would be requiring importers to first run import torch

Cc @younesbelkada @SunMarc @amyeroberts @ArthurZucker based on recent PR reviewers for this file

[ArthurZucker]

I can reproduce. Thanks for fixing, this would need a patch cc @younesbelkada
We can also just import torch if torch is available

[ArthurZucker]

Failing test is unrelated!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.