ImportError: cannot import name 'is_flash_attn_available' from 'transformers.utils' (~/lib/python3.10/site-packages/transformers/utils/__init__.py)

[Rajmehta123]

System Info

transformers: 4.35.0
python: 3.10.13
Platform: Linux

Who can help?

@Narsil @ArthurZucker @SunMarc

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
import torch
model = AutoModelForCausalLM.from_pretrained("01-ai/Yi-6B-200K", trust_remote_code=True,torch_dtype=torch.bfloat16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("01-ai/Yi-6B-200K",trust_remote_code=True)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)

This works fine with 4.34.1 transformers verion

Expected behavior

The model gets loaded correctly

BUT I got this error:

ImportError: cannot import name 'is_flash_attn_available' from 'transformers.utils' (~/lib/python3.10/site-packages/transformers/utils/init.py)

This works fine with 4.34.1 transformers verion

[SunMarc]

Hi @Rajmehta123 ! Thanks for reporting this issue. This happens because in the latest version of transformers, we removed is_flash_attn_available() in favor of is_flash_attn_2_available(). See related PR. I guess the remote code that you are executing is using is_flash_attn_available, hence the error. I've opened an PR to put back the function and make it go through a deprecation cycle !

[regineshalom]

Hi @SunMarc , I have an error as well but its on the flip side

from transformers import pipeline
classifier = pipeline('summarization')

RuntimeError: Failed to import transformers.models.bart.modeling_bart because of the following error (look up to see its traceback):
cannot import name 'is_flash_attn_2_available' from 'transformers.utils'

Transformers: Version: 4.35.0
Python: 3.11.4

[SunMarc]

Hi @regineshalom, i'm unable to reproduce your error on my local setup. If you can reproduce the error in a colab and send it to me, that would be great.

[yecphaha]

ImportError: cannot import name 'is_flash_attn_available' from 'transformers.utils'
答：pip install transformers==4.34.1，transformers的版本必须是4.34，不能是4.31、4.32、4.33

[curlup]

I got the same cannot import name 'is_flash_attn_2_available' from 'transformers.utils
What version i downgrade to to get it working? thanks

[ArthurZucker]

I think upgrading would be better 😉

[imneov]

I got the same cannot import name 'is_flash_attn_2_available' from 'transformers.utils What version i downgrade to to get it working? thanks

# Install the specific version using pip
pip install transformers==4.34.1

@curlup It work fine to me.