FP8 causes exception: name te not defined

[JulesGM]

System Info

Accelerate 0.18.0
Transformers 4.27.4
Torch 2.0.0

Reproduction

When launching with mixed_precision=fp8 in the config_file, I get

[muellerzr]

Hi @JulesGM, this should be solved via #1283

Can you do pip install git+https://github.com/huggingface/accelerate to double check? :)

[JulesGM]

hello @muellerzr , I get ImportError: Using 'has_transformer_engine_layers' requires transformer_engine to be installed.

[JulesGM]

[muellerzr]

That is correct, have you installed the transformer engine on your system? :) (As the error suggests, both it and the required hardware are needed)

[JulesGM]

No I have not. But it is a weird error to get. I did not know I needed transformer_engine to use fp8, I just selected the option because I saw it in accelerate config

[JulesGM]

hmm also, it's only on H100s, ok, sad. I was hoping it would also be ok with A100 somehow.

[JulesGM]

No I have not. But it is a weird error to get. I did not know I needed transformer_engine to use fp8, I just selected the option because I saw it in accelerate config.

I suggest that the library checks if transformer_engine is installed, and raises if it's not, explaining that it's required. It would be more clean than this.

[JulesGM]

Like this, it makes one think it could be a bug.

[muellerzr]

Thanks @JulesGM, that does indeed make sense. Should be addressed via #1287, need to see with Sylvain if we want to raise that in the CLI as well (since config files can be swapped/passed through machines). However this will raise the right error when running the script now more clearly

[JulesGM]

cool thanks. I can close this if you want

[muellerzr]

Sure :)