-
Notifications
You must be signed in to change notification settings - Fork 28.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace accelerator.use_fp16
in examples
#33513
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thx for fixing !
If we are using fp8, we need to pad to 16. Since these are simple examples + did not take into account fp8 from the start, I guess we can leave it like that. Otherwise, we would have to do the following if accelerator.mixed_precision == "fp8":
pad_to_multiple_of = 16
elif accelerator.mixed_precision != "no":
pad_to_multiple_of = 8
else:
pad_to_multiple_of = None |
831a6e1
to
51bc1b9
Compare
Thanks for the review, I've gone ahead and added |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You know a lot more than me about this one @SunMarc ! Juste wondering when was mixed_precision
introduced? (before 1.0?)
Looks like |
Thanks for looking into ! Looks like this was done in 2022, so with <= accelerate v.0.20. The min version of accelerate in transformers is v0.26.0. Merging this since this should be safe to do |
Thanks both for checking! |
* Replace `accelerator.use_fp16` in examples * pad_to_multiple_of=16 for fp8
What does this PR do?
examples_torch
tests are currently failing due to usage ofaccelerator.use_fp16
,use_fp16
was removed in accelerate#3098 andtransformers
CI uses accelerate@main so the property is no longer available.This PR replaces
accelerator.use_fp16
withaccelerator.mixed_precision != "no"
which is equivalent to the removeduse_fp16
property.Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
cc @muellerzr