-
Notifications
You must be signed in to change notification settings - Fork 27.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GPTQ
, CompressedTensors
] Fix unsafe imports and metada check
#34815
Conversation
3c927c3
to
f3c7f78
Compare
GPTQ
, CompressedTensors
] Fix unsafe imports and metada check
Thanks for the fix @vasqu, LGTM ! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fixes !
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good but I think we should always call validate_environment
before even initializing the quantizer (so probably update super (HfQuantizer
) to call it at init time no?
@ArthurZucker
The problem is some quantizers need additional information that is not available at creation; another option I could see it to add a post_init to create those unsafe parts :) |
gentle ping @ArthurZucker |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for explaining
Not sure if it was fixed by recent quantizer compressed changes, let's resolve conflicts and good to merge |
Forgot about this PR myself 😅 should be good to merge now! Thx for the merges here, ci sometimes do be flaky tho |
will merge it once the ci is green @vasqu 🤗 |
What does this PR do?
Fixes some unsafe imports for GPTQ and CompressedTensors as well as an unsafe metadata check for GPTQ, i.e. metadata can't be checked if the package doesn't exist...
I tried looking into integrating the env validation function before even creating the quantizers but that would require a complete restructure, especially since the env is checked after creation to get additional info like device map etc. So I kept it as simple as I could and wrapped the unsafe stuff.
Minimal reproducible script to trigger gptq errors (when optimum is not installed):
Fixes #34765
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@SunMarc @MekkCyber