pass module to Params4bit.from_prequantized to ensure quant_state

[winglian]

What does this PR do?

see #32477 #32477

Let's start with making sure pre-quantized models can be loaded properly for FSDP. This at least unlocks pre-quant models with qlora+fsdp where CPU RAM isn't a blocker.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[amyeroberts]

cc @SunMarc

[SunMarc]

Thanks for the work ! Left a comment !

[SunMarc]

Can we just do what was done in the reverted PR ?

        param_kwargs = {}
            sig = inspect.signature(bnb.nn.Params4bit.from_prequantized)
            if "module" in sig.parameters:
                param_kwargs["module"] = module

or we could perform the version check here to device if we pass module or not !

[SunMarc]

Nice ! Thanks for fixing @winglian ! I left a minor question !

[SunMarc]

Quick question about inspect.signature, does this make the quantization slower ? I'm a bit concerned since create_quantized_param is called for each param cc @muellerzr as I remember that you prefer not to use inspect

[winglian]

@SunMarc also happy to do it as

    @cached_property
    def is_bnb_gte_0_43_3(self) -> bool:
        if self._is_bnb_gte_0_43_3 is not None:
            return self._is_bnb_gte_0_43_3

        self._is_bnb_gte_0_43_3 = version.parse(importlib.metadata.version("bitsandbytes")) >= version.parse("0.43.3")
        return self._is_bnb_gte_0_43_3

and then we can use self. is_bnb_gte_0_43_3()

[muellerzr]

Version parsing is far more preferred!

[muellerzr]

Though generally this would exist in import_utils as a min_version to is_bnb_available(). (We should then cache the bnb_version if we don't already, similar to _torch_version)

[matthewdouglas]

Minor nit: I think if there's a variable or function named for this that the name should be more meaningful than is_bnb_gte_0_43_3. It should describe what that version check is about, e.g. 'bnb_can_load_from_float_storage`.

That said, I like the suggestion to model it the way I see is_accelerate_available(min_version) in import_utils.

[ArthurZucker]

can we do the version parsing? 🤗

[muellerzr]

Thanks!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ArthurZucker]

cool thanks