Qwen2VL: skip base input_ids-inputs_embeds equivalence check
#34535
Conversation
| @@ -1610,7 +1610,7 @@ def test_generate_from_inputs_embeds(self, _, num_beams): | |||
| inputs_dict.pop("pixel_values_images", None) | |||
| # 2.C - No easy fix, let's skip the check that compares the outputs from `input_ids` and `inputs_embeds` | |||
| has_complex_embeds_computation = any( | |||
| model_name in model_class.__name__.lower() for model_name in ["moshi"] | |||
| model_name in model_class.__name__.lower() for model_name in ["moshi", "qwen2vl"] | |||
There was a problem hiding this comment.
This flag skips the check that passing inputs_embeds = model.get_input_embeddings()(input_ids) should be equivalent to passing input_ids to generate.
This is not true in Qwen2VL, whose computation of inputs_embeds is more complex
There was a problem hiding this comment.
OK for me. Just curious: would this be addressed in the future to make it work
There was a problem hiding this comment.
Not unless we create a model function to embed inputs, similar to def get_input_embeddings(self).
(I think we should do it eventually, we're starting to have a few models with non-standard inputs embeddings creation!)
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
…gingface#34535) it has complex inputs_embeds computation
What does this PR do?
Computing
inputs_embedsinQwen2VLis more complex than simply embeddinginput_ids(seeQwen2VLForConditionalGeneration), so the basic check doesn't apply :)Fixes: