Paligemma: fix generation with Gemma2 #36044
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
molbap
reviewed
Feb 5, 2025
ArthurZucker
reviewed
Feb 6, 2025
|
I think making it explicit that kwargs will be used by an only an LM was better |
Cyrilvallez
approved these changes
Feb 6, 2025
ArthurZucker
approved these changes
Feb 6, 2025
There was a problem hiding this comment.
let's say that an integration test is most welcome as well!
There was a problem hiding this comment.
yeah, was quite low in priority for the patch so I decided to skip it for now :)
|
For transparency, this commit needs to be modified for the patch, only applying changes for PaliGemnma2 |
ArthurZucker
pushed a commit
that referenced
this pull request
Feb 6, 2025
* fix paligemma * nit * use `kwargs` in models that can load any LM * update changes to only affect Paligenma
MekkCyber
pushed a commit
that referenced
this pull request
Feb 7, 2025
* fix paligemma * nit * use `kwargs` in models that can load any LM
elvircrn
pushed a commit
to elvircrn/transformers
that referenced
this pull request
Feb 13, 2025
* fix paligemma * nit * use `kwargs` in models that can load any LM
sbucaille
pushed a commit
to sbucaille/transformers
that referenced
this pull request
Feb 16, 2025
* fix paligemma * nit * use `kwargs` in models that can load any LM
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #36029 and adds tests for the model, imo we need tests with different LM backbone because Gemma-2 is special
This is a quick fix but I think we should make this kind of fix on LM work out-of-the-box, by adding it as
kwargsfor example. Most LMs acceptloss_kwargsthus we can make all multimodal models also accept kwargs that are simply passed further to the LM. WDYT?