Fix: Mamba2 generation mismatch between input_ids and inputs_embeds

[vasqu]

What does this PR do?

Fixes the failing generation test in mamba2 comparing the generation between passing input ids or passing the embeds directly. The cause of the mismatch was a false calculation of past_len which can be solved by changing the if check slightly.

Fixes #32555

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@molbap @ArthurZucker

[vasqu]

Lmk if there's a better way to do this but I think this should be ok. At least the cause of the issue is apparent now.

[molbap]

That's a nice fix, thanks! I wonder if we can use cache_position instead, it would be cleaner/more aligned with the rest of the lib, other that that LGTM

[ArthurZucker]

Can you add a small test. They exist for other model in the Mixin, https://github.com/huggingface/transformers/blob/main/tests/test_modeling_common.py#L2777 should be tested for mamba models

[ArthurZucker]

this is no longer the past length, but the current length 😉 Otherwise as this is used to create the cache positions, this is valid 😉

[vasqu]

Fair enough :p

[vasqu]

@ArthurZucker Isn't it also tested for all mamba-related models already? I've run the mamba2 tests locally for example and the test you linked was included in them (can also be seen in the CI of this PR over here).

Maybe I've misunderstood something 😅 Could you clarify the test?

[ArthurZucker]

You are right I forgot that it was failing ever since we added it!

[ArthurZucker]

Thanks @vasqu for maintaining mamba2 😉

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.