owlvit/2 dynamic input resolution.

[bastrob]

What does this PR do?

Towards #30579
Hey, this is a draft, Im wondering how can we manage the variables impacted by the dynamic input change in the OwlViTForObjectDetection class (self.sqrt_num_patches, self.box_bias) ?
Is there a better way to handle this ?

The interpolate_pos_encoding allows new input size respecting height==width strictly ?
In that case I should ensure this.
If not, I think sqrt_num_patches needs to be decomposed too (_h, _w), some examples where it might throws exc:
https://github.com/bastrob/transformers/blob/30f3c2d56729974ec0d1d9e2fc4fd633ab697eb2/src/transformers/models/owlvit/modeling_owlvit.py#L1355
https://github.com/bastrob/transformers/blob/30f3c2d56729974ec0d1d9e2fc4fd633ab697eb2/src/transformers/models/owlvit/modeling_owlvit.py#L1459
https://github.com/bastrob/transformers/blob/30f3c2d56729974ec0d1d9e2fc4fd633ab697eb2/src/transformers/models/owlvit/modeling_owlvit.py#L1719

Fixes: #34622

Before submitting

• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@amyeroberts
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[qubvel]

Hi @bastrob! Thanks for opening PR!

It would be great to enable any image resolution, but If you will not find a way to manage height != width images we can limit it to a square input image size. We just have to make sure the proper error is raised.

[bastrob]

Yes, Im agree it makes sense to enable any image resolution, i just wanted to clarify this point. I will work on it, thanks for your feedback !

[bastrob]

Hi @qubvel, I pushed a version to handle image size at any resolution, what do you think about it ?

[qubvel]

Hi @bastrob, thanks for digging into it and adding tests! Overall looks great 🚀 just added some nits regarding variable naming.

Could you please also push an empty commit with the message[run-slow] owlvit, owlv2 to trigger slow tests (this commit should be the last one, so I can approve the CI run). Thanks!

[bastrob]

If it sounds good to you @qubvel, Im ready to push the last commit :)

[qubvel]

Hi @bastrob, thanks for the update! Please, see the comments below 🤗

[bastrob]

Hello @qubvel, box_predictor now takes interpolate_pos_encoding as extra parameter to enable the dynamic computation of box_bias inside.
Plus, I've added # copied from statement.

[qubvel]

Thanks, looks good to me, just a few notes re the style! Also tested locally and predictions look good 👍

can you please also push empty commit with the message [run-slow] owlvit, owlv2 to trigger slow tests for these models in CI? (it should be the last commit at the moment I approve the run of CI)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qubvel]

Thanks for handling this non-standard case! Looks great to me!

[ArthurZucker]

Super good that things are copied from one another! And thanks for also adding the tests. I am usually not a fan of adding codepaths, so if there is a way to avoid checking if interpolate pos encoding at 4 differents places, to just do it when we actually do the interpolation, it would be a lot better!
We can merge whatsoever 🤗