Fix swin embeddings interpolation

[amyeroberts]

What does this PR do?

Fixes broken tests on main introduced by #30656

The PR mistakenly introduced interpolation to the patch embeddings, which shouldn't be done, and also adds a verification check which doesn't account for the padding in _maybe_pad

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ydshieh]

Thanks for the fix!

(why the tests are failing however? The interpolate_pos_encoding is False by default no? Or it is set to True in some tests?)

[amyeroberts]

@ydshieh Yes, it's set to False by default. There's a few things happening here:

• The interpolation logic was added for the patch embeddings (which shouldn't have happened)
• Swin actually already handles images of different sizes (providing they're divisible by 32), so in the pipeline tests, images different from those the model was trained on were being passed in
• The additional interpolation logic added a check to verify the input images size if interpolation was False (as before). However, because swin can handle some images of different sizes, and interpolate_pos_encoding was False by default this new check was triggered and would fail