[bug] Fix float/int guidance scale not working in StableVideoDiffusionPipeline

[JinayJain]

What does this PR do?

fa3c86b caused a regression in the support for disabling CFG. Specifically, do_classifier_free_guidance was no longer returning a boolean value and causing the following error when using max_guidance_scale = 1.

    507 latent_model_input = self.scheduler.scale_model_input(latent_model_input, t)
    509 # Concatenate image_latents over channels dimention
--> 510 latent_model_input = torch.cat([latent_model_input, image_latents], dim=2)
    512 # predict the noise residual
    513 noise_pred = self.unet(
    514     latent_model_input,
    515     t,
   (...)
    518     return_dict=False,
    519 )[0]

RuntimeError: Sizes of tensors must match except in dimension 2. Expected size 1 but got size 2 for tensor number 1 in the list.

Adding this change allows the pipeline to run without error.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Tagging @patrickvonplaten who wrote the original commit.

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[sayakpaul]

Nice catch. We should definitely add a test case for this as well here: https://github.com/huggingface/diffusers/blob/main/tests/pipelines/stable_video_diffusion/test_stable_video_diffusion.py.

@yiyixuxu WDYT?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[DN6]

Looking good 👍🏽 I think a test to check disabling cfg would be a nice addition.

[JinayJain]

@DN6 Added a test, let me know how it looks.

[sayakpaul]

This is looking solid. Thanks!

[sayakpaul]

@DN6 feel free to merge once you're good!

[JinayJain]

Side note: I was looking at the tests and see that the test config sets both height and width 32. It's probably a good idea to change those to different numbers to catch shape errors where the height/width are switched mistakenly.

[sayakpaul]

Welcome you to add them in a separate PR.