v1-5 docs updates #921

apolinario · 2022-10-20T08:08:14Z

No description provided.

Additionally add FLAX so the model card can be slimmer and point to this page

HuggingFaceDocBuilderDev · 2022-10-20T08:11:33Z

The documentation is not available anymore as the PR was closed or merged.

pcuenca

I fixed a couple of references to v1-4 (in the text, not the links) :)

pcuenca · 2022-10-20T10:28:49Z

README.md

@@ -64,44 +64,54 @@ In order to get started, we recommend taking a look at two notebooks:
 - The [Training a diffusers model](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/training_example.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/training_example.ipynb) notebook summarizes diffusion models training methods. This notebook takes a step-by-step approach to training your
  diffusion models on an image dataset, with explanatory graphics. 

-## **New** Stable Diffusion is now fully compatible with `diffusers`!  
+## Stable Diffusion is fully compatible with `diffusers`!  


pcuenca · 2022-10-20T10:30:04Z

README.md


-Stable Diffusion is a text-to-image latent diffusion model created by the researchers and engineers from [CompVis](https://github.com/CompVis), [Stability AI](https://stability.ai/) and [LAION](https://laion.ai/). It's trained on 512x512 images from a subset of the [LAION-5B](https://laion.ai/blog/laion-5b/) database. This model uses a frozen CLIP ViT-L/14 text encoder to condition the model on text prompts. With its 860M UNet and 123M text encoder, the model is relatively lightweight and runs on a GPU with at least 10GB VRAM.
+Stable Diffusion is a text-to-image latent diffusion model created by the researchers and engineers from [CompVis](https://github.com/CompVis), [Stability AI](https://stability.ai/), [LAION](https://laion.ai/) and [RunwayML](https://runwayml.com/). It's trained on 512x512 images from a subset of the [LAION-5B](https://laion.ai/blog/laion-5b/) database. This model uses a frozen CLIP ViT-L/14 text encoder to condition the model on text prompts. With its 860M UNet and 123M text encoder, the model is relatively lightweight and runs on a GPU with at least 10GB VRAM.


~4 now I think, with attention slicing, but probably better to keep this simple.

README.md

docs/source/quicktour.mdx

pcuenca · 2022-10-20T10:38:35Z

examples/text_to_image/README.md

@@ -25,7 +25,7 @@ accelerate config

 ### Pokemon example

-You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/CompVis/stable-diffusion-v1-4), read the license and tick the checkbox if you agree. 
+You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree. 


Suggested change

You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree.

You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-5`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree.

pcuenca · 2022-10-20T10:38:50Z

examples/textual_inversion/README.md

@@ -29,7 +29,7 @@ accelerate config

 ### Cat toy example

-You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/CompVis/stable-diffusion-v1-4), read the license and tick the checkbox if you agree. 
+You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree. 


Suggested change

You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree.

You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-5`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree.

patrickvonplaten

Looks good to me! Thanks a lot for taking this one @apolinario :-)

patrickvonplaten · 2022-10-20T11:32:48Z

README.md

+prompt_ids = shard(prompt_ids)
+
+images = pipeline(prompt_ids, params, prng_seed, num_inference_steps, jit=True).images
+images = pipeline.numpy_to_pil(np.asarray(images.reshape((num_samples,) + images.shape[-3:])))


Unrelated to this PR: Think we should move the reshape functionality in the numpy_to_pil function (maybe we could open an issue for this)

README.md

patrickvonplaten

Let's wait with this one until we're sure it's better than v1-4 and naming is confirmed

patil-suraj

Looks good to me, thanks a lot! Just left couple of comments.

README.md

patil-suraj · 2022-10-21T10:27:11Z

examples/dreambooth/README.md

@@ -22,7 +22,7 @@ accelerate config

 ### Dog toy example

-You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/CompVis/stable-diffusion-v1-4), read the license and tick the checkbox if you agree. 
+You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree. 


I'm not sure if we want to change the model in examples. Think this is up to the user. The commands have specific hprams tested with specific checkpoint, and are supposed to to work with that checkpoint out of the box, in case some users want to just follow the example.

IMO would make sense to have the latest model in the examples. What kind of hparams are tuned to v1-4? I remember when we were preparing SD for release with diffusers we were working on v1-3 and v1-4 dropped quite late, did we swap a lot of stuff?

+1 on Suraj here

Suraj "tuned" the long blog post on v1-4, so I think let's leave it there for now

patil-suraj · 2022-10-21T10:27:54Z

examples/text_to_image/README.md

@@ -25,7 +25,7 @@ accelerate config

 ### Pokemon example

-You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/CompVis/stable-diffusion-v1-4), read the license and tick the checkbox if you agree. 
+You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree. 


Same comment as above, let's not change the model in the examples, unless we run the experiments with it.

patil-suraj · 2022-10-21T10:28:13Z

examples/textual_inversion/README.md

@@ -29,7 +29,7 @@ accelerate config

 ### Cat toy example

-You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/CompVis/stable-diffusion-v1-4), read the license and tick the checkbox if you agree. 
+You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree. 


Same comment as above.

patrickvonplaten · 2022-10-21T10:40:37Z

examples/community/interpolate_stable_diffusion.py

@@ -69,7 +69,7 @@ class StableDiffusionWalkPipeline(DiffusionPipeline):
            [`DDIMScheduler`], [`LMSDiscreteScheduler`], or [`PNDMScheduler`].
        safety_checker ([`StableDiffusionSafetyChecker`]):
            Classification module that estimates whether generated images could be considered offensive or harmful.
-            Please, refer to the [model card](https://huggingface.co/CompVis/stable-diffusion-v1-4) for details.
+            Please, refer to the [model card](https://huggingface.co/runwayml/stable-diffusion-v1-5) for details.


Let's maybe revert this as well as the community pipeline author should decide here

patrickvonplaten · 2022-10-21T10:40:44Z

examples/community/lpw_stable_diffusion.py

@@ -389,7 +389,7 @@ class StableDiffusionLongPromptWeightingPipeline(DiffusionPipeline):
            [`DDIMScheduler`], [`LMSDiscreteScheduler`], or [`PNDMScheduler`].
        safety_checker ([`StableDiffusionSafetyChecker`]):
            Classification module that estimates whether generated images could be considered offensive or harmful.
-            Please, refer to the [model card](https://huggingface.co/CompVis/stable-diffusion-v1-4) for details.
+            Please, refer to the [model card](https://huggingface.co/runwayml/stable-diffusion-v1-5) for details.


Let's maybe revert this as well as the community pipeline author should decide here

patrickvonplaten · 2022-10-21T10:41:11Z

examples/community/README.md

@@ -39,7 +39,7 @@ clip_model = CLIPModel.from_pretrained("laion/CLIP-ViT-B-32-laion2B-s34B-b79K",


 guided_pipeline = DiffusionPipeline.from_pretrained(
-    "CompVis/stable-diffusion-v1-4",
+    "runwayml/stable-diffusion-v1-5",


@patil-suraj do you want to change this?

we can change this, it should be fine here.

patrickvonplaten · 2022-10-21T10:41:37Z

examples/community/README.md

@@ -202,7 +202,7 @@ from diffusers import DiffusionPipeline
 import torch

 pipe = DiffusionPipeline.from_pretrained(
-    'CompVis/stable-diffusion-v1-4',
+    'runwayml/stable-diffusion-v1-5',


Let's maybe revert this as well as the community pipeline author should decide here

patrickvonplaten · 2022-10-21T10:41:44Z

examples/community/README.md

@@ -139,7 +139,7 @@ def download_image(url):
    response = requests.get(url)
    return PIL.Image.open(BytesIO(response.content)).convert("RGB")

-pipe = DiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", custom_pipeline="stable_diffusion_mega", torch_dtype=torch.float16, revision="fp16")
+pipe = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", custom_pipeline="stable_diffusion_mega", torch_dtype=torch.float16, revision="fp16")


patrickvonplaten · 2022-10-21T10:41:51Z

examples/community/README.md

@@ -97,7 +97,7 @@ from diffusers import DiffusionPipeline
 import torch

 pipe = DiffusionPipeline.from_pretrained(
-    "CompVis/stable-diffusion-v1-4",
+    "runwayml/stable-diffusion-v1-5",


Let's maybe revert this as well as the community pipeline author should decide here

patrickvonplaten · 2022-10-21T10:42:17Z

docs/source/using-diffusers/custom_pipelines.mdx

@@ -58,7 +58,7 @@ feature_extractor = CLIPFeatureExtractor.from_pretrained(clip_model_id)
 clip_model = CLIPModel.from_pretrained(clip_model_id)

 pipeline = DiffusionPipeline.from_pretrained(
-    "CompVis/stable-diffusion-v1-4",
+    "runwayml/stable-diffusion-v1-5",


@patil-suraj should we change or leave this?

okay to chnage this!

patrickvonplaten · 2022-10-21T10:43:10Z

docs/source/training/text_inversion.mdx

@@ -64,7 +64,7 @@ accelerate config

 ### Cat toy example

-You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/CompVis/stable-diffusion-v1-4), read the license and tick the checkbox if you agree. 
+You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree.


Let's revert this until we've tested it with v1-5

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

Co-authored-by: Suraj Patil <surajp815@gmail.com>

… into v1-5-updates

apolinario · 2022-10-21T10:53:08Z

Thank you so much for reviewing! I think I acted on all the reversals discussed!

README.md

patrickvonplaten

Thanks a lot @apolinario !

* Update README.md Additionally add FLAX so the model card can be slimmer and point to this page * Find and replace all * v-1-5 -> v1-5 * revert test changes * Update README.md Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * Update docs/source/quicktour.mdx Co-authored-by: Pedro Cuenca <pedro@huggingface.co> * Update README.md Co-authored-by: Pedro Cuenca <pedro@huggingface.co> * Update docs/source/quicktour.mdx Co-authored-by: Pedro Cuenca <pedro@huggingface.co> * Update README.md Co-authored-by: Suraj Patil <surajp815@gmail.com> * Revert certain references to v1-5 * Docs changes * Apply suggestions from code review Co-authored-by: apolinario <joaopaulo.passos+multimodal@gmail.com> Co-authored-by: anton-l <anton@huggingface.co> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> Co-authored-by: Pedro Cuenca <pedro@huggingface.co> Co-authored-by: Suraj Patil <surajp815@gmail.com>

Update README.md

118b5e4

Additionally add FLAX so the model card can be slimmer and point to this page

apolinario and others added 2 commits October 20, 2022 10:13

Find and replace all

7778c2b

v-1-5 -> v1-5

355746f

anton-l changed the title ~~v1-5 updates~~ v1-5 docs updates Oct 20, 2022

anton-l marked this pull request as ready for review October 20, 2022 10:26

revert test changes

af65886

pcuenca approved these changes Oct 20, 2022

View reviewed changes

patrickvonplaten approved these changes Oct 20, 2022

View reviewed changes

patrickvonplaten reviewed Oct 20, 2022

View reviewed changes

README.md Outdated Show resolved Hide resolved

patrickvonplaten suggested changes Oct 20, 2022

View reviewed changes

patil-suraj approved these changes Oct 21, 2022

View reviewed changes

patrickvonplaten reviewed Oct 21, 2022

View reviewed changes

apolinario and others added 8 commits October 21, 2022 12:47

Update README.md

3d31560

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

Update docs/source/quicktour.mdx

35086d2

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

Update README.md

6a0eb1c

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

Update docs/source/quicktour.mdx

e943d9c

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

Update README.md

a2ff989

Co-authored-by: Suraj Patil <surajp815@gmail.com>

Revert certain references to v1-5

6f0d34e

Merge branch 'v1-5-updates' of https://github.com/huggingface/diffusers…

0af885c

… into v1-5-updates

Docs changes

f48fc0d

patrickvonplaten reviewed Oct 24, 2022

View reviewed changes

README.md Outdated Show resolved Hide resolved

Apply suggestions from code review

a77496b

patrickvonplaten approved these changes Oct 24, 2022

View reviewed changes

patrickvonplaten merged commit 8aac1f9 into main Oct 24, 2022

patrickvonplaten deleted the v1-5-updates branch October 24, 2022 20:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v1-5 docs updates #921

v1-5 docs updates #921

apolinario commented Oct 20, 2022

HuggingFaceDocBuilderDev commented Oct 20, 2022 •

edited

Loading

pcuenca left a comment

pcuenca Oct 20, 2022

pcuenca Oct 20, 2022

pcuenca Oct 20, 2022

pcuenca Oct 20, 2022

patrickvonplaten left a comment

patrickvonplaten Oct 20, 2022

patrickvonplaten left a comment

patil-suraj left a comment

patil-suraj Oct 21, 2022

apolinario Oct 21, 2022 •

edited

Loading

patrickvonplaten Oct 21, 2022

patrickvonplaten Oct 21, 2022

patil-suraj Oct 21, 2022

patrickvonplaten Oct 21, 2022

patil-suraj Oct 21, 2022

patrickvonplaten Oct 21, 2022

patrickvonplaten Oct 21, 2022

patrickvonplaten Oct 21, 2022

patil-suraj Oct 21, 2022

patrickvonplaten Oct 21, 2022

patrickvonplaten Oct 21, 2022

patrickvonplaten Oct 21, 2022

patrickvonplaten Oct 21, 2022

patil-suraj Oct 21, 2022

patrickvonplaten Oct 21, 2022

patil-suraj Oct 21, 2022

apolinario commented Oct 21, 2022

patrickvonplaten left a comment


		Stable Diffusion is a text-to-image latent diffusion model created by the researchers and engineers from [CompVis](https://github.com/CompVis), [Stability AI](https://stability.ai/) and [LAION](https://laion.ai/). It's trained on 512x512 images from a subset of the [LAION-5B](https://laion.ai/blog/laion-5b/) database. This model uses a frozen CLIP ViT-L/14 text encoder to condition the model on text prompts. With its 860M UNet and 123M text encoder, the model is relatively lightweight and runs on a GPU with at least 10GB VRAM.
		Stable Diffusion is a text-to-image latent diffusion model created by the researchers and engineers from [CompVis](https://github.com/CompVis), [Stability AI](https://stability.ai/), [LAION](https://laion.ai/) and [RunwayML](https://runwayml.com/). It's trained on 512x512 images from a subset of the [LAION-5B](https://laion.ai/blog/laion-5b/) database. This model uses a frozen CLIP ViT-L/14 text encoder to condition the model on text prompts. With its 860M UNet and 123M text encoder, the model is relatively lightweight and runs on a GPU with at least 10GB VRAM.

	You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-4`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree.
	You need to accept the model license before downloading or using the weights. In this example we'll use model version `v1-5`, so you'll need to visit [its card](https://huggingface.co/runwayml/stable-diffusion-v1-5), read the license and tick the checkbox if you agree.

v1-5 docs updates #921

v1-5 docs updates #921

Conversation

apolinario commented Oct 20, 2022

HuggingFaceDocBuilderDev commented Oct 20, 2022 • edited Loading

pcuenca left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

patrickvonplaten left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

patrickvonplaten left a comment

Choose a reason for hiding this comment

patil-suraj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apolinario Oct 21, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apolinario commented Oct 21, 2022

patrickvonplaten left a comment

Choose a reason for hiding this comment

HuggingFaceDocBuilderDev commented Oct 20, 2022 •

edited

Loading

apolinario Oct 21, 2022 •

edited

Loading