Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Community tips #7137

Merged
merged 3 commits into from
Mar 7, 2024
Merged

[docs] Community tips #7137

merged 3 commits into from
Mar 7, 2024

Conversation

stevhliu
Copy link
Member

@stevhliu stevhliu commented Feb 28, 2024

This PR adds a callback for displaying image after each generation step.

@ultranity
Copy link
Contributor

Hi! Is there any detailed description or citation of negative noise, and example code of how to make it? Seems to be missing for now

@stevhliu
Copy link
Member Author

As pointed out by @asomoza, the negative noise feature is only documented here. This PR is still a WIP so that's why it is currently missing. This feature is not quite ready yet in Diffusers, so I'm going to remove it for now! There should be a discussion soon about it 🙂

@stevhliu stevhliu requested review from asomoza, sayakpaul and yiyixuxu and removed request for asomoza February 29, 2024 21:15
Copy link
Member

@asomoza asomoza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all looks good to me, thank you.

Comment on lines 71 to 76
image = pipe(
prompt = "A croissant shaped like a cute bear."
negative_prompt = "Deformed, ugly, bad anatomy"
callback_on_step_end=decode_tensors,
callback_on_step_end_tensor_inputs=["latents"],
).images[0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
image = pipe(
prompt = "A croissant shaped like a cute bear."
negative_prompt = "Deformed, ugly, bad anatomy"
callback_on_step_end=decode_tensors,
callback_on_step_end_tensor_inputs=["latents"],
).images[0]
image = pipe(
prompt="A croissant shaped like a cute bear."
negative_prompt="Deformed, ugly, bad anatomy"
callback_on_step_end=decode_tensors,
callback_on_step_end_tensor_inputs=["latents"],
).images[0]

Because I am a Pythonista :v

pipeline.load_lora_weights('.', weight_name="marcille.safetensors")
```

3. Load a scheduler and set `use_karras_sigmas=True` to use the DPM++ 2M Karras scheduler (take a look at this [scheduler table](../api/schedulers/overview.) to find the A1111 equivalent scheduler in Diffusers).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can directly link to DPMSolverMultistepScheduler here no?

Comment on lines 136 to 140
```py
pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config)
pipeline.scheduler.config.use_karras_sigmas=True
pipeline.to("cuda")
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
```py
pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config)
pipeline.scheduler.config.use_karras_sigmas=True
pipeline.to("cuda")
```
```py
pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config)
pipeline.scheduler.config.use_karras_sigmas = True
pipeline.to("cuda")

Copy link
Member

@sayakpaul sayakpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking solid. Thanks so much for putting this together.

@asomoza hat-tip 🫡

Copy link
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think about moving the callback examples to callback page? https://huggingface.co/docs/diffusers/main/en/using-diffusers/callback

we can create a section there called "community examples"?

the animte example we don't have to include in the doc right now, we can discuss a little bit more on how to organizing these contents

@sayakpaul
Copy link
Member

what do you think about moving the callback examples to callback page? https://huggingface.co/docs/diffusers/main/en/using-diffusers/callback

we can create a section there called "community examples"?

the animte example we don't have to include in the doc right now, we can discuss a little bit more on how to organizing these contents

Not all the tips here belong to the usage of callbacks no?

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yiyixuxu
Copy link
Collaborator

yiyixuxu commented Mar 1, 2024

@ultranity
it's here for now! #7167

is there any detailed description or citation of negative noise, and example code of how to make it?

@yiyixuxu
Copy link
Collaborator

yiyixuxu commented Mar 1, 2024

@sayakpaul

Not all the tips here belong to the usage of callbacks no?

yes yes -

I'm just not sure about the best way to organize these contents yet. I'm hoping to see a lot more of these cool community examples and I don't think we should have one page of a random collection of them

@stevhliu
Copy link
Member Author

stevhliu commented Mar 7, 2024

I just added the"display after each generation step" callback for now to our callback doc, and we can add the other features from @asomoza in a later PR :)

@stevhliu stevhliu requested a review from yiyixuxu March 7, 2024 22:13
Copy link
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ohh thanks!

@stevhliu stevhliu merged commit f6df16c into huggingface:main Mar 7, 2024
1 check passed
@stevhliu stevhliu deleted the tips branch March 7, 2024 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants