Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add (Scheduled) Pseudo-Huber Loss training scripts to research projects #7527

Merged
merged 21 commits into from
Apr 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
61e90e5
add scheduled pseudo-huber loss training scripts
kabachuha Mar 30, 2024
e8aec5f
add reduction modes to huber loss
kabachuha Mar 31, 2024
f1ae40f
Merge branch 'main' into huber-losses
sayakpaul Mar 31, 2024
fe19ec3
[DB Lora] *2 multiplier to huber loss cause of 1/2 a^2 conv.
kabachuha Apr 1, 2024
7c0ed59
[DB Lora] add option for smooth l1 (huber / delta)
kabachuha Apr 1, 2024
f56319e
[DB Lora] unify huber scheduling
kabachuha Apr 1, 2024
d354b2b
[DB Lora] add snr huber scheduler
kabachuha Apr 1, 2024
48b40a9
fixup examples link
kabachuha Apr 1, 2024
58fafab
Merge branch 'main' into huber-losses
sayakpaul Apr 2, 2024
c12dc96
use snr schedule by default in DB
kabachuha Apr 7, 2024
292199a
update all huber scripts with snr
kabachuha Apr 8, 2024
24bad91
Merge branch 'huber-losses' of https://github.com/kabachuha/diffusers…
kabachuha Apr 8, 2024
93ced70
Merge remote-tracking branch 'upstream/main' into huber-losses
kabachuha Apr 8, 2024
c82b524
Merge branch 'main' into huber-losses
sayakpaul Apr 9, 2024
e567e87
code quality
kabachuha Apr 9, 2024
91ea34d
Merge branch 'huber-losses' of https://github.com/kabachuha/diffusers…
kabachuha Apr 9, 2024
11c11a7
Merge branch 'main' into huber-losses
kabachuha Apr 10, 2024
7b133bd
Merge branch 'main' into huber-losses
sayakpaul Apr 11, 2024
c8e69ce
Merge branch 'main' into huber-losses
sayakpaul Apr 12, 2024
5c64eee
huber: make style && make quality
kabachuha Apr 12, 2024
3cc2207
Merge branch 'main' into huber-losses
kabachuha Apr 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions examples/research_projects/scheduled_huber_loss_training/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Scheduled Pseudo-Huber Loss for Diffusers

These are the modifications of to include the possibility of training text2image models with Scheduled Pseudo Huber loss, introduced in https://arxiv.org/abs/2403.16728. (https://github.com/kabachuha/SPHL-for-stable-diffusion)

## Why this might be useful?

- If you suspect that the part of the training dataset might be corrupted, and you don't want these outliers to distort the model's supposed output

- If you want to improve the aesthetic quality of pictures by helping the model disentangle concepts and be less influenced by another sorts of pictures.

See https://github.com/huggingface/diffusers/issues/7488 for the detailed description.

## Instructions

The same usage as in the case of the corresponding vanilla Diffusers scripts https://github.com/huggingface/diffusers/tree/main/examples
Loading
Loading