Skip to content

Commit

Permalink
fix for docs
Browse files Browse the repository at this point in the history
  • Loading branch information
winglian committed Jun 1, 2024
1 parent 2f28abc commit 7f5516e
Showing 1 changed file with 1 addition and 3 deletions.
4 changes: 1 addition & 3 deletions docs/source/en/trainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -436,9 +436,7 @@ trainer.train()

The Schedule Free optimizers have been introduced in [The Road Less Scheduled](https://hf.co/papers/2405.15682).
Schedule-Free learning replaces the momentum of the base optimizer with a combination of averaging and interpolation, to completely remove the need to anneal the learning rate with a traditional schedule.
Supported optimizers for SFO are `"schedule_free_adamw"` and `"schedule_free_sgd"`. First install schedulefree from pypi `pip install schedulefree`.

</Tip>
Supported optimizers for SFO are `"schedule_free_adamw"` and `"schedule_free_sgd"`. First install schedulefree from pypi `pip install schedulefree`.

Below is a simple script to demonstrate how to fine-tune [google/gemma-2b](https://huggingface.co/google/gemma-2b) on IMDB dataset in full precision:

Expand Down

0 comments on commit 7f5516e

Please sign in to comment.