Refactor of models and trainers with base class for common methods #306
Conversation
PierpaoloSorbellini
commented
Mar 27, 2023
•
- Refactor models and trainers to avoid code replication.
- Added logs with loguru package.
- Fix logs with MultiGPU trainers.
- Added support for LoRA with PEFT library.
- Added support for load_8bit option with HF models.
- Added self-instruct dataset of HF.
- Added CerebrasGPT and Decapoda LLaMA models from HF.
- Added mixed-precision training to reduce GPU memory requirements.
- Fixed RLHF KL divergence equation.
- Added support to keep only the last n checkpoints for all training.
- Added generation of negative examples when creating the reward dataset to improve the quality of the reward model.
- Improved stability of MultiGPU training with both Accelerate form HF and DeepSpeed.
This reverts commit 156fa19.
| # pytorch mixed precison | ||
| with torch.autocast( | ||
| device_type=self.config.device_type, | ||
| dtype=torch.float16, |
There was a problem hiding this comment.
do we need to auto-cast to fp16 all the tensors? Shouldn't this be a config param?
There was a problem hiding this comment.
just following documentation...
https://pytorch.org/docs/stable/notes/amp_examples.html
wrt to casting manually the tensors, this is better with less problem with types in the embedding.
It is not a config param because if you do not use fp16 you would use fp32 and is probably worse.
not seen the point of adding the option for fp32.
There was a problem hiding this comment.
But what if I want to train the model in fp32 precision? (DeepSpeed for instance allows the user to select the precision)
|
@PierpaoloSorbellini please add a description of what this PR is adding in terms of features and which bugs it is fixing. |