[Feature]Modification of the finetune mechanism #177
Conversation
…ing string; adjust config; adjust trainer and context; create optimizer in the beginning of a routine
# Conflicts: # federatedscope/core/trainers/trainer.py
| @@ -0,0 +1,20 @@ | |||
| def use_diff(func): | |||
There was a problem hiding this comment.
@rayrayraykk Please help me check this modification. Make sure it is consistent with my original implementation. Thanks!
There was a problem hiding this comment.
This part looks good to me.
There was a problem hiding this comment.
Good job! It might help us to split the functionalities of the trainer. And @yxdyc can double-check the details of the trainer.
Please note that the modification of config (such as cfg.optimizer -> cfg.traine.optimizer) would affect a large range of existing files, and we should make sure the provided examples and the declarations have been updated.
| g['lr'] = cfg.personalization.lr | ||
| ctx.pFedMe_outer_lr = cfg.optimizer.lr | ||
| g['lr'] = ctx.cfg.personalization.lr | ||
| ctx.pFedMe_outer_lr = ctx.cfg.train.optimizer.lr |
There was a problem hiding this comment.
Shall we have ctx.cfg.xxx ? @yxdyc
There was a problem hiding this comment.
Both are ok, as we don't usually change cfg.personalization.lr during the FL courses
|
LGTM. The modification of configs can invalid the scripts of our three FL benchmarks. We can discuss this later that whether we need to make our three benchmarks compatible with FS 0.2.0. |
# Conflicts: # federatedscope/core/auxiliaries/utils.py # federatedscope/core/configs/cfg_fl_algo.py # federatedscope/core/configs/cfg_fl_setting.py # federatedscope/core/configs/cfg_hpo.py # federatedscope/core/configs/cfg_training.py # federatedscope/core/trainers/context.py # federatedscope/core/trainers/torch_trainer.py # federatedscope/core/trainers/trainer.py # federatedscope/core/worker/client.py
|
LGTM, but please rebase the master to request a new CI-test for format checks, thanks. |
modified accordingly |
This PR is mainly for the modification of the finetune mechansim (#148 ), but we also make small change for other functions as following
Finetune
cfg.federateintocfg.trainas they are more relevant to the training, includinglocal_update_stepbatch_or_epochcfg.finetuneandcfg.trainin the config to support different parameters for finetuning and training (e.g.optimizer.lr)finetunefunction in the basic trainerbenchmark)Enums and Decorators
enums.pyto avoid using string and the inconsistency issuesdecorators.pyto keep the code cleanOptimizer
ctx.optimizerin the beginning of the routine function rather than within thecontextto solve The optimizer may track some values across different training routines #136To be discussed
@joneswong please check if the following modifications are appropriate
use_diffis implemented by a decoratoruse_diff.