About Hyper-paramters

[maple-zhou]

Hi, sorry to bother.
I really appreciate that this is a wonderful work, but I'm wondering how do you confirm the hyper-parameters of the distill.py?
e.g. lr_img, lr_lr, lr_teacher, max_start_epoch, and so on.
Do you use the grid search on the hyper parameters? If so, how do you implement it? Since I notice that there are a number of hyper-parameters needing to be set, I think it would take quite a few time to get a good setting.
Thanks!

[GeorgeCazenavette]

Hi, apologies for the late reply.

For the smaller experiments, we did a decently sized grid search. However, we could not afford to do this for the larger experiments.

I just ran a bunch of experiments and tracked them with wandb.

Ideally, the method should not be as sensitive to hparams. Hopefully future work can resolve this :)