How to train the adapter?

[sipie800]

I do some edition in the train.py, which involves your adapter model and adapter agent. But training can't launch.

w = bool_masked_pos.unsqueeze(-1).type_as(mask_token).reshape(-1, Hp, Wp, 1)
RuntimeError: shape '[-1, 84, 28, 1]' is invalid for input of size 1568

Does it just need more code because your code is incompleted?

As far as I'm concerned, the learnable image tensor means to replace the input prompt image and mask, is this true?
If that's true. We need to change the training dataloader so that it yields one single traning sample instead of paired samples?

[SteveImmanuel]

As far as I'm concerned, the learnable image tensor means to replace the input prompt image and mask, is this true?

Yes you are correct. The training dataloader indeed needs to be changed to only output a single image.
I forgot to implement such dataloader for this repository, I will update it when I get the chance.

For now, you can see my other repo for the implementation for reference
https://github.com/SteveImmanuel/OEM-Few-Shot-Learnable-Prompt/blob/1335420b65cdfe2acbca0d2ec338b5a08cc2020e/data.py#L608