[Chatllama] Should I add a <end_of_text> at the end of sentence?

[bino282]

When I train a actor model with bloom-560M. I realize that , the model generates the text repeated at the end. The model always generates enough words predefined by max_length but it;s not stop early. Should I add a <end_of_text> at the end of sentence when training?

[PierpaoloSorbellini]

@bino282 Yes, with HF models the text is repeated, we are aware of the problem and will be releasing a patch for this very soon. We will get back to you as soon as possible.

[allaccs]

I am having the same problem of repeated text, what is the current workaround?

[PierpaoloSorbellini]

Hi @bino282 @allaccs
Yes we have the same issue too with some HF models.
currently we have tried to add a EOS to each sequence to make the model understand where to put this token,
and added some parameters to generate function of the actor form HF that should help in removing the repetition.
The last version is in the PR #306 please refer to that and contact me again if you notice that the problem persist.
thanks for feedback!