this training process did not consider decoder_attention_mask?

[zlh1992]

I see that :

def model_forward(model, inputs):
h = inputs
h = h.to(model.base_model.model.model.embed_tokens.weight.device)
h = model.base_model.model.model.embed_tokens(h)
for layer in model.base_model.model.model.layers:
h = h.to(layer.input_layernorm.weight.device)
h = layer(h)[0]
h = h.to(model.base_model.model.model.norm.weight.device)
h = model.base_model.model.model.norm(h)
h = model.base_model.model.lm_head(h)
return h

the output of this model comes from all sequence?

Maybe you need add _prepare_decoder_attention_mask(h) to avoid this...