dpo metrics error

[yhhit]

Reminder

• I have read the README and searched the existing issues.

System Info

• llamafactory version: 0.9.1.dev0
• Platform: macOS-15.1-arm64-arm-64bit
• Python version: 3.12.7
• PyTorch version: 2.5.1
• Transformers version: 4.46.1
• Datasets version: 2.21.0
• Accelerate version: 1.0.1
• PEFT version: 0.12.0
• TRL version: 0.9.6
• Bitsandbytes version: 0.42.0

Reproduction

this problem in "LLaMA-Factory/src/llamafactory/train/dpo/trainer.py"
def get_batch_loss_metrics

# in def get_batch_loss_metrics
        metrics[f"{prefix}logps/rejected"] = policy_chosen_logps.mean().item()
        metrics[f"{prefix}logps/chosen"] = policy_rejected_logps.mean().item()
        metrics[f"{prefix}logits/rejected"] = policy_chosen_logits.mean().item()
        metrics[f"{prefix}logits/chosen"] = policy_rejected_logits.mean().item()

it should be

        metrics[f"{prefix}logps/rejected"] = policy_rejected_logps.mean().item()
        metrics[f"{prefix}logps/chosen"] = policy_chosen_logps.mean().item()
        metrics[f"{prefix}logits/rejected"] = policy_rejected_logits.mean().item()
        metrics[f"{prefix}logits/chosen"] = policy_chosen_logits.mean().item()

Expected behavior

No response

Others

No response

[hiyouga]

thanks for your reporting