BEYOND DIALOGUE: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model
📄 Paper · 🗂️ Dataset · 🤗 Models · 🏆 Evaluation
We introduce BEYOND DIALOGUE, a novel framework designed to revolutionize role-playing model training by addressing key challenges in current approaches. Traditional methods that rely on predefined role profiles, which often lead to inconsistencies and biases between predefined profiles and scenario dialogues, BEYOND DIALOGUE introduces a unique approach by aligning dialogues with role profile traits specific to each scenario. This approach ensures fine-grained profile-dialogue alignment at the sentence level, fully automated and cost-effective. Our framework outperforms existing baselines in adhering to various role profile dimensions. For more details, please refer to the paper.
- [2024/08/30] Our models are released.
- [2024/08/29] Our dataset construction and evaluation
codeare released. - [2024/08/29] Our dataset is released.
- [2024/08/22] Our paper is released.
- Using a predefined role profile to prompt dialogue training for specific scenarios usually leads to inconsistencies and even conflicts between the dialogue and the profile, resulting in training biases.
- The model learns to imitate the role based solely on the profile, neglecting profile-dialogue alignment at the sentence level.
- We use an innovative prompting mechanism in GPT-4o to generate fine-grained CSERP alignment tasks as "beyond dialogue" training data. This approach ensures detailed alignment between profiles and dialogues, enhancing the model’s reasoning capabilities and adherence to profiles.
- Taking inspiration from actors learning to play different roles -- understanding the performance of various role traits in scenarios to enhance their portrayal -- we also employ fine-grained alignment tasks to train the role-playing model.
- The left side shows the training phases, which include role-playing dialogue, chit-chat, and profile alignment. The profile alignment results are utilized to adjust each scenario’s dialogue profiles, eliminating training biases.
- On the right, the LLM generates random scenarios and roles for multi-turn dialogues with the model, followed by an evaluation using objective questions (such as multiple-choice questions, judgmental questions) to obtain quantitative metrics of the model’s role-playing capabilities.
-
For more information about the dataset construction process and detailed statistics, please refer to our paper and code
-
The constructed dataset is available in the Hugging Face Datasets repository.
We use objective questions to assess eight dimensions: Character, Style, Emotion, Relationship, Personality, Human-likeness, Coherence, and Role Consistency.
-
Automated Dialogue Generation:
- Generate a role and its description aligned with its worldview.
- Create a dialogue scenario based on role profiles, design emotions, and define role relationships.
- Engage two models in multi-turn dialogues to produce a dialogue corpus.
-
Evaluation Approach:
- CSERP: Evaluate dialogues based on the five alignment dimensions (Character, Style, Emotion, Relationship, Personality) that are consistent with the alignment tasks used in our study.
- Human-likeness: Evaluate if outputs match human expression.
- Coherence: Assess dialogue continuity.
- Role-based Multiple-Choice: Measure role consistency across dialogues.
For more details, please refer to our evaluation code
Please cite our work if you found the resources in this repository useful:
@article{yu2024beyond,
title = {BEYOND DIALOGUE: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model},
author = {Yu, Yeyong and Yu, Runsheng and Wei, Haojie and Zhang, Zhanqiu and Qian, Quan},
year = {2024},
journal = {arXiv preprint arXiv:2408.10903},
}We would like to express our sincere gratitude to Tencent LightSpeed Studios for their invaluable support in this project. Their contributions and encouragement have been instrumental in the successful completion of our work.