[New Model]: Qwen2.5-Math-PRM-7B, Qwen2.5-Math-PRM-72B

[HaitaoWuTJU]

🚀 The feature, motivation and pitch

Qwen/Qwen2.5-Math-RM-72B has been supported.
New PRM model: Qwen2.5-Math-PRM-7B, Qwen2.5-Math-PRM-72B are not supported.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[HaitaoWuTJU]

Qwen2.5-Math-PRM-7B
Qwen2.5-Math-PRM-72B

[DarkLight1337]

Can you try passing --hf-overrides '{"architectures": ["Qwen2ForRewardModel"]}' to use the existing vLLM implementation?

[HaitaoWuTJU]

model = "Qwen/Qwen2.5-Math-PRM-7B"
task = 'embedding'
hf_overrides = {"architectures": ["Qwen2ForRewardModel"]}
llm = LLM(model=model, task=task, tensor_parallel_size=4, hf_overrides=hf_overrides)

[Isotr0py]

Seems that it's because of the num_labels in score layer, RM is 1, while PRM 2.

[DarkLight1337]

Unfortunately it doesn't seem like their config has num_labels field. Maybe we have to provide this through hf_overrides and read this config value in our model implementation.

[DarkLight1337]

Never mind, num_labels is defined correctly after loading AutoConfig.

Actually num_labels is set to 2 for both Qwen/Qwen2.5-Math-RM-72B and Qwen2.5-Math-PRM-7B, so we can't use this directly...