[Model] Add support for Qwen2 for embeddings

[mgoin]

FIX #5600
FIX #5827
FIX #6015

This works for ssmits/Qwen2-7B-Instruct-embed-base

>>> from vllm import LLM
>>> model = LLM("ssmits/Qwen2-7B-Instruct-embed-base")
>>> outputs = model.encode("Hello!")
Processed prompts: 100%|█████████████████████████████████████| 1/1 [00:00<00:00,  2.10it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
>>> outputs
[EmbeddingRequestOutput(request_id='0', outputs=EmbeddingOutput(embedding=3584), prompt_token_ids=[9707, 0], finished=True)]

However, this doesn't work for Alibaba-NLP/gte-Qwen2-7B-instruct since its config still says it is a Qwen2ForCausalLM, where we have been relying on embedding models to be of the type XModel: https://huggingface.co/Alibaba-NLP/gte-Qwen2-7B-instruct/blob/main/config.json

[prattcmp]

+1 this would be great

[waters222]

I think ppl can edit the config file to change it from Qwen2ForCausalLM to Qwen2Model

[Semihal]

Should we expect a merge in the near future?

[mgoin]

@Semihal thanks for the reminder! I just merged with main so if it is green we can land

[0xWelt]

Can you pass "pytest tests/models/test_embedding.py"? I got "AssertionError: Not all values are within 0.01 of 1.0" with model "ssmits/Qwen2-7B-Instruct-embed-base" .

[mgoin]

@Nickydusk No it does not pass, it seems there is a correctness issue between the implementations so this is not ready to land. This is low on my priority to investigate so if anyone would like to take over this PR, feel welcome and ping me for review.

[starmemda]

@youkaichao @WoosukKwon @zhuohan123 @simon-mo @DarkLight1337
Could you please check this PR and merge it? gte-Qwen2 is a Qwen2ForCausalLM embedding model which is quite efficient.

[waters222]

need to add following line for loading model Alibaba-NLP/gte-Qwen2-7B-instruct

if "lm_head.weight" in name:
  continue

[0xWelt]

@mgoin @prattcmp @waters222 @Semihal @starmemda

I have opened a new PR #6282 to directly support 'gte-Qwen2' embedding models without modifying its Qwen2ForCausalLM architecture. We can collaborate on reviewing the code and merge it in the near future.

[ybbz]

Can this feature be merged?

[mgoin]

@ybbz The output is not correct/matching HF, as seen in the test. Anybody is welcome to debug it!

[github-actions]

This pull request has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this pull request should remain open. Thank you!

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. @mgoin please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[DarkLight1337]

Ping in case you forgot about this @mgoin . It should be quite straightforward to add embedding models now.

[DarkLight1337]

Closing as superseded by #10184