Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 #9944

Merged
merged 10 commits into from
Nov 13, 2024
Prev Previous commit
Next Next commit
Fix documentation
Signed-off-by: FurtherAI <austin.veselka@lighton.ai>
  • Loading branch information
FurtherAI committed Nov 7, 2024
commit 433066703872dac32c72f26b837cbf43384d0c91
3 changes: 3 additions & 0 deletions docs/source/models/vlm.rst
Original file line number Diff line number Diff line change
Expand Up @@ -313,14 +313,17 @@ Since the request schema is not defined by OpenAI client, we post a request to t
Here is an example for serving the ``MrLight/dse-qwen2-2b-mrl-v1`` model.

.. code-block:: bash

vllm serve MrLight/dse-qwen2-2b-mrl-v1 --task embedding \
--trust-remote-code --max-model-len 8192 --chat-template examples/template_dse_qwen2_vl.jinja

.. important::

Like with VLM2Vec, we have to explicitly pass ``--task embedding``. Additionally, ``MrLight/dse-qwen2-2b-mrl-v1`` requires an EOS token for embeddings,
which is handled by the jinja template.

.. important::

Also important, ``MrLight/dse-qwen2-2b-mrl-v1`` requires a placeholder image of the minimum image size for text query embeddings. See the full code
example below for details.

Expand Down