From 94162beb9f454403d68ec009bb5572ee560d7603 Mon Sep 17 00:00:00 2001 From: Jiaxin Shan Date: Tue, 16 Jul 2024 10:11:04 -0700 Subject: [PATCH] [Doc] Fix the lora adapter path in server startup script (#6230) --- docs/source/models/lora.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/source/models/lora.rst b/docs/source/models/lora.rst index 934887a607a6a..5cc3076073fbd 100644 --- a/docs/source/models/lora.rst +++ b/docs/source/models/lora.rst @@ -64,7 +64,10 @@ LoRA adapted models can also be served with the Open-AI compatible vLLM server. python -m vllm.entrypoints.openai.api_server \ --model meta-llama/Llama-2-7b-hf \ --enable-lora \ - --lora-modules sql-lora=~/.cache/huggingface/hub/models--yard1--llama-2-7b-sql-lora-test/ + --lora-modules sql-lora=$HOME/.cache/huggingface/hub/models--yard1--llama-2-7b-sql-lora-test/snapshots/0dfa347e8877a4d4ed19ee56c140fa518470028c/ + +.. note:: + The commit ID `0dfa347e8877a4d4ed19ee56c140fa518470028c` may change over time. Please check the latest commit ID in your environment to ensure you are using the correct one. The server entrypoint accepts all other LoRA configuration parameters (``max_loras``, ``max_lora_rank``, ``max_cpu_loras``, etc.), which will apply to all forthcoming requests. Upon querying the ``/models`` endpoint, we should see our LoRA along