We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The output of `python collect_env.py`
start vllm with env export VLLM_USE_MODELSCOPE=True, got errors:
export VLLM_USE_MODELSCOPE=True
INFO 07-24 08:44:25 model_runner.py:680] Starting to load model LLM-Research/Meta-Llama-3.1-8B-Instruct... [rank0]: Traceback (most recent call last): [rank0]: File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main [rank0]: return _run_code(code, main_globals, None, [rank0]: File "/usr/lib/python3.10/runpy.py", line 86, in _run_code [rank0]: exec(code, run_globals) [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 317, in <module> [rank0]: run_server(args) [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 231, in run_server [rank0]: if llm_engine is not None else AsyncLLMEngine.from_engine_args( [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/engine/async_llm_engine.py", line 466, in from_engine_args [rank0]: engine = cls( [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/engine/async_llm_engine.py", line 380, in __init__ [rank0]: self.engine = self._init_engine(*args, **kwargs) [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/engine/async_llm_engine.py", line 547, in _init_engine [rank0]: return engine_class(*args, **kwargs) [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 251, in __init__ [rank0]: self.model_executor = executor_class( [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/executor/executor_base.py", line 47, in __init__ [rank0]: self._init_executor() [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/executor/gpu_executor.py", line 36, in _init_executor [rank0]: self.driver_worker.load_model() [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker.py", line 139, in load_model [rank0]: self.model_runner.load_model() [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 682, in load_model [rank0]: self.model = get_model(model_config=self.model_config, [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/__init__.py", line 21, in get_model [rank0]: return loader.load_model(model_config=model_config, [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/loader.py", line 284, in load_model [rank0]: self._get_weights_iterator(model_config.model, [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/loader.py", line 245, in _get_weights_iterator [rank0]: hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/loader.py", line 177, in _prepare_weights [rank0]: model_name_or_path = self._maybe_download_from_modelscope( [rank0]: File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/loader.py", line 159, in _maybe_download_from_modelscope [rank0]: model_path = snapshot_download( [rank0]: TypeError: snapshot_download() got an unexpected keyword argument 'ignore_patterns'
vllm try to use args
vllm/vllm/model_executor/model_loader/loader.py
Line 164 in ccc4a73
but modelscope provides https://github.com/modelscope/modelscope/blob/210ab40c546b67ba2de81f8e5fd4db43e9724d91/modelscope/hub/snapshot_download.py#L31
The text was updated successfully, but these errors were encountered:
Successfully merging a pull request may close this issue.
Your current environment
🐛 Describe the bug
start vllm with env
export VLLM_USE_MODELSCOPE=True
, got errors:vllm try to use args
vllm/vllm/model_executor/model_loader/loader.py
Line 164 in ccc4a73
but modelscope provides https://github.com/modelscope/modelscope/blob/210ab40c546b67ba2de81f8e5fd4db43e9724d91/modelscope/hub/snapshot_download.py#L31
The text was updated successfully, but these errors were encountered: