Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Installation]: pip install vllm-0.6.2.zip err:setuptools-scm was unable to detect version for /tmp/pip-req-build-7ptioibj #9182

Closed
1 task done
uRENu opened this issue Oct 9, 2024 · 7 comments · Fixed by #11435
Labels
installation Installation problems

Comments

@uRENu
Copy link

uRENu commented Oct 9, 2024

Your current environment

pip install auto_gptq modelscope xformers==0.0.27.post2 torchvision==0.19 torchaudio torch==2.4.0 torchtext numpy wheel
pip install setuptools>=74.1.1 setuptools_scm==8.1.0 pyportfolioopt
pip install "pillow==10.*" -U
pip install sentencepiece charset_normalizer cpm_kernels tiktoken -U
pip install matplotlib scikit-learn tqdm tensorboard -U
pip install datasets huggingface-hub transformers==4.45.0 -U
pip install accelerate transformers_stream_generator -U
pip install pydantic==1.7.4 typer==0.3.0
pip install fastrlock cupy-cuda11x==12.1.0 
pip install vllm-flash-attn==2.6.1 datamodel_code_generator

How you are installing vllm

pip install vllm-0.6.2.zip

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
@uRENu uRENu added the installation Installation problems label Oct 9, 2024
@uRENu
Copy link
Author

uRENu commented Oct 9, 2024

err:

Processing vllm-0.6.2.zip
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [26 lines of output]
/tmp/pip-build-env-nr7os5qd/overlay/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:258: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:84.)
cpu = _conversion_method_template(device=torch.device("cpu"))
Traceback (most recent call last):
File "/home/wangrenzhong/anaconda3/envs/py310/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in
main()
File "/home/wangrenzhong/anaconda3/envs/py310/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/wangrenzhong/anaconda3/envs/py310/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-nr7os5qd/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
File "/tmp/pip-build-env-nr7os5qd/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 302, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-nr7os5qd/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 318, in run_setup
exec(code, locals())
File "", line 485, in
File "", line 357, in get_vllm_version
File "/tmp/pip-build-env-nr7os5qd/overlay/lib/python3.10/site-packages/setuptools_scm/_get_version_impl.py", line 158, in get_version
_version_missing(config)
File "/tmp/pip-build-env-nr7os5qd/overlay/lib/python3.10/site-packages/setuptools_scm/_get_version_impl.py", line 112, in _version_missing
raise LookupError(
LookupError: setuptools-scm was unable to detect version for /tmp/pip-req-build-caovnllw.

  Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.
  
  For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

@DarkLight1337
Copy link
Member

cc @dtrifiro

@dtrifiro
Copy link
Contributor

dtrifiro commented Oct 9, 2024

Are you trying to build from source? If so, the error is telling you what's wrong:

  Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.
  
  For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
  [end of output]

Which can be solved by installing from a git+https ref:

pip install git+https://github.com/vllm-project/vllm@v0.6.2

If you're not trying to build from source, I recommend installing the prebuilt wheel

@simon-mo
Copy link
Collaborator

simon-mo commented Oct 9, 2024

Hmm can we still make this workflow work? I would imagine there are folks trying to vendor vLLM without git.

@dtrifiro
Copy link
Contributor

dtrifiro commented Oct 10, 2024

@simon-mo

The right way to provide a tarball, is building a source distribution:

python setup.py sdist
# or, if using modern tools:
python -m build --sdist

this will create a tar archive which can be built. It will look something like this: vllm-0.6.3.dev146+g9b064faa6.d20241010.tar.gz

Another approach would be hacking setup.py to ignore the LookupError:

    try:
        version = get_version(
        write_to="vllm/_version.py",  # TODO: move this to pyproject.toml
    )
    except LookupError:
        version = "0.0.0"

Although this is not ideal:

  • this is hacky
  • we lose any kind of versioning information
  • this would only work temporarily: I'm not sure if this can be done when moving this definition to pyproject.toml
  • I'm sure there's plenty other ways this could backfire

@simon-mo
Copy link
Collaborator

We do provide the tarball on PyPI

@paulcx
Copy link

paulcx commented Oct 16, 2024

@dtrifiro This issue exsited in 0.6.3 docker image.

16T06:22:59.319607711Z /usr/local/lib/python3.12/dist-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:

16T06:22:59.319678095Z No module named 'vllm._version'

16T06:22:59.319688347Z   from vllm.version import __version__ as VLLM_VERSION

BKitor pushed a commit to BKitor/vllm that referenced this issue Dec 30, 2024
hongxiayang pushed a commit to ROCm/vllm that referenced this issue Jan 15, 2025
* [Misc] Move weights mapper (vllm-project#11443)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Bugfix] Fix issues in CPU build Dockerfile. Fixes vllm-project#9182 (vllm-project#11435)

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

* [Model] Automatic conversion of classification and reward models (vllm-project#11469)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (vllm-project#11472)

* [Misc] Update disaggregation benchmark scripts and test logs (vllm-project#11456)

Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>

* [Frontend] Enable decord to load video from base64 (vllm-project#11492)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Improve GitHub links (vllm-project#11491)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Misc] Move some multimodal utils to modality-specific modules (vllm-project#11494)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* Mypy checking for vllm/compilation (vllm-project#11496)

Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>

* [Misc][LoRA] Fix LoRA weight mapper (vllm-project#11495)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Doc] Add `QVQ` and `QwQ` to the list of supported models (vllm-project#11509)

Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

* [V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (vllm-project#10681)

Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [Model]  Modify MolmoForCausalLM MLP  (vllm-project#11510)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Misc] Add placeholder module (vllm-project#11501)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Add video example to openai client for multimodal (vllm-project#11521)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [1/N] API Server  (Remove Proxy) (vllm-project#11529)

* [Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (vllm-project#11523)

Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>

* [2/N] API Server: Avoid ulimit footgun (vllm-project#11530)

* Deepseek v3 (vllm-project#11502)

Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>

* [Docs] Document Deepseek V3 support (vllm-project#11535)

Signed-off-by: simon-mo <simon.mo@hey.com>

* Update openai_compatible_server.md (vllm-project#11536)

Co-authored-by: Simon Mo <simon.mo@hey.com>

* [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (vllm-project#11394)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [V1] Fix yapf (vllm-project#11538)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [CI] Fix broken CI (vllm-project#11543)

* [misc] fix typing (vllm-project#11540)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (vllm-project#11534)

* [BugFix] Fix quantization for all other methods (vllm-project#11547)

* [Platform] Move model arch check to platform (vllm-project#11503)

Signed-off-by: Mengqing Cao <cmq0113@163.com>

* Update deploying_with_k8s.md with AMD ROCm GPU example (vllm-project#11465)

Signed-off-by: Alex He <alehe@amd.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [Bugfix] Fix TeleChat2ForCausalLM weights mapper (vllm-project#11546)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Misc] Abstract the logic for reading and writing media content (vllm-project#11527)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc]  Add xgrammar in doc (vllm-project#11549)

Signed-off-by: ccjincong <chenjincong11@gmail.com>

* [VLM] Support caching in merged multi-modal processor (vllm-project#11396)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [MODEL] LoRA support for Jamba model (vllm-project#11209)

Signed-off-by: Erez Schwartz <erezs@ai21.com>

* [Misc]Add BNB quantization for MolmoForCausalLM  (vllm-project#11551)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Misc] Improve BNB loader to handle mixture of sharded and merged weights with same suffix (vllm-project#11566)

Signed-off-by: Isotr0py <2037008807@qq.com>

* [Bugfix] Fix for ROCM compressed tensor support (vllm-project#11561)

* [Doc] Update mllama example based on official doc (vllm-project#11567)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* [V1] [4/N] API Server: ZMQ/MP Utilities (vllm-project#11541)

* [Bugfix] Last token measurement fix (vllm-project#11376)

Signed-off-by: rajveerb <46040700+rajveerb@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

* [Model] Support InternLM2 Reward models (vllm-project#11571)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [Model] Remove hardcoded image tokens ids from Pixtral (vllm-project#11582)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [Hardware][AMD]: Replace HIPCC version with more precise ROCm version (vllm-project#11515)

Signed-off-by: hjwei <hjwei_xd@163.com>

* [V1][Minor] Set pin_memory=False for token_ids_cpu tensor (vllm-project#11581)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [Doc] Minor documentation fixes (vllm-project#11580)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [bugfix] interleaving sliding window for cohere2 model (vllm-project#11583)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [V1] [5/N] API Server: unify `Detokenizer` and  `EngineCore` input (vllm-project#11545)

Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

* [Doc] Convert list tables to MyST (vllm-project#11594)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [v1][bugfix] fix cudagraph with inplace buffer assignment (vllm-project#11596)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Misc] KV cache transfer connector registry (vllm-project#11481)

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

* Remove print statement in DeepseekScalingRotaryEmbedding (vllm-project#11604)

* [v1] fix compilation cache (vllm-project#11598)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Docker] bump up neuron sdk v2.21 (vllm-project#11593)

Signed-off-by: Liangfu Chen <liangfc@amazon.com>

* [Build][Kernel] Update CUTLASS to v3.6.0 (vllm-project#11607)

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

* [CI/Build][CPU] Fix CPU CI by lazy importing triton FP8 kernels (vllm-project#11618)

Signed-off-by: jiang1.li <jiang1.li@intel.com>

* [platforms] enable platform plugins (vllm-project#11602)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [VLM] Abstract out multi-modal data parsing in merged processor (vllm-project#11620)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [V1] [6/N] API Server: Better Shutdown (vllm-project#11586)

* [Bugfix] Validate and concatenate image embeddings in MiniCPMVBaseModel (vllm-project#11631)

* [benchmark] Remove dependency for H100 benchmark step (vllm-project#11572)

* [Model][LoRA]LoRA support added for MolmoForCausalLM (vllm-project#11439)

Signed-off-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

* [Bugfix] Fix OpenAI parallel sampling when using xgrammar (vllm-project#11637)

Signed-off-by: mgoin <michael@neuralmagic.com>

* [Misc][LoRA] Support Rank Stabilized LoRA (RSLoRA) (vllm-project#6909)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

* [Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to after the check for allocating new blocks. (vllm-project#11565)

* [V1] Simpify vision block hash for prefix caching by removing offset from hash (vllm-project#11646)

* [V1][VLM] V1 support for selected single-image models. (vllm-project#11632)

Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>

* [Benchmark] Add benchmark script for CPU offloading  (vllm-project#11533)

Signed-off-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: KuntaiDu <kuntai@uchicago.edu>

* [Bugfix][Refactor] Unify model management in frontend (vllm-project#11660)

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>

* [VLM] Add max-count checking in data parser for single image models (vllm-project#11661)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

* [Misc] Optimize Qwen2-VL LoRA test (vllm-project#11663)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Misc] Replace space with - in the file names (vllm-project#11667)

Signed-off-by: Lu Fang <lufang@fb.com>

* [Doc] Fix typo (vllm-project#11666)

Signed-off-by: Kazuhiro Serizawa <nserihiro@gmail.com>

* [V1] Implement Cascade Attention (vllm-project#11635)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [VLM] Move supported limits and max tokens to merged multi-modal processor (vllm-project#11669)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

* [VLM][Bugfix] Multi-modal processor compatible with V1 multi-input (vllm-project#11674)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [mypy] Pass type checking in vllm/inputs (vllm-project#11680)

Signed-off-by: Tobias Pitters <tobias.pitters@gmail.com>

* [VLM] Merged multi-modal processor for LLaVA-NeXT (vllm-project#11682)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* According to vllm.EngineArgs, the name should be distributed_executor_backend (vllm-project#11689)

* [Bugfix] Free cross attention block table for preempted-for-recompute sequence group. (vllm-project#10013)

Signed-off-by: Kathy Yu <feiyangyu@google.com>

* [V1][Minor] Optimize token_ids_cpu copy (vllm-project#11692)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [Bugfix] Change kv scaling factor by param json on nvidia gpu (vllm-project#11688)

Signed-off-by: bjmsong <bjmsong@126.com>
Co-authored-by: bjmsong <bjmsong@126.com>

* Resolve race conditions in Marlin kernel (vllm-project#11493)

Signed-off-by: wchen61 <wchen61@foxmail.com>

* [Misc] Minimum requirements for SageMaker compatibility (vllm-project#11576)

* Update default max_num_batch_tokens for chunked prefill (vllm-project#11694)

* [Bugfix] Check chain_speculative_sampling before calling it (vllm-project#11673)

Signed-off-by: Lu Fang <lufang@fb.com>

* [perf-benchmark] Fix dependency for steps in benchmark pipeline (vllm-project#11710)

* [Model] Whisper model implementation (vllm-project#11280)

Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>

* [V1] Simplify Shutdown (vllm-project#11659)

* [Bugfix] Fix ColumnParallelLinearWithLoRA slice (vllm-project#11708)

Signed-off-by: ZincCat <zincchloride@outlook.com>

* [V1] Improve TP>1 Error Handling + Stack Trace (vllm-project#11721)

Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

* [Misc]Add BNB quantization for Qwen2VL (vllm-project#11719)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

* Update requirements-tpu.txt to support python 3.9 and 3.11 (vllm-project#11695)

Signed-off-by: mgoin <michael@neuralmagic.com>

* [V1] Chore: cruft removal (vllm-project#11724)

* [V1] log GPU blocks num for MultiprocExecutor (vllm-project#11656)

* Update tool_calling.md (vllm-project#11701)

* Update bnb.md with example for OpenAI (vllm-project#11718)

* [V1] Add `RayExecutor` support for `AsyncLLM` (api server) (vllm-project#11712)

* [V1] Add kv cache utils tests. (vllm-project#11513)

Signed-off-by: xcnick <xcnick0412@gmail.com>

* [Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (vllm-project#11233)

Signed-off-by: Yan Burman <yanburman@users.noreply.github.com>
Signed-off-by: Ido Asraff <idoa@atero.ai>

* [VLM] Merged multi-modal processors for LLaVA-NeXT-Video and LLaVA-OneVision (vllm-project#11717)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Bugfix] Fix precision error in LLaVA-NeXT (vllm-project#11735)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Model] Remove unnecessary weight initialization logic (vllm-project#11736)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

* [Bugfix][V1] Fix test_kv_cache_utils.py (vllm-project#11738)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [MISC] Replace c10::optional with std::optional (vllm-project#11730)

Signed-off-by: Lu Fang <lufang@fb.com>

* [distributed] remove pynccl's redundant stream (vllm-project#11744)

* fix: [doc] fix typo (vllm-project#11751)

Co-authored-by: Lancer <maruixiang6688@gmail.com>

* [Frontend] Improve `StreamingResponse` Exception Handling (vllm-project#11752)

* [distributed] remove pynccl's redundant change_state (vllm-project#11749)

* [Doc] [1/N] Reorganize Getting Started section (vllm-project#11645)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Bugfix] Remove block size constraint (vllm-project#11723)

* [V1] Add BlockTable class (vllm-project#11693)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [Misc] Fix typo for valid_tool_parses  (vllm-project#11753)

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>

* [V1] Refactor get_executor_cls (vllm-project#11754)

* [mypy] Forward pass function type hints in lora (vllm-project#11740)

Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>

* k8s-config: Update the secret to use stringData (vllm-project#11679)

Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com>

* [VLM] Separate out profiling-related logic (vllm-project#11746)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc][2/N] Reorganize Models and Usage sections (vllm-project#11755)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Bugfix] Fix max image size for LLaVA-Onevision (vllm-project#11769)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [doc] explain how to add interleaving sliding window support (vllm-project#11771)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Bugfix][V1] Fix molmo text-only inputs (vllm-project#11676)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Kernel] Move attn_type to Attention.__init__() (vllm-project#11690)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* format

* [V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (vllm-project#11685)

Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

* deepseek overflow fix (#349)

* [Bugfix] Fix LLaVA-NeXT feature size precision error (for real) (vllm-project#11772)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Model] Future-proof Qwen2-Audio multi-modal processor (vllm-project#11776)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [XPU] Make pp group initilized for pipeline-parallelism (vllm-project#11648)

Signed-off-by: yisheng <yi.sheng@intel.com>

* [Doc][3/N] Reorganize Serving section (vllm-project#11766)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Kernel][LoRA]Punica prefill  kernels fusion (vllm-project#11234)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Abatom <abzhonghua@gmail.com>
Co-authored-by: Zhonghua Deng <abatom@163.com>

* [Bugfix] Update attention interface in `Whisper` (vllm-project#11784)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [CI] Fix neuron CI and run offline tests (vllm-project#11779)

Signed-off-by: Liangfu Chen <liangfc@amazon.com>

* fix init error for MessageQueue when n_local_reader is zero (vllm-project#11768)

* [Doc] Create a vulnerability management team (vllm-project#9925)

Signed-off-by: Russell Bryant <rbryant@redhat.com>

* [CI][CPU] adding build number to docker image name (vllm-project#11788)

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

* [V1][Doc] Update V1 support for `LLaVa-NeXT-Video` (vllm-project#11798)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [Bugfix] Comprehensively test and fix LLaVA-NeXT feature size calculation (vllm-project#11800)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [doc] add doc to explain how to use uv (vllm-project#11773)

Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [V1] Support audio language models on V1 (vllm-project#11733)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [doc] update how pip can install nightly wheels (vllm-project#11806)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Doc] Add note to `gte-Qwen2` models (vllm-project#11808)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [optimization] remove python function call for custom op (vllm-project#11750)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Bugfix] update the prefix for qwen2 (vllm-project#11795)

Co-authored-by: jiadi.jjd <jiadi.jjd@antgroup.com>

* [Doc]Add documentation for using EAGLE in vLLM (vllm-project#11417)

Signed-off-by: Sourashis Roy <sroy@roblox.com>

* [Bugfix] Significant performance drop on CPUs with --num-scheduler-steps > 1 (vllm-project#11794)

* [Doc] Group examples into categories (vllm-project#11782)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

* [Bugfix] Fix image input for Pixtral-HF (vllm-project#11741)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Misc] sort torch profiler table by kernel timing (vllm-project#11813)

* Remove the duplicate imports of MultiModalKwargs and PlaceholderRange… (vllm-project#11824)

* Fixed docker build for ppc64le (vllm-project#11518)

Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>

* [OpenVINO] Fixed Docker.openvino build (vllm-project#11732)

Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

* [Bugfix] Add checks for LoRA and CPU offload (vllm-project#11810)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Docs] reorganize sponsorship page (vllm-project#11639)

Signed-off-by: simon-mo <simon.mo@hey.com>

* [Bug] Fix pickling of `ModelConfig` when RunAI Model Streamer is used (vllm-project#11825)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [misc] improve memory profiling (vllm-project#11809)

Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [doc] update wheels url (vllm-project#11830)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Docs] Update sponsor name: 'Novita' to 'Novita AI' (vllm-project#11833)

* [Hardware][Apple] Native support for macOS Apple Silicon (vllm-project#11696)

Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

* [torch.compile] consider relevant code in compilation cache (vllm-project#11614)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [VLM] Reorganize profiling/processing-related code (vllm-project#11812)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Move examples into categories (vllm-project#11840)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

* [Doc][4/N] Reorganize API Reference (vllm-project#11843)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [CI/Build][Bugfix] Fix CPU CI image clean up (vllm-project#11836)

Signed-off-by: jiang1.li <jiang1.li@intel.com>

* [Bugfix][XPU] fix silu_and_mul (vllm-project#11823)

Signed-off-by: yan ma <yan.ma@intel.com>

* [Misc] Move some model utils into vision file (vllm-project#11848)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Expand Multimodal API Reference (vllm-project#11852)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Misc]add some explanations for BlockHashType (vllm-project#11847)

* [TPU][Quantization] TPU `W8A8` (vllm-project#11785)

Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [Kernel][Triton][AMD] Use block size heuristic for avg 2.8x speedup for int8 models (vllm-project#11698)

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

* [Docs] Add Google Cloud Meetup (vllm-project#11864)

* Revert nccl changes (#351)

* Revert "[distributed] remove pynccl's redundant change_state (vllm-project#11749)"

This reverts commit 9e764e7.

* Revert "[distributed] remove pynccl's redundant stream (vllm-project#11744)"

This reverts commit 635b897.

* [CI] Turn on basic correctness tests for V1 (vllm-project#10864)

* treat do_lower_case in the same way as the sentence-transformers library (vllm-project#11815)

Signed-off-by: Max de Bayser <mbayser@br.ibm.com>

* [Doc] Recommend uv and python 3.12 for quickstart guide (vllm-project#11849)

Signed-off-by: mgoin <michael@neuralmagic.com>

* [Misc] Move `print_*_once` from utils to logger (vllm-project#11298)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>

* [Doc] Intended links Python multiprocessing library (vllm-project#11878)

* [perf]fix current stream (vllm-project#11870)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Bugfix] Override dunder methods of placeholder modules (vllm-project#11882)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Bugfix] fix beam search input errors and latency benchmark script (vllm-project#11875)

Signed-off-by: Ye Qi <yeq@meta.com>
Co-authored-by: yeq <yeq@devgpu004.lla3.facebook.com>

* [Doc] Add model development API Reference (vllm-project#11884)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [platform] Allow platform specify attention backend (vllm-project#11609)

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>

* [ci]try to fix flaky multi-step tests (vllm-project#11894)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Misc] Provide correct Pixtral-HF chat template (vllm-project#11891)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* fp8 support (#352)

Co-authored-by: Yida Wu <yidawu@amd.com>

* [Docs] Add Modal to deployment frameworks (vllm-project#11907)

* [Doc][5/N] Move Community and API Reference to the bottom (vllm-project#11896)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Simon Mo <simon.mo@hey.com>

* [VLM] Enable tokenized inputs for merged multi-modal processor (vllm-project#11900)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Show default pooling method in a table (vllm-project#11904)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [torch.compile] Hide KV cache behind torch.compile boundary (vllm-project#11677)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* [Bugfix] Validate lora adapters to avoid crashing server (vllm-project#11727)

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

* [BUGFIX] Fix `UnspecifiedPlatform` package name (vllm-project#11916)

Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>

* [ci] fix gh200 tests (vllm-project#11919)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [misc] remove python function call for custom activation op (vllm-project#11885)

Co-authored-by: youkaichao <youkaichao@gmail.com>

* [platform] support pytorch custom op pluggable (vllm-project#11328)

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

* Replace "online inference" with "online serving" (vllm-project#11923)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

* [ci] Fix sampler tests (vllm-project#11922)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Doc] [1/N] Initial guide for merged multi-modal processor (vllm-project#11925)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [platform] support custom torch.compile backend key (vllm-project#11318)

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

* [Doc] Rename offline inference examples (vllm-project#11927)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

* [Docs] Fix docstring in `get_ip` function (vllm-project#11932)

Signed-off-by: Kuntai Du <kuntai@uchicago.edu>

* Doc fix in `benchmark_long_document_qa_throughput.py` (vllm-project#11933)

Signed-off-by: Kuntai Du <kuntai@uchicago.edu>

* [Hardware][CPU] Support MOE models on x86 CPU (vllm-project#11831)

Signed-off-by: jiang1.li <jiang1.li@intel.com>

* [Misc] Clean up debug code in Deepseek-V3 (vllm-project#11930)

Signed-off-by: Isotr0py <2037008807@qq.com>

* [Misc] Update benchmark_prefix_caching.py fixed example usage (vllm-project#11920)

Signed-off-by: Ren MinMin <renmm6@chinaunicom.cn>
Co-authored-by: Ren MinMin <renmm6@chinaunicom.cn>

* [Bugfix] Check that number of images matches number of <|image|> tokens with mllama (vllm-project#11939)

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>

* [mypy] Fix mypy warnings in api_server.py (vllm-project#11941)

Signed-off-by: Fred Reiss <frreiss@us.ibm.com>

* [ci] fix broken distributed-tests-4-gpus (vllm-project#11937)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Bugfix][SpecDecode] Adjust Eagle model architecture to align with intended design (vllm-project#11672)

Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>

* [Bugfix] fused_experts_impl wrong compute type for float32 (vllm-project#11921)

Signed-off-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Co-authored-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>

* [CI/Build] Move model-specific multi-modal processing tests (vllm-project#11934)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Basic guide for writing unit tests for new models (vllm-project#11951)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Bugfix] Fix RobertaModel loading (vllm-project#11940)

Signed-off-by: NickLucche <nlucches@redhat.com>

* [Model] Add cogagent model support vLLM (vllm-project#11742)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

* [V1] Avoid sending text prompt to core engine (vllm-project#11963)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [CI/Build] Add markdown linter (vllm-project#11857)

Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>

* [Model] Initialize support for Deepseek-VL2 models (vllm-project#11578)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [Hardware][CPU] Multi-LoRA implementation for the CPU backend (vllm-project#11100)

Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

* [Hardware][TPU] workaround fix for MoE on TPU (vllm-project#11764)

* [V1][Core][1/n] Logging and Metrics (vllm-project#11962)

Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

* [Model] Support GGUF models newly added in `transformers` 4.46.0 (vllm-project#9685)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [V1] [2/n] Logging and Metrics - `OutputProcessor` Abstraction (vllm-project#11973)

Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

* [MISC] fix typo in kv transfer send recv test (vllm-project#11983)

* [Bug] Fix usage of `.transpose()` and `.view()` consecutively. (vllm-project#11979)

* [CI][Spec Decode] fix: broken test for EAGLE model (vllm-project#11972)

Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>

* [Misc] Fix Deepseek V2 fp8 kv-scale remapping (vllm-project#11947)

Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>

* [Misc]Minor Changes about Worker (vllm-project#11555)

Signed-off-by: Chenguang Li <757486878@qq.com>

* [platform] add ray_device_key (vllm-project#11948)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* Fix Max Token ID for Qwen-VL-Chat (vllm-project#11980)

Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>

* [Kernel] unified_attention for Attention.forward (vllm-project#11967)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* [Doc][V1] Update model implementation guide for V1 support (vllm-project#11998)

Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

* [Doc] Organise installation documentation into categories and tabs (vllm-project#11935)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

* [platform] add device_control env var (vllm-project#12009)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Platform] Move get_punica_wrapper() function to Platform (vllm-project#11516)

Signed-off-by: Shanshan Shen <467638484@qq.com>

* bugfix: Fix signature mismatch in benchmark's `get_tokenizer` function (vllm-project#11982)

Signed-off-by: elijah <f1renze.142857@gmail.com>

* Using list

* Revert "[misc] improve memory profiling (vllm-project#11809)"

This reverts commit 889e662.

* Multi-lingual P3L (#356)

* Commiting the *multilingual* P3L test.

* Created a *multi-lingual* P3L test.

* Making ruff happy.

* .

* Added a reference to the language-scripture Confluence table.

* Typo fixing.

* Harmonizing naming.

* Fixing comments in the header.

---------

Co-authored-by: Alexei V. Ivanov <alivanov@banff-cyxtera-s65-4.amd.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>

* Trying to make scales work with compileable attention

* Docs lint

* linter formatting bug fixes

* inherit config file updates under fused_moe from main branch.

* match tests for the MOE layers with main.

---------

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: Alex He <alehe@amd.com>
Signed-off-by: ccjincong <chenjincong11@gmail.com>
Signed-off-by: Erez Schwartz <erezs@ai21.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: rajveerb <46040700+rajveerb@users.noreply.github.com>
Signed-off-by: hjwei <hjwei_xd@163.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Kazuhiro Serizawa <nserihiro@gmail.com>
Signed-off-by: Tobias Pitters <tobias.pitters@gmail.com>
Signed-off-by: Kathy Yu <feiyangyu@google.com>
Signed-off-by: bjmsong <bjmsong@126.com>
Signed-off-by: wchen61 <wchen61@foxmail.com>
Signed-off-by: ZincCat <zincchloride@outlook.com>
Signed-off-by: xcnick <xcnick0412@gmail.com>
Signed-off-by: Yan Burman <yanburman@users.noreply.github.com>
Signed-off-by: Ido Asraff <idoa@atero.ai>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com>
Signed-off-by: yisheng <yi.sheng@intel.com>
Signed-off-by: Abatom <abzhonghua@gmail.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>
Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Signed-off-by: yan ma <yan.ma@intel.com>
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Signed-off-by: Ye Qi <yeq@meta.com>
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
Signed-off-by: Ren MinMin <renmm6@chinaunicom.cn>
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Fred Reiss <frreiss@us.ibm.com>
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
Signed-off-by: Chenguang Li <757486878@qq.com>
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Signed-off-by: Shanshan Shen <467638484@qq.com>
Signed-off-by: elijah <f1renze.142857@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Yuan Tang <terrytangyuan@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com>
Co-authored-by: Jiaxin Shan <seedjeffwan@gmail.com>
Co-authored-by: Lucas Tucker <47258766+lucas-tucker@users.noreply.github.com>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: sroy745 <142070531+sroy745@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: AlexHe99 <alehe@amd.com>
Co-authored-by: Chen1022 <112855051+ccjincong@users.noreply.github.com>
Co-authored-by: ErezSC42 <erezs@ai21.com>
Co-authored-by: Selali <selali.adobor@gmail.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Rajveer Bachkaniwala <46040700+rajveerb@users.noreply.github.com>
Co-authored-by: hj-wei <hjwei_xd@163.com>
Co-authored-by: Kuntai Du <kuntai@uchicago.edu>
Co-authored-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
Co-authored-by: whyiug <whyiug@hotmail.com>
Co-authored-by: Kevin H. Luu <kevin@anyscale.com>
Co-authored-by: Matthias Vogler <60004995+ayylemao@users.noreply.github.com>
Co-authored-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Co-authored-by: John Giorgi <johnmgiorgi@gmail.com>
Co-authored-by: sakunkun <zhou.qianjun@zte.com.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Yihua Cheng <yihua98@uchicago.edu>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: Kazuhiro Serizawa <nserihiro@gmail.com>
Co-authored-by: Tobias Pitters <31857876+CloseChoice@users.noreply.github.com>
Co-authored-by: Chunyang Wen <chunyang.wen@gmail.com>
Co-authored-by: Kathy Yu <143133934+kathyyu-google@users.noreply.github.com>
Co-authored-by: bjmsong <wq.songbob@gmail.com>
Co-authored-by: bjmsong <bjmsong@126.com>
Co-authored-by: wchen61 <wchen61@foxmail.com>
Co-authored-by: Nathan Azrak <42650258+nathan-az@users.noreply.github.com>
Co-authored-by: Sachin Varghese <sachin.mathew31@gmail.com>
Co-authored-by: Aurick Qiao <aurickq@users.noreply.github.com>
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
Co-authored-by: ZincCat <52513999+zinccat@users.noreply.github.com>
Co-authored-by: WangErXiao <863579016@qq.com>
Co-authored-by: Hust_YangXian <bryceyx@gmail.com>
Co-authored-by: Alberto Ferrer <albertof@barrahome.org>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: xcnick <xcnick0412@gmail.com>
Co-authored-by: Yan Burman <yanburman@users.noreply.github.com>
Co-authored-by: cennn <61925104+cennn@users.noreply.github.com>
Co-authored-by: Lancer <402430575@qq.com>
Co-authored-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Suraj Deshmukh <surajd.service@gmail.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Concurrensee <yida.wu@amd.com>
Co-authored-by: YiSheng5 <yi.sheng@intel.com>
Co-authored-by: Zhonghua Deng <abatom@163.com>
Co-authored-by: XiaobingZhang <xiaobingzhangupc@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Yuan <yuan.zhou@intel.com>
Co-authored-by: jiangjiadi <34134495+jiangjiadi@users.noreply.github.com>
Co-authored-by: jiadi.jjd <jiadi.jjd@antgroup.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
Co-authored-by: Jie Fu (傅杰) <jiefu@tencent.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Nishidha <nishidha.panpaliya@partner.ibm.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: Wallas Henrique <wallashss@users.noreply.github.com>
Co-authored-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: rasmith <Randall.Smith@amd.com>
Co-authored-by: Maximilien de Bayser <mbayser@br.ibm.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Guspan Tanadi <36249910+guspan-tanadi@users.noreply.github.com>
Co-authored-by: Ye (Charlotte) Qi <ye.charlotte.qi@gmail.com>
Co-authored-by: yeq <yeq@devgpu004.lla3.facebook.com>
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: Yida Wu <yidawu@amd.com>
Co-authored-by: Charles Frye <cfrye59@gmail.com>
Co-authored-by: minmin <rmm0811@gmail.com>
Co-authored-by: Ren MinMin <renmm6@chinaunicom.cn>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Fred Reiss <frreiss@us.ibm.com>
Co-authored-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Co-authored-by: shaochangxu <85155497+shaochangxu@users.noreply.github.com>
Co-authored-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: sixgod <evethwillbeok@outlook.com>
Co-authored-by: Rafael Vasquez <rafvasq21@gmail.com>
Co-authored-by: Akshat Tripathi <Akshat.tripathi6568@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Avshalom Manevich <12231371+avshalomman@users.noreply.github.com>
Co-authored-by: Yangcheng Li <liyangcheng.lyc@alibaba-inc.com>
Co-authored-by: Siyuan Li <94890248+liaoyanqing666@users.noreply.github.com>
Co-authored-by: Chenguang Li <757486878@qq.com>
Co-authored-by: Alex Brooks <alex.brooks@ibm.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: elijah <30852919+e1ijah1@users.noreply.github.com>
Co-authored-by: Alexei-V-Ivanov-AMD <156011006+Alexei-V-Ivanov-AMD@users.noreply.github.com>
Co-authored-by: Alexei V. Ivanov <alivanov@banff-cyxtera-s65-4.amd.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
terrytangyuan added a commit to terrytangyuan/vllm that referenced this issue Jan 17, 2025
terrytangyuan added a commit to terrytangyuan/vllm that referenced this issue Jan 17, 2025
…file. Fixes vllm-project#9182

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
joennlae pushed a commit to 44ai-labs/vllm that referenced this issue Jan 19, 2025
joennlae pushed a commit to 44ai-labs/vllm that referenced this issue Jan 19, 2025
abmfy pushed a commit to abmfy/vllm-flashinfer that referenced this issue Jan 24, 2025
…llm-project#11435)

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
Signed-off-by: Bowen Wang <abmfy@icloud.com>
abmfy pushed a commit to abmfy/vllm-flashinfer that referenced this issue Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
installation Installation problems
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants