Skip to content

Commit

Permalink
fix whisper k_proj btw
Browse files Browse the repository at this point in the history
Signed-off-by: Isotr0py <2037008807@qq.com>
  • Loading branch information
Isotr0py committed Feb 28, 2025
1 parent 8433813 commit 1826b32
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions vllm/model_executor/models/whisper.py
Original file line number Diff line number Diff line change
Expand Up @@ -748,11 +748,11 @@ def _create_fake_bias_for_k_proj(
weights: Iterable[Tuple[str, torch.Tensor]]
) -> Iterable[Tuple[str, torch.Tensor]]:
"""
Create full zeros bias for k_proj weight in self-attention layers.
Create full zeros bias for k_proj weight in self-attn and x-attn layers.
So that the bias for k_proj in qkv_proj can be initialized with zeros.
"""
for name, weight in weights:
if name.endswith(".self_attn.k_proj.weight"):
if name.endswith(".k_proj.weight"):
bias = torch.zeros(weight.size(0))
bias_name = name.replace("weight", "bias")
yield from [(name, weight), (bias_name, bias)]
Expand Down

0 comments on commit 1826b32

Please sign in to comment.