Code

Fix: Mamba2 norm_before_gate usage#32686

Code

Merged

ArthurZucker merged 3 commits intohuggingface:mainhuggingface/transformers:mainfrom vasqu:mamba2-gated-norm-fixvasqu/transformers:mamba2-gated-norm-fixCopy head branch name to clipboardAug 20, 2024

+1-6

Commits

Commits on Aug 20, 2024

• mamba2 uses norm_before_gate=False

  vasqu

  committed
  b72e876

  Copy full SHA for b72e876

• small nit

  vasqu

  committed
  4f0ce72

  Copy full SHA for 4f0ce72

• remove norm_before_gate flag and follow False path only

  vasqu

  committed
  7d01af0

  Copy full SHA for 7d01af0