Skip to content

Commit

Permalink
convert: Fix handling of LLAMA2 vocab_size = -1
Browse files Browse the repository at this point in the history
When vocab_size is detected to be -1 simply remove its value from the
parsed params.json and fallback to using the tok_embeddings.weight.

Fixes  ggerganov#3900
  • Loading branch information
Nikolay Borisov committed Nov 6, 2023
1 parent d6d905b commit f36a777
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions convert.py
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,11 @@ def loadOriginalParamsJson(model: LazyModel, config_path: Path) -> Params:
elif config["norm_eps"] in (1e-05, 1e-06):
# LLaMA v2
n_ctx = 4096
# For some reason FB writes -1 to vocab size for their LLAMA2 models
# simply remove this bogus value and let the return statement belo
# figure it out
if config["vocab_size"] == -1:
del config["vocab_size"]
else:
# LLaMA v1
n_ctx = 2048
Expand Down

0 comments on commit f36a777

Please sign in to comment.