Skip to content

Commit

Permalink
comment
Browse files Browse the repository at this point in the history
- adding # NEW for assert num_heads % num_kv_groups == 0
  • Loading branch information
casinca authored Nov 17, 2024
1 parent 4925790 commit 7292733
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion ch05/07_gpt_to_llama/converting-llama2-to-llama3.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -481,7 +481,7 @@
" ):\n",
" super().__init__()\n",
" assert d_out % num_heads == 0, \"d_out must be divisible by num_heads\"\n",
" assert num_heads % num_kv_groups == 0, \"num_heads must be divisible by num_kv_groups\"\n",
" assert num_heads % num_kv_groups == 0, \"num_heads must be divisible by num_kv_groups\" # NEW\n",
"\n",
" self.d_out = d_out\n",
" self.num_heads = num_heads\n",
Expand Down

0 comments on commit 7292733

Please sign in to comment.