Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Count model parameters and size using only the GGUF tensors #10285

Closed
slaren opened this issue Nov 14, 2024 Discussed in #10274 · 1 comment
Closed

Count model parameters and size using only the GGUF tensors #10285

slaren opened this issue Nov 14, 2024 Discussed in #10274 · 1 comment
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@slaren
Copy link
Collaborator

slaren commented Nov 14, 2024

Discussed in #10274

The number of parameters and size of the model currently is calculated from the tensors created after the model is loaded, which in some cases may contain duplicated tensors, resulting in an inaccurate and inconsistent reporting of the model size. To address this, llama_model_n_params and llama_model_size should be modified to return the value as calculated in llama_model_loader::n_elements and n_bytes, which could be stored in llama_model while loading the model.
@slaren slaren added bug Something isn't working good first issue Good for newcomers labels Nov 14, 2024
@FirstTimeEZ FirstTimeEZ mentioned this issue Nov 14, 2024
Merged
1 task
@FirstTimeEZ
Copy link
Contributor

This issue is fixed by #10286

slaren pushed a commit that referenced this issue Nov 16, 2024
@FirstTimeEZ
llama : save number of parameters and the size in llama_model (#10286)
89e4caa 
fixes #10285
arthw pushed a commit to arthw/llama.cpp that referenced this issue Nov 18, 2024
@FirstTimeEZ @arthw
llama : save number of parameters and the size in llama_model (ggerga…
0d12d86 
…nov#10286)

fixes ggerganov#10285
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@slaren @FirstTimeEZ