Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Misc. bug: Some server response JSON still not restored #10728

Closed
CentricStorm opened this issue Dec 9, 2024 · 3 comments · Fixed by #10818
Closed

Misc. bug: Some server response JSON still not restored #10728

CentricStorm opened this issue Dec 9, 2024 · 3 comments · Fixed by #10818
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@CentricStorm
Copy link
Contributor

Name and Version

version: 4291 (ce8784b)
built with MSVC 19.29.30157.0 for x64

Operating systems

Windows

Which llama.cpp modules do you know to be affected?

llama-server

Problem description & steps to reproduce

After #10722, there are still some issues when using the native /completion endpoint:

  • The has_new_line property is set to 1 instead of true like it used to be.
  • The model property is set to gpt-3.5-turbo-0613 even though this isn't the OpenAI-compatible API.

First Bad Commit

No response

Relevant log output

No response

@ggerganov ggerganov added bug Something isn't working good first issue Good for newcomers and removed bug-unconfirmed labels Dec 9, 2024
@ggerganov
Copy link
Owner

PRs welcome

@MichelleTanPY
Copy link
Contributor

Hi @ggerganov , I saw the 'good first issue' tag on this. I'm new to this project and thought to give this a try. I manage to build the llama-server, but I'm getting this strange error when get the server up and running.
I'm following the README here : https://github.com/ggerganov/llama.cpp/tree/master/examples/server and getting the following error.
llama_model_load: error loading model: missing tensor 'token_embd.weight' llama_load_model_from_file: failed to load model common_init_from_params: failed to load model '.\models\ggml-vocab-deepseek-llm.gguf' srv load_model: failed to load model, '.\models\ggml-vocab-deepseek-llm.gguf' main: exiting due to model loading error
Any pointers on how I should proceed? Building with MSVC 19.42.34435.0 x64 Windows machine.

@ngxson
Copy link
Collaborator

ngxson commented Dec 13, 2024

  • The model property is set to gpt-3.5-turbo-0613 even though this isn't the OpenAI-compatible API.

This is to prepare for adding OAI-compat to /v1/completions endpoint. If you're searching for the model path, please have a look at /props endpoint

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants