readme : refresh #10587
readme : refresh #10587
Conversation
README.md
Outdated
|
|
||
| ### Obtaining and quantizing models | ||
|
|
||
| The [Hugging Face](https://huggingface.co) platform hosts a [large amount of LLMs](https://huggingface.co/models?library=gguf&sort=trending) compatible with `llama.cpp` - simply search for the [GGUF](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md) file format. |
There was a problem hiding this comment.
| The [Hugging Face](https://huggingface.co) platform hosts a [large amount of LLMs](https://huggingface.co/models?library=gguf&sort=trending) compatible with `llama.cpp` - simply search for the [GGUF](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md) file format. | |
| The [Hugging Face](https://huggingface.co) platform hosts a [large number of LLMs](https://huggingface.co/models?library=gguf&sort=trending) compatible with `llama.cpp` - simply search for the [GGUF](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md) file format. |
There was a problem hiding this comment.
Strictly speaking this is not 100% correct since you could store arbitrary data in GGUF files (and some people unfortunately upload broken GGUFs immediately after a new model release). As of right now though you can reasonably assume that any GGUF model will run with llama.cpp. But we should keep in mind to update this if GGUF models for other projects ever become popular.
There was a problem hiding this comment.
Hmm right, I can see from the link above that flux gguf is one of the trending model, which is not compat with llama.cpp. But I think it's acceptable for now, as this link is here to help user find commonly used GGUF models.
Indeed, on HF hub, we do have a measure to filter only llama.cpp-compatible models. Maybe we could add a specific query parameter for it in the future (for example, &compatible=llama-cpp). CC @julien-c too!
There was a problem hiding this comment.
I've updated the wording to no imply that all GGUF files will work with llama.cpp. Feel free to improve this in the future.
README.md
Outdated
| @@ -309,7 +264,7 @@ See [this page](./examples/main/README.md) for a full list of parameters. | |||
|
|
|||
| ### Conversation mode | |||
|
|
|||
| If you want a more ChatGPT-like experience, you can run in conversation mode by passing `-cnv` as a parameter: | |||
| For a more ChatGPT-like experience, run `llama-cli` in conversation mode by passing `-cnv` as a parameter: | |||
There was a problem hiding this comment.
I would say nowadays the "ChatGPT-like" experience is the server.
There was a problem hiding this comment.
I removed the "ChatGPT" term all together.
* readme : refresh * readme : move section [no ci] * readme : clarify [no ci] * readme : fixes [no ci] * readme : more fixes [no ci] * readme : simplify [no ci] * readme : clarify GGUF
Clean-up some old stuff from the readme and reorganize the information a little bit