Support speculative decoding with llama.cpp #197
Labels
needs-kind
Indicates a PR lacks a label and requires one.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
What would you like to be added:
See comment here: ggerganov/llama.cpp#5877 (comment), llama.cpp use another command
llama-speculative
to process the technology.Why is this needed:
Completion requirements:
This enhancement requires the following artifacts:
The artifacts should be linked in subsequent comments.
The text was updated successfully, but these errors were encountered: