Add Support for Llama-cpp-python Grammar

[Kaotic3]

Hi There,

Love this server, super fast and really one of the few that utilises the GPUs I am using to their full capacity.

The one problem I am having is that I use Grammar from Llama-cpp-python to control the output from the LLM and force it into a JSON format. Which I can parse.

I have tried without it, and the formatting is just so poor that the remedial work required makes any time saving from the faster server a wash, bearing in mind I am dealing with thousands of requests not just one or two.

It would be great if we could use grammar with vLLM and get back the responses we need.

Appreciate the consideration.

[lzl12051]

I second this, Grammar is super useful when you need to process LLM's results in a deterministic way. Appreciate it.

[hmellor]

vLLM supports guided decoding via outlines #2819