server: add KV cache quantization options #8932
| Job | Run time |
|---|---|
| 2m 13s | |
| 2m 0s | |
| 2m 1s | |
| 2m 11s | |
| 5m 38s | |
| 4m 24s | |
| 17m 43s | |
| 5m 13s | |
| 8m 31s | |
| 5m 18s | |
| 59s | |
| 5m 5s | |
| 3m 57s | |
| 1m 15s | |
| 2m 0s | |
| 6m 9s | |
| 2m 9s | |
| 5m 13s | |
| 5m 17s | |
| 2m 4s | |
| 1m 46s | |
| 3m 54s | |
| 18m 37s | |
| 4m 45s | |
| 24m 14s | |
| 4m 20s | |
| 13m 2s | |
| 2m 52s | |
| 13m 17s | |
| 4m 17s | |
| 3m 52s | |
| 4m 27s | |
| 0s | |
| 3h 8m 43s |