Skip to content

[Speculative decoding] Add ngram prompt lookup decoding#4237

Merged
cadedaniel merged 2 commits intovllm-project:mainfrom leiwen83:ngram_lookahead_specinferMay 1, 2024
+1,004-319

Commits

Commits on May 1, 2024