Text stream support #3

snwfdhmp started this conversation in General

snwfdhmp
Jun 9, 2023
Maintainer

Most models integrate a stream mode to stream text as it is generated.

It is a must-have for llm since it avoids having to wait for the whole output to be finished, which is an issue for long outputs.

How to integrate that into llm ?

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment