Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add two more flags for language models: streaming & PDFs #593

Open
krschacht opened this issue Jan 13, 2025 · 4 comments
Open

Add two more flags for language models: streaming & PDFs #593

krschacht opened this issue Jan 13, 2025 · 4 comments

Comments

@krschacht
Copy link
Contributor

There are API models that do not support streaming (specifically OpenAI o1) but our backend always sets the streaming flag to true. Similarly, there are now API models that support non-image attachments (specifically Claude Sonnet) but our app only allows you to attach images.

We should migrate the langauge_model table to have two new booleans, maybe called supports_streaming (default to true) and supports_pdfs (default to false).

Then our language model settings page should have checkboxes for these.

Then our app internals should be updated to reflect these.

Anyone want to take this one?
@mattlindsey @duckworth @papayalabs @drnic @voodoo @sergiobayona

@krschacht
Copy link
Contributor Author

This is another possible one maybe @draknor, @ceicke, @stephan-buckmaster, @jasonpaulso or @htruong might want to help with?

@stephan-buckmaster
Copy link
Contributor

I can work on this.

@stephan-buckmaster
Copy link
Contributor

Sorry, I'm not getting to this, it also looks not straightforward.

@krschacht
Copy link
Contributor Author

No problem. And I actually think we should drop the streaming boolean flag idea. OpenAI just announced that they’re new o3 model now supports streaming, and this makes their o1 model obsolete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants