add back streaming for base o3 #8361
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
Addresses issue #8360 |
| if ( | ||
| model and "o3" in model | ||
| ): # o3 models support streaming - https://github.com/BerriAI/litellm/issues/8274 | ||
| return False | ||
| supported_stream_models = ["o1-mini", "o1-preview"] |
There was a problem hiding this comment.
you just need to add it here, instead of another if block @mbosc
There was a problem hiding this comment.
you want me to add the model to the list? I did it like this because I saw that the AzureOpenAI version of o_series_transformation uses this block, so I wanted to keep the code aligned.
https://github.com/BerriAI/litellm/blob/16be203283b25f1116c6acc9144b19711b10e909/litellm/llms/azure/chat/o_series_transformation.py#L37C1-L41C25
If you prefer I'll just add o3-mini and o3-mini-2025-01-31 to the supported_stream_models list at line 61.
There was a problem hiding this comment.
Sure let's start there
There was a problem hiding this comment.
Perhaps we can update the check to see if the substring o1-preview/o3-mini etc. is in the model name.
It should ensure it works for future versions
There was a problem hiding this comment.
Ok, I'll rebase and re-commit
31ee977
into
BerriAI:litellm_dev_02_07_2025_p3
* add back streaming for base o3 (#8361) * test(base_llm_unit_tests.py): add base test for o-series models - ensure streaming always works * fix(base_llm_unit_tests.py): fix test for o series models * refactor: move test --------- Co-authored-by: Matteo Boschini <12133566+mbosc@users.noreply.github.com>
* add back streaming for base o3 (BerriAI#8361) * test(base_llm_unit_tests.py): add base test for o-series models - ensure streaming always works * fix(base_llm_unit_tests.py): fix test for o series models * refactor: move test --------- Co-authored-by: Matteo Boschini <12133566+mbosc@users.noreply.github.com>
No description provided.