v1.61.4-nightly
What's Changed
- docs(perplexity.md): removing
return_citations
documentation by @miraclebakelaser in #8527 - (docs - cookbook) litellm proxy x langfuse by @ishaan-jaff in #8541
- UI Fixes and Improvements (02/14/2025) p1 by @krrishdholakia in #8546
- (Feat) - Add
/bedrock/meta.llama3-3-70b-instruct-v1:0
tool calling support + cost tracking + base llm unit test for tool calling by @ishaan-jaff in #8545 - fix(general_settings.tsx): filter out empty dictionaries post fallbac… by @krrishdholakia in #8550
- (perf) Fix memory leak on
/completions
route by @ishaan-jaff in #8551 - Org Flow Improvements by @krrishdholakia in #8549
- feat(openai/o_series_transformation.py): support native streaming for o1 by @krrishdholakia in #8552
- fix(team_endpoints.py): fix team info check to handle team keys by @krrishdholakia in #8529
- build: ui build update by @krrishdholakia in #8553
Full Changelog: v1.61.3...v1.61.4-nightly
## Docker Run LiteLLM Proxy
```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.4-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Failed ❌ | 190.0 | 216.89425311206062 | 6.2617791082055785 | 6.2617791082055785 | 1874 | 1874 | 143.52555700003222 | 3508.21726800001 |
Aggregated | Failed ❌ | 190.0 | 216.89425311206062 | 6.2617791082055785 | 6.2617791082055785 | 1874 | 1874 | 143.52555700003222 | 3508.21726800001 |