feat:Updated Deep Infra models #121
Conversation
WalkthroughThe changes in this pull request involve the addition of new enumerations and updates to existing entries within the Changes
Possibly related PRs
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (3)
🔇 Additional comments (5)src/DeepInfra/src/DeepInfraModelProvider.cs (2)
The model is properly configured with:
Several Meta Llama models have different completion costs despite being similar in architecture:
Run the following script to check cost patterns across Meta Llama models: ✅ Verification successfulCost variations in Meta Llama models are consistent with model sizes and capabilities The cost differences between Meta Llama models in DeepInfra are justified:
The pricing structure logically scales with model size and optimization level. 🏁 Scripts executedThe following scripts were executed for the analysis: Script: #!/bin/bash
# Description: Check cost patterns across Meta Llama models
rg "Llama.*Instruct.*Completion Cost:" -A 1
Length of output: 45 Script: #!/bin/bash
# Let's search for model costs in the codebase using a more general pattern
rg "ToMetadata.*meta-llama.*Instruct" -A 1
Length of output: 6263 Script: #!/bin/bash
# Let's also check the model IDs and their definitions
ast-grep --pattern 'public static class DeepInfraModelIds { $$$ }'
Length of output: 68 Script: #!/bin/bash
# Search for any cost-related documentation or comments
rg -i "cost|pricing|token" --type cs
Length of output: 66606 Script: #!/bin/bash
# Let's search for Meta Llama model costs in DeepInfra provider
rg "Meta.*Llama.*Instruct.*ToMetadata" src/DeepInfra/src/DeepInfraModelProvider.cs
Length of output: 1126 Script: #!/bin/bash
# Let's also check if there are any comments or documentation about pricing in the DeepInfra provider
rg -i "cost|price|token" src/DeepInfra/src/DeepInfraModelProvider.cs
Length of output: 344 src/DeepInfra/src/Predefined/AllModels.cs (1)
The new Llama3370BInstructModel and Llama3370BInstructTurboModel classes:
src/DeepInfra/src/DeepInfraModelIds.cs (2)
The new Llama3370BInstruct and Llama3370BInstructTurbo enums have comprehensive documentation including:
Significant cost reductions noted for Mistral models:
Please confirm these are intentional price changes. Run the following script to check historical cost changes: Also applies to: 618-619 ✅ Verification successfulCost verification for Mistral models confirmed The costs in the code match exactly with what was mentioned in the review comment:
🏁 Scripts executedThe following scripts were executed for the analysis: Script: #!/bin/bash
# Description: Check cost history for Mistral models
git log -p -- "src/DeepInfra/src/DeepInfraModelIds.cs" | rg "Mistral.*Cost:"
Length of output: 0 Script: #!/bin/bash
# Let's try a different approach to check the current costs for Mistral models
rg -A 1 "Mistral.*Instruct.*{" src/DeepInfra/src/DeepInfraModelIds.cs
Length of output: 175 Script: #!/bin/bash
# Also check the costs directly
rg "Cost.*MTok" src/DeepInfra/src/DeepInfraModelIds.cs
Length of output: 5072 Script: #!/bin/bash
# Let's also check if there are any other cost-related comments for these models
rg "Mistral" src/DeepInfra/src/DeepInfraModelIds.cs
Length of output: 2878 Script: #!/bin/bash
# Let's get more context around the Mistral models to see their costs
rg -B 2 -A 4 "Mistral7BInstructV03|MistralNemoInstruct2407" src/DeepInfra/src/DeepInfraModelIds.cs
Length of output: 721 Script: #!/bin/bash
# Let's try to find the cost information near these model definitions
rg -B 5 -A 5 "Mistral7BInstructV03" src/DeepInfra/src/DeepInfraModelIds.cs
Length of output: 686 Script: #!/bin/bash
# And for the other model
rg -B 5 -A 5 "MistralNemoInstruct2407" src/DeepInfra/src/DeepInfraModelIds.cs
Length of output: 658 Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
Created by Github Actions
Summary by CodeRabbit
Llama3370BInstructandLlama3370BInstructTurbo.DeepseekV25from the available models list.