Bugfix: Groq via LiteLLM 🚅

[JGalego]

Fixes #429

Example

"""
smolagents meets Groq via LiteLLM

Adapted from https://github.com/huggingface/smolagents/issues/429
"""

import os

from smolagents import CodeAgent, LiteLLMModel

# Initialize the model with Groq
model = LiteLLMModel(
    "groq/deepseek-r1-distill-llama-70b",
    api_base="https://api.groq.com/openai/v1",
    api_key=os.getenv("GROQ_API_KEY")
)

# Create a minimal agent
agent = CodeAgent(
    tools=[],  # No tools needed to demonstrate the issue
    model=model,
    add_base_tools=False,
    verbosity_level=2
)

# Try to run a simple task
try:
    result = agent.run("Say hello!")
    print(result)
except Exception as e:  # pylint: disable=broad-exception-caught
    print(f"Error: {str(e)}")

Output

╭───────────────────────────────────────────────────── New run ─────────────────────────────────────────────────────╮
│                                                                                                                   │
│ Say hello!                                                                                                        │
│                                                                                                                   │
╰─ LiteLLMModel - groq/deepseek-r1-distill-llama-70b ───────────────────────────────────────────────────────────────╯
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Output message of the LLM: ──────────────────────────────────────────────────────────────────────────────────────────
<think>                                                                                                              
Okay, so I need to solve the task "Say hello!" and provide a final answer using the tools available. Let me think    
about how to approach this.                                                                                          
                                                                                                                     
First, the task is pretty straightforward—just say hello. But since I'm supposed to use the tools provided, I should 
figure out which tool is appropriate here. The only tool I have is `final_answer`, which is used to provide the final
answer.                                                                                                              
                                                                                                                     
I remember that the `final_answer` tool takes an argument 'answer' which can be of any type. So, I can simply pass   
the string "Hello!" to it.                                                                                           
                                                                                                                     
Wait, but do I need to do anything else before that? Like, maybe check if there's a specific format or if I need to  
process something. But the task is just to say hello, so no additional processing is required.                       
                                                                                                                     
I should make sure that I'm following the rules correctly. I need to have a 'Thought:' section and a 'Code:' section.
In the code, I'll just call `final_answer` with "Hello!".                                                            
                                                                                                                     
I don't think I need any other tools here because the task doesn't require any computation or data retrieval. It's a 
direct response.                                                                                                     
                                                                                                                     
So, putting it all together, I'll write the code that uses `final_answer` with the message. That should complete the 
task.                                                                                                                
</think>                                                                                                             
                                                                                                                     
To solve the task of saying hello, I will use the `final_answer` tool to provide the greeting.                       
                                                                                                                     
**Code:**                                                                                                            
\```python                                                                                                            
final_answer("Hello!")                                                                                               
\```                                                                                                                  
 ─ Executing parsed code: ────────────────────────────────────────────────────────────────────────────────────────── 
  final_answer("Hello!")                                                                                             
 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
Out - Final answer: Hello!
[Step 0: Duration 1.60 seconds| Input tokens: 1,959 | Output tokens: 312]
Hello!

[sysradium]

Sweet. I would have used a tuple here, but that still works. Thanks for the fix.

[JGalego]

For future reference, here's a quick n' easy way to test this fix:

uv run https://gist.github.com/JGalego/df3422a325e2dd9126883ffef6ea44b1/raw/smol_groq.py