Claude Sonnet 3.5 (Bedrock) Always announces which tool it's going to use.

[KentonParton]

I have configured a streamText endpoint which uses AWS Bedrock with Claude Sonnet 3.5 V1 and no matter what system prompt I use, it always start with something like, To answer your question about ..., I'll need to consult our documentation. Let me fetch that information for you., it then invokes the tool, then proceed to answer the question.

When I switch to OpenAI gpt4o, it invokes the tools immediately without the announcement which is the desired outcome.

Package Versions:
"ai": "^3.4.9",
"@ai-sdk/amazon-bedrock": "^0.0.28",

Has anyone encountered this and figured out how to correct this? Thanks!

[BenGrabowski]

I've been dealing with the same issue and I just switched to use Sonnet 3.5 v2. I also added the following to my system prompt:

"Do not show your "chain of thought" before choosing a tool. If you decide to call a tool, the first chunks should be tool-call and not text-delta."

I got the idea from this portion of Anthropic's documentation: https://docs.anthropic.com/en/docs/build-with-claude/tool-use#chain-of-thought

I'm now getting tool-call and tool-result chunks streamed before text-delta chunks

[KentonParton]

Literally just upgraded to V2, included "Get straight to the point - do not announce that you are going to look up information
or consult documentation. Simply provide the relevant information directly." and that did the trick. I like your prompt more though. Will give it a try. Thanks Ben.

[yych42]

I was having the same issue with models provided directly by Anthropic. The workaround I found was to...

• downgrade the model version to claude-3-5-sonnet-20240620
• add "If you want to take use tools, it must happen at the start of your response" to my system prompt
• then, emphasize again in the tool description with "[...] Only run this at the start of your response, after the user has provided new information. This means if you asked a question, you probably don't want to run this tool until after the user has responded."

The tool description part stabilized the model's behavior a lot.