Anthropic-specific prompts needed for classification and agent tasks

[dhirenmathur]

Issue Summary

While PR #203 fixed basic Anthropic integration, the current prompts are not optimized for Anthropic models. We need to update classification prompts and agent task descriptions to improve performance with Anthropic models while work continues on dynamic prompt injection (#189).

Current Status

• Anthropic integration is technically working after PR Support for anthropic, req pinning #203
• Existing prompts designed for other models perform sub-optimally with Anthropic
• Work in progress on dynamic prompt injection based on model name (Centralised prompt management #189)
• Classification and agent task prompts need immediate updates

Action Items

• Review and update classification prompts for Anthropic compatibility
• Revise agent task descriptions to align with Anthropic's capabilities
• Test updated prompts with Anthropic models
• Document Anthropic-specific prompt patterns that work well

Related Issues/PRs

• Centralised prompt management #189 Dynamic prompt injection based on model name
• Support for anthropic, req pinning #203 Basic Anthropic integration fix

Additional Context

This is an interim solution while the more comprehensive dynamic prompt injection system is being developed. Once #189 is completed, these Anthropic-specific prompts can be integrated into the dynamic system.

[simin75simin]

@dhirenmathur I'd like to contribute to this issue. Could you assigned this to me? Thanks!

[dhirenmathur]

Gladly! Thank you and all the best!

[dhirenmathur]

Hey @simin75simin what were your final findings?

[simin75simin]

could not test comprehensively right now cause an issue in local repo reads: 'Repo' object has no method 'get_contents'

it seems there are two packages used: gitPython (import git) and pyGithub (import github) and pyGithub has get_contents for production mode, which grabs the repo online.

trying to install production mode for now. @dhirenmathur

[dhirenmathur]

@simin75simin understood, we didn't face this bug when we were testing it that day correct? Did you try to test something different? Could you please provide steps to reproduce and open an issue for it? Thanks!

[simin75simin]

it seemed this bug is only present for anthropic. still trying to pin down how exactly it happened. maybe it's some network issue on my local PC that uses a proxy & WSL 2. but this does not look likely because i can use litellm & anthropic from python shell in the same terminal. maybe something else.

on my PC this error was triggered by going into the postgres docker container and added the preferred AI for the defaultuser to be 'anthropic' and followed readme. @dhirenmathur

[dhirenmathur]

If you remove the preferred provider from db it works end to end like we tried previously?

[simin75simin]

yes, and if i change it to openai it also works, but if i change it to anthropic it gives UNEXPECTED_EOF_WHILE_READING and even though sometimes the call to the AI works Anthropic gives responses suggesting it cannot access context i.e. the repo. no problem with openai though.

[dhirenmathur]

And to confirm, you used the API to set the provider in the database? (To make sure that the format is correct in the database)

[simin75simin]

no but i can try

also if it's openai it works, like

UPDATE user_preferences
SET preferences = '{"llm_provider": "openai"}'
WHERE user_id = 'defaultuser';

but if i change the word openai to anthropic it throws UNEXPECTED_EOF_WHILE_READING

[simin75simin]

so i tried using API to set provider in database but same error. i got a 200 from Anthropic but then started getting one UNEXPECTED_EOF_WHILE_READING. looking into this.

[simin75simin]

looks like some format conversion mistake with Pydantic with PydanticOutputParser