Function calling response message

[0xffmeta]

I'm trying to implemnt a very simple agent which only enables one tool to create a markdown document:

1. user asks agent to create a document reagarding xx topic
2. agent calls the tool with correct parameters - this actually calls another LLM model to get the long markdown format document as the response
3. agent returns the response of the action

Seems in step 3, the response content always contains the result from step 2, e.g.

The new document is created as follow:
NEW_DOCUMENT_CONTENT

I have tried to optimize the system instructions like this

You are a helpful AI assistant specialized in writing and research.
Your responses are clear, precise, and well-structured.
When you invoke tools, especially `modify_content`, follow these steps:
1. Call the tool and wait for its completion
2. After the tool finishes, provide a brief summary of the changes made
3. Ask if the user would like to make any additional modifications

You should return `finish` when the tool is finished.

But seems the step 3 response is not changed too much.

[0xffmeta]

I followed the kernel exection steps a little bit, seems the result from step 2 will be passed to step 3 when agent trys to summarize the action. Not sure if this is the correct understanding and how to optimize this.

[moonbox3]

Hi @0xffmeta, which SK language are you using? Are you using SK's agent framework or the chat completion methods? It sounds like you would not like the agent to return the final response which is passed through the LLM, is that correct? If that's the case, you can use a filter/hook to simply return the result from the tool call. This would return the properly formatted tool call response (the long markdown) instead of having that go through the LLM for a final response.

Depending upon which SK language you're using, and the method (agent framework or chat completion) I can help guide you further.

[0xffmeta]

Thanks for the help @moonbox3! I'm using python with ChatCompletionAgent.

you can use a filter/hook to simply return the result from the tool call

I would like the tool call response is streamed to the UI as a result to the user, but not sure how to just return a static response to LLM in that tool call.

[moonbox3]

Thanks for your response, @0xffmeta. Given that the ChatCompletionAgent is based on the chat completion endpoint, the filters should work fine for this scenario. You can enable the streaming auto invocation filter following this sample.