Feeding tool result back into the messages

[msjoshlopez]

[elitan]

EDIT: It's supported here: https://sdk.vercel.ai/examples/node/tools/call-tools-with-automatic-roundtrips

OLD ANSWER:

Here's what I'm experimenting with right now:

async function main() {

  let messages: CoreMessage[] = [
    { role: "user", content: "Hi!" },
    { role: "assistant", content: "Hello, how can I help?" },
    {
      role: "user",
      content: "What's the weather in New York?",
    },
  ];

  let textResultIsPresent = false;
  let textResult = "";

  while (!textResultIsPresent) {
    await sleep(2000);
    const { text, toolCalls, toolResults } = await generateText({
      model,
      tools: {
        weather: tool({
          description: "Get the weather in a location",
          parameters: z.object({
            location: z
              .string()
              .describe("The location to get the weather for"),
          }),
          execute: async ({ location }) => ({
            location,
            temperature: 72 + Math.floor(Math.random() * 21) - 10,
          }),
        }),
      },
      messages,
    });

    if (toolResults.length > 0 && toolCalls.length > 0) {
      messages.push({
        role: "assistant" as const,
        content: toolCalls,
      });

      messages.push({
        role: "tool" as const,
        content: toolResults,
      });
    } else {
      textResult = text;
      textResultIsPresent = true;
    }

    console.log(JSON.stringify(messages, null, 2));
  }

  console.log(textResult);
}

Output:

[
  {
    "role": "user",
    "content": "Hi!"
  },
  {
    "role": "assistant",
    "content": "Hello, how can I help?"
  },
  {
    "role": "user",
    "content": "What's the weather in New York?"
  },
  {
    "role": "assistant",
    "content": [
      {
        "type": "tool-call",
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": [
      {
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        },
        "result": {
          "location": "New York",
          "temperature": 66
        }
      }
    ]
  }
]
[
  {
    "role": "user",
    "content": "Hi!"
  },
  {
    "role": "assistant",
    "content": "Hello, how can I help?"
  },
  {
    "role": "user",
    "content": "What's the weather in New York?"
  },
  {
    "role": "assistant",
    "content": [
      {
        "type": "tool-call",
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": [
      {
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        },
        "result": {
          "location": "New York",
          "temperature": 66
        }
      }
    ]
  }
]

The current temperature in New York is 66°F.

[zineanteoh]

Is there anyway to stream the final output?

The current temperature in New York is 66°F.

[benjaminalgreen]

Have been exploring how to do exactly this, thanks!

Just curious, how good/bad is the AI at being able to find the right info from previous tool responses when appropriate? E.g. Let's say you ask it to find the weather for New York. Then you ask it later how the weather in NY compares to SF. Does it know to search for SF and use the previous data from NY if prompted well?

[lgrammel]

With useChat, you can now do automatic roundtrips: https://sdk.vercel.ai/docs/ai-sdk-ui/chatbot-with-tool-calling

Server-side roundtrip support in streamText and generateText is planned

[benjaminalgreen]

lgrammel, does streamText with maxToolRoundtrips on the server support sending toolResults back to the frontend?

[lgrammel]

yes

[castortech]

yes

First of, thanks so much for having this in place and on schedule. Really appreciated.

Right now I am getting an empty array for toolCalls and toolResults is returning a never[] | undefined

I am on the latest versions from 1 day ago.

[hminle]

how to use this one maxToolRoundtrips?

[lgrammel]

https://sdk.vercel.ai/docs/ai-sdk-core/tools-and-tool-calling#multi-step-calls