Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add tool calling support in ChatOllama #505

Merged
merged 1 commit into from
Jul 24, 2024
Merged

Conversation

davidmigloz
Copy link
Owner

@davidmigloz davidmigloz commented Jul 24, 2024

Tool calling allows a model to respond to a given prompt by generating output that matches a user-defined schema, that you can then use to call the tools in your code and return the result back to the model to complete the conversation.

Notes:

  • Tool calling requires Ollama 0.2.8 or newer.
  • Streaming tool calls is not supported at the moment.
  • Not all models support tool calls. Check the Ollama catalogue for models that have the Tools tag (e.g. llama3.1).

Check the docs for more information on how to use tools.

const tool = ToolSpec(
  name: 'get_current_weather',
  description: 'Get the current weather in a given location',
  inputJsonSchema: {
    'type': 'object',
    'properties': {
      'location': {
        'type': 'string',
        'description': 'The city and country, e.g. San Francisco, US',
      },
    },
    'required': ['location'],
  },
);

final chatModel = ChatOllama(
  defaultOptions: ChatOllamaOptions(
    model: 'llama3.1',
    temperature: 0,
    tools: [tool],
  ),
);

final res = await chatModel.invoke(
  PromptValue.string('What’s the weather like in Boston and Madrid right now in celsius?'),
);
print(res.output.toolCalls);
// [AIChatMessageToolCall{
//   id: a621064b-03b3-4ca6-8278-f37504901034,
//   name: get_current_weather,
//   arguments: {location: Boston, US},
// }, 
// AIChatMessageToolCall{
//   id: f160d9ba-ae7d-4abc-a910-2b6cd503ec53,
//   name: get_current_weather,
//   arguments: {location: Madrid, ES},
// }]

@davidmigloz davidmigloz self-assigned this Jul 24, 2024
@davidmigloz davidmigloz added t:enhancement New feature or request c:tools Tools. p:langchain_ollama langchain_ollama package. labels Jul 24, 2024
@davidmigloz davidmigloz added this to the v0.8.0 milestone Jul 24, 2024
@davidmigloz davidmigloz merged commit 6ffde20 into main Jul 24, 2024
1 check passed
@davidmigloz davidmigloz deleted the chat-ollama-tools branch July 24, 2024 21:54
KennethKnudsen97 pushed a commit to KennethKnudsen97/langchain_dart that referenced this pull request Oct 1, 2024
KennethKnudsen97 pushed a commit to KennethKnudsen97/langchain_dart that referenced this pull request Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c:tools Tools. p:langchain_ollama langchain_ollama package. t:enhancement New feature or request
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

1 participant