Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add tool calling support in ollama_dart #504

Merged
merged 1 commit into from
Jul 24, 2024
Merged

Conversation

davidmigloz
Copy link
Owner

@davidmigloz davidmigloz commented Jul 24, 2024

Tool calling allows a model to respond to a given prompt by generating output that matches a user-defined schema, that you can then use to call the tools in your code and return the result back to the model to complete the conversation.

Notes:

  • Tool calling requires Ollama 0.2.8 or newer.
  • Streaming tool calls is not supported at the moment.
  • Not all models support tool calls. Check the Ollama catalogue for models that have the Tools tag (e.g. llama3.1).
const tool = Tool(
  function: ToolFunction(
    name: 'get_current_weather',
    description: 'Get the current weather in a given location',
    parameters: {
      'type': 'object',
      'properties': {
        'location': {
          'type': 'string',
          'description': 'The city and country, e.g. San Francisco, US',
        },
        'unit': {
          'type': 'string',
          'description': 'The unit of temperature to return',
          'enum': ['celsius', 'fahrenheit'],
        },
      },
      'required': ['location'],
    },
  ),
);

const userMsg = Message(
  role: MessageRole.user,
  content: 'What’s the weather like in Barcelona in celsius?',
);

final res1 = await client.generateChatCompletion(
  request: GenerateChatCompletionRequest(
    model: 'llama3.1',
    messages: [userMsg],
    tools: [tool],
  ),
);

print(res1.message.toolCalls);
// [
//   ToolCall(
//     function:
//       ToolCallFunction(
//         name: get_current_weather,
//         arguments: {
//           location: Barcelona, ES,
//           unit: celsius
//         }
//       )
//   )
// ]

// Call your tool here. For this example, we'll just mock the response.
const toolResult = '{"location": "Barcelona, ES", "temperature": 20, "unit": "celsius"}';

// Submit the response of the tool call to the model
final res2 = await client.generateChatCompletion(
  request: GenerateChatCompletionRequest(
    model: 'llama3.1',
    messages: [
      userMsg,
      res1.message,
      Message(
        role: MessageRole.tool,
        content: toolResult,
      ),
    ],
  ),
);
print(res2.message.content);
// The current weather in Barcelona is 20°C.

@davidmigloz davidmigloz self-assigned this Jul 24, 2024
@davidmigloz davidmigloz added t:enhancement New feature or request p:ollama_dart ollama_dart package. labels Jul 24, 2024
@davidmigloz davidmigloz force-pushed the ollama-tools branch 2 times, most recently from 1953eb3 to 3c1032e Compare July 24, 2024 21:04
@davidmigloz davidmigloz added this to the v0.8.0 milestone Jul 24, 2024
@davidmigloz davidmigloz merged commit 1ffdb41 into main Jul 24, 2024
1 check passed
@davidmigloz davidmigloz deleted the ollama-tools branch July 24, 2024 21:42
@davidmigloz
Copy link
Owner Author

(@HavenDV tagging you in case you want to regenerate your client)

@HavenDV
Copy link
Contributor

HavenDV commented Jul 25, 2024

Thank you very much, I'll implement this.
A little later, I planned to implement a periodic poll of the specification and create a new PR/auto-merge/release a new preview version based on this, as I do with the OpenAI specification, but for now your ping helps

KennethKnudsen97 pushed a commit to KennethKnudsen97/langchain_dart that referenced this pull request Oct 1, 2024
KennethKnudsen97 pushed a commit to KennethKnudsen97/langchain_dart that referenced this pull request Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p:ollama_dart ollama_dart package. t:enhancement New feature or request
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants