Skip to content

Commit

Permalink
doc: Update SelectorGroupChat doc on how to use O3-mini model. (#5657)
Browse files Browse the repository at this point in the history 
Resolves #5408

Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>
  • Loading branch information
@ekzhu @jackgerrits
ekzhu and jackgerrits authored Feb 25, 2025
1 parent a14aeab commit c302b54
Showing 1 changed file with 188 additions and 8 deletions.
196 changes: 188 additions & 8 deletions .../packages/autogen-core/docs/src/user-guide/agentchat-user-guide/selector-group-chat.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -90,7 +90,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -125,7 +125,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -228,7 +228,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -250,7 +250,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -267,6 +267,25 @@
"\"\"\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```{tip}\n",
"Try not to overload the model with too much instruction in the selector prompt.\n",
"\n",
"What is too much? It depends on the capabilities of the model you are using.\n",
"For GPT-4o and equivalents, you can use a selector prompt with a condition for when each speaker should be selected.\n",
"For smaller models such as Phi-4, you should keep the selector prompt as simple as possible\n",
"such as the one used in this example.\n",
"\n",
"Generally, if you find yourself writing multiple conditions for each agent,\n",
"it is a sign that you should consider using a custom selection function,\n",
"or breaking down the task into smaller, sequential tasks to be handled by\n",
"separate agents or teams.\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down Expand Up @@ -300,7 +319,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -410,8 +429,10 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Often times we want better control over the selection process. \n",
"Often times we want better control over the selection process.\n",
"To this end, we can set the `selector_func` argument with a custom selector function to override the default model-based selection.\n",
"This allows us to implement more complex selection logic and state-based transitions.\n",
"\n",
"For instance, we want the Planning Agent to speak immediately after any specialized agent to check the progress.\n",
"\n",
"```{note}\n",
Expand Down Expand Up @@ -518,7 +539,17 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"You can see from the conversation log that the Planning Agent always speaks immediately after the specialized agents."
"You can see from the conversation log that the Planning Agent always speaks immediately after the specialized agents.\n",
"\n",
"```{tip}\n",
"Each participant agent only makes one step (executing tools, generating a response, etc.)\n",
"on each turn. \n",
"If you want an {py:class}`~autogen_agentchat.agents.AssistantAgent` to repeat\n",
"until it stop returning a {py:class}`~autogen_agentchat.messages.ToolCallSummaryMessage`\n",
"when it has finished running all the tools it needs to run, you can do so by\n",
"checking the last message and returning the agent if it is a\n",
"{py:class}`~autogen_agentchat.messages.ToolCallSummaryMessage`.\n",
"```"
]
},
{
Expand Down Expand Up @@ -669,6 +700,155 @@
"Now, the user's feedback is incorporated into the conversation flow,\n",
"and the user can approve or reject the planning agent's decisions."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Using Reasoning Models\n",
"\n",
"So far in the examples, we have used a `gpt-4o` model. Models like `gpt-4o`\n",
"and `gemini-1.5-flash` are great at following instructions, so you can\n",
"have relatively detailed instructions in the selector prompt for the team and the \n",
"system messages for each agent to guide their behavior.\n",
"\n",
"However, if you are using a reasoning model like `o3-mini`, you will need to\n",
"keep the selector prompt and system messages as simple and to the point as possible.\n",
"This is because the reasoning models are already good at coming up with their own \n",
"instructions given the context provided to them.\n",
"\n",
"This also means that we don't need a planning agent to break down the task\n",
"anymore, since the {py:class}`~autogen_agentchat.teams.SelectorGroupChat` that\n",
"uses a reasoning model can do that on its own.\n",
"\n",
"In the following example, we will use `o3-mini` as the model for the\n",
"agents and the team, and we will not use a planning agent.\n",
"Also, we are keeping the selector prompt and system messages as simple as possible."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"model_client = OpenAIChatCompletionClient(model=\"o3-mini\")\n",
"\n",
"web_search_agent = AssistantAgent(\n",
" \"WebSearchAgent\",\n",
" description=\"An agent for searching information on the web.\",\n",
" tools=[search_web_tool],\n",
" model_client=model_client,\n",
" system_message=\"\"\"Use web search tool to find information.\"\"\",\n",
")\n",
"\n",
"data_analyst_agent = AssistantAgent(\n",
" \"DataAnalystAgent\",\n",
" description=\"An agent for performing calculations.\",\n",
" model_client=model_client,\n",
" tools=[percentage_change_tool],\n",
" system_message=\"\"\"Use tool to perform calculation. If you have not seen the data, ask for it.\"\"\",\n",
")\n",
"\n",
"user_proxy_agent = UserProxyAgent(\n",
" \"UserProxyAgent\",\n",
" description=\"A user to approve or disapprove tasks.\",\n",
")\n",
"\n",
"selector_prompt = \"\"\"Select an agent to perform task.\n",
"\n",
"{roles}\n",
"\n",
"Current conversation context:\n",
"{history}\n",
"\n",
"Read the above conversation, then select an agent from {participants} to perform the next task.\n",
"When the task is complete, let the user approve or disapprove the task.\n",
"\"\"\"\n",
"\n",
"team = SelectorGroupChat(\n",
" [web_search_agent, data_analyst_agent, user_proxy_agent],\n",
" model_client=model_client,\n",
" termination_condition=termination, # Use the same termination condition as before.\n",
" selector_prompt=selector_prompt,\n",
" allow_repeated_speaker=True,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"---------- user ----------\n",
"Who was the Miami Heat player with the highest points in the 2006-2007 season, and what was the percentage change in his total rebounds between the 2007-2008 and 2008-2009 seasons?\n",
"---------- WebSearchAgent ----------\n",
"[FunctionCall(id='call_hl7EP6Lp5jj5wEdxeNHTwUVG', arguments='{\"query\": \"Who was the Miami Heat player with the highest points in the 2006-2007 season Miami Heat statistics Dwyane Wade rebounds percentage change 2007-2008 2008-2009 seasons\"}', name='search_web_tool')]\n",
"---------- WebSearchAgent ----------\n",
"[FunctionExecutionResult(content='Here are the total points scored by Miami Heat players in the 2006-2007 season:\\n Udonis Haslem: 844 points\\n Dwayne Wade: 1397 points\\n James Posey: 550 points\\n ...\\n ', call_id='call_hl7EP6Lp5jj5wEdxeNHTwUVG', is_error=False)]\n",
"---------- WebSearchAgent ----------\n",
"Here are the total points scored by Miami Heat players in the 2006-2007 season:\n",
" Udonis Haslem: 844 points\n",
" Dwayne Wade: 1397 points\n",
" James Posey: 550 points\n",
" ...\n",
" \n",
"---------- DataAnalystAgent ----------\n",
"I found that in the 2006–2007 season the player with the highest points was Dwyane Wade (with 1,397 points). Could you please provide Dwyane Wade’s total rebounds for the 2007–2008 and the 2008–2009 seasons so I can calculate the percentage change?\n",
"---------- WebSearchAgent ----------\n",
"[FunctionCall(id='call_lppGTILXDvO9waPwKO66ehK6', arguments='{\"query\": \"Dwyane Wade total rebounds 2007-2008 and 2008-2009 seasons for Miami Heat\"}', name='search_web_tool')]\n",
"---------- WebSearchAgent ----------\n",
"[FunctionExecutionResult(content='The number of total rebounds for Dwayne Wade in the Miami Heat season 2007-2008 is 214.', call_id='call_lppGTILXDvO9waPwKO66ehK6', is_error=False)]\n",
"---------- WebSearchAgent ----------\n",
"The number of total rebounds for Dwayne Wade in the Miami Heat season 2007-2008 is 214.\n",
"---------- DataAnalystAgent ----------\n",
"Could you please provide Dwyane Wade’s total rebounds in the 2008-2009 season?\n",
"---------- WebSearchAgent ----------\n",
"[FunctionCall(id='call_r8DBcbJtQfdtugLtyTrqOvoK', arguments='{\"query\": \"Dwyane Wade total rebounds 2008-2009 season Miami Heat\"}', name='search_web_tool')]\n",
"---------- WebSearchAgent ----------\n",
"[FunctionExecutionResult(content='The number of total rebounds for Dwayne Wade in the Miami Heat season 2008-2009 is 398.', call_id='call_r8DBcbJtQfdtugLtyTrqOvoK', is_error=False)]\n",
"---------- WebSearchAgent ----------\n",
"The number of total rebounds for Dwayne Wade in the Miami Heat season 2008-2009 is 398.\n",
"---------- DataAnalystAgent ----------\n",
"[FunctionCall(id='call_4jejv1wM7V1osbBCxJze8aQM', arguments='{\"start\": 214, \"end\": 398}', name='percentage_change_tool')]\n",
"---------- DataAnalystAgent ----------\n",
"[FunctionExecutionResult(content='85.98130841121495', call_id='call_4jejv1wM7V1osbBCxJze8aQM', is_error=False)]\n",
"---------- DataAnalystAgent ----------\n",
"85.98130841121495\n",
"---------- DataAnalystAgent ----------\n",
"Dwyane Wade was the Miami Heat player with the highest total points (1,397) during the 2006-2007 season. His total rebounds increased by approximately 86% from 214 in the 2007-2008 season to 398 in the 2008-2009 season.\n",
"---------- UserProxyAgent ----------\n",
"Approve. TERMINATE\n"
]
},
{
"data": {
"text/plain": [
"TaskResult(messages=[TextMessage(source='user', models_usage=None, content='Who was the Miami Heat player with the highest points in the 2006-2007 season, and what was the percentage change in his total rebounds between the 2007-2008 and 2008-2009 seasons?', type='TextMessage'), ToolCallRequestEvent(source='WebSearchAgent', models_usage=RequestUsage(prompt_tokens=103, completion_tokens=384), content=[FunctionCall(id='call_hl7EP6Lp5jj5wEdxeNHTwUVG', arguments='{\"query\": \"Who was the Miami Heat player with the highest points in the 2006-2007 season Miami Heat statistics Dwyane Wade rebounds percentage change 2007-2008 2008-2009 seasons\"}', name='search_web_tool')], type='ToolCallRequestEvent'), ToolCallExecutionEvent(source='WebSearchAgent', models_usage=None, content=[FunctionExecutionResult(content='Here are the total points scored by Miami Heat players in the 2006-2007 season:\\n Udonis Haslem: 844 points\\n Dwayne Wade: 1397 points\\n James Posey: 550 points\\n ...\\n ', call_id='call_hl7EP6Lp5jj5wEdxeNHTwUVG', is_error=False)], type='ToolCallExecutionEvent'), ToolCallSummaryMessage(source='WebSearchAgent', models_usage=None, content='Here are the total points scored by Miami Heat players in the 2006-2007 season:\\n Udonis Haslem: 844 points\\n Dwayne Wade: 1397 points\\n James Posey: 550 points\\n ...\\n ', type='ToolCallSummaryMessage'), TextMessage(source='DataAnalystAgent', models_usage=RequestUsage(prompt_tokens=183, completion_tokens=1038), content='I found that in the 2006–2007 season the player with the highest points was Dwyane Wade (with 1,397 points). Could you please provide Dwyane Wade’s total rebounds for the 2007–2008 and the 2008–2009 seasons so I can calculate the percentage change?', type='TextMessage'), ToolCallRequestEvent(source='WebSearchAgent', models_usage=RequestUsage(prompt_tokens=299, completion_tokens=109), content=[FunctionCall(id='call_lppGTILXDvO9waPwKO66ehK6', arguments='{\"query\": \"Dwyane Wade total rebounds 2007-2008 and 2008-2009 seasons for Miami Heat\"}', name='search_web_tool')], type='ToolCallRequestEvent'), ToolCallExecutionEvent(source='WebSearchAgent', models_usage=None, content=[FunctionExecutionResult(content='The number of total rebounds for Dwayne Wade in the Miami Heat season 2007-2008 is 214.', call_id='call_lppGTILXDvO9waPwKO66ehK6', is_error=False)], type='ToolCallExecutionEvent'), ToolCallSummaryMessage(source='WebSearchAgent', models_usage=None, content='The number of total rebounds for Dwayne Wade in the Miami Heat season 2007-2008 is 214.', type='ToolCallSummaryMessage'), TextMessage(source='DataAnalystAgent', models_usage=RequestUsage(prompt_tokens=291, completion_tokens=224), content='Could you please provide Dwyane Wade’s total rebounds in the 2008-2009 season?', type='TextMessage'), ToolCallRequestEvent(source='WebSearchAgent', models_usage=RequestUsage(prompt_tokens=401, completion_tokens=37), content=[FunctionCall(id='call_r8DBcbJtQfdtugLtyTrqOvoK', arguments='{\"query\": \"Dwyane Wade total rebounds 2008-2009 season Miami Heat\"}', name='search_web_tool')], type='ToolCallRequestEvent'), ToolCallExecutionEvent(source='WebSearchAgent', models_usage=None, content=[FunctionExecutionResult(content='The number of total rebounds for Dwayne Wade in the Miami Heat season 2008-2009 is 398.', call_id='call_r8DBcbJtQfdtugLtyTrqOvoK', is_error=False)], type='ToolCallExecutionEvent'), ToolCallSummaryMessage(source='WebSearchAgent', models_usage=None, content='The number of total rebounds for Dwayne Wade in the Miami Heat season 2008-2009 is 398.', type='ToolCallSummaryMessage'), ToolCallRequestEvent(source='DataAnalystAgent', models_usage=RequestUsage(prompt_tokens=353, completion_tokens=158), content=[FunctionCall(id='call_4jejv1wM7V1osbBCxJze8aQM', arguments='{\"start\": 214, \"end\": 398}', name='percentage_change_tool')], type='ToolCallRequestEvent'), ToolCallExecutionEvent(source='DataAnalystAgent', models_usage=None, content=[FunctionExecutionResult(content='85.98130841121495', call_id='call_4jejv1wM7V1osbBCxJze8aQM', is_error=False)], type='ToolCallExecutionEvent'), ToolCallSummaryMessage(source='DataAnalystAgent', models_usage=None, content='85.98130841121495', type='ToolCallSummaryMessage'), TextMessage(source='DataAnalystAgent', models_usage=RequestUsage(prompt_tokens=394, completion_tokens=138), content='Dwyane Wade was the Miami Heat player with the highest total points (1,397) during the 2006-2007 season. His total rebounds increased by approximately 86% from 214 in the 2007-2008 season to 398 in the 2008-2009 season.', type='TextMessage'), UserInputRequestedEvent(source='UserProxyAgent', models_usage=None, request_id='b3b05408-73fc-47d4-b832-16c9f447cd6e', content='', type='UserInputRequestedEvent'), TextMessage(source='UserProxyAgent', models_usage=None, content='Approve. TERMINATE', type='TextMessage')], stop_reason=\"Text 'TERMINATE' mentioned\")"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"await Console(team.run_stream(task=task))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```{tip}\n",
"For more guidance on how to prompt reasoning models, see the\n",
"Azure AI Services Blog on [Prompt Engineering for OpenAI's O1 and O3-mini Reasoning Models](https://techcommunity.microsoft.com/blog/azure-ai-services-blog/prompt-engineering-for-openai%E2%80%99s-o1-and-o3-mini-reasoning-models/4374010)\n",
"```"
]
}
],
"metadata": {
Expand Down

0 comments on commit c302b54

Please sign in to comment.