Structured outputs with GPT-4o/Structured outputs with Azure OpenAI GPT-4o 2024-08-06.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9b078a9b",
   "metadata": {},
   "source": [
    "# Structured outputs with Azure OpenAI GPT-4o 2024-08-06\n",
    "\n",
    "The GPT-4o-2024-08-06 model is designed to perform a wide range of tasks with minimal cost and latency, making it perfect for applications that require fast, real-time text responses. With the introduction of JSON Structured Outputs, it delivers 100% reliability in evaluations, ensuring outputs perfectly match the defined output schemas. This innovation enhances the efficiency and accuracy of AI-powered applications across diverse use cases.\n",
    "\n",
    "### Structured Outputs: Revolutionizing AI Development\n",
    "What are Structured Outputs? Structured Outputs allow developers to define the exact output format using a JSON Schema, making it easier to generate data in the desired structure for various applications. This feature is ideal for seamless integration with other systems.\n",
    "\n",
    "### Key Benefits for Developers\n",
    "- Consistent Data Formats: Outputs adhere to a defined JSON Schema, minimizing the need for manual post-processing.\n",
    "- Flexible Tool Integrations: Supports specific function signatures, enabling precise outputs for tool integrations.\n",
    "- Streamlined Application Development: Simplifies the creation of data payloads, enhancing the overall user experience.\n",
    "\n",
    "### Example Use Case\n",
    "A customer support chatbot can use Structured Outputs to generate responses formatted specifically for logging and analytics. This approach allows for seamless data integration, simplifying the process of logging, analyzing, and taking action based on the data.\n",
    "\n",
    "### Documentation\n",
    "https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/structured-outputs?tabs=python-secure\n",
    "\n",
    "### Regions\n",
    "Available for standard and global deployments in all US regions and Sweden Central.\n",
    "This model adds support for structured outputs when used in conjunction with the new 2024-08-01-preview inference API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a7e2c497",
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime\n",
    "import json\n",
    "import openai\n",
    "import sys\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "from enum import Enum\n",
    "from openai import AzureOpenAI\n",
    "from pydantic import BaseModel\n",
    "from typing import Union"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "11fe3ddf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python version: 3.10.11 (main, May 16 2023, 00:28:57) [GCC 11.2.0]\n",
      "OpenAI version: 1.43.0\n"
     ]
    }
   ],
   "source": [
    "print(f\"Python version: {sys.version}\")\n",
    "print(f\"OpenAI version: {openai.__version__}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1a12cec1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Today is 06-Sep-2024 08:31:49\n"
     ]
    }
   ],
   "source": [
    "print(f\"Today is {datetime.datetime.today().strftime('%d-%b-%Y %H:%M:%S')}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42e877d1",
   "metadata": {},
   "source": [
    "## Azure OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5eb663f8",
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv(\"azure.env\")\n",
    "\n",
    "# Azure OpenAI\n",
    "AZURE_OPENAI_ENDPOINT: str = os.getenv(\"AZURE_OPENAI_API_ENDPOINT\")\n",
    "AZURE_OPENAI_API_KEY: str = os.getenv(\"AZURE_OPENAI_API_KEY\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "5f4c394a",
   "metadata": {},
   "outputs": [],
   "source": [
    "MODEL:str = \"gpt-4o-2024-08-06\" # Your Azure OpenAI model\n",
    "API_VERSION:str = \"2024-08-01-preview\" # Subject to change in the future"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "3cf79125",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Azure OpenAI client\n",
    "client = AzureOpenAI(azure_endpoint=AZURE_OPENAI_ENDPOINT,\n",
    "                     api_key=AZURE_OPENAI_API_KEY,\n",
    "                     api_version=API_VERSION)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d2009e4",
   "metadata": {},
   "source": [
    "## Example 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "46581fac",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Structured outputs\n",
    "\n",
    "class GetInformations(BaseModel):\n",
    "    weather_location: str\n",
    "    country_location: str\n",
    "    country_language: str\n",
    "\n",
    "\n",
    "tools = [openai.pydantic_function_tool(GetInformations)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ca4a3178",
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = []\n",
    "\n",
    "messages.append({\n",
    "    \"role\": \"system\",\n",
    "    \"content\": \"You are an AI helpful assistant.\"\n",
    "})\n",
    "\n",
    "messages.append({\n",
    "    \"role\": \"user\",\n",
    "    \"content\": \"Hello, what is the current weather for the capital of France?\"\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "007ef542",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.chat.completions.create(model=MODEL,\n",
    "                                          messages=messages,\n",
    "                                          tools=tools)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "62f778a3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ChatCompletion(id='chatcmpl-A4OfdQw8pj2NiO8jMnp3qTglw2b7D', choices=[Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content=None, refusal=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_GRryPcMz0VNyPxjWOc2lwtyU', function=Function(arguments='{\"weather_location\":\"Paris\",\"country_location\":\"France\",\"country_language\":\"French\"}', name='GetInformations'), type='function')]))], created=1725611509, model='gpt-4o-2024-08-06', object='chat.completion', service_tier=None, system_fingerprint='fp_b2ffeb16ee', usage=CompletionUsage(completion_tokens=25, prompt_tokens=82, total_tokens=107))"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d91cf269",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;34mFunction(arguments='{\"weather_location\":\"Paris\",\"country_location\":\"France\",\"country_language\":\"French\"}', name='GetInformations')\n"
     ]
    }
   ],
   "source": [
    "print(\"\\033[1;34m\", end=\"\")\n",
    "print(response.choices[0].message.tool_calls[0].function)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "db18cd2d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;34m{\"weather_location\":\"Paris\",\"country_location\":\"France\",\"country_language\":\"French\"}\n"
     ]
    }
   ],
   "source": [
    "print(\"\\033[1;34m\", end=\"\")\n",
    "print(response.choices[0].message.tool_calls[0].function.arguments)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "92e57509",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;34mLocation: Paris\n",
      "Country: France\n",
      "Language: French\n"
     ]
    }
   ],
   "source": [
    "results = json.loads(response.choices[0].message.tool_calls[0].function.arguments)\n",
    "\n",
    "print(\"\\033[1;34m\", end=\"\")\n",
    "print(f\"Location: {results['weather_location']}\")\n",
    "print(f\"Country: {results['country_location']}\")\n",
    "print(f\"Language: {results['country_language']}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "d6c55cfa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;34m{\n",
      "   \"id\": \"chatcmpl-A4OfdQw8pj2NiO8jMnp3qTglw2b7D\",\n",
      "   \"choices\": [\n",
      "      {\n",
      "         \"finish_reason\": \"tool_calls\",\n",
      "         \"index\": 0,\n",
      "         \"logprobs\": null,\n",
      "         \"message\": {\n",
      "            \"content\": null,\n",
      "            \"refusal\": null,\n",
      "            \"role\": \"assistant\",\n",
      "            \"function_call\": null,\n",
      "            \"tool_calls\": [\n",
      "               {\n",
      "                  \"id\": \"call_GRryPcMz0VNyPxjWOc2lwtyU\",\n",
      "                  \"function\": {\n",
      "                     \"arguments\": \"{\\\"weather_location\\\":\\\"Paris\\\",\\\"country_location\\\":\\\"France\\\",\\\"country_language\\\":\\\"French\\\"}\",\n",
      "                     \"name\": \"GetInformations\"\n",
      "                  },\n",
      "                  \"type\": \"function\"\n",
      "               }\n",
      "            ]\n",
      "         }\n",
      "      }\n",
      "   ],\n",
      "   \"created\": 1725611509,\n",
      "   \"model\": \"gpt-4o-2024-08-06\",\n",
      "   \"object\": \"chat.completion\",\n",
      "   \"service_tier\": null,\n",
      "   \"system_fingerprint\": \"fp_b2ffeb16ee\",\n",
      "   \"usage\": {\n",
      "      \"completion_tokens\": 25,\n",
      "      \"prompt_tokens\": 82,\n",
      "      \"total_tokens\": 107\n",
      "   }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print(\"\\033[1;34m\", end=\"\")\n",
    "print(response.model_dump_json(indent=3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6178cb36",
   "metadata": {},
   "source": [
    "## Example 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "44c4cd8f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Structured outputs\n",
    "\n",
    "class ProductSupport(BaseModel):\n",
    "    product: str\n",
    "    brand: str\n",
    "    issue: str\n",
    "\n",
    "\n",
    "tools = [openai.pydantic_function_tool(ProductSupport)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "5d4eb159",
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = []\n",
    "\n",
    "messages.append({\n",
    "    \"role\": \"system\",\n",
    "    \"content\": \"You are an AI helpful assistant.\"\n",
    "})\n",
    "\n",
    "messages.append({\n",
    "    \"role\": \"user\",\n",
    "    \"content\": \"My Contoso headphones are not working anymore. No sound on the left.\"\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "0f481dc7",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.chat.completions.create(model=MODEL,\n",
    "                                          messages=messages,\n",
    "                                          tools=tools)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "58dc7fc7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ChatCompletion(id='chatcmpl-A4OfeX5Q5rZC6S2mfyiSF8m0D0SmM', choices=[Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content=None, refusal=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_GRryPcMz0VNyPxjWOc2lwtyU', function=Function(arguments='{\"product\":\"headphones\",\"brand\":\"Contoso\",\"issue\":\"No sound on the left\"}', name='ProductSupport'), type='function')]))], created=1725611510, model='gpt-4o-2024-08-06', object='chat.completion', service_tier=None, system_fingerprint='fp_b2ffeb16ee', usage=CompletionUsage(completion_tokens=28, prompt_tokens=78, total_tokens=106))"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "d71da1bd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;34m{\"product\":\"headphones\",\"brand\":\"Contoso\",\"issue\":\"No sound on the left\"}\n"
     ]
    }
   ],
   "source": [
    "print(\"\\033[1;34m\", end=\"\")\n",
    "print(response.choices[0].message.tool_calls[0].function.arguments)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "b2d20bb5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;34mProduct: headphones\n",
      "Brand: Contoso\n",
      "issue: No sound on the left\n"
     ]
    }
   ],
   "source": [
    "results = json.loads(response.choices[0].message.tool_calls[0].function.arguments)\n",
    "\n",
    "print(\"\\033[1;34m\", end=\"\")\n",
    "print(f\"Product: {results['product']}\")\n",
    "print(f\"Brand: {results['brand']}\")\n",
    "print(f\"issue: {results['issue']}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "0e005c30",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;34m{\n",
      "   \"id\": \"chatcmpl-A4OfeX5Q5rZC6S2mfyiSF8m0D0SmM\",\n",
      "   \"choices\": [\n",
      "      {\n",
      "         \"finish_reason\": \"tool_calls\",\n",
      "         \"index\": 0,\n",
      "         \"logprobs\": null,\n",
      "         \"message\": {\n",
      "            \"content\": null,\n",
      "            \"refusal\": null,\n",
      "            \"role\": \"assistant\",\n",
      "            \"function_call\": null,\n",
      "            \"tool_calls\": [\n",
      "               {\n",
      "                  \"id\": \"call_GRryPcMz0VNyPxjWOc2lwtyU\",\n",
      "                  \"function\": {\n",
      "                     \"arguments\": \"{\\\"product\\\":\\\"headphones\\\",\\\"brand\\\":\\\"Contoso\\\",\\\"issue\\\":\\\"No sound on the left\\\"}\",\n",
      "                     \"name\": \"ProductSupport\"\n",
      "                  },\n",
      "                  \"type\": \"function\"\n",
      "               }\n",
      "            ]\n",
      "         }\n",
      "      }\n",
      "   ],\n",
      "   \"created\": 1725611510,\n",
      "   \"model\": \"gpt-4o-2024-08-06\",\n",
      "   \"object\": \"chat.completion\",\n",
      "   \"service_tier\": null,\n",
      "   \"system_fingerprint\": \"fp_b2ffeb16ee\",\n",
      "   \"usage\": {\n",
      "      \"completion_tokens\": 28,\n",
      "      \"prompt_tokens\": 78,\n",
      "      \"total_tokens\": 106\n",
      "   }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print(\"\\033[1;34m\", end=\"\")\n",
    "print(response.model_dump_json(indent=3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2ef94d4",
   "metadata": {},
   "source": [
    "## Example 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "3feb2ae1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Structured outputs\n",
    "\n",
    "class CallCenter(BaseModel):\n",
    "    travel_type: str\n",
    "    from_city: str\n",
    "    to_city: str\n",
    "    travel_class:str\n",
    "    travel_date:str\n",
    "    from_country:str\n",
    "    to_country:str\n",
    "    airplane_company:str\n",
    "\n",
    "\n",
    "tools = [openai.pydantic_function_tool(CallCenter)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "ffc51fb0",
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = []\n",
    "\n",
    "messages.append({\n",
    "    \"role\": \"system\",\n",
    "    \"content\": \"You are an AI helpful assistant.\"\n",
    "})\n",
    "\n",
    "messages.append({\n",
    "    \"role\": \"user\",\n",
    "    \"content\": \"Hi, I need to purchase an Air France flight ticket from Paris to Seattle for Tuesday. \\\n",
    "I want a first class ticket.\"\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "a2406209",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.chat.completions.create(model=MODEL,\n",
    "                                          messages=messages,\n",
    "                                          tools=tools)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "08443f68",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ChatCompletion(id='chatcmpl-A4OfeJxhJTT2cAhlgq0WPSWwhZYJi', choices=[Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content=None, refusal=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_QzcRPtRB9FbgexZqy6vW6sS9', function=Function(arguments='{\"travel_type\":\"Air\",\"from_city\":\"Paris\",\"to_city\":\"Seattle\",\"travel_class\":\"First\",\"travel_date\":\"2023-10-10\",\"from_country\":\"France\",\"to_country\":\"United States\",\"airplane_company\":\"Air France\"}', name='CallCenter'), type='function')]))], created=1725611510, model='gpt-4o-2024-08-06', object='chat.completion', service_tier=None, system_fingerprint='fp_b2ffeb16ee', usage=CompletionUsage(completion_tokens=58, prompt_tokens=136, total_tokens=194))"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "41d98955",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;34m{\"travel_type\":\"Air\",\"from_city\":\"Paris\",\"to_city\":\"Seattle\",\"travel_class\":\"First\",\"travel_date\":\"2023-10-10\",\"from_country\":\"France\",\"to_country\":\"United States\",\"airplane_company\":\"Air France\"}\n"
     ]
    }
   ],
   "source": [
    "print(\"\\033[1;34m\", end=\"\")\n",
    "print(response.choices[0].message.tool_calls[0].function.arguments)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "aa451d33",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;34m{\n",
      "   \"id\": \"chatcmpl-A4OfeJxhJTT2cAhlgq0WPSWwhZYJi\",\n",
      "   \"choices\": [\n",
      "      {\n",
      "         \"finish_reason\": \"tool_calls\",\n",
      "         \"index\": 0,\n",
      "         \"logprobs\": null,\n",
      "         \"message\": {\n",
      "            \"content\": null,\n",
      "            \"refusal\": null,\n",
      "            \"role\": \"assistant\",\n",
      "            \"function_call\": null,\n",
      "            \"tool_calls\": [\n",
      "               {\n",
      "                  \"id\": \"call_QzcRPtRB9FbgexZqy6vW6sS9\",\n",
      "                  \"function\": {\n",
      "                     \"arguments\": \"{\\\"travel_type\\\":\\\"Air\\\",\\\"from_city\\\":\\\"Paris\\\",\\\"to_city\\\":\\\"Seattle\\\",\\\"travel_class\\\":\\\"First\\\",\\\"travel_date\\\":\\\"2023-10-10\\\",\\\"from_country\\\":\\\"France\\\",\\\"to_country\\\":\\\"United States\\\",\\\"airplane_company\\\":\\\"Air France\\\"}\",\n",
      "                     \"name\": \"CallCenter\"\n",
      "                  },\n",
      "                  \"type\": \"function\"\n",
      "               }\n",
      "            ]\n",
      "         }\n",
      "      }\n",
      "   ],\n",
      "   \"created\": 1725611510,\n",
      "   \"model\": \"gpt-4o-2024-08-06\",\n",
      "   \"object\": \"chat.completion\",\n",
      "   \"service_tier\": null,\n",
      "   \"system_fingerprint\": \"fp_b2ffeb16ee\",\n",
      "   \"usage\": {\n",
      "      \"completion_tokens\": 58,\n",
      "      \"prompt_tokens\": 136,\n",
      "      \"total_tokens\": 194\n",
      "   }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print(\"\\033[1;34m\", end=\"\")\n",
    "print(response.model_dump_json(indent=3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c053494f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}