Chapter-10/simple-sequential-chain-with-llama-2-and-langchain.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bae624bb",
   "metadata": {
    "papermill": {
     "duration": 0.00579,
     "end_time": "2023-10-28T10:17:30.513094",
     "exception": false,
     "start_time": "2023-10-28T10:17:30.507304",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Introduction\n",
    "\n",
    "\n",
    "## Objective  \n",
    "\n",
    "Use Llama 2 and Langchain to create a multi-step task chain. \n",
    "\n",
    "## Models details  \n",
    "\n",
    "* **Model #1**: Llama 2  \n",
    "* **Variation**: 7b-chat-hf    \n",
    "* **Version**: V1  \n",
    "* **Framework**: PyTorch  \n",
    "\n",
    "\n",
    "LlaMA 2 model is pretrained and fine-tuned with 2 Trillion tokens and 7 to 70 Billion parameters which makes it one of the powerful open source models. It is a highly improvement over LlaMA 1 model. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d430968",
   "metadata": {
    "papermill": {
     "duration": 0.005365,
     "end_time": "2023-10-28T10:17:30.524269",
     "exception": false,
     "start_time": "2023-10-28T10:17:30.518904",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# InstalIing, imports, utils"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bc4d9a0",
   "metadata": {
    "papermill": {
     "duration": 0.004932,
     "end_time": "2023-10-28T10:17:30.534431",
     "exception": false,
     "start_time": "2023-10-28T10:17:30.529499",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Install packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "5e896a86",
   "metadata": {
    "_kg_hide-input": true,
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2023-10-28T10:17:30.546487Z",
     "iopub.status.busy": "2023-10-28T10:17:30.546107Z",
     "iopub.status.idle": "2023-10-28T10:21:05.257281Z",
     "shell.execute_reply": "2023-10-28T10:21:05.256188Z"
    },
    "papermill": {
     "duration": 214.720154,
     "end_time": "2023-10-28T10:21:05.259905",
     "exception": false,
     "start_time": "2023-10-28T10:17:30.539751",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: transformers==4.33.0 in /opt/conda/lib/python3.10/site-packages (4.33.0)\r\n",
      "Requirement already satisfied: accelerate==0.22.0 in /opt/conda/lib/python3.10/site-packages (0.22.0)\r\n",
      "Collecting einops==0.6.1\r\n",
      "  Downloading einops-0.6.1-py3-none-any.whl (42 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m42.2/42.2 kB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting langchain==0.0.300\r\n",
      "  Downloading langchain-0.0.300-py3-none-any.whl (1.7 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m24.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting xformers==0.0.21\r\n",
      "  Downloading xformers-0.0.21-cp310-cp310-manylinux2014_x86_64.whl (167.0 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m167.0/167.0 MB\u001b[0m \u001b[31m8.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting bitsandbytes==0.41.1\r\n",
      "  Downloading bitsandbytes-0.41.1-py3-none-any.whl (92.6 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.6/92.6 MB\u001b[0m \u001b[31m13.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting sentence_transformers==2.2.2\r\n",
      "  Downloading sentence-transformers-2.2.2.tar.gz (85 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.0/86.0 kB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l-\b \bdone\r\n",
      "\u001b[?25hCollecting chromadb==0.4.12\r\n",
      "  Downloading chromadb-0.4.12-py3-none-any.whl (426 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m426.5/426.5 kB\u001b[0m \u001b[31m28.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hRequirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (3.12.2)\r\n",
      "Requirement already satisfied: huggingface-hub<1.0,>=0.15.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (0.16.4)\r\n",
      "Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (1.23.5)\r\n",
      "Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (21.3)\r\n",
      "Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (6.0)\r\n",
      "Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (2023.6.3)\r\n",
      "Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (2.31.0)\r\n",
      "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (0.13.3)\r\n",
      "Requirement already satisfied: safetensors>=0.3.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (0.3.3)\r\n",
      "Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (4.66.1)\r\n",
      "Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from accelerate==0.22.0) (5.9.3)\r\n",
      "Requirement already satisfied: torch>=1.10.0 in /opt/conda/lib/python3.10/site-packages (from accelerate==0.22.0) (2.0.0)\r\n",
      "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (2.0.17)\r\n",
      "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (3.8.4)\r\n",
      "Requirement already satisfied: anyio<4.0 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (3.7.0)\r\n",
      "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (4.0.2)\r\n",
      "Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (0.6.0)\r\n",
      "Collecting jsonpatch<2.0,>=1.33 (from langchain==0.0.300)\r\n",
      "  Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\r\n",
      "Collecting langsmith<0.1.0,>=0.0.38 (from langchain==0.0.300)\r\n",
      "  Downloading langsmith-0.0.53-py3-none-any.whl (43 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.3/43.3 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hRequirement already satisfied: numexpr<3.0.0,>=2.8.4 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (2.8.5)\r\n",
      "Requirement already satisfied: pydantic<3,>=1 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (1.10.9)\r\n",
      "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (8.2.2)\r\n",
      "Collecting torch>=1.10.0 (from accelerate==0.22.0)\r\n",
      "  Downloading torch-2.0.1-cp310-cp310-manylinux1_x86_64.whl (619.9 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m619.9/619.9 MB\u001b[0m \u001b[31m1.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hRequirement already satisfied: torchvision in /opt/conda/lib/python3.10/site-packages (from sentence_transformers==2.2.2) (0.15.1)\r\n",
      "Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.10/site-packages (from sentence_transformers==2.2.2) (1.2.2)\r\n",
      "Requirement already satisfied: scipy in /opt/conda/lib/python3.10/site-packages (from sentence_transformers==2.2.2) (1.11.2)\r\n",
      "Requirement already satisfied: nltk in /opt/conda/lib/python3.10/site-packages (from sentence_transformers==2.2.2) (3.2.4)\r\n",
      "Requirement already satisfied: sentencepiece in /opt/conda/lib/python3.10/site-packages (from sentence_transformers==2.2.2) (0.1.99)\r\n",
      "Collecting chroma-hnswlib==0.7.3 (from chromadb==0.4.12)\r\n",
      "  Downloading chroma_hnswlib-0.7.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.4/2.4 MB\u001b[0m \u001b[31m9.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hRequirement already satisfied: fastapi<0.100.0,>=0.95.2 in /opt/conda/lib/python3.10/site-packages (from chromadb==0.4.12) (0.98.0)\r\n",
      "Requirement already satisfied: uvicorn[standard]>=0.18.3 in /opt/conda/lib/python3.10/site-packages (from chromadb==0.4.12) (0.22.0)\r\n",
      "Collecting posthog>=2.4.0 (from chromadb==0.4.12)\r\n",
      "  Downloading posthog-3.0.2-py2.py3-none-any.whl (37 kB)\r\n",
      "Requirement already satisfied: typing-extensions>=4.5.0 in /opt/conda/lib/python3.10/site-packages (from chromadb==0.4.12) (4.6.3)\r\n",
      "Collecting pulsar-client>=3.1.0 (from chromadb==0.4.12)\r\n",
      "  Downloading pulsar_client-3.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.4/5.4 MB\u001b[0m \u001b[31m6.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting onnxruntime>=1.14.1 (from chromadb==0.4.12)\r\n",
      "  Downloading onnxruntime-1.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.2 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.2/6.2 MB\u001b[0m \u001b[31m56.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting pypika>=0.48.9 (from chromadb==0.4.12)\r\n",
      "  Downloading PyPika-0.48.9.tar.gz (67 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.3/67.3 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25h  Installing build dependencies ... \u001b[?25l-\b \b\\\b \b|\b \b/\b \bdone\r\n",
      "\u001b[?25h  Getting requirements to build wheel ... \u001b[?25l-\b \bdone\r\n",
      "\u001b[?25h  Preparing metadata (pyproject.toml) ... \u001b[?25l-\b \bdone\r\n",
      "\u001b[?25hCollecting overrides>=7.3.1 (from chromadb==0.4.12)\r\n",
      "  Downloading overrides-7.4.0-py3-none-any.whl (17 kB)\r\n",
      "Requirement already satisfied: importlib-resources in /opt/conda/lib/python3.10/site-packages (from chromadb==0.4.12) (5.12.0)\r\n",
      "Collecting bcrypt>=4.0.1 (from chromadb==0.4.12)\r\n",
      "  Downloading bcrypt-4.0.1-cp36-abi3-manylinux_2_28_x86_64.whl (593 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m593.7/593.7 kB\u001b[0m \u001b[31m44.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hRequirement already satisfied: typer>=0.9.0 in /opt/conda/lib/python3.10/site-packages (from chromadb==0.4.12) (0.9.0)\r\n",
      "Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.22.0) (1.12)\r\n",
      "Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.22.0) (3.1)\r\n",
      "Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.22.0) (3.1.2)\r\n",
      "Collecting nvidia-cuda-nvrtc-cu11==11.7.99 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.0/21.0 MB\u001b[0m \u001b[31m57.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting nvidia-cuda-runtime-cu11==11.7.99 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m849.3/849.3 kB\u001b[0m \u001b[31m50.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting nvidia-cuda-cupti-cu11==11.7.101 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading nvidia_cuda_cupti_cu11-11.7.101-py3-none-manylinux1_x86_64.whl (11.8 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.8/11.8 MB\u001b[0m \u001b[31m75.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting nvidia-cudnn-cu11==8.5.0.96 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m557.1/557.1 MB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting nvidia-cublas-cu11==11.10.3.66 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m317.1/317.1 MB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting nvidia-cufft-cu11==10.9.0.58 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading nvidia_cufft_cu11-10.9.0.58-py3-none-manylinux1_x86_64.whl (168.4 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m168.4/168.4 MB\u001b[0m \u001b[31m7.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting nvidia-curand-cu11==10.2.10.91 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading nvidia_curand_cu11-10.2.10.91-py3-none-manylinux1_x86_64.whl (54.6 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m54.6/54.6 MB\u001b[0m \u001b[31m22.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting nvidia-cusolver-cu11==11.4.0.1 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading nvidia_cusolver_cu11-11.4.0.1-2-py3-none-manylinux1_x86_64.whl (102.6 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m102.6/102.6 MB\u001b[0m \u001b[31m12.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting nvidia-cusparse-cu11==11.7.4.91 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading nvidia_cusparse_cu11-11.7.4.91-py3-none-manylinux1_x86_64.whl (173.2 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m173.2/173.2 MB\u001b[0m \u001b[31m7.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting nvidia-nccl-cu11==2.14.3 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading nvidia_nccl_cu11-2.14.3-py3-none-manylinux1_x86_64.whl (177.1 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m177.1/177.1 MB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting nvidia-nvtx-cu11==11.7.91 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading nvidia_nvtx_cu11-11.7.91-py3-none-manylinux1_x86_64.whl (98 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m98.6/98.6 kB\u001b[0m \u001b[31m7.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting triton==2.0.0 (from torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading triton-2.0.0-1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.3 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m63.3/63.3 MB\u001b[0m \u001b[31m20.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hRequirement already satisfied: setuptools in /opt/conda/lib/python3.10/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.10.0->accelerate==0.22.0) (68.0.0)\r\n",
      "Requirement already satisfied: wheel in /opt/conda/lib/python3.10/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.10.0->accelerate==0.22.0) (0.40.0)\r\n",
      "Collecting cmake (from triton==2.0.0->torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading cmake-3.27.7-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (26.0 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m26.0/26.0 MB\u001b[0m \u001b[31m50.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting lit (from triton==2.0.0->torch>=1.10.0->accelerate==0.22.0)\r\n",
      "  Downloading lit-17.0.3.tar.gz (154 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m154.7/154.7 kB\u001b[0m \u001b[31m13.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25h  Installing build dependencies ... \u001b[?25l-\b \b\\\b \b|\b \bdone\r\n",
      "\u001b[?25h  Getting requirements to build wheel ... \u001b[?25l-\b \bdone\r\n",
      "\u001b[?25h  Installing backend dependencies ... \u001b[?25l-\b \b\\\b \bdone\r\n",
      "\u001b[?25h  Preparing metadata (pyproject.toml) ... \u001b[?25l-\b \bdone\r\n",
      "\u001b[?25hRequirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (23.1.0)\r\n",
      "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (3.1.0)\r\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (6.0.4)\r\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (1.9.2)\r\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (1.3.3)\r\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (1.3.1)\r\n",
      "Requirement already satisfied: idna>=2.8 in /opt/conda/lib/python3.10/site-packages (from anyio<4.0->langchain==0.0.300) (3.4)\r\n",
      "Requirement already satisfied: sniffio>=1.1 in /opt/conda/lib/python3.10/site-packages (from anyio<4.0->langchain==0.0.300) (1.3.0)\r\n",
      "Requirement already satisfied: exceptiongroup in /opt/conda/lib/python3.10/site-packages (from anyio<4.0->langchain==0.0.300) (1.1.1)\r\n",
      "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /opt/conda/lib/python3.10/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain==0.0.300) (3.20.1)\r\n",
      "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /opt/conda/lib/python3.10/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain==0.0.300) (0.9.0)\r\n",
      "Requirement already satisfied: starlette<0.28.0,>=0.27.0 in /opt/conda/lib/python3.10/site-packages (from fastapi<0.100.0,>=0.95.2->chromadb==0.4.12) (0.27.0)\r\n",
      "Requirement already satisfied: fsspec in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.15.1->transformers==4.33.0) (2023.9.0)\r\n",
      "Requirement already satisfied: jsonpointer>=1.9 in /opt/conda/lib/python3.10/site-packages (from jsonpatch<2.0,>=1.33->langchain==0.0.300) (2.0)\r\n",
      "Collecting coloredlogs (from onnxruntime>=1.14.1->chromadb==0.4.12)\r\n",
      "  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hRequirement already satisfied: flatbuffers in /opt/conda/lib/python3.10/site-packages (from onnxruntime>=1.14.1->chromadb==0.4.12) (23.5.26)\r\n",
      "Requirement already satisfied: protobuf in /opt/conda/lib/python3.10/site-packages (from onnxruntime>=1.14.1->chromadb==0.4.12) (3.20.3)\r\n",
      "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.10/site-packages (from packaging>=20.0->transformers==4.33.0) (3.0.9)\r\n",
      "Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.10/site-packages (from posthog>=2.4.0->chromadb==0.4.12) (1.16.0)\r\n",
      "Collecting monotonic>=1.5 (from posthog>=2.4.0->chromadb==0.4.12)\r\n",
      "  Downloading monotonic-1.6-py2.py3-none-any.whl (8.2 kB)\r\n",
      "Requirement already satisfied: backoff>=1.10.0 in /opt/conda/lib/python3.10/site-packages (from posthog>=2.4.0->chromadb==0.4.12) (2.2.1)\r\n",
      "Requirement already satisfied: python-dateutil>2.1 in /opt/conda/lib/python3.10/site-packages (from posthog>=2.4.0->chromadb==0.4.12) (2.8.2)\r\n",
      "Requirement already satisfied: certifi in /opt/conda/lib/python3.10/site-packages (from pulsar-client>=3.1.0->chromadb==0.4.12) (2023.7.22)\r\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers==4.33.0) (1.26.15)\r\n",
      "Requirement already satisfied: greenlet!=0.4.17 in /opt/conda/lib/python3.10/site-packages (from SQLAlchemy<3,>=1.4->langchain==0.0.300) (2.0.2)\r\n",
      "Requirement already satisfied: click<9.0.0,>=7.1.1 in /opt/conda/lib/python3.10/site-packages (from typer>=0.9.0->chromadb==0.4.12) (8.1.7)\r\n",
      "Requirement already satisfied: h11>=0.8 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (0.14.0)\r\n",
      "Requirement already satisfied: httptools>=0.5.0 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (0.6.0)\r\n",
      "Requirement already satisfied: python-dotenv>=0.13 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (1.0.0)\r\n",
      "Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (0.17.0)\r\n",
      "Requirement already satisfied: watchfiles>=0.13 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (0.20.0)\r\n",
      "Requirement already satisfied: websockets>=10.4 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (11.0.3)\r\n",
      "Requirement already satisfied: joblib>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from scikit-learn->sentence_transformers==2.2.2) (1.3.2)\r\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from scikit-learn->sentence_transformers==2.2.2) (3.1.0)\r\n",
      "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/lib/python3.10/site-packages (from torchvision->sentence_transformers==2.2.2) (9.5.0)\r\n",
      "Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/lib/python3.10/site-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain==0.0.300) (1.0.0)\r\n",
      "Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime>=1.14.1->chromadb==0.4.12)\r\n",
      "  Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hRequirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->accelerate==0.22.0) (2.1.3)\r\n",
      "Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.10/site-packages (from sympy->torch>=1.10.0->accelerate==0.22.0) (1.3.0)\r\n",
      "Building wheels for collected packages: sentence_transformers, pypika, lit\r\n",
      "  Building wheel for sentence_transformers (setup.py) ... \u001b[?25l-\b \b\\\b \bdone\r\n",
      "\u001b[?25h  Created wheel for sentence_transformers: filename=sentence_transformers-2.2.2-py3-none-any.whl size=125926 sha256=b55ab8c0f93f78940fa8228f795e959633caac0e1e4877793cb3469402fbca43\r\n",
      "  Stored in directory: /root/.cache/pip/wheels/62/f2/10/1e606fd5f02395388f74e7462910fe851042f97238cbbd902f\r\n",
      "  Building wheel for pypika (pyproject.toml) ... \u001b[?25l-\b \bdone\r\n",
      "\u001b[?25h  Created wheel for pypika: filename=PyPika-0.48.9-py2.py3-none-any.whl size=53723 sha256=527e2a8965ddf20f3f35a8182c626c1c697ff94017bf1b9fd832f9ad0ef1244e\r\n",
      "  Stored in directory: /root/.cache/pip/wheels/e1/26/51/d0bffb3d2fd82256676d7ad3003faea3bd6dddc9577af665f4\r\n",
      "  Building wheel for lit (pyproject.toml) ... \u001b[?25l-\b \bdone\r\n",
      "\u001b[?25h  Created wheel for lit: filename=lit-17.0.3-py3-none-any.whl size=93257 sha256=54a7c943028c0bbb8b720eb0fbbf018245cea1ec0c0e65c89f5dd1c95f86a537\r\n",
      "  Stored in directory: /root/.cache/pip/wheels/ac/b8/42/f6f56aba870f9f3cc895b2e0c970ececaafc7d191217fa10a4\r\n",
      "Successfully built sentence_transformers pypika lit\r\n",
      "Installing collected packages: pypika, monotonic, lit, cmake, bitsandbytes, pulsar-client, overrides, nvidia-nvtx-cu11, nvidia-nccl-cu11, nvidia-cusparse-cu11, nvidia-curand-cu11, nvidia-cufft-cu11, nvidia-cuda-runtime-cu11, nvidia-cuda-nvrtc-cu11, nvidia-cuda-cupti-cu11, nvidia-cublas-cu11, jsonpatch, humanfriendly, einops, chroma-hnswlib, bcrypt, posthog, nvidia-cusolver-cu11, nvidia-cudnn-cu11, langsmith, coloredlogs, onnxruntime, langchain, chromadb, triton, torch, xformers, sentence_transformers\r\n",
      "  Attempting uninstall: overrides\r\n",
      "    Found existing installation: overrides 6.5.0\r\n",
      "    Uninstalling overrides-6.5.0:\r\n",
      "      Successfully uninstalled overrides-6.5.0\r\n",
      "  Attempting uninstall: jsonpatch\r\n",
      "    Found existing installation: jsonpatch 1.32\r\n",
      "    Uninstalling jsonpatch-1.32:\r\n",
      "      Successfully uninstalled jsonpatch-1.32\r\n",
      "  Attempting uninstall: torch\r\n",
      "    Found existing installation: torch 2.0.0\r\n",
      "    Uninstalling torch-2.0.0:\r\n",
      "      Successfully uninstalled torch-2.0.0\r\n",
      "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\r\n",
      "google-cloud-pubsublite 1.8.2 requires overrides<7.0.0,>=6.0.1, but you have overrides 7.4.0 which is incompatible.\r\n",
      "jupyterlab-lsp 4.2.0 requires jupyter-lsp>=2.0.0, but you have jupyter-lsp 1.5.1 which is incompatible.\r\n",
      "torchdata 0.6.0 requires torch==2.0.0, but you have torch 2.0.1 which is incompatible.\u001b[0m\u001b[31m\r\n",
      "\u001b[0mSuccessfully installed bcrypt-4.0.1 bitsandbytes-0.41.1 chroma-hnswlib-0.7.3 chromadb-0.4.12 cmake-3.27.7 coloredlogs-15.0.1 einops-0.6.1 humanfriendly-10.0 jsonpatch-1.33 langchain-0.0.300 langsmith-0.0.53 lit-17.0.3 monotonic-1.6 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 onnxruntime-1.16.1 overrides-7.3.1 posthog-3.0.2 pulsar-client-3.3.0 pypika-0.48.9 sentence_transformers-2.2.2 torch-2.0.1 triton-2.0.0 xformers-0.0.21\r\n"
     ]
    }
   ],
   "source": [
    "!pip install transformers==4.33.0 accelerate==0.22.0 einops==0.6.1 langchain==0.0.300 xformers==0.0.21 \\\n",
    "bitsandbytes==0.41.1 sentence_transformers==2.2.2 chromadb==0.4.12"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d03abc71",
   "metadata": {
    "papermill": {
     "duration": 0.068293,
     "end_time": "2023-10-28T10:21:05.398368",
     "exception": false,
     "start_time": "2023-10-28T10:21:05.330075",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Import packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "073d37f7",
   "metadata": {
    "_kg_hide-input": true,
    "execution": {
     "iopub.execute_input": "2023-10-28T10:21:05.540353Z",
     "iopub.status.busy": "2023-10-28T10:21:05.539869Z",
     "iopub.status.idle": "2023-10-28T10:21:15.102902Z",
     "shell.execute_reply": "2023-10-28T10:21:15.101762Z"
    },
    "papermill": {
     "duration": 9.637119,
     "end_time": "2023-10-28T10:21:15.105580",
     "exception": false,
     "start_time": "2023-10-28T10:21:05.468461",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from torch import cuda, bfloat16\n",
    "import torch\n",
    "import transformers\n",
    "from transformers import AutoTokenizer\n",
    "from time import time\n",
    "\n",
    "from langchain.llms import HuggingFacePipeline\n",
    "from langchain.chains import LLMChain, SimpleSequentialChain\n",
    "from langchain import PromptTemplate\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebc9009d",
   "metadata": {
    "papermill": {
     "duration": 0.06812,
     "end_time": "2023-10-28T10:21:15.248194",
     "exception": false,
     "start_time": "2023-10-28T10:21:15.180074",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Initialize model, tokenizer, query pipeline  \n",
    "\n",
    "Define the model, the device, and the bitsandbytes configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0a5d51ae",
   "metadata": {
    "_kg_hide-input": true,
    "execution": {
     "iopub.execute_input": "2023-10-28T10:21:15.388623Z",
     "iopub.status.busy": "2023-10-28T10:21:15.387585Z",
     "iopub.status.idle": "2023-10-28T10:21:15.562790Z",
     "shell.execute_reply": "2023-10-28T10:21:15.562008Z"
    },
    "papermill": {
     "duration": 0.247439,
     "end_time": "2023-10-28T10:21:15.564844",
     "exception": false,
     "start_time": "2023-10-28T10:21:15.317405",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_1_id = '/kaggle/input/llama-2/pytorch/7b-chat-hf/1'\n",
    "\n",
    "model_2_id = '/kaggle/input/llama-2/pytorch/13b-chat-hf/1'\n",
    "\n",
    "device = f'cuda:{cuda.current_device()}' if cuda.is_available() else 'cpu'\n",
    "\n",
    "# set quantization configuration to load large model with less GPU memory\n",
    "# this requires the `bitsandbytes` library\n",
    "bnb_config = transformers.BitsAndBytesConfig(\n",
    "    load_in_4bit=True,\n",
    "    bnb_4bit_quant_type='nf4',\n",
    "    bnb_4bit_use_double_quant=True,\n",
    "    bnb_4bit_compute_dtype=bfloat16\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6087ae7",
   "metadata": {
    "papermill": {
     "duration": 0.076058,
     "end_time": "2023-10-28T10:21:15.711091",
     "exception": false,
     "start_time": "2023-10-28T10:21:15.635033",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Prepare the model and the tokenizer.  \n",
    "\n",
    "We perform this operation for both models (7b & 13b)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7158c5fc",
   "metadata": {
    "_kg_hide-input": true,
    "execution": {
     "iopub.execute_input": "2023-10-28T10:21:15.851140Z",
     "iopub.status.busy": "2023-10-28T10:21:15.850824Z",
     "iopub.status.idle": "2023-10-28T10:23:29.634127Z",
     "shell.execute_reply": "2023-10-28T10:23:29.633124Z"
    },
    "papermill": {
     "duration": 133.855565,
     "end_time": "2023-10-28T10:23:29.636407",
     "exception": false,
     "start_time": "2023-10-28T10:21:15.780842",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n",
      "  warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1739d6d931da47098573cab0ad340914",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:362: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.\n",
      "  warnings.warn(\n",
      "/opt/conda/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:367: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prepare model #1, tokenizer: 133.778 sec.\n"
     ]
    }
   ],
   "source": [
    "time_1 = time()\n",
    "model_1_config = transformers.AutoConfig.from_pretrained(\n",
    "    model_1_id,\n",
    ")\n",
    "model_1 = transformers.AutoModelForCausalLM.from_pretrained(\n",
    "    model_1_id,\n",
    "    trust_remote_code=True,\n",
    "    config=model_1_config,\n",
    "    quantization_config=None,\n",
    "    device_map='auto',\n",
    ")\n",
    "tokenizer_1 = AutoTokenizer.from_pretrained(model_1_id)\n",
    "time_2 = time()\n",
    "print(f\"Prepare model #1, tokenizer: {round(time_2-time_1, 3)} sec.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2a10212",
   "metadata": {
    "papermill": {
     "duration": 0.070773,
     "end_time": "2023-10-28T10:23:29.780436",
     "exception": false,
     "start_time": "2023-10-28T10:23:29.709663",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Define a pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d1992340",
   "metadata": {
    "_kg_hide-input": true,
    "execution": {
     "iopub.execute_input": "2023-10-28T10:23:29.926925Z",
     "iopub.status.busy": "2023-10-28T10:23:29.926541Z",
     "iopub.status.idle": "2023-10-28T10:23:32.489698Z",
     "shell.execute_reply": "2023-10-28T10:23:32.488664Z"
    },
    "papermill": {
     "duration": 2.638961,
     "end_time": "2023-10-28T10:23:32.492063",
     "exception": false,
     "start_time": "2023-10-28T10:23:29.853102",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prepare pipeline #1: 2.557 sec.\n"
     ]
    }
   ],
   "source": [
    "time_1 = time()\n",
    "query_pipeline_1 = transformers.pipeline(\n",
    "        \"text-generation\",\n",
    "        model=model_1,\n",
    "        tokenizer=tokenizer_1,\n",
    "        torch_dtype=torch.float16,\n",
    "        device_map=\"auto\",)\n",
    "time_2 = time()\n",
    "print(f\"Prepare pipeline #1: {round(time_2-time_1, 3)} sec.\")\n",
    "\n",
    "llm_1 = HuggingFacePipeline(pipeline=query_pipeline_1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1056f98d",
   "metadata": {
    "papermill": {
     "duration": 0.071161,
     "end_time": "2023-10-28T10:23:32.634396",
     "exception": false,
     "start_time": "2023-10-28T10:23:32.563235",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "\n",
    "We test it by running a simple query."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "54baadbe",
   "metadata": {
    "_kg_hide-input": true,
    "execution": {
     "iopub.execute_input": "2023-10-28T10:23:32.777862Z",
     "iopub.status.busy": "2023-10-28T10:23:32.777520Z",
     "iopub.status.idle": "2023-10-28T10:23:37.222522Z",
     "shell.execute_reply": "2023-10-28T10:23:37.221627Z"
    },
    "papermill": {
     "duration": 4.518662,
     "end_time": "2023-10-28T10:23:37.225900",
     "exception": false,
     "start_time": "2023-10-28T10:23:32.707238",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.10/site-packages/transformers/generation/utils.py:1417: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation )\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'\\n\\nAnswer: Escargots (Snails)'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# checking model #1\n",
    "llm_1(prompt=\"What is the most popular food in France for tourists? Just return the name of the food.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84737a41",
   "metadata": {
    "papermill": {
     "duration": 0.06945,
     "end_time": "2023-10-28T10:23:37.370551",
     "exception": false,
     "start_time": "2023-10-28T10:23:37.301101",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Define and execute the sequential chain\n",
    "\n",
    "\n",
    "We define a chain with two tasks in sequence.  \n",
    "The input for the second step is the output of the first step.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7c89c875",
   "metadata": {
    "_kg_hide-input": true,
    "execution": {
     "iopub.execute_input": "2023-10-28T10:23:37.509274Z",
     "iopub.status.busy": "2023-10-28T10:23:37.508570Z",
     "iopub.status.idle": "2023-10-28T10:23:37.515475Z",
     "shell.execute_reply": "2023-10-28T10:23:37.514591Z"
    },
    "papermill": {
     "duration": 0.078417,
     "end_time": "2023-10-28T10:23:37.517523",
     "exception": false,
     "start_time": "2023-10-28T10:23:37.439106",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def sequential_chain(country, llm):\n",
    "    \"\"\"\n",
    "    Args:\n",
    "        country: country selected\n",
    "    Returns:\n",
    "        None\n",
    "    \"\"\"\n",
    "    time_1 = time()\n",
    "    template = \"What is the most popular food in {country} for tourists? Just return the name of the food.\"\n",
    "\n",
    "    #  first task in chain\n",
    "    first_prompt = PromptTemplate(\n",
    "\n",
    "    input_variables=[\"country\"],\n",
    "\n",
    "    template=template)\n",
    "\n",
    "    chain_one = LLMChain(llm = llm, prompt = first_prompt)\n",
    "\n",
    "    # second step in chain\n",
    "    second_prompt = PromptTemplate(\n",
    "\n",
    "    input_variables=[\"food\"],\n",
    "\n",
    "    template=\"What are the top three ingredients in {food}. Just return the answer as three bullet points.\",)\n",
    "\n",
    "    chain_two = LLMChain(llm=llm, prompt=second_prompt)\n",
    "\n",
    "    # combine the two steps and run the chain sequence\n",
    "    overall_chain = SimpleSequentialChain(chains=[chain_one, chain_two], verbose=True)\n",
    "    overall_chain.run(country)\n",
    "    time_2 = time()\n",
    "    print(f\"Run sequential chain: {round(time_2-time_1, 3)} sec.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87d33ebb",
   "metadata": {
    "papermill": {
     "duration": 0.068935,
     "end_time": "2023-10-28T10:23:37.654599",
     "exception": false,
     "start_time": "2023-10-28T10:23:37.585664",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Test the sequence with Llama v2 **7b** chat HF model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "34927f50",
   "metadata": {
    "_kg_hide-input": true,
    "execution": {
     "iopub.execute_input": "2023-10-28T10:23:37.802445Z",
     "iopub.status.busy": "2023-10-28T10:23:37.801716Z",
     "iopub.status.idle": "2023-10-28T10:23:41.381087Z",
     "shell.execute_reply": "2023-10-28T10:23:41.379990Z"
    },
    "papermill": {
     "duration": 3.657049,
     "end_time": "2023-10-28T10:23:41.383781",
     "exception": false,
     "start_time": "2023-10-28T10:23:37.726732",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new SimpleSequentialChain chain...\u001b[0m\n",
      "\u001b[36;1m\u001b[1;3m\n",
      "\n",
      "Answer: Escargots (Snails)\u001b[0m\n",
      "\u001b[33;1m\u001b[1;3m\n",
      "\n",
      "* Escargots (Snails)\n",
      "* Garlic\n",
      "* Butter\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "Run sequential chain: 3.574 sec.\n"
     ]
    }
   ],
   "source": [
    "final_answer = sequential_chain(\"France\", llm_1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb3170a0",
   "metadata": {
    "papermill": {
     "duration": 0.069883,
     "end_time": "2023-10-28T10:23:41.524669",
     "exception": false,
     "start_time": "2023-10-28T10:23:41.454786",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Test the sequence with Llama v2 **7b** chat HF model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "cf2f5f6f",
   "metadata": {
    "_kg_hide-input": true,
    "execution": {
     "iopub.execute_input": "2023-10-28T10:23:41.666058Z",
     "iopub.status.busy": "2023-10-28T10:23:41.665517Z",
     "iopub.status.idle": "2023-10-28T10:23:45.558857Z",
     "shell.execute_reply": "2023-10-28T10:23:45.557820Z"
    },
    "papermill": {
     "duration": 3.966798,
     "end_time": "2023-10-28T10:23:45.561170",
     "exception": false,
     "start_time": "2023-10-28T10:23:41.594372",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new SimpleSequentialChain chain...\u001b[0m\n",
      "\u001b[36;1m\u001b[1;3m\n",
      "\n",
      "Answer: Pizza.\u001b[0m\n",
      "\u001b[33;1m\u001b[1;3m\n",
      "\n",
      "Top three ingredients in pizza:\n",
      "\n",
      "• Cheese\n",
      "• Tomato sauce\n",
      "• Pepperoni\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "Run sequential chain: 3.888 sec.\n"
     ]
    }
   ],
   "source": [
    "final_answer = sequential_chain(\"Italy\", llm_1)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 383.55476,
   "end_time": "2023-10-28T10:23:49.607319",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2023-10-28T10:17:26.052559",
   "version": "2.4.0"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "1739d6d931da47098573cab0ad340914": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_39fe281d1671426fb49c749eb237af17",
        "IPY_MODEL_321596f2aecc4e55a16e097fc02eea1f",
        "IPY_MODEL_ceeb1053899544edbb0e6cb2c91972d0"
       ],
       "layout": "IPY_MODEL_be3f1dbc68e240178370d761cf503557"
      }
     },
     "21aac0ab02754ee882119cbb37e935eb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "321596f2aecc4e55a16e097fc02eea1f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_bd966b0a8f1a43a482faf04945cd95af",
       "max": 2.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_21aac0ab02754ee882119cbb37e935eb",
       "value": 2.0
      }
     },
     "39fe281d1671426fb49c749eb237af17": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_4ff0d2c66f7f48e4904a559c188aa065",
       "placeholder": "​",
       "style": "IPY_MODEL_9fa91fec96eb4fb391fec606f7f35cbc",
       "value": "Loading checkpoint shards: 100%"
      }
     },
     "4ff0d2c66f7f48e4904a559c188aa065": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "7d7b290e35fe41a5a7c66a8d16a2d20a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "9876275b49904ffda61c4e9465e490c7": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "9fa91fec96eb4fb391fec606f7f35cbc": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "bd966b0a8f1a43a482faf04945cd95af": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "be3f1dbc68e240178370d761cf503557": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ceeb1053899544edbb0e6cb2c91972d0": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_9876275b49904ffda61c4e9465e490c7",
       "placeholder": "​",
       "style": "IPY_MODEL_7d7b290e35fe41a5a7c66a8d16a2d20a",
       "value": " 2/2 [01:49&lt;00:00, 50.11s/it]"
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}