studio-notebook-fine-tuning/falcon-40b-qlora-finetune-summarize.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "36a2f3b4",
   "metadata": {},
   "source": [
    "# Fine-tune Falcon-40B with QLoRA"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "639d3904",
   "metadata": {},
   "source": [
    "--- \n",
    "\n",
    "In this demo notebook, we demonstrate how to fine-tune the Falcon-40B model using QLoRA, Hugging Face PEFT, and bitsandbytes. \n",
    "\n",
    "---"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2ec1c5bb",
   "metadata": {},
   "source": [
    "This notebook was tested in Amazon SageMaker Studio with Python 3 (Data Science 3.0) kernel with a ml.g5.12xlarge instance."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "8df4c2bc",
   "metadata": {},
   "source": [
    "Install the required libriaries, including the Hugging Face libraries, and restart the kernel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "153d07ab-3e43-45b1-89fd-140352b1d6ce",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install -q -U torch==2.0.1 bitsandbytes==0.40.2\n",
    "%pip install -q -U transformers==4.31.0 peft==0.4.0 accelerate==0.21.0\n",
    "%pip install -q -U datasets py7zr einops tensorboardX"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f5185bb8-3c83-483c-a145-dd4c0b9941f2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Add installed cuda runtime to path for bitsandbytes \n",
    "import os\n",
    "import nvidia\n",
    "\n",
    "cuda_install_dir = '/'.join(nvidia.__file__.split('/')[:-1]) + '/cuda_runtime/lib/'\n",
    "os.environ['LD_LIBRARY_PATH'] =  cuda_install_dir"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "70b26c91",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "To train our model, we need to convert our inputs (text) to token IDs. This is done by a Hugging Face Transformers Tokenizer. In addition to QLoRA, we will use bitsanbytes 4-bit precision to quantize out frozen LLM to 4-bit and attach LoRA adapters on it.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "85946e42-9f8a-488c-8b90-8afedf311b02",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "===================================BUG REPORT===================================\n",
      "Welcome to bitsandbytes. For bug reports, please run\n",
      "\n",
      "python -m bitsandbytes\n",
      "\n",
      " and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues\n",
      "================================================================================\n",
      "bin /opt/conda/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so\n",
      "CUDA SETUP: CUDA runtime path found: /opt/conda/lib/python3.10/site-packages/nvidia/cuda_runtime/lib/libcudart.so.11.0\n",
      "CUDA SETUP: Highest compute capability among GPUs detected: 8.0\n",
      "CUDA SETUP: Detected CUDA version 117\n",
      "CUDA SETUP: Loading binary /opt/conda/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9e550a7701564161982282643eebbc14",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/9 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import torch\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig\n",
    "\n",
    "model_id = \"tiiuae/falcon-40b\"\n",
    "bnb_config = BitsAndBytesConfig(\n",
    "    load_in_4bit=True,\n",
    "    bnb_4bit_use_double_quant=True,\n",
    "    bnb_4bit_quant_type=\"nf4\",\n",
    "    bnb_4bit_compute_dtype=torch.bfloat16\n",
    ")\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_id)\n",
    "\n",
    "# Falcon requires you to allow remote code execution. This is because the model uses a new architecture that is not part of transformers yet.\n",
    "# The code is provided by the model authors in the repo.\n",
    "model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True, quantization_config=bnb_config, device_map=\"auto\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "804a368e-e578-4b95-8442-a9c2f2095659",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Set the Falcon tokenizer\n",
    "tokenizer.pad_token = tokenizer.eos_token"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "364f130c-ed00-475f-aae7-585d75c0ae09",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from peft import prepare_model_for_kbit_training\n",
    "\n",
    "model.gradient_checkpointing_enable()\n",
    "model = prepare_model_for_kbit_training(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b98b0095-4bac-48a1-bb29-8d48a7daa258",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def print_trainable_parameters(model):\n",
    "    \"\"\"\n",
    "    Prints the number of trainable parameters in the model.\n",
    "    \"\"\"\n",
    "    trainable_params = 0\n",
    "    all_param = 0\n",
    "    for _, param in model.named_parameters():\n",
    "        all_param += param.numel()\n",
    "        if param.requires_grad:\n",
    "            trainable_params += param.numel()\n",
    "    print(\n",
    "        f\"trainable params: {trainable_params} || all params: {all_param} || trainable%: {100 * trainable_params / all_param}\"\n",
    "    )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1a260ca0",
   "metadata": {},
   "source": [
    "Prepare the model for LoRA training using PEFT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "from peft import LoraConfig, get_peft_model\n",
    "\n",
    "config = LoraConfig(\n",
    "    r=8,\n",
    "    lora_alpha=32,\n",
    "    target_modules=[\n",
    "        \"query_key_value\",\n",
    "        \"dense\",\n",
    "        \"dense_h_to_4h\",\n",
    "        \"dense_4h_to_h\",\n",
    "        ],\n",
    "    lora_dropout=0.05,\n",
    "    bias=\"none\",\n",
    "    task_type=\"CAUSAL_LM\"\n",
    ")\n",
    "\n",
    "model = get_peft_model(model, config)\n",
    "print_trainable_parameters(model)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "462c9410",
   "metadata": {},
   "source": [
    "To load the samsum dataset, we use the load_dataset() method from the Hugging Face Datasets library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "e1498a1a-3cac-40ba-b7f5-1e626d7ac7ca",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Found cached dataset samsum (/root/.cache/huggingface/datasets/samsum/samsum/0.0.0/f1d7c6b7353e6de335d444e424dc002ef70d1277109031327bc9cc6af5d3d46e)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "dd4a004d38464d9395cedc2dcc2ce4a9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train dataset size: 14732\n",
      "Test dataset size: 819\n"
     ]
    }
   ],
   "source": [
    "from datasets import load_dataset\n",
    "\n",
    "# Load dataset from the hub\n",
    "dataset = load_dataset(\"samsum\")\n",
    "\n",
    "print(f\"Train dataset size: {len(dataset['train'])}\")\n",
    "print(f\"Test dataset size: {len(dataset['test'])}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "116773e2",
   "metadata": {},
   "source": [
    "Create a prompt template and load the dataset with a random sample to try summarization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fd7c487d-3ff7-4a64-9d87-143515edb984",
   "metadata": {
    "tags": [],
    "pycharm": {
     "is_executing": true
    }
   },
   "outputs": [],
   "source": [
    "from random import randint\n",
    "\n",
    "# custom instruct prompt start\n",
    "prompt_template = f\"Summarize the chat dialogue:\\n{{dialogue}}\\n---\\nSummary:\\n{{summary}}{{eos_token}}\"\n",
    "\n",
    "# template dataset to add prompt to each sample\n",
    "def template_dataset(sample):\n",
    "    sample[\"text\"] = prompt_template.format(dialogue=sample[\"dialogue\"],\n",
    "                                            summary=sample[\"summary\"],\n",
    "                                            eos_token=tokenizer.eos_token)\n",
    "    return sample\n",
    "\n",
    "\n",
    "# apply prompt template per sample\n",
    "train_dataset = dataset[\"train\"].map(template_dataset, remove_columns=list(dataset[\"train\"].features))\n",
    "\n",
    "print(train_dataset[randint(0, len(dataset))][\"text\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "14571066-34fe-4fe1-97a9-697f1de79218",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Loading cached processed dataset at /root/.cache/huggingface/datasets/samsum/samsum/0.0.0/f1d7c6b7353e6de335d444e424dc002ef70d1277109031327bc9cc6af5d3d46e/cache-7b2e1367f7be0c6a.arrow\n"
     ]
    }
   ],
   "source": [
    "# apply prompt template per sample\n",
    "test_dataset = dataset[\"test\"].map(template_dataset, remove_columns=list(dataset[\"test\"].features))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "1c197761-eb0c-4712-a2e7-40c6a40eb74f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/14732 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Loading cached processed dataset at /root/.cache/huggingface/datasets/samsum/samsum/0.0.0/f1d7c6b7353e6de335d444e424dc002ef70d1277109031327bc9cc6af5d3d46e/cache-915c1bd91b5622ba.arrow\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of train samples: 14732\n"
     ]
    }
   ],
   "source": [
    " # tokenize and chunk dataset\n",
    "lm_train_dataset = train_dataset.map(\n",
    "    lambda sample: tokenizer(sample[\"text\"]), batched=True, batch_size=24, remove_columns=list(train_dataset.features)\n",
    ")\n",
    "\n",
    "\n",
    "lm_test_dataset = test_dataset.map(\n",
    "    lambda sample: tokenizer(sample[\"text\"]), batched=True, remove_columns=list(test_dataset.features)\n",
    ")\n",
    "\n",
    "# Print total number of samples\n",
    "print(f\"Total number of train samples: {len(lm_train_dataset)}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ade7f4fd",
   "metadata": {},
   "source": [
    "Create a S3 bucket to log our metrics to TensorBoard."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "a3864a9d-e17b-437d-bd68-52ee8e134b41",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "bucket = <YOUR-S3-BUCKET>\n",
    "log_bucket = f\"s3://{bucket}/falcon-40b-qlora-finetune\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a97f4fc7",
   "metadata": {},
   "source": [
    "Use the Hugging Face Trainer class to fine-tune the model. Define the hyperparameters we want to use. We also create a DataCollator that will take care of padding our inputs and labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "ca6e7c03-58b3-4057-afde-e8e2e249143d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import transformers\n",
    "\n",
    "# We set num_train_epochs=1 simply to run a demonstration\n",
    "\n",
    "trainer = transformers.Trainer(\n",
    "    model=model,\n",
    "    train_dataset=lm_train_dataset,\n",
    "    eval_dataset=lm_test_dataset,\n",
    "    args=transformers.TrainingArguments(\n",
    "        per_device_train_batch_size=8,\n",
    "        per_device_eval_batch_size=8,\n",
    "        logging_dir=log_bucket,\n",
    "        logging_steps=2,\n",
    "        num_train_epochs=1,\n",
    "        learning_rate=2e-4,\n",
    "        bf16=True,\n",
    "        save_strategy = \"no\",\n",
    "        output_dir=\"outputs\",\n",
    "        report_to=\"tensorboard\",\n",
    "    ),\n",
    "    data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),\n",
    ")\n",
    "model.config.use_cache = False  # silence the warnings. Please re-enable for inference!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Start training\n",
    "trainer.train()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "b12267be-fbfe-4aeb-bb11-3454d9a70416",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='103' max='103' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [103/103 03:54]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "{'eval_loss': 1.7204433679580688,\n",
       " 'eval_runtime': 236.733,\n",
       " 'eval_samples_per_second': 3.46,\n",
       " 'eval_steps_per_second': 0.435,\n",
       " 'epoch': 0.03}"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#evaluate and return the metrics\n",
    "trainer.evaluate()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9351292f",
   "metadata": {},
   "source": [
    "Load a test dataset and try a random sample for summarization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "2f220ba3-f782-4a86-a349-ed8d8cda405e",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Found cached dataset samsum (/root/.cache/huggingface/datasets/samsum/samsum/0.0.0/f1d7c6b7353e6de335d444e424dc002ef70d1277109031327bc9cc6af5d3d46e)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Summarize the chat dialogue:\n",
      "Richie: Pogba\n",
      "Clay: Pogboom\n",
      "Richie: what a s strike yoh!\n",
      "Clay: was off the seat the moment he chopped the ball back to his right foot\n",
      "Richie: me too dude\n",
      "Clay: hope his form lasts\n",
      "Richie: This season he's more mature\n",
      "Clay: Yeah, Jose has his trust in him\n",
      "Richie: everyone does\n",
      "Clay: yeah, he really deserved to score after his first 60 minutes\n",
      "Richie: reward\n",
      "Clay: yeah man\n",
      "Richie: cool then \n",
      "Clay: cool\n",
      "---\n",
      "Summary:\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Load dataset from the hub\n",
    "test_dataset = load_dataset(\"samsum\", split=\"test\")\n",
    "\n",
    "# select a random test sample\n",
    "sample = test_dataset[randint(0, len(test_dataset))]\n",
    "\n",
    "# format sample\n",
    "prompt_template = f\"Summarize the chat dialogue:\\n{{dialogue}}\\n---\\nSummary:\\n\"\n",
    "\n",
    "test_sample = prompt_template.format(dialogue=sample[\"dialogue\"])\n",
    "\n",
    "print(test_sample)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "413c44ae-35b3-47fa-b3b4-9eaeafec9da6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "input_ids = tokenizer(test_sample, return_tensors=\"pt\").input_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "1af86337-48b6-4165-a105-d4650eb07762",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
      "Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Summarize the chat dialogue:\n",
      "Richie: Pogba\n",
      "Clay: Pogboom\n",
      "Richie: what a s strike yoh!\n",
      "Clay: was off the seat the moment he chopped the ball back to his right foot\n",
      "Richie: me too dude\n",
      "Clay: hope his form lasts\n",
      "Richie: This season he's more mature\n",
      "Clay: Yeah, Jose has his trust in him\n",
      "Richie: everyone does\n",
      "Clay: yeah, he really deserved to score after his first 60 minutes\n",
      "Richie: reward\n",
      "Clay: yeah man\n",
      "Richie: cool then \n",
      "Clay: cool\n",
      "---\n",
      "Summary:\n",
      "Richie and Clay talk about Paul Pogba's beautiful first goal against Man Utd this season. Richie and Clay are hoping his form will\n"
     ]
    }
   ],
   "source": [
    "#set the tokens for the summary evaluation\n",
    "tokens_for_summary = 30\n",
    "output_tokens = input_ids.shape[1] + tokens_for_summary\n",
    "\n",
    "outputs = model.generate(inputs=input_ids, do_sample=True, max_length=output_tokens)\n",
    "gen_text = tokenizer.batch_decode(outputs)[0]\n",
    "print(gen_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "117c8857-5f32-4052-8013-bc496d7c5a84",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 96
   }
  ],
  "instance_type": "ml.g5.12xlarge",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}