use-code-llama-to-generate-python-code.ipynb

{"cells":[{"source":"<a href=\"https://www.kaggle.com/code/gpreda/use-code-llama-to-generate-python-code?scriptVersionId=148267875\" target=\"_blank\"><img align=\"left\" alt=\"Kaggle\" title=\"Open in Kaggle\" src=\"https://kaggle.com/static/images/open-in-kaggle.svg\"></a>","metadata":{},"cell_type":"markdown"},{"cell_type":"markdown","id":"d911dccf","metadata":{"papermill":{"duration":0.004473,"end_time":"2023-10-27T21:10:09.38086","exception":false,"start_time":"2023-10-27T21:10:09.376387","status":"completed"},"tags":[]},"source":["# Introduction  \n","\n","\n","Code generation is a very useful feature that LLMs can be used for.  \n","In this Notebook we will test one of the available models on Kaggle for code generation.  \n","\n","\n","Let's check the **Code LLama** for code generation.  \n","\n","The model details are: \n","\n","* **Model**: Code Llama 2\n","* **Variation**: 7b-python-hf (7b: 7B dimm. hf: HuggingFace build)   \n","* **Version**: V1  \n","* **Framework**: PyTorch\n","\n"]},{"cell_type":"markdown","id":"d5ad6524","metadata":{"papermill":{"duration":0.003656,"end_time":"2023-10-27T21:10:09.388592","exception":false,"start_time":"2023-10-27T21:10:09.384936","status":"completed"},"tags":[]},"source":["# Import packages"]},{"cell_type":"code","execution_count":1,"id":"d335ac9b","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2023-10-27T21:10:09.39839Z","iopub.status.busy":"2023-10-27T21:10:09.397684Z","iopub.status.idle":"2023-10-27T21:10:15.417666Z","shell.execute_reply":"2023-10-27T21:10:15.416772Z"},"papermill":{"duration":6.027478,"end_time":"2023-10-27T21:10:15.420142","exception":false,"start_time":"2023-10-27T21:10:09.392664","status":"completed"},"tags":[]},"outputs":[],"source":["from transformers import AutoTokenizer\n","import transformers\n","import torch\n","from time import time"]},{"cell_type":"markdown","id":"b4d405f2","metadata":{"papermill":{"duration":0.003574,"end_time":"2023-10-27T21:10:15.427748","exception":false,"start_time":"2023-10-27T21:10:15.424174","status":"completed"},"tags":[]},"source":["# Prepare the tokenizer and pipeline"]},{"cell_type":"code","execution_count":2,"id":"b3fd33db","metadata":{"execution":{"iopub.execute_input":"2023-10-27T21:10:15.436976Z","iopub.status.busy":"2023-10-27T21:10:15.436359Z","iopub.status.idle":"2023-10-27T21:12:09.213482Z","shell.execute_reply":"2023-10-27T21:12:09.212455Z"},"papermill":{"duration":113.784097,"end_time":"2023-10-27T21:12:09.215721","exception":false,"start_time":"2023-10-27T21:10:15.431624","status":"completed"},"tags":[]},"outputs":[{"name":"stderr","output_type":"stream","text":["Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n","/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n","  warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n"]},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"e9fa7fbe01c5492dbc555bc582b7a4ed","version_major":2,"version_minor":0},"text/plain":["Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"]},"metadata":{},"output_type":"display_data"},{"name":"stderr","output_type":"stream","text":["Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"]},{"name":"stdout","output_type":"stream","text":["Tokenizer & pipeline: 114 sec.\n"]}],"source":["model = '/kaggle/input/codellama/pytorch/7b-python-hf/1'\n","\n","time_1 = time()\n","tokenizer = AutoTokenizer.from_pretrained(model)\n","pipeline = transformers.pipeline(\n","    \"text-generation\",\n","    model=model,\n","    torch_dtype=torch.float16,\n","    device_map=\"auto\",\n",")\n","print(f\"Tokenizer & pipeline: {round(time() - time_1)} sec.\")"]},{"cell_type":"markdown","id":"3c321e13","metadata":{"papermill":{"duration":0.003925,"end_time":"2023-10-27T21:12:09.224118","exception":false,"start_time":"2023-10-27T21:12:09.220193","status":"completed"},"tags":[]},"source":["# Test with a code prompt"]},{"cell_type":"code","execution_count":3,"id":"3e2e6719","metadata":{"execution":{"iopub.execute_input":"2023-10-27T21:12:09.234041Z","iopub.status.busy":"2023-10-27T21:12:09.233715Z","iopub.status.idle":"2023-10-27T21:12:09.238031Z","shell.execute_reply":"2023-10-27T21:12:09.237138Z"},"papermill":{"duration":0.011741,"end_time":"2023-10-27T21:12:09.24013","exception":false,"start_time":"2023-10-27T21:12:09.228389","status":"completed"},"tags":[]},"outputs":[],"source":["prompt = 'Write the code for a function to compute the area of circle.'"]},{"cell_type":"code","execution_count":4,"id":"0d60edf4","metadata":{"execution":{"iopub.execute_input":"2023-10-27T21:12:09.249632Z","iopub.status.busy":"2023-10-27T21:12:09.249306Z","iopub.status.idle":"2023-10-27T21:12:24.768757Z","shell.execute_reply":"2023-10-27T21:12:24.767765Z"},"papermill":{"duration":15.526995,"end_time":"2023-10-27T21:12:24.771286","exception":false,"start_time":"2023-10-27T21:12:09.244291","status":"completed"},"tags":[]},"outputs":[{"name":"stderr","output_type":"stream","text":["Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"]}],"source":["sequences = pipeline(\n","    prompt,\n","    do_sample=True,\n","    top_k=10,\n","    temperature=0.1,\n","    top_p=0.95,\n","    num_return_sequences=1,\n","    eos_token_id=tokenizer.eos_token_id,\n","    max_length=200,\n",")"]},{"cell_type":"code","execution_count":5,"id":"d58dccf1","metadata":{"execution":{"iopub.execute_input":"2023-10-27T21:12:24.781923Z","iopub.status.busy":"2023-10-27T21:12:24.781625Z","iopub.status.idle":"2023-10-27T21:12:24.786695Z","shell.execute_reply":"2023-10-27T21:12:24.785657Z"},"papermill":{"duration":0.012602,"end_time":"2023-10-27T21:12:24.78876","exception":false,"start_time":"2023-10-27T21:12:24.776158","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["Result: Write the code for a function to compute the area of circle.\n","\n","# import math\n","# def area(r):\n","#     return math.pi * r * r\n","\n","# print(area(5))\n","\n","# Write the code for a function to compute the circumference of a circle.\n","\n","# import math\n","# def circumference(r):\n","#     return 2 * math.pi * r\n","\n","# print(circumference(5))\n","\n","# Write the code for a function to compute the volume of a sphere.\n","\n","# import math\n","# def volume(r):\n","#     return (4/3) * math.pi * r * r * r\n","\n","# print(volume(5))\n","\n","# Write the code for a function to compute the average of three numbers.\n","\n","# def average(a, b, c):\n","#     return (a + b + c) / 3\n","\n"]}],"source":["for seq in sequences:\n","    print(f\"Result: {seq['generated_text']}\")"]},{"cell_type":"code","execution_count":6,"id":"f213cf07","metadata":{"execution":{"iopub.execute_input":"2023-10-27T21:12:24.798384Z","iopub.status.busy":"2023-10-27T21:12:24.79811Z","iopub.status.idle":"2023-10-27T21:12:24.802474Z","shell.execute_reply":"2023-10-27T21:12:24.801526Z"},"papermill":{"duration":0.011778,"end_time":"2023-10-27T21:12:24.804712","exception":false,"start_time":"2023-10-27T21:12:24.792934","status":"completed"},"tags":[]},"outputs":[],"source":["prompt = 'from sklearn.decomposition import PCA\\n\\ndef apply_pca(X, n_components=2):\\n\\n#Just show the apply_cpa function code.'"]},{"cell_type":"code","execution_count":7,"id":"e7cce8fc","metadata":{"execution":{"iopub.execute_input":"2023-10-27T21:12:24.814874Z","iopub.status.busy":"2023-10-27T21:12:24.814603Z","iopub.status.idle":"2023-10-27T21:12:37.695819Z","shell.execute_reply":"2023-10-27T21:12:37.695025Z"},"papermill":{"duration":12.888903,"end_time":"2023-10-27T21:12:37.698159","exception":false,"start_time":"2023-10-27T21:12:24.809256","status":"completed"},"tags":[]},"outputs":[{"name":"stderr","output_type":"stream","text":["Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"]}],"source":["sequences = pipeline(\n","    prompt,\n","    do_sample=True,\n","    top_k=10,\n","    temperature=0.01,\n","    top_p=0.95,\n","    num_return_sequences=1,\n","    eos_token_id=tokenizer.eos_token_id,\n","    max_length=256,\n",")"]},{"cell_type":"code","execution_count":8,"id":"0c2a933e","metadata":{"execution":{"iopub.execute_input":"2023-10-27T21:12:37.708441Z","iopub.status.busy":"2023-10-27T21:12:37.708164Z","iopub.status.idle":"2023-10-27T21:12:37.712827Z","shell.execute_reply":"2023-10-27T21:12:37.711955Z"},"papermill":{"duration":0.012095,"end_time":"2023-10-27T21:12:37.715065","exception":false,"start_time":"2023-10-27T21:12:37.70297","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["Result: from sklearn.decomposition import PCA\n","\n","def apply_pca(X, n_components=2):\n","\n","#Just show the apply_cpa function code.\n","\n","    pca = PCA(n_components=n_components)\n","    pca.fit(X)\n","    X_pca = pca.transform(X)\n","\n","    return X_pca\n","\n","\n","def plot_pca(X, y, n_components=2):\n","\n","    X_pca = apply_pca(X, n_components=n_components)\n","\n","    fig, ax = plt.subplots(figsize=(12, 8))\n","    for c in np.unique(y):\n","        ix = np.where(y == c)\n","        ax.scatter(X_pca[ix, 0], X_pca[ix, 1], label=c)\n","    ax.set_title(\"PCA of dataset\")\n","    ax.set_xlabel(\"1st eigenvector\")\n","    ax.set_ylabel(\"2nd eigenvector\")\n","    ax.legend(fontsize=1\n"]}],"source":["for seq in sequences:\n","    print(f\"Result: {seq['generated_text']}\")"]},{"cell_type":"markdown","id":"13402579","metadata":{"papermill":{"duration":0.004422,"end_time":"2023-10-27T21:12:37.724","exception":false,"start_time":"2023-10-27T21:12:37.719578","status":"completed"},"tags":[]},"source":["# Conclusions\n","\n","The code generated looks functional, but rather than getting only the objective as described in the prompt, the code generated looks like an excerpt from a list of routines, without stopping after the code targeted is fully generated (and continue with functions that were not asked for).  \n","\n","We will need to continue the investigation, to see if use of larger Code Llama 2 models results in a better quality code generation.\n"]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"},"papermill":{"default_parameters":{},"duration":155.817572,"end_time":"2023-10-27T21:12:41.455441","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2023-10-27T21:10:05.637869","version":"2.4.0"},"widgets":{"application/vnd.jupyter.widget-state+json":{"state":{"00262764f7f448dd943b7e0bf1f048b8":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1e7307012f8a4075a65728033ab98e53":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"314fb341502a41beaca8bf4c2b20ef2d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_1e7307012f8a4075a65728033ab98e53","max":2.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_d44b6cf6e3c94ae092778480265c9cf7","value":2.0}},"501346d4e0204066b9681a8c1d35d1db":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"838a8a94c07a4a398734a6de7602bdb7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ad8816e6de704141b36516fa7c0d4cd9","placeholder":"​","style":"IPY_MODEL_c08b29b9523542999fdc190dcb58d79a","value":" 2/2 [01:31&lt;00:00, 41.96s/it]"}},"ad8816e6de704141b36516fa7c0d4cd9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c08b29b9523542999fdc190dcb58d79a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c3981bc59abe48aaad57e44d8061214b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d44b6cf6e3c94ae092778480265c9cf7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"e9fa7fbe01c5492dbc555bc582b7a4ed":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f5d0733b2b484a17937707063543e4c0","IPY_MODEL_314fb341502a41beaca8bf4c2b20ef2d","IPY_MODEL_838a8a94c07a4a398734a6de7602bdb7"],"layout":"IPY_MODEL_00262764f7f448dd943b7e0bf1f048b8"}},"f5d0733b2b484a17937707063543e4c0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_501346d4e0204066b9681a8c1d35d1db","placeholder":"​","style":"IPY_MODEL_c3981bc59abe48aaad57e44d8061214b","value":"Loading checkpoint shards: 100%"}}},"version_major":2,"version_minor":0}}},"nbformat":4,"nbformat_minor":5}