google-ai4code-xgbrank.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a096e8ce",
   "metadata": {
    "papermill": {
     "duration": 0.042919,
     "end_time": "2022-05-15T12:32:21.199006",
     "exception": false,
     "start_time": "2022-05-15T12:32:21.156087",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<h1 style=\"color:#fc3f51\"> Google AI4Code : The results of my first experiments. </h1>\n",
    "\n",
    "The majority of this content comes from [Getting Started with AI4Code](https://www.kaggle.com/code/ryanholbrook/getting-started-with-ai4code) Notebook.\n",
    "\n",
    "\n",
    "My contribution :\n",
    "\n",
    "- New params for TfidfVectorizer (In Cell 11)\n",
    "- Better params for XGBRanker Model (In Cell 14)\n",
    "\n",
    "I tried OPTUNA to look for better parameters, but I did not use it in this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2529d519",
   "metadata": {
    "papermill": {
     "duration": 0.039412,
     "end_time": "2022-05-15T12:32:21.280635",
     "exception": false,
     "start_time": "2022-05-15T12:32:21.241223",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "314583af",
   "metadata": {
    "papermill": {
     "duration": 0.039938,
     "end_time": "2022-05-15T12:32:21.360331",
     "exception": false,
     "start_time": "2022-05-15T12:32:21.320393",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Welcome to the Google AI4Code Competition! #\n",
    "\n",
    "In this competition you're challenged to reconstruct the order of Kaggle notebooks whose cells have been shuffled. Check out the [Competition Pages](https://www.kaggle.com/competitions/AI4Code/overview) for a complete overview.\n",
    "\n",
    "This notebook will walk you through making a submission with a simple ranking model. We'll look at how to:\n",
    "- Wrangle the competition data and create validation splits,\n",
    "- Represent the code cell orders with a feature,\n",
    "- Build a ranking model with XGBoost,\n",
    "- Evaluate predictions with a Python implementation of the competition metric, and,\n",
    "- Format predictions to make a successful submission.\n",
    "\n",
    "Our model will be able to learn roughly where a cell should go in a notebook based on what words it contains -- that, for example, cells containing \"Introduction\" or `import` should usually be near the beginning, while cells containing \"Submit\" or `submission.csv` should usually be near the end. These simple features are effective at reconstructing the global order of typical data science workflows. An understanding of the *interactions* or *relationships between cells*, however, will be required of the most successful solutions. We encourage you therefore to explore things like modern neural network language models for learning the relationships between natural language and computer code."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "987e1674",
   "metadata": {
    "papermill": {
     "duration": 0.039433,
     "end_time": "2022-05-15T12:32:21.439731",
     "exception": false,
     "start_time": "2022-05-15T12:32:21.400298",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Setup #"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "29a7bb6c",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:32:21.527688Z",
     "iopub.status.busy": "2022-05-15T12:32:21.526942Z",
     "iopub.status.idle": "2022-05-15T12:32:21.620921Z",
     "shell.execute_reply": "2022-05-15T12:32:21.620057Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "papermill": {
     "duration": 0.14375,
     "end_time": "2022-05-15T12:32:21.623436",
     "exception": false,
     "start_time": "2022-05-15T12:32:21.479686",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import json\n",
    "from pathlib import Path\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from scipy import sparse\n",
    "from tqdm import tqdm\n",
    "\n",
    "pd.options.display.width = 180\n",
    "pd.options.display.max_colwidth = 120\n",
    "\n",
    "data_dir = Path('../input/AI4Code')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7d1791c",
   "metadata": {
    "papermill": {
     "duration": 0.039345,
     "end_time": "2022-05-15T12:32:21.702653",
     "exception": false,
     "start_time": "2022-05-15T12:32:21.663308",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Load Data #"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8730b0b",
   "metadata": {
    "papermill": {
     "duration": 0.039337,
     "end_time": "2022-05-15T12:32:21.781812",
     "exception": false,
     "start_time": "2022-05-15T12:32:21.742475",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The notebooks are stored as individiual JSON files. They've been cleaned of the usual metadata present in Jupyter notebooks, leaving only the `cell_type` and `source`. The [Data](https://www.kaggle.com/competitions/AI4Code/data) page on the competition website has the full documentation of this dataset.\n",
    "\n",
    "We'll load the notebooks here and join them into a dataframe for easier processing. The full set of training data takes quite a while to load, so we'll just use a subset for this demonstration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a0829145",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:32:21.863866Z",
     "iopub.status.busy": "2022-05-15T12:32:21.863348Z",
     "iopub.status.idle": "2022-05-15T12:33:56.347284Z",
     "shell.execute_reply": "2022-05-15T12:33:56.346518Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "papermill": {
     "duration": 94.528346,
     "end_time": "2022-05-15T12:33:56.349845",
     "exception": false,
     "start_time": "2022-05-15T12:32:21.821499",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Train NBs: 100%|██████████| 10000/10000 [01:28<00:00, 112.79it/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>cell_type</th>\n",
       "      <th>source</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th>cell_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">000c0a9b2fef4d</th>\n",
       "      <th>1087237d</th>\n",
       "      <td>code</td>\n",
       "      <td># Data manipulation\\nimport pandas as pd\\nimport numpy as np\\n\\n# Data visualization\\nimport matplotlib.pyplot as pl...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d7209f1f</th>\n",
       "      <td>code</td>\n",
       "      <td>fifa_raw_dataset = pd.read_csv('../input/data.csv')</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>daf5b8ee</th>\n",
       "      <td>code</td>\n",
       "      <td>fifa_raw_dataset.head()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e404213c</th>\n",
       "      <td>code</td>\n",
       "      <td>fifa_raw_dataset.info()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2bad59b0</th>\n",
       "      <td>code</td>\n",
       "      <td>fifa_raw_dataset.shape</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">fffc63ff750064</th>\n",
       "      <th>56aa8da7</th>\n",
       "      <td>code</td>\n",
       "      <td>\\nsubmission.to_csv('house_price_rf.csv', index = False)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>411b85d9</th>\n",
       "      <td>markdown</td>\n",
       "      <td>1. # Data exploration</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e7e67119</th>\n",
       "      <td>markdown</td>\n",
       "      <td># # Data preprocessing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8b54cf58</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Post-process for submission</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b3c6bc16</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Define and fit model</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>461166 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                        cell_type                                                                                                                   source\n",
       "id             cell_id                                                                                                                                    \n",
       "000c0a9b2fef4d 1087237d      code  # Data manipulation\\nimport pandas as pd\\nimport numpy as np\\n\\n# Data visualization\\nimport matplotlib.pyplot as pl...\n",
       "               d7209f1f      code                                                                      fifa_raw_dataset = pd.read_csv('../input/data.csv')\n",
       "               daf5b8ee      code                                                                                                  fifa_raw_dataset.head()\n",
       "               e404213c      code                                                                                                  fifa_raw_dataset.info()\n",
       "               2bad59b0      code                                                                                                   fifa_raw_dataset.shape\n",
       "...                           ...                                                                                                                      ...\n",
       "fffc63ff750064 56aa8da7      code                                                                 \\nsubmission.to_csv('house_price_rf.csv', index = False)\n",
       "               411b85d9  markdown                                                                                                    1. # Data exploration\n",
       "               e7e67119  markdown                                                                                                   # # Data preprocessing\n",
       "               8b54cf58  markdown                                                                                            # Post-process for submission\n",
       "               b3c6bc16  markdown                                                                                                   # Define and fit model\n",
       "\n",
       "[461166 rows x 2 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "NUM_TRAIN = 10000\n",
    "\n",
    "\n",
    "def read_notebook(path):\n",
    "    return (\n",
    "        pd.read_json(\n",
    "            path,\n",
    "            dtype={'cell_type': 'category', 'source': 'str'})\n",
    "        .assign(id=path.stem)\n",
    "        .rename_axis('cell_id')\n",
    "    )\n",
    "\n",
    "\n",
    "paths_train = list((data_dir / 'train').glob('*.json'))[:NUM_TRAIN]\n",
    "notebooks_train = [\n",
    "    read_notebook(path) for path in tqdm(paths_train, desc='Train NBs')\n",
    "]\n",
    "df = (\n",
    "    pd.concat(notebooks_train)\n",
    "    .set_index('id', append=True)\n",
    "    .swaplevel()\n",
    "    .sort_index(level='id', sort_remaining=False)\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b06bf29b",
   "metadata": {
    "papermill": {
     "duration": 0.261123,
     "end_time": "2022-05-15T12:33:56.874774",
     "exception": false,
     "start_time": "2022-05-15T12:33:56.613651",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Each notebook has all the code cells given first with the markdown cells following. The code cells are in the correct relative order, while the markdown cells are shuffled. In the next section, we'll see how to recover the correct orderings for notebooks in the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "ef12728f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:33:57.404972Z",
     "iopub.status.busy": "2022-05-15T12:33:57.404101Z",
     "iopub.status.idle": "2022-05-15T12:33:57.430638Z",
     "shell.execute_reply": "2022-05-15T12:33:57.429857Z"
    },
    "papermill": {
     "duration": 0.291889,
     "end_time": "2022-05-15T12:33:57.433213",
     "exception": false,
     "start_time": "2022-05-15T12:33:57.141324",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Notebook: 00290ddf866418\n",
      "The disordered notebook:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cell_type</th>\n",
       "      <th>source</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cell_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4e6f32f6</th>\n",
       "      <td>code</td>\n",
       "      <td># This Python 3 environment comes with many helpful analytics libraries installed\\n# It is defined by the kaggle/pyt...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0aeca210</th>\n",
       "      <td>code</td>\n",
       "      <td>import warnings\\nimport random\\n\\nSEED=44\\nrandom.seed(SEED)\\nnp.random.seed(SEED)\\npd.set_option('display.width', N...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cadfdb16</th>\n",
       "      <td>code</td>\n",
       "      <td>train = pd.read_csv('/kaggle/input/tabular-playground-series-mar-2021/train.csv')\\ntest = pd.read_csv('/kaggle/input...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fe39c117</th>\n",
       "      <td>code</td>\n",
       "      <td>train.info()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bbc5f229</th>\n",
       "      <td>code</td>\n",
       "      <td>y = train.iloc[:,-1]\\nX = train.iloc[:,:-1]\\nZ = test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46cc92d1</th>\n",
       "      <td>code</td>\n",
       "      <td>def get_obj_cols(df):\\n    return [col for col in df.columns if df.dtypes[col] == np.object]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9d6ea72b</th>\n",
       "      <td>code</td>\n",
       "      <td>X_objs = get_obj_cols(X)\\nX_objs_idx = [X.columns.get_loc(col) for col in X_objs]\\nZ_objs = get_obj_cols(Z)\\nZ_objs_...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d0f88604</th>\n",
       "      <td>code</td>\n",
       "      <td>for obj in X_objs:\\n    X[obj] = X[obj].astype('category').cat.codes\\nfor obj in Z_objs:\\n    Z[obj] = Z[obj].astype...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8df28832</th>\n",
       "      <td>code</td>\n",
       "      <td>X.drop('id', axis=1, inplace=True)\\nZ.drop('id', axis=1, inplace=True)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8c01934e</th>\n",
       "      <td>code</td>\n",
       "      <td>from sklearn.model_selection import StratifiedKFold, cross_val_score\\nfrom sklearn.metrics import roc_auc_score\\nK = 10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18b4e8d6</th>\n",
       "      <td>code</td>\n",
       "      <td>skf = StratifiedKFold(n_splits=K, shuffle=True, random_state=SEED)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c012109d</th>\n",
       "      <td>code</td>\n",
       "      <td>df_model = pd.DataFrame(columns = [*[f'model_{i}' for i in range(K)], 'average_auc'])\\ndf_model</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83ff0598</th>\n",
       "      <td>code</td>\n",
       "      <td>import lightgbm as lgbm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d35f13e5</th>\n",
       "      <td>code</td>\n",
       "      <td>params = {\\n    'learning_rate':1e-2,\\n    'max_bin':24,\\n    'n_estimators':1000,\\n    'device_type':'gpu',\\n    'm...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cfba39db</th>\n",
       "      <td>code</td>\n",
       "      <td>ctr = 0\\nmodel = []\\nauc_score = []\\nfor train_idx, val_idx in skf.split(X, y):\\n    Xt, Xv = X.loc[train_idx], X.lo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fbc1c206</th>\n",
       "      <td>code</td>\n",
       "      <td>df_model.loc[df_model.shape[0]] = [\\n    *model,\\n    np.mean(auc_score)\\n]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ace03b55</th>\n",
       "      <td>code</td>\n",
       "      <td>df_model['average_auc']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>671b78af</th>\n",
       "      <td>code</td>\n",
       "      <td>def predict(X):\\n    y_prob = []\\n    for i in range(K):\\n        y_prob.append(df_model.loc[0, f'model_{i}'].predic...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61d3ad05</th>\n",
       "      <td>code</td>\n",
       "      <td>from sklearn.model_selection import train_test_split</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6d6683ea</th>\n",
       "      <td>code</td>\n",
       "      <td>Xt, Xv, yt, yv = train_test_split(X, y, test_size=0.2, random_state=SEED)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2f41ebe2</th>\n",
       "      <td>code</td>\n",
       "      <td>yv_prob = predict(Xv)\\nmetrics(yv, yv_prob)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9dae7ae9</th>\n",
       "      <td>code</td>\n",
       "      <td>test_prob = predict(Z)\\nsubmission = pd.read_csv(\"../input/tabular-playground-series-mar-2021/sample_submission.csv\"...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d3345791</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91194a54</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Categorical</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91d97bb2</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Read Data</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         cell_type                                                                                                                   source\n",
       "cell_id                                                                                                                                    \n",
       "4e6f32f6      code  # This Python 3 environment comes with many helpful analytics libraries installed\\n# It is defined by the kaggle/pyt...\n",
       "0aeca210      code  import warnings\\nimport random\\n\\nSEED=44\\nrandom.seed(SEED)\\nnp.random.seed(SEED)\\npd.set_option('display.width', N...\n",
       "cadfdb16      code  train = pd.read_csv('/kaggle/input/tabular-playground-series-mar-2021/train.csv')\\ntest = pd.read_csv('/kaggle/input...\n",
       "fe39c117      code                                                                                                             train.info()\n",
       "bbc5f229      code                                                                    y = train.iloc[:,-1]\\nX = train.iloc[:,:-1]\\nZ = test\n",
       "46cc92d1      code                             def get_obj_cols(df):\\n    return [col for col in df.columns if df.dtypes[col] == np.object]\n",
       "9d6ea72b      code  X_objs = get_obj_cols(X)\\nX_objs_idx = [X.columns.get_loc(col) for col in X_objs]\\nZ_objs = get_obj_cols(Z)\\nZ_objs_...\n",
       "d0f88604      code  for obj in X_objs:\\n    X[obj] = X[obj].astype('category').cat.codes\\nfor obj in Z_objs:\\n    Z[obj] = Z[obj].astype...\n",
       "8df28832      code                                                   X.drop('id', axis=1, inplace=True)\\nZ.drop('id', axis=1, inplace=True)\n",
       "8c01934e      code  from sklearn.model_selection import StratifiedKFold, cross_val_score\\nfrom sklearn.metrics import roc_auc_score\\nK = 10\n",
       "18b4e8d6      code                                                       skf = StratifiedKFold(n_splits=K, shuffle=True, random_state=SEED)\n",
       "c012109d      code                          df_model = pd.DataFrame(columns = [*[f'model_{i}' for i in range(K)], 'average_auc'])\\ndf_model\n",
       "83ff0598      code                                                                                                  import lightgbm as lgbm\n",
       "d35f13e5      code  params = {\\n    'learning_rate':1e-2,\\n    'max_bin':24,\\n    'n_estimators':1000,\\n    'device_type':'gpu',\\n    'm...\n",
       "cfba39db      code  ctr = 0\\nmodel = []\\nauc_score = []\\nfor train_idx, val_idx in skf.split(X, y):\\n    Xt, Xv = X.loc[train_idx], X.lo...\n",
       "fbc1c206      code                                              df_model.loc[df_model.shape[0]] = [\\n    *model,\\n    np.mean(auc_score)\\n]\n",
       "ace03b55      code                                                                                                  df_model['average_auc']\n",
       "671b78af      code  def predict(X):\\n    y_prob = []\\n    for i in range(K):\\n        y_prob.append(df_model.loc[0, f'model_{i}'].predic...\n",
       "61d3ad05      code                                                                     from sklearn.model_selection import train_test_split\n",
       "6d6683ea      code                                                Xt, Xv, yt, yv = train_test_split(X, y, test_size=0.2, random_state=SEED)\n",
       "2f41ebe2      code                                                                              yv_prob = predict(Xv)\\nmetrics(yv, yv_prob)\n",
       "9dae7ae9      code  test_prob = predict(Z)\\nsubmission = pd.read_csv(\"../input/tabular-playground-series-mar-2021/sample_submission.csv\"...\n",
       "d3345791  markdown                                                                                                               # Training\n",
       "91194a54  markdown                                                                                                            # Categorical\n",
       "91d97bb2  markdown                                                                                                              # Read Data"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# Get an example notebook\n",
    "nb_id = df.index.unique('id')[6]\n",
    "print('Notebook:', nb_id)\n",
    "\n",
    "print(\"The disordered notebook:\")\n",
    "nb = df.loc[nb_id, :]\n",
    "display(nb)\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dde4ea6c",
   "metadata": {
    "papermill": {
     "duration": 0.262362,
     "end_time": "2022-05-15T12:33:57.958941",
     "exception": false,
     "start_time": "2022-05-15T12:33:57.696579",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Ordering the Cells #"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5fbd3e8",
   "metadata": {
    "papermill": {
     "duration": 0.264839,
     "end_time": "2022-05-15T12:33:58.490373",
     "exception": false,
     "start_time": "2022-05-15T12:33:58.225534",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "In the `train_orders.csv` file we have, for notebooks in the training set, the correct ordering of cells in terms of the cell ids."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "de136e2a",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:33:59.022445Z",
     "iopub.status.busy": "2022-05-15T12:33:59.022000Z",
     "iopub.status.idle": "2022-05-15T12:34:01.851983Z",
     "shell.execute_reply": "2022-05-15T12:34:01.851115Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "papermill": {
     "duration": 3.099165,
     "end_time": "2022-05-15T12:34:01.854015",
     "exception": false,
     "start_time": "2022-05-15T12:33:58.754850",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id\n",
       "00001756c60be8    [1862f0a6, 448eb224, 2a9e43d6, 7e2f170a, 038b763d, 77e56113, 2eefe0ef, 1ae087ab, 0beab1cd, 8ffe0b25, 9a78ab76, 0d136...\n",
       "00015c83e2717b    [2e94bd7a, 3e99dee9, b5e286ea, da4f7550, c417225b, 51e3cd89, 2600b4eb, 75b65993, cf195f8b, 25699d02, 72b3201a, f2c75...\n",
       "0001bdd4021779    [3fdc37be, 073782ca, 8ea7263c, 80543cd8, 38310c80, 073e27e5, 015d52a4, ad7679ef, 7fde4f04, 07c52510, 0a1a7a39, 0bcd3...\n",
       "0001daf4c2c76d    [97266564, a898e555, 86605076, 76cc2642, ef279279, df6c939f, 2476da96, 00f87d0a, ae93e8e6, 58aadb1d, d20b0094, 986fd...\n",
       "0002115f48f982                                 [9ec225f0, 18281c6c, e3b6b115, 4a044c54, 365fe576, a3188e54, b3f6e12d, ee7655ca, 84125b7a]\n",
       "                                                                           ...                                                           \n",
       "fffc30d5a0bc46    [09727c0c, ff1ea6a0, ddfef603, a01ce9b3, 3ba953ee, bf92a015, f4a0492a, 095812e6, 53125cfe, aa32a700, 63340e73, 06d8c...\n",
       "fffc3b44869198    [978a5137, faa48f03, 28dfb12a, eea2e812, 64fef97c, 4e0d1510, 58e68f2c, 8784e700, 4bd5a4cf, dc14bfec, 2aff7603, 8047d...\n",
       "fffc63ff750064    [5015c300, 411b85d9, 8238198c, f4781d1d, b5532930, e1f223e5, e7e67119, 4aaf741d, 7229cce6, a7fa3628, e4c2fa86, 1f8f9...\n",
       "fffcd063cda949    [7e6266ad, d8281fc5, d4ffcaef, 3e0e4a47, 21387fc8, cc229f9a, baed9c8b, d1bb21aa, 82979992, 65f95dad, eba4fa9e, c97e2...\n",
       "fffe1d764579d5    [1a63248d, 9c3b96a5, 1398a873, 4e2d4c2d, f71c538e, 8b44a5e8, 385dff7a, b8254ef8, 4d0e433e, debc496c, e15ae953, e4d79...\n",
       "Name: cell_order, Length: 139256, dtype: object"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_orders = pd.read_csv(\n",
    "    data_dir / 'train_orders.csv',\n",
    "    index_col='id',\n",
    "    squeeze=True,\n",
    ").str.split()  # Split the string representation of cell_ids into a list\n",
    "\n",
    "df_orders"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "914f98e0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:34:02.397575Z",
     "iopub.status.busy": "2022-05-15T12:34:02.397260Z",
     "iopub.status.idle": "2022-05-15T12:34:02.411599Z",
     "shell.execute_reply": "2022-05-15T12:34:02.410961Z"
    },
    "papermill": {
     "duration": 0.286454,
     "end_time": "2022-05-15T12:34:02.413634",
     "exception": false,
     "start_time": "2022-05-15T12:34:02.127180",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The ordered notebook:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cell_type</th>\n",
       "      <th>source</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cell_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>91d97bb2</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Read Data</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4e6f32f6</th>\n",
       "      <td>code</td>\n",
       "      <td># This Python 3 environment comes with many helpful analytics libraries installed\\n# It is defined by the kaggle/pyt...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0aeca210</th>\n",
       "      <td>code</td>\n",
       "      <td>import warnings\\nimport random\\n\\nSEED=44\\nrandom.seed(SEED)\\nnp.random.seed(SEED)\\npd.set_option('display.width', N...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cadfdb16</th>\n",
       "      <td>code</td>\n",
       "      <td>train = pd.read_csv('/kaggle/input/tabular-playground-series-mar-2021/train.csv')\\ntest = pd.read_csv('/kaggle/input...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fe39c117</th>\n",
       "      <td>code</td>\n",
       "      <td>train.info()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bbc5f229</th>\n",
       "      <td>code</td>\n",
       "      <td>y = train.iloc[:,-1]\\nX = train.iloc[:,:-1]\\nZ = test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91194a54</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Categorical</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46cc92d1</th>\n",
       "      <td>code</td>\n",
       "      <td>def get_obj_cols(df):\\n    return [col for col in df.columns if df.dtypes[col] == np.object]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9d6ea72b</th>\n",
       "      <td>code</td>\n",
       "      <td>X_objs = get_obj_cols(X)\\nX_objs_idx = [X.columns.get_loc(col) for col in X_objs]\\nZ_objs = get_obj_cols(Z)\\nZ_objs_...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d0f88604</th>\n",
       "      <td>code</td>\n",
       "      <td>for obj in X_objs:\\n    X[obj] = X[obj].astype('category').cat.codes\\nfor obj in Z_objs:\\n    Z[obj] = Z[obj].astype...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8df28832</th>\n",
       "      <td>code</td>\n",
       "      <td>X.drop('id', axis=1, inplace=True)\\nZ.drop('id', axis=1, inplace=True)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d3345791</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8c01934e</th>\n",
       "      <td>code</td>\n",
       "      <td>from sklearn.model_selection import StratifiedKFold, cross_val_score\\nfrom sklearn.metrics import roc_auc_score\\nK = 10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18b4e8d6</th>\n",
       "      <td>code</td>\n",
       "      <td>skf = StratifiedKFold(n_splits=K, shuffle=True, random_state=SEED)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c012109d</th>\n",
       "      <td>code</td>\n",
       "      <td>df_model = pd.DataFrame(columns = [*[f'model_{i}' for i in range(K)], 'average_auc'])\\ndf_model</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83ff0598</th>\n",
       "      <td>code</td>\n",
       "      <td>import lightgbm as lgbm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d35f13e5</th>\n",
       "      <td>code</td>\n",
       "      <td>params = {\\n    'learning_rate':1e-2,\\n    'max_bin':24,\\n    'n_estimators':1000,\\n    'device_type':'gpu',\\n    'm...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cfba39db</th>\n",
       "      <td>code</td>\n",
       "      <td>ctr = 0\\nmodel = []\\nauc_score = []\\nfor train_idx, val_idx in skf.split(X, y):\\n    Xt, Xv = X.loc[train_idx], X.lo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fbc1c206</th>\n",
       "      <td>code</td>\n",
       "      <td>df_model.loc[df_model.shape[0]] = [\\n    *model,\\n    np.mean(auc_score)\\n]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ace03b55</th>\n",
       "      <td>code</td>\n",
       "      <td>df_model['average_auc']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>671b78af</th>\n",
       "      <td>code</td>\n",
       "      <td>def predict(X):\\n    y_prob = []\\n    for i in range(K):\\n        y_prob.append(df_model.loc[0, f'model_{i}'].predic...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61d3ad05</th>\n",
       "      <td>code</td>\n",
       "      <td>from sklearn.model_selection import train_test_split</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6d6683ea</th>\n",
       "      <td>code</td>\n",
       "      <td>Xt, Xv, yt, yv = train_test_split(X, y, test_size=0.2, random_state=SEED)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2f41ebe2</th>\n",
       "      <td>code</td>\n",
       "      <td>yv_prob = predict(Xv)\\nmetrics(yv, yv_prob)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9dae7ae9</th>\n",
       "      <td>code</td>\n",
       "      <td>test_prob = predict(Z)\\nsubmission = pd.read_csv(\"../input/tabular-playground-series-mar-2021/sample_submission.csv\"...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         cell_type                                                                                                                   source\n",
       "cell_id                                                                                                                                    \n",
       "91d97bb2  markdown                                                                                                              # Read Data\n",
       "4e6f32f6      code  # This Python 3 environment comes with many helpful analytics libraries installed\\n# It is defined by the kaggle/pyt...\n",
       "0aeca210      code  import warnings\\nimport random\\n\\nSEED=44\\nrandom.seed(SEED)\\nnp.random.seed(SEED)\\npd.set_option('display.width', N...\n",
       "cadfdb16      code  train = pd.read_csv('/kaggle/input/tabular-playground-series-mar-2021/train.csv')\\ntest = pd.read_csv('/kaggle/input...\n",
       "fe39c117      code                                                                                                             train.info()\n",
       "bbc5f229      code                                                                    y = train.iloc[:,-1]\\nX = train.iloc[:,:-1]\\nZ = test\n",
       "91194a54  markdown                                                                                                            # Categorical\n",
       "46cc92d1      code                             def get_obj_cols(df):\\n    return [col for col in df.columns if df.dtypes[col] == np.object]\n",
       "9d6ea72b      code  X_objs = get_obj_cols(X)\\nX_objs_idx = [X.columns.get_loc(col) for col in X_objs]\\nZ_objs = get_obj_cols(Z)\\nZ_objs_...\n",
       "d0f88604      code  for obj in X_objs:\\n    X[obj] = X[obj].astype('category').cat.codes\\nfor obj in Z_objs:\\n    Z[obj] = Z[obj].astype...\n",
       "8df28832      code                                                   X.drop('id', axis=1, inplace=True)\\nZ.drop('id', axis=1, inplace=True)\n",
       "d3345791  markdown                                                                                                               # Training\n",
       "8c01934e      code  from sklearn.model_selection import StratifiedKFold, cross_val_score\\nfrom sklearn.metrics import roc_auc_score\\nK = 10\n",
       "18b4e8d6      code                                                       skf = StratifiedKFold(n_splits=K, shuffle=True, random_state=SEED)\n",
       "c012109d      code                          df_model = pd.DataFrame(columns = [*[f'model_{i}' for i in range(K)], 'average_auc'])\\ndf_model\n",
       "83ff0598      code                                                                                                  import lightgbm as lgbm\n",
       "d35f13e5      code  params = {\\n    'learning_rate':1e-2,\\n    'max_bin':24,\\n    'n_estimators':1000,\\n    'device_type':'gpu',\\n    'm...\n",
       "cfba39db      code  ctr = 0\\nmodel = []\\nauc_score = []\\nfor train_idx, val_idx in skf.split(X, y):\\n    Xt, Xv = X.loc[train_idx], X.lo...\n",
       "fbc1c206      code                                              df_model.loc[df_model.shape[0]] = [\\n    *model,\\n    np.mean(auc_score)\\n]\n",
       "ace03b55      code                                                                                                  df_model['average_auc']\n",
       "671b78af      code  def predict(X):\\n    y_prob = []\\n    for i in range(K):\\n        y_prob.append(df_model.loc[0, f'model_{i}'].predic...\n",
       "61d3ad05      code                                                                     from sklearn.model_selection import train_test_split\n",
       "6d6683ea      code                                                Xt, Xv, yt, yv = train_test_split(X, y, test_size=0.2, random_state=SEED)\n",
       "2f41ebe2      code                                                                              yv_prob = predict(Xv)\\nmetrics(yv, yv_prob)\n",
       "9dae7ae9      code  test_prob = predict(Z)\\nsubmission = pd.read_csv(\"../input/tabular-playground-series-mar-2021/sample_submission.csv\"..."
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get the correct order\n",
    "cell_order = df_orders.loc[nb_id]\n",
    "\n",
    "print(\"The ordered notebook:\")\n",
    "nb.loc[cell_order, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db2cc175",
   "metadata": {
    "papermill": {
     "duration": 0.264085,
     "end_time": "2022-05-15T12:34:02.969183",
     "exception": false,
     "start_time": "2022-05-15T12:34:02.705098",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The correct numeric position of a cell we will call the **rank** of the cell. We can find the ranks of the cells within a notebook by referencing the true ordering of cell ids as given in `train_orders.csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8c76168f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:34:03.504613Z",
     "iopub.status.busy": "2022-05-15T12:34:03.504116Z",
     "iopub.status.idle": "2022-05-15T12:34:03.516300Z",
     "shell.execute_reply": "2022-05-15T12:34:03.515713Z"
    },
    "papermill": {
     "duration": 0.280371,
     "end_time": "2022-05-15T12:34:03.518574",
     "exception": false,
     "start_time": "2022-05-15T12:34:03.238203",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rank</th>\n",
       "      <th>cell_type</th>\n",
       "      <th>source</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cell_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4e6f32f6</th>\n",
       "      <td>1</td>\n",
       "      <td>code</td>\n",
       "      <td># This Python 3 environment comes with many helpful analytics libraries installed\\n# It is defined by the kaggle/pyt...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0aeca210</th>\n",
       "      <td>2</td>\n",
       "      <td>code</td>\n",
       "      <td>import warnings\\nimport random\\n\\nSEED=44\\nrandom.seed(SEED)\\nnp.random.seed(SEED)\\npd.set_option('display.width', N...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cadfdb16</th>\n",
       "      <td>3</td>\n",
       "      <td>code</td>\n",
       "      <td>train = pd.read_csv('/kaggle/input/tabular-playground-series-mar-2021/train.csv')\\ntest = pd.read_csv('/kaggle/input...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fe39c117</th>\n",
       "      <td>4</td>\n",
       "      <td>code</td>\n",
       "      <td>train.info()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bbc5f229</th>\n",
       "      <td>5</td>\n",
       "      <td>code</td>\n",
       "      <td>y = train.iloc[:,-1]\\nX = train.iloc[:,:-1]\\nZ = test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46cc92d1</th>\n",
       "      <td>7</td>\n",
       "      <td>code</td>\n",
       "      <td>def get_obj_cols(df):\\n    return [col for col in df.columns if df.dtypes[col] == np.object]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9d6ea72b</th>\n",
       "      <td>8</td>\n",
       "      <td>code</td>\n",
       "      <td>X_objs = get_obj_cols(X)\\nX_objs_idx = [X.columns.get_loc(col) for col in X_objs]\\nZ_objs = get_obj_cols(Z)\\nZ_objs_...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d0f88604</th>\n",
       "      <td>9</td>\n",
       "      <td>code</td>\n",
       "      <td>for obj in X_objs:\\n    X[obj] = X[obj].astype('category').cat.codes\\nfor obj in Z_objs:\\n    Z[obj] = Z[obj].astype...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8df28832</th>\n",
       "      <td>10</td>\n",
       "      <td>code</td>\n",
       "      <td>X.drop('id', axis=1, inplace=True)\\nZ.drop('id', axis=1, inplace=True)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8c01934e</th>\n",
       "      <td>12</td>\n",
       "      <td>code</td>\n",
       "      <td>from sklearn.model_selection import StratifiedKFold, cross_val_score\\nfrom sklearn.metrics import roc_auc_score\\nK = 10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18b4e8d6</th>\n",
       "      <td>13</td>\n",
       "      <td>code</td>\n",
       "      <td>skf = StratifiedKFold(n_splits=K, shuffle=True, random_state=SEED)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c012109d</th>\n",
       "      <td>14</td>\n",
       "      <td>code</td>\n",
       "      <td>df_model = pd.DataFrame(columns = [*[f'model_{i}' for i in range(K)], 'average_auc'])\\ndf_model</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83ff0598</th>\n",
       "      <td>15</td>\n",
       "      <td>code</td>\n",
       "      <td>import lightgbm as lgbm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d35f13e5</th>\n",
       "      <td>16</td>\n",
       "      <td>code</td>\n",
       "      <td>params = {\\n    'learning_rate':1e-2,\\n    'max_bin':24,\\n    'n_estimators':1000,\\n    'device_type':'gpu',\\n    'm...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cfba39db</th>\n",
       "      <td>17</td>\n",
       "      <td>code</td>\n",
       "      <td>ctr = 0\\nmodel = []\\nauc_score = []\\nfor train_idx, val_idx in skf.split(X, y):\\n    Xt, Xv = X.loc[train_idx], X.lo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fbc1c206</th>\n",
       "      <td>18</td>\n",
       "      <td>code</td>\n",
       "      <td>df_model.loc[df_model.shape[0]] = [\\n    *model,\\n    np.mean(auc_score)\\n]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ace03b55</th>\n",
       "      <td>19</td>\n",
       "      <td>code</td>\n",
       "      <td>df_model['average_auc']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>671b78af</th>\n",
       "      <td>20</td>\n",
       "      <td>code</td>\n",
       "      <td>def predict(X):\\n    y_prob = []\\n    for i in range(K):\\n        y_prob.append(df_model.loc[0, f'model_{i}'].predic...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61d3ad05</th>\n",
       "      <td>21</td>\n",
       "      <td>code</td>\n",
       "      <td>from sklearn.model_selection import train_test_split</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6d6683ea</th>\n",
       "      <td>22</td>\n",
       "      <td>code</td>\n",
       "      <td>Xt, Xv, yt, yv = train_test_split(X, y, test_size=0.2, random_state=SEED)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2f41ebe2</th>\n",
       "      <td>23</td>\n",
       "      <td>code</td>\n",
       "      <td>yv_prob = predict(Xv)\\nmetrics(yv, yv_prob)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9dae7ae9</th>\n",
       "      <td>24</td>\n",
       "      <td>code</td>\n",
       "      <td>test_prob = predict(Z)\\nsubmission = pd.read_csv(\"../input/tabular-playground-series-mar-2021/sample_submission.csv\"...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d3345791</th>\n",
       "      <td>11</td>\n",
       "      <td>markdown</td>\n",
       "      <td># Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91194a54</th>\n",
       "      <td>6</td>\n",
       "      <td>markdown</td>\n",
       "      <td># Categorical</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91d97bb2</th>\n",
       "      <td>0</td>\n",
       "      <td>markdown</td>\n",
       "      <td># Read Data</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          rank cell_type                                                                                                                   source\n",
       "cell_id                                                                                                                                          \n",
       "4e6f32f6     1      code  # This Python 3 environment comes with many helpful analytics libraries installed\\n# It is defined by the kaggle/pyt...\n",
       "0aeca210     2      code  import warnings\\nimport random\\n\\nSEED=44\\nrandom.seed(SEED)\\nnp.random.seed(SEED)\\npd.set_option('display.width', N...\n",
       "cadfdb16     3      code  train = pd.read_csv('/kaggle/input/tabular-playground-series-mar-2021/train.csv')\\ntest = pd.read_csv('/kaggle/input...\n",
       "fe39c117     4      code                                                                                                             train.info()\n",
       "bbc5f229     5      code                                                                    y = train.iloc[:,-1]\\nX = train.iloc[:,:-1]\\nZ = test\n",
       "46cc92d1     7      code                             def get_obj_cols(df):\\n    return [col for col in df.columns if df.dtypes[col] == np.object]\n",
       "9d6ea72b     8      code  X_objs = get_obj_cols(X)\\nX_objs_idx = [X.columns.get_loc(col) for col in X_objs]\\nZ_objs = get_obj_cols(Z)\\nZ_objs_...\n",
       "d0f88604     9      code  for obj in X_objs:\\n    X[obj] = X[obj].astype('category').cat.codes\\nfor obj in Z_objs:\\n    Z[obj] = Z[obj].astype...\n",
       "8df28832    10      code                                                   X.drop('id', axis=1, inplace=True)\\nZ.drop('id', axis=1, inplace=True)\n",
       "8c01934e    12      code  from sklearn.model_selection import StratifiedKFold, cross_val_score\\nfrom sklearn.metrics import roc_auc_score\\nK = 10\n",
       "18b4e8d6    13      code                                                       skf = StratifiedKFold(n_splits=K, shuffle=True, random_state=SEED)\n",
       "c012109d    14      code                          df_model = pd.DataFrame(columns = [*[f'model_{i}' for i in range(K)], 'average_auc'])\\ndf_model\n",
       "83ff0598    15      code                                                                                                  import lightgbm as lgbm\n",
       "d35f13e5    16      code  params = {\\n    'learning_rate':1e-2,\\n    'max_bin':24,\\n    'n_estimators':1000,\\n    'device_type':'gpu',\\n    'm...\n",
       "cfba39db    17      code  ctr = 0\\nmodel = []\\nauc_score = []\\nfor train_idx, val_idx in skf.split(X, y):\\n    Xt, Xv = X.loc[train_idx], X.lo...\n",
       "fbc1c206    18      code                                              df_model.loc[df_model.shape[0]] = [\\n    *model,\\n    np.mean(auc_score)\\n]\n",
       "ace03b55    19      code                                                                                                  df_model['average_auc']\n",
       "671b78af    20      code  def predict(X):\\n    y_prob = []\\n    for i in range(K):\\n        y_prob.append(df_model.loc[0, f'model_{i}'].predic...\n",
       "61d3ad05    21      code                                                                     from sklearn.model_selection import train_test_split\n",
       "6d6683ea    22      code                                                Xt, Xv, yt, yv = train_test_split(X, y, test_size=0.2, random_state=SEED)\n",
       "2f41ebe2    23      code                                                                              yv_prob = predict(Xv)\\nmetrics(yv, yv_prob)\n",
       "9dae7ae9    24      code  test_prob = predict(Z)\\nsubmission = pd.read_csv(\"../input/tabular-playground-series-mar-2021/sample_submission.csv\"...\n",
       "d3345791    11  markdown                                                                                                               # Training\n",
       "91194a54     6  markdown                                                                                                            # Categorical\n",
       "91d97bb2     0  markdown                                                                                                              # Read Data"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def get_ranks(base, derived):\n",
    "    return [base.index(d) for d in derived]\n",
    "\n",
    "cell_ranks = get_ranks(cell_order, list(nb.index))\n",
    "nb.insert(0, 'rank', cell_ranks)\n",
    "\n",
    "nb"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c637206c",
   "metadata": {
    "papermill": {
     "duration": 0.264875,
     "end_time": "2022-05-15T12:34:04.052759",
     "exception": false,
     "start_time": "2022-05-15T12:34:03.787884",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Sorting a notebook by the cell ranks is another way to order the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e81828a5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:34:04.588441Z",
     "iopub.status.busy": "2022-05-15T12:34:04.588201Z",
     "iopub.status.idle": "2022-05-15T12:34:04.597108Z",
     "shell.execute_reply": "2022-05-15T12:34:04.596494Z"
    },
    "papermill": {
     "duration": 0.279854,
     "end_time": "2022-05-15T12:34:04.599097",
     "exception": false,
     "start_time": "2022-05-15T12:34:04.319243",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from pandas.testing import assert_frame_equal\n",
    "\n",
    "assert_frame_equal(nb.loc[cell_order, :], nb.sort_values('rank'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71b5baa7",
   "metadata": {
    "papermill": {
     "duration": 0.267998,
     "end_time": "2022-05-15T12:34:05.130614",
     "exception": false,
     "start_time": "2022-05-15T12:34:04.862616",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The algorithm we'll be using for our baseline model uses the cell ranks as the target, so let's create a dataframe of the ranks for each notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "d7fdbe32",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:34:05.660598Z",
     "iopub.status.busy": "2022-05-15T12:34:05.660053Z",
     "iopub.status.idle": "2022-05-15T12:34:08.026279Z",
     "shell.execute_reply": "2022-05-15T12:34:08.025413Z"
    },
    "papermill": {
     "duration": 2.633683,
     "end_time": "2022-05-15T12:34:08.028419",
     "exception": false,
     "start_time": "2022-05-15T12:34:05.394736",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>rank</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th>cell_id</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">000c0a9b2fef4d</th>\n",
       "      <th>1087237d</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d7209f1f</th>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>daf5b8ee</th>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e404213c</th>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2bad59b0</th>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">fffc63ff750064</th>\n",
       "      <th>56aa8da7</th>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>411b85d9</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e7e67119</th>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8b54cf58</th>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b3c6bc16</th>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>461166 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                        rank\n",
       "id             cell_id      \n",
       "000c0a9b2fef4d 1087237d    2\n",
       "               d7209f1f    4\n",
       "               daf5b8ee    6\n",
       "               e404213c    7\n",
       "               2bad59b0    8\n",
       "...                      ...\n",
       "fffc63ff750064 56aa8da7   25\n",
       "               411b85d9    1\n",
       "               e7e67119    6\n",
       "               8b54cf58   22\n",
       "               b3c6bc16   18\n",
       "\n",
       "[461166 rows x 1 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_orders_ = df_orders.to_frame().join(\n",
    "    df.reset_index('cell_id').groupby('id')['cell_id'].apply(list),\n",
    "    how='right',\n",
    ")\n",
    "\n",
    "ranks = {}\n",
    "for id_, cell_order, cell_id in df_orders_.itertuples():\n",
    "    ranks[id_] = {'cell_id': cell_id, 'rank': get_ranks(cell_order, cell_id)}\n",
    "\n",
    "df_ranks = (\n",
    "    pd.DataFrame\n",
    "    .from_dict(ranks, orient='index')\n",
    "    .rename_axis('id')\n",
    "    .apply(pd.Series.explode)\n",
    "    .set_index('cell_id', append=True)\n",
    ")\n",
    "\n",
    "df_ranks"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7df674ec",
   "metadata": {
    "papermill": {
     "duration": 0.274944,
     "end_time": "2022-05-15T12:34:08.569954",
     "exception": false,
     "start_time": "2022-05-15T12:34:08.295010",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Splits #"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e49e4479",
   "metadata": {
    "papermill": {
     "duration": 0.26489,
     "end_time": "2022-05-15T12:34:09.101204",
     "exception": false,
     "start_time": "2022-05-15T12:34:08.836314",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The `df_ancestors.csv` file identifies groups of notebooks derived from a common origin, that is, notebooks belonging to the same forking tree."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "78b1b70b",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:34:09.631611Z",
     "iopub.status.busy": "2022-05-15T12:34:09.631223Z",
     "iopub.status.idle": "2022-05-15T12:34:09.867396Z",
     "shell.execute_reply": "2022-05-15T12:34:09.866700Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "papermill": {
     "duration": 0.504446,
     "end_time": "2022-05-15T12:34:09.870068",
     "exception": false,
     "start_time": "2022-05-15T12:34:09.365622",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ancestor_id</th>\n",
       "      <th>parent_id</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>00001756c60be8</th>\n",
       "      <td>945aea18</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>00015c83e2717b</th>\n",
       "      <td>aa2da37e</td>\n",
       "      <td>317b65d12af9df</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0001bdd4021779</th>\n",
       "      <td>a7711fde</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0001daf4c2c76d</th>\n",
       "      <td>090152ca</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0002115f48f982</th>\n",
       "      <td>272b483a</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fffc30d5a0bc46</th>\n",
       "      <td>6aed207b</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fffc3b44869198</th>\n",
       "      <td>a6aaa8d7</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fffc63ff750064</th>\n",
       "      <td>0a1b5b65</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fffcd063cda949</th>\n",
       "      <td>d971e960</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fffe1d764579d5</th>\n",
       "      <td>3c40bfa6</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>139256 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "               ancestor_id       parent_id\n",
       "id                                        \n",
       "00001756c60be8    945aea18             NaN\n",
       "00015c83e2717b    aa2da37e  317b65d12af9df\n",
       "0001bdd4021779    a7711fde             NaN\n",
       "0001daf4c2c76d    090152ca             NaN\n",
       "0002115f48f982    272b483a             NaN\n",
       "...                    ...             ...\n",
       "fffc30d5a0bc46    6aed207b             NaN\n",
       "fffc3b44869198    a6aaa8d7             NaN\n",
       "fffc63ff750064    0a1b5b65             NaN\n",
       "fffcd063cda949    d971e960             NaN\n",
       "fffe1d764579d5    3c40bfa6             NaN\n",
       "\n",
       "[139256 rows x 2 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_ancestors = pd.read_csv(data_dir / 'train_ancestors.csv', index_col='id')\n",
    "df_ancestors"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60e942cd",
   "metadata": {
    "papermill": {
     "duration": 0.267075,
     "end_time": "2022-05-15T12:34:10.405897",
     "exception": false,
     "start_time": "2022-05-15T12:34:10.138822",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "To prevent leakage, the test set has no notebook with an ancestor in the training set. We therefore form a validation split using `ancestor_id` as a grouping factor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1b847468",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:34:10.938860Z",
     "iopub.status.busy": "2022-05-15T12:34:10.938285Z",
     "iopub.status.idle": "2022-05-15T12:34:11.872716Z",
     "shell.execute_reply": "2022-05-15T12:34:11.871801Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "papermill": {
     "duration": 1.202913,
     "end_time": "2022-05-15T12:34:11.875152",
     "exception": false,
     "start_time": "2022-05-15T12:34:10.672239",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GroupShuffleSplit\n",
    "\n",
    "NVALID = 0.1  # size of validation set\n",
    "\n",
    "splitter = GroupShuffleSplit(n_splits=1, test_size=NVALID, random_state=0)\n",
    "\n",
    "# Split, keeping notebooks with a common origin (ancestor_id) together\n",
    "ids = df.index.unique('id')\n",
    "ancestors = df_ancestors.loc[ids, 'ancestor_id']\n",
    "ids_train, ids_valid = next(splitter.split(ids, groups=ancestors))\n",
    "ids_train, ids_valid = ids[ids_train], ids[ids_valid]\n",
    "\n",
    "df_train = df.loc[ids_train, :]\n",
    "df_valid = df.loc[ids_valid, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5908fdd",
   "metadata": {
    "papermill": {
     "duration": 0.264884,
     "end_time": "2022-05-15T12:34:12.412084",
     "exception": false,
     "start_time": "2022-05-15T12:34:12.147200",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Feature Engineering #\n",
    "\n",
    "Let's generate [tf-idf features](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfTransformer.html#sklearn.feature_extraction.text.TfidfTransformer) to use with our ranking model. These features will help our model learn what kinds of words tend to occur most often at various positions within a notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b2166c49",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:34:12.948282Z",
     "iopub.status.busy": "2022-05-15T12:34:12.947784Z",
     "iopub.status.idle": "2022-05-15T12:34:25.562613Z",
     "shell.execute_reply": "2022-05-15T12:34:25.561718Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "papermill": {
     "duration": 12.883541,
     "end_time": "2022-05-15T12:34:25.564929",
     "exception": false,
     "start_time": "2022-05-15T12:34:12.681388",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "\n",
    "# Training set\n",
    "tfidf = TfidfVectorizer(min_df=0.001357, lowercase=True)\n",
    "X_train = tfidf.fit_transform(df_train['source'].astype(str))\n",
    "# Rank of each cell within the notebook\n",
    "y_train = df_ranks.loc[ids_train].to_numpy()\n",
    "# Number of cells in each notebook\n",
    "groups = df_ranks.loc[ids_train].groupby('id').size().to_numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a7d168e",
   "metadata": {
    "papermill": {
     "duration": 0.26783,
     "end_time": "2022-05-15T12:34:26.098828",
     "exception": false,
     "start_time": "2022-05-15T12:34:25.830998",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now let's add the code cell ordering as a feature. We'll append a column that enumerates the code cells in the correct order, like `1, 2, 3, 4, ...`, while having the dummy value `0` for all markdown cells. This feature will help the model learn to put the code cells in the correct order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0d89bfcd",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:34:26.665021Z",
     "iopub.status.busy": "2022-05-15T12:34:26.664437Z",
     "iopub.status.idle": "2022-05-15T12:34:27.002963Z",
     "shell.execute_reply": "2022-05-15T12:34:27.001792Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "papermill": {
     "duration": 0.63946,
     "end_time": "2022-05-15T12:34:27.005297",
     "exception": false,
     "start_time": "2022-05-15T12:34:26.365837",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(416586, 1680)\n"
     ]
    }
   ],
   "source": [
    "# Add code cell ordering\n",
    "X_train = sparse.hstack((\n",
    "    X_train,\n",
    "    np.where(\n",
    "        df_train['cell_type'] == 'code',\n",
    "        df_train.groupby(['id', 'cell_type']).cumcount().to_numpy() + 1,\n",
    "        0,\n",
    "    ).reshape(-1, 1)\n",
    "))\n",
    "print(X_train.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "49e2066f",
   "metadata": {
    "_kg_hide-output": true,
    "collapsed": true,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:34:27.593333Z",
     "iopub.status.busy": "2022-05-15T12:34:27.592416Z",
     "iopub.status.idle": "2022-05-15T12:34:27.607658Z",
     "shell.execute_reply": "2022-05-15T12:34:27.607011Z"
    },
    "jupyter": {
     "outputs_hidden": true
    },
    "papermill": {
     "duration": 0.319644,
     "end_time": "2022-05-15T12:34:27.609772",
     "exception": false,
     "start_time": "2022-05-15T12:34:27.290128",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>cell_type</th>\n",
       "      <th>source</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th>cell_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">000c0a9b2fef4d</th>\n",
       "      <th>1087237d</th>\n",
       "      <td>code</td>\n",
       "      <td># Data manipulation\\nimport pandas as pd\\nimport numpy as np\\n\\n# Data visualization\\nimport matplotlib.pyplot as pl...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d7209f1f</th>\n",
       "      <td>code</td>\n",
       "      <td>fifa_raw_dataset = pd.read_csv('../input/data.csv')</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>daf5b8ee</th>\n",
       "      <td>code</td>\n",
       "      <td>fifa_raw_dataset.head()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e404213c</th>\n",
       "      <td>code</td>\n",
       "      <td>fifa_raw_dataset.info()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2bad59b0</th>\n",
       "      <td>code</td>\n",
       "      <td>fifa_raw_dataset.shape</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">fffc63ff750064</th>\n",
       "      <th>56aa8da7</th>\n",
       "      <td>code</td>\n",
       "      <td>\\nsubmission.to_csv('house_price_rf.csv', index = False)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>411b85d9</th>\n",
       "      <td>markdown</td>\n",
       "      <td>1. # Data exploration</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e7e67119</th>\n",
       "      <td>markdown</td>\n",
       "      <td># # Data preprocessing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8b54cf58</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Post-process for submission</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b3c6bc16</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Define and fit model</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>416586 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                        cell_type                                                                                                                   source\n",
       "id             cell_id                                                                                                                                    \n",
       "000c0a9b2fef4d 1087237d      code  # Data manipulation\\nimport pandas as pd\\nimport numpy as np\\n\\n# Data visualization\\nimport matplotlib.pyplot as pl...\n",
       "               d7209f1f      code                                                                      fifa_raw_dataset = pd.read_csv('../input/data.csv')\n",
       "               daf5b8ee      code                                                                                                  fifa_raw_dataset.head()\n",
       "               e404213c      code                                                                                                  fifa_raw_dataset.info()\n",
       "               2bad59b0      code                                                                                                   fifa_raw_dataset.shape\n",
       "...                           ...                                                                                                                      ...\n",
       "fffc63ff750064 56aa8da7      code                                                                 \\nsubmission.to_csv('house_price_rf.csv', index = False)\n",
       "               411b85d9  markdown                                                                                                    1. # Data exploration\n",
       "               e7e67119  markdown                                                                                                   # # Data preprocessing\n",
       "               8b54cf58  markdown                                                                                            # Post-process for submission\n",
       "               b3c6bc16  markdown                                                                                                   # Define and fit model\n",
       "\n",
       "[416586 rows x 2 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1efb9156",
   "metadata": {
    "papermill": {
     "duration": 0.284571,
     "end_time": "2022-05-15T12:34:28.174630",
     "exception": false,
     "start_time": "2022-05-15T12:34:27.890059",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Train #"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fbd5826",
   "metadata": {
    "papermill": {
     "duration": 0.276095,
     "end_time": "2022-05-15T12:34:28.724158",
     "exception": false,
     "start_time": "2022-05-15T12:34:28.448063",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We'll use the ranking algorithm provided by XGBoost."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "693c78ff",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:34:29.261518Z",
     "iopub.status.busy": "2022-05-15T12:34:29.261228Z",
     "iopub.status.idle": "2022-05-15T12:35:05.585231Z",
     "shell.execute_reply": "2022-05-15T12:35:05.584318Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "papermill": {
     "duration": 36.927588,
     "end_time": "2022-05-15T12:35:05.918178",
     "exception": false,
     "start_time": "2022-05-15T12:34:28.990590",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "XGBRanker(base_score=0.5, booster='gbtree', callbacks=None, colsample_bylevel=1,\n",
       "          colsample_bynode=1, colsample_bytree=1, early_stopping_rounds=None,\n",
       "          enable_categorical=False, eval_metric=None, gamma=0, gpu_id=-1,\n",
       "          grow_policy='depthwise', importance_type=None,\n",
       "          interaction_constraints='', learning_rate=0.300000012, max_bin=256,\n",
       "          max_cat_to_onehot=4, max_delta_step=0, max_depth=6, max_leaves=0,\n",
       "          min_child_weight=12, missing=nan, monotone_constraints='()',\n",
       "          n_estimators=100, n_jobs=0, num_parallel_tree=1, predictor='auto',\n",
       "          random_state=0, reg_alpha=0, reg_lambda=1, ...)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from xgboost import XGBRanker\n",
    "\n",
    "#params = {'subsample': 0.9587444099995703, 'min_child_weight': 58, 'tree_method':'hist'}\n",
    "\n",
    "model = XGBRanker(#**params\n",
    "    min_child_weight=12,\n",
    "    subsample=0.55,\n",
    "    tree_method='hist',\n",
    ")\n",
    "model.fit(X_train, y_train, group=groups)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "cdf529b3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:06.469121Z",
     "iopub.status.busy": "2022-05-15T12:35:06.468546Z",
     "iopub.status.idle": "2022-05-15T12:35:06.472754Z",
     "shell.execute_reply": "2022-05-15T12:35:06.471994Z"
    },
    "papermill": {
     "duration": 0.276947,
     "end_time": "2022-05-15T12:35:06.474700",
     "exception": false,
     "start_time": "2022-05-15T12:35:06.197753",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_squared_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "de68f763",
   "metadata": {
    "_kg_hide-input": false,
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:07.013152Z",
     "iopub.status.busy": "2022-05-15T12:35:07.012594Z",
     "iopub.status.idle": "2022-05-15T12:35:07.020058Z",
     "shell.execute_reply": "2022-05-15T12:35:07.019168Z"
    },
    "papermill": {
     "duration": 0.280228,
     "end_time": "2022-05-15T12:35:07.022094",
     "exception": false,
     "start_time": "2022-05-15T12:35:06.741866",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"\\nimport optuna\\n#optuna.logging.set_verbosity(optuna.logging.WARNING)\\n\\n\\n# 1. Define an objective function to be maximized.\\ndef objective(trial):\\n    \\n    params = {\\n        'tree_method':'hist',\\n        'subsample': trial.suggest_float('subsample', 0.02, 1),\\n        'min_child_weight': trial.suggest_int('min_child_weight', 10, 100),\\n        \\n        'learning_rate': trial.suggest_float('learning_rate', 0, 1),\\n        'n_estimators': trial.suggest_int('n_estimators', 1, 2500),\\n        #'max_depth': trial.suggest_int('max_depth', 3, 16),\\n    }\\n\\n    model = XGBRanker(\\n        **params\\n        #min_child_weight=12,\\n    )\\n    model.fit(X_train, y_train, group=groups)\\n    y_pred = model.predict(X_train)\\n\\n    return 1 - mean_squared_error(y_train, y_pred)\\n\\n# 3. Create a study object and optimize the objective function.\\nstudy = optuna.create_study(direction='maximize')\\nstudy.optimize(objective, n_trials=125)\\n\""
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"\n",
    "import optuna\n",
    "#optuna.logging.set_verbosity(optuna.logging.WARNING)\n",
    "\n",
    "\n",
    "# 1. Define an objective function to be maximized.\n",
    "def objective(trial):\n",
    "    \n",
    "    params = {\n",
    "        'tree_method':'hist',\n",
    "        'subsample': trial.suggest_float('subsample', 0.02, 1),\n",
    "        'min_child_weight': trial.suggest_int('min_child_weight', 10, 100),\n",
    "        \n",
    "        'learning_rate': trial.suggest_float('learning_rate', 0, 1),\n",
    "        'n_estimators': trial.suggest_int('n_estimators', 1, 2500),\n",
    "        #'max_depth': trial.suggest_int('max_depth', 3, 16),\n",
    "    }\n",
    "\n",
    "    model = XGBRanker(\n",
    "        **params\n",
    "        #min_child_weight=12,\n",
    "    )\n",
    "    model.fit(X_train, y_train, group=groups)\n",
    "    y_pred = model.predict(X_train)\n",
    "\n",
    "    return 1 - mean_squared_error(y_train, y_pred)\n",
    "\n",
    "# 3. Create a study object and optimize the objective function.\n",
    "study = optuna.create_study(direction='maximize')\n",
    "study.optimize(objective, n_trials=125)\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "56f66409",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:07.567070Z",
     "iopub.status.busy": "2022-05-15T12:35:07.566792Z",
     "iopub.status.idle": "2022-05-15T12:35:07.570412Z",
     "shell.execute_reply": "2022-05-15T12:35:07.569542Z"
    },
    "papermill": {
     "duration": 0.280149,
     "end_time": "2022-05-15T12:35:07.572289",
     "exception": false,
     "start_time": "2022-05-15T12:35:07.292140",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#params = study.best_params\n",
    "#params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "e1bc3aec",
   "metadata": {
    "_kg_hide-input": true,
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:08.113515Z",
     "iopub.status.busy": "2022-05-15T12:35:08.112938Z",
     "iopub.status.idle": "2022-05-15T12:35:08.118231Z",
     "shell.execute_reply": "2022-05-15T12:35:08.117627Z"
    },
    "papermill": {
     "duration": 0.280522,
     "end_time": "2022-05-15T12:35:08.120090",
     "exception": false,
     "start_time": "2022-05-15T12:35:07.839568",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"\\n#params = {'subsample': 0.9587444099995703, 'min_child_weight': 58, 'tree_method':'hist'}\\n\\nmodel = XGBRanker(**params)\\nmodel.fit(X_train, y_train, group=groups)\\n\""
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"\n",
    "#params = {'subsample': 0.9587444099995703, 'min_child_weight': 58, 'tree_method':'hist'}\n",
    "\n",
    "model = XGBRanker(**params)\n",
    "model.fit(X_train, y_train, group=groups)\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f8ead17",
   "metadata": {
    "papermill": {
     "duration": 0.2992,
     "end_time": "2022-05-15T12:35:08.688677",
     "exception": false,
     "start_time": "2022-05-15T12:35:08.389477",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Evaluate #"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21754df5",
   "metadata": {
    "papermill": {
     "duration": 0.271442,
     "end_time": "2022-05-15T12:35:09.227632",
     "exception": false,
     "start_time": "2022-05-15T12:35:08.956190",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now let's see how well our model learned to order Kaggle notebook cells. We'll evaluate predictions on the validation set with a variant of the Kendall tau correlation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65094491",
   "metadata": {
    "papermill": {
     "duration": 0.267336,
     "end_time": "2022-05-15T12:35:09.762129",
     "exception": false,
     "start_time": "2022-05-15T12:35:09.494793",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Validation set ##"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50e6101f",
   "metadata": {
    "papermill": {
     "duration": 0.268279,
     "end_time": "2022-05-15T12:35:10.301687",
     "exception": false,
     "start_time": "2022-05-15T12:35:10.033408",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "First we'll create features for the validation set just like we did for the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "0a69d07f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:10.838010Z",
     "iopub.status.busy": "2022-05-15T12:35:10.837752Z",
     "iopub.status.idle": "2022-05-15T12:35:12.115780Z",
     "shell.execute_reply": "2022-05-15T12:35:12.115016Z"
    },
    "papermill": {
     "duration": 1.549184,
     "end_time": "2022-05-15T12:35:12.118035",
     "exception": false,
     "start_time": "2022-05-15T12:35:10.568851",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Validation set\n",
    "X_valid = tfidf.transform(df_valid['source'].astype(str))\n",
    "# The metric uses cell ids\n",
    "y_valid = df_orders.loc[ids_valid]\n",
    "\n",
    "X_valid = sparse.hstack((\n",
    "    X_valid,\n",
    "    np.where(\n",
    "        df_valid['cell_type'] == 'code',\n",
    "        df_valid.groupby(['id', 'cell_type']).cumcount().to_numpy() + 1,\n",
    "        0,\n",
    "    ).reshape(-1, 1)\n",
    "))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dac79fd9",
   "metadata": {
    "papermill": {
     "duration": 0.269296,
     "end_time": "2022-05-15T12:35:12.656191",
     "exception": false,
     "start_time": "2022-05-15T12:35:12.386895",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Here we'll use the model to predict the rank of each cell within its notebook and then convert these ranks into a list of ordered cell ids."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "8738c567",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:13.197472Z",
     "iopub.status.busy": "2022-05-15T12:35:13.197036Z",
     "iopub.status.idle": "2022-05-15T12:35:13.374102Z",
     "shell.execute_reply": "2022-05-15T12:35:13.373306Z"
    },
    "papermill": {
     "duration": 0.451147,
     "end_time": "2022-05-15T12:35:13.376031",
     "exception": false,
     "start_time": "2022-05-15T12:35:12.924884",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id\n",
       "0019c0de64fe80    [76b81a6e, c06f3027, 2e8ddcdc, 0bdb7484, 4470e13e, 12d2c276, 62cd27a1, 975dc007, 6a2f9600, 153d9221, 342a10b0, 175ac...\n",
       "0098e6a711804b    [0b57c7d1, bac84fb8, 7befb981, 2b1c4938, ec8c11d4, c85adef0, 92aebe06, 1f2574e6, b39006ed, 0b34ec73, d6e30791, ad148...\n",
       "0115938e54b661    [7bdb2279, 26008f09, 1393af9d, bc18f233, 2b20e565, 4ae6d198, e7596437, 60f096be, e2b23e8b, 38a4d1e2, 3912c173, 4247f...\n",
       "01a86eb72c41c6    [43e4bff0, bd5969e9, 6f3a8cf6, e155f365, 7af1cc87, c6e3de3d, e0c51b34, c05bfc2a, 904ebd55, 1c3732fa, 86e80ef2, 1e0ef...\n",
       "01b0f2f0cc925b    [e1ed7e8f, 4f96d02a, 46e763b2, ce25eefa, 6e0ade08, a720b51d, 8107082f, 55dbfb53, 26d53ebe, c4a12ce1, 786f1175, e3eec...\n",
       "021671a4f2e18c    [ebdb791f, 9572d02e, 296fb2fe, cbb7476c, a1d6a390, 298e543f, 24afcc91, c784ebec, 88a62103, 65d9ae24, c46226f3, 987aa...\n",
       "0245a6f3fe3b0f    [d868648f, 1dcf6f80, 70cd8db6, b7c21c6d, 201e80f2, 5411f98a, 906911bf, 86405f4a, 5df866b3, 1c7a4a93, 63a94813, 37a54...\n",
       "02462dad8226f3    [94503eaf, 9f9e9cd2, 2737a864, 3c7d380e, 68bb7490, bc9ff4aa, 8f668846, d8983c3a, 321b53ad, 9572c462, d33bb033, 70cb0...\n",
       "027887a911092e    [e84775a2, f4f088b2, 4389b63f, 1a9b7d44, 2f87a4ad, 8b1b5945, d1b5be9b, b877bdf1, c7c5c317, f7f7779e, 93a9ff7f, 2fd5e...\n",
       "02b11b6f20832b    [a97f4eb3, 37416f41, 02ba2843, 9f5cd934, 84d3366e, 6a4d026a, ae937e23, 0dd584fd, 48d93530, e6441161, 9745cd9f, 116ca...\n",
       "Name: cell_id, dtype: object"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred = pd.DataFrame({'rank': model.predict(X_valid)}, index=df_valid.index)\n",
    "y_pred = (\n",
    "    y_pred\n",
    "    .sort_values(['id', 'rank'])  # Sort the cells in each notebook by their rank.\n",
    "                                  # The cell_ids are now in the order the model predicted.\n",
    "    .reset_index('cell_id')  # Convert the cell_id index into a column.\n",
    "    .groupby('id')['cell_id'].apply(list)  # Group the cell_ids for each notebook into a list.\n",
    ")\n",
    "y_pred.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0dc4814",
   "metadata": {
    "papermill": {
     "duration": 0.273979,
     "end_time": "2022-05-15T12:35:13.923198",
     "exception": false,
     "start_time": "2022-05-15T12:35:13.649219",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now let's examine a notebook to see how the model did."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "af8100aa",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:14.468044Z",
     "iopub.status.busy": "2022-05-15T12:35:14.467520Z",
     "iopub.status.idle": "2022-05-15T12:35:14.491602Z",
     "shell.execute_reply": "2022-05-15T12:35:14.491026Z"
    },
    "papermill": {
     "duration": 0.296907,
     "end_time": "2022-05-15T12:35:14.493334",
     "exception": false,
     "start_time": "2022-05-15T12:35:14.196427",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cell_type</th>\n",
       "      <th>source</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cell_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>e84775a2</th>\n",
       "      <td>code</td>\n",
       "      <td># Bread and butter of Machine learning\\nimport pandas as pd\\nimport numpy as np\\nimport matplotlib.pyplot as plt\\n\\n...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>f4f088b2</th>\n",
       "      <td>code</td>\n",
       "      <td>train = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\\ntest = pd.read_csv('/kaggle/input/digit-recognizer/...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1a9b7d44</th>\n",
       "      <td>code</td>\n",
       "      <td># train.shape[0] is the number of images in the training data, and test.shape[0] holds the number of images in the t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2f87a4ad</th>\n",
       "      <td>code</td>\n",
       "      <td>def to_tensor(data):\\n    return [torch.FloatTensor(point) for point in data]\\n\\nclass MNISTData(Dataset):\\n    def ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8b1b5945</th>\n",
       "      <td>code</td>\n",
       "      <td># We'll split our data into 90% training and 10% test data \\nsplit = int(0.9 * len(train))\\nvalid_data = train[split...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d1b5be9b</th>\n",
       "      <td>code</td>\n",
       "      <td># Getting features of the image (pixel 0-783, 784 pixels in total for a 28*28 image)\\nX_col = list(train.columns[1:]...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c7c5c317</th>\n",
       "      <td>code</td>\n",
       "      <td>class dummyModel(nn.Module):\\n    def __init__(self):\\n        super(dummyModel, self).__init__()\\n        \\n       ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93a9ff7f</th>\n",
       "      <td>code</td>\n",
       "      <td>network = dummyModel()\\nprint(network)\\nnum_epochs = 5\\nfor epoch in range(num_epochs):\\n    for train_batch in trai...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2fd5e864</th>\n",
       "      <td>code</td>\n",
       "      <td>class model(nn.Module):\\n    def __init__(self):\\n        super(model, self).__init__()\\n        self.conv1 = nn.Seq...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67a541a1</th>\n",
       "      <td>code</td>\n",
       "      <td># Before we start training our neural network, we'll define our accuracy function\\ndef acc(y_true, y_pred):\\n    y_t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83c4fae9</th>\n",
       "      <td>code</td>\n",
       "      <td>network = model()\\n# Loss and optimizer\\ncriterion = nn.CrossEntropyLoss()\\noptimizer = torch.optim.Adam(network.par...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45c85876</th>\n",
       "      <td>code</td>\n",
       "      <td># Gotta prepare the test set first\\ntest[y_col] = [-1]*len(test)\\n\\ntest_set = MNISTData(test, X_col, y_col)\\ntest_l...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>685d0859</th>\n",
       "      <td>code</td>\n",
       "      <td>test_X, _ = next(iter(test_loader))\\ntest_X = test_X[:36]\\nfig, ax = plt.subplots(nrows=6, ncols=6, figsize=(15, 15)...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b886158b</th>\n",
       "      <td>code</td>\n",
       "      <td>submissions = pd.read_csv('../input/digit-recognizer/sample_submission.csv')\\nlabels = [x.item() for x in test_pred]...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6204b7f6</th>\n",
       "      <td>code</td>\n",
       "      <td>submissions.to_csv(\"submission.csv\", index=False)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2c0fb149</th>\n",
       "      <td>markdown</td>\n",
       "      <td>&gt; Well, that's pretty accurate!\\nLet's submit our predictions\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d628dab8</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Training our Neural Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4fddc47f</th>\n",
       "      <td>markdown</td>\n",
       "      <td>&gt; Oh Man, If you get these numbers for any dataset other than this, it's a major **RED** Flag!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3409c80e</th>\n",
       "      <td>markdown</td>\n",
       "      <td>Our Tensor is in the shape  $(N, C, H, W)$,  where\\n  * $N$ = batch size\\n  * $C$ = Number of channels\\n  * $H$ = He...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b877bdf1</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Converting the data in csv files to images\\n&gt; Let's convert our data from pixel values in csv to actual images\\n\\n...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6cb797dc</th>\n",
       "      <td>markdown</td>\n",
       "      <td>&gt; This was my first notebbok using Pytorch. Leave a upvote if you found it helpful!\\n\\nNote: You can always bump up ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8db4e028</th>\n",
       "      <td>markdown</td>\n",
       "      <td>&gt; Let's see our predictions\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85826b0f</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Making a Dummy Neural Network\\n&gt; So we're gonna define a model with 2 convolutional layers and 2 Fully Connected l...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5ab2119c</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Let's make Predictions</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>f7f7779e</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Meeting the data</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9e56a707</th>\n",
       "      <td>markdown</td>\n",
       "      <td>&gt; All the above data looks like garbage to us. We'll need to convert them to images.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4389b63f</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Mnist Handwritten Digit dataset with Pytorch (CNN)\\n&gt; Hi Everyone, in this notebook we're going to train a neural ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>f0a02a1b</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Training our (Dummy) Convolutional Neural Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70226413</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Creating our Model for training\\n&gt; Playtime is over, let's get down to business</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         cell_type                                                                                                                   source\n",
       "cell_id                                                                                                                                    \n",
       "e84775a2      code  # Bread and butter of Machine learning\\nimport pandas as pd\\nimport numpy as np\\nimport matplotlib.pyplot as plt\\n\\n...\n",
       "f4f088b2      code  train = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\\ntest = pd.read_csv('/kaggle/input/digit-recognizer/...\n",
       "1a9b7d44      code  # train.shape[0] is the number of images in the training data, and test.shape[0] holds the number of images in the t...\n",
       "2f87a4ad      code  def to_tensor(data):\\n    return [torch.FloatTensor(point) for point in data]\\n\\nclass MNISTData(Dataset):\\n    def ...\n",
       "8b1b5945      code  # We'll split our data into 90% training and 10% test data \\nsplit = int(0.9 * len(train))\\nvalid_data = train[split...\n",
       "d1b5be9b      code  # Getting features of the image (pixel 0-783, 784 pixels in total for a 28*28 image)\\nX_col = list(train.columns[1:]...\n",
       "c7c5c317      code  class dummyModel(nn.Module):\\n    def __init__(self):\\n        super(dummyModel, self).__init__()\\n        \\n       ...\n",
       "93a9ff7f      code  network = dummyModel()\\nprint(network)\\nnum_epochs = 5\\nfor epoch in range(num_epochs):\\n    for train_batch in trai...\n",
       "2fd5e864      code  class model(nn.Module):\\n    def __init__(self):\\n        super(model, self).__init__()\\n        self.conv1 = nn.Seq...\n",
       "67a541a1      code  # Before we start training our neural network, we'll define our accuracy function\\ndef acc(y_true, y_pred):\\n    y_t...\n",
       "83c4fae9      code  network = model()\\n# Loss and optimizer\\ncriterion = nn.CrossEntropyLoss()\\noptimizer = torch.optim.Adam(network.par...\n",
       "45c85876      code  # Gotta prepare the test set first\\ntest[y_col] = [-1]*len(test)\\n\\ntest_set = MNISTData(test, X_col, y_col)\\ntest_l...\n",
       "685d0859      code  test_X, _ = next(iter(test_loader))\\ntest_X = test_X[:36]\\nfig, ax = plt.subplots(nrows=6, ncols=6, figsize=(15, 15)...\n",
       "b886158b      code  submissions = pd.read_csv('../input/digit-recognizer/sample_submission.csv')\\nlabels = [x.item() for x in test_pred]...\n",
       "6204b7f6      code                                                                        submissions.to_csv(\"submission.csv\", index=False)\n",
       "2c0fb149  markdown                                                          > Well, that's pretty accurate!\\nLet's submit our predictions\\n\n",
       "d628dab8  markdown                                                                                            # Training our Neural Network\n",
       "4fddc47f  markdown                           > Oh Man, If you get these numbers for any dataset other than this, it's a major **RED** Flag!\n",
       "3409c80e  markdown  Our Tensor is in the shape  $(N, C, H, W)$,  where\\n  * $N$ = batch size\\n  * $C$ = Number of channels\\n  * $H$ = He...\n",
       "b877bdf1  markdown  # Converting the data in csv files to images\\n> Let's convert our data from pixel values in csv to actual images\\n\\n...\n",
       "6cb797dc  markdown  > This was my first notebbok using Pytorch. Leave a upvote if you found it helpful!\\n\\nNote: You can always bump up ...\n",
       "8db4e028  markdown                                                                                            > Let's see our predictions\\n\n",
       "85826b0f  markdown  # Making a Dummy Neural Network\\n> So we're gonna define a model with 2 convolutional layers and 2 Fully Connected l...\n",
       "5ab2119c  markdown                                                                                                 # Let's make Predictions\n",
       "f7f7779e  markdown                                                                                                       # Meeting the data\n",
       "9e56a707  markdown                                     > All the above data looks like garbage to us. We'll need to convert them to images.\n",
       "4389b63f  markdown  # Mnist Handwritten Digit dataset with Pytorch (CNN)\\n> Hi Everyone, in this notebook we're going to train a neural ...\n",
       "f0a02a1b  markdown                                                                      # Training our (Dummy) Convolutional Neural Network\n",
       "70226413  markdown                                        # Creating our Model for training\\n> Playtime is over, let's get down to business"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cell_type</th>\n",
       "      <th>source</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cell_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>e84775a2</th>\n",
       "      <td>code</td>\n",
       "      <td># Bread and butter of Machine learning\\nimport pandas as pd\\nimport numpy as np\\nimport matplotlib.pyplot as plt\\n\\n...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>f4f088b2</th>\n",
       "      <td>code</td>\n",
       "      <td>train = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\\ntest = pd.read_csv('/kaggle/input/digit-recognizer/...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4389b63f</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Mnist Handwritten Digit dataset with Pytorch (CNN)\\n&gt; Hi Everyone, in this notebook we're going to train a neural ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1a9b7d44</th>\n",
       "      <td>code</td>\n",
       "      <td># train.shape[0] is the number of images in the training data, and test.shape[0] holds the number of images in the t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2f87a4ad</th>\n",
       "      <td>code</td>\n",
       "      <td>def to_tensor(data):\\n    return [torch.FloatTensor(point) for point in data]\\n\\nclass MNISTData(Dataset):\\n    def ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8b1b5945</th>\n",
       "      <td>code</td>\n",
       "      <td># We'll split our data into 90% training and 10% test data \\nsplit = int(0.9 * len(train))\\nvalid_data = train[split...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d1b5be9b</th>\n",
       "      <td>code</td>\n",
       "      <td># Getting features of the image (pixel 0-783, 784 pixels in total for a 28*28 image)\\nX_col = list(train.columns[1:]...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b877bdf1</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Converting the data in csv files to images\\n&gt; Let's convert our data from pixel values in csv to actual images\\n\\n...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c7c5c317</th>\n",
       "      <td>code</td>\n",
       "      <td>class dummyModel(nn.Module):\\n    def __init__(self):\\n        super(dummyModel, self).__init__()\\n        \\n       ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>f7f7779e</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Meeting the data</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93a9ff7f</th>\n",
       "      <td>code</td>\n",
       "      <td>network = dummyModel()\\nprint(network)\\nnum_epochs = 5\\nfor epoch in range(num_epochs):\\n    for train_batch in trai...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2fd5e864</th>\n",
       "      <td>code</td>\n",
       "      <td>class model(nn.Module):\\n    def __init__(self):\\n        super(model, self).__init__()\\n        self.conv1 = nn.Seq...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4fddc47f</th>\n",
       "      <td>markdown</td>\n",
       "      <td>&gt; Oh Man, If you get these numbers for any dataset other than this, it's a major **RED** Flag!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9e56a707</th>\n",
       "      <td>markdown</td>\n",
       "      <td>&gt; All the above data looks like garbage to us. We'll need to convert them to images.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67a541a1</th>\n",
       "      <td>code</td>\n",
       "      <td># Before we start training our neural network, we'll define our accuracy function\\ndef acc(y_true, y_pred):\\n    y_t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3409c80e</th>\n",
       "      <td>markdown</td>\n",
       "      <td>Our Tensor is in the shape  $(N, C, H, W)$,  where\\n  * $N$ = batch size\\n  * $C$ = Number of channels\\n  * $H$ = He...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45c85876</th>\n",
       "      <td>code</td>\n",
       "      <td># Gotta prepare the test set first\\ntest[y_col] = [-1]*len(test)\\n\\ntest_set = MNISTData(test, X_col, y_col)\\ntest_l...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b886158b</th>\n",
       "      <td>code</td>\n",
       "      <td>submissions = pd.read_csv('../input/digit-recognizer/sample_submission.csv')\\nlabels = [x.item() for x in test_pred]...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>685d0859</th>\n",
       "      <td>code</td>\n",
       "      <td>test_X, _ = next(iter(test_loader))\\ntest_X = test_X[:36]\\nfig, ax = plt.subplots(nrows=6, ncols=6, figsize=(15, 15)...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d628dab8</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Training our Neural Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>f0a02a1b</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Training our (Dummy) Convolutional Neural Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83c4fae9</th>\n",
       "      <td>code</td>\n",
       "      <td>network = model()\\n# Loss and optimizer\\ncriterion = nn.CrossEntropyLoss()\\noptimizer = torch.optim.Adam(network.par...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85826b0f</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Making a Dummy Neural Network\\n&gt; So we're gonna define a model with 2 convolutional layers and 2 Fully Connected l...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70226413</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Creating our Model for training\\n&gt; Playtime is over, let's get down to business</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6cb797dc</th>\n",
       "      <td>markdown</td>\n",
       "      <td>&gt; This was my first notebbok using Pytorch. Leave a upvote if you found it helpful!\\n\\nNote: You can always bump up ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8db4e028</th>\n",
       "      <td>markdown</td>\n",
       "      <td>&gt; Let's see our predictions\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5ab2119c</th>\n",
       "      <td>markdown</td>\n",
       "      <td># Let's make Predictions</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2c0fb149</th>\n",
       "      <td>markdown</td>\n",
       "      <td>&gt; Well, that's pretty accurate!\\nLet's submit our predictions\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6204b7f6</th>\n",
       "      <td>code</td>\n",
       "      <td>submissions.to_csv(\"submission.csv\", index=False)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         cell_type                                                                                                                   source\n",
       "cell_id                                                                                                                                    \n",
       "e84775a2      code  # Bread and butter of Machine learning\\nimport pandas as pd\\nimport numpy as np\\nimport matplotlib.pyplot as plt\\n\\n...\n",
       "f4f088b2      code  train = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\\ntest = pd.read_csv('/kaggle/input/digit-recognizer/...\n",
       "4389b63f  markdown  # Mnist Handwritten Digit dataset with Pytorch (CNN)\\n> Hi Everyone, in this notebook we're going to train a neural ...\n",
       "1a9b7d44      code  # train.shape[0] is the number of images in the training data, and test.shape[0] holds the number of images in the t...\n",
       "2f87a4ad      code  def to_tensor(data):\\n    return [torch.FloatTensor(point) for point in data]\\n\\nclass MNISTData(Dataset):\\n    def ...\n",
       "8b1b5945      code  # We'll split our data into 90% training and 10% test data \\nsplit = int(0.9 * len(train))\\nvalid_data = train[split...\n",
       "d1b5be9b      code  # Getting features of the image (pixel 0-783, 784 pixels in total for a 28*28 image)\\nX_col = list(train.columns[1:]...\n",
       "b877bdf1  markdown  # Converting the data in csv files to images\\n> Let's convert our data from pixel values in csv to actual images\\n\\n...\n",
       "c7c5c317      code  class dummyModel(nn.Module):\\n    def __init__(self):\\n        super(dummyModel, self).__init__()\\n        \\n       ...\n",
       "f7f7779e  markdown                                                                                                       # Meeting the data\n",
       "93a9ff7f      code  network = dummyModel()\\nprint(network)\\nnum_epochs = 5\\nfor epoch in range(num_epochs):\\n    for train_batch in trai...\n",
       "2fd5e864      code  class model(nn.Module):\\n    def __init__(self):\\n        super(model, self).__init__()\\n        self.conv1 = nn.Seq...\n",
       "4fddc47f  markdown                           > Oh Man, If you get these numbers for any dataset other than this, it's a major **RED** Flag!\n",
       "9e56a707  markdown                                     > All the above data looks like garbage to us. We'll need to convert them to images.\n",
       "67a541a1      code  # Before we start training our neural network, we'll define our accuracy function\\ndef acc(y_true, y_pred):\\n    y_t...\n",
       "3409c80e  markdown  Our Tensor is in the shape  $(N, C, H, W)$,  where\\n  * $N$ = batch size\\n  * $C$ = Number of channels\\n  * $H$ = He...\n",
       "45c85876      code  # Gotta prepare the test set first\\ntest[y_col] = [-1]*len(test)\\n\\ntest_set = MNISTData(test, X_col, y_col)\\ntest_l...\n",
       "b886158b      code  submissions = pd.read_csv('../input/digit-recognizer/sample_submission.csv')\\nlabels = [x.item() for x in test_pred]...\n",
       "685d0859      code  test_X, _ = next(iter(test_loader))\\ntest_X = test_X[:36]\\nfig, ax = plt.subplots(nrows=6, ncols=6, figsize=(15, 15)...\n",
       "d628dab8  markdown                                                                                            # Training our Neural Network\n",
       "f0a02a1b  markdown                                                                      # Training our (Dummy) Convolutional Neural Network\n",
       "83c4fae9      code  network = model()\\n# Loss and optimizer\\ncriterion = nn.CrossEntropyLoss()\\noptimizer = torch.optim.Adam(network.par...\n",
       "85826b0f  markdown  # Making a Dummy Neural Network\\n> So we're gonna define a model with 2 convolutional layers and 2 Fully Connected l...\n",
       "70226413  markdown                                        # Creating our Model for training\\n> Playtime is over, let's get down to business\n",
       "6cb797dc  markdown  > This was my first notebbok using Pytorch. Leave a upvote if you found it helpful!\\n\\nNote: You can always bump up ...\n",
       "8db4e028  markdown                                                                                            > Let's see our predictions\\n\n",
       "5ab2119c  markdown                                                                                                 # Let's make Predictions\n",
       "2c0fb149  markdown                                                          > Well, that's pretty accurate!\\nLet's submit our predictions\\n\n",
       "6204b7f6      code                                                                        submissions.to_csv(\"submission.csv\", index=False)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "nb_id = df_valid.index.get_level_values('id').unique()[8]\n",
    "\n",
    "display(df.loc[nb_id])\n",
    "display(df.loc[nb_id].loc[y_pred.loc[nb_id]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03d6fceb",
   "metadata": {
    "papermill": {
     "duration": 0.27418,
     "end_time": "2022-05-15T12:35:15.045259",
     "exception": false,
     "start_time": "2022-05-15T12:35:14.771079",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Metric ##\n",
    "\n",
    "This competition uses a variant of the [Kendall tau correlation](https://www.kaggle.com/competitions/AI4Code/overview/evaluation), which will measure how close to the correct order our predicted orderings are. See this notebook for more on this metric: [Competition Metric - Kendall Tau Correlation](https://www.kaggle.com/code/ryanholbrook/competition-metric-kendall-tau-correlation/notebook)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "168611a6",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:15.592819Z",
     "iopub.status.busy": "2022-05-15T12:35:15.592339Z",
     "iopub.status.idle": "2022-05-15T12:35:15.598049Z",
     "shell.execute_reply": "2022-05-15T12:35:15.597493Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "papermill": {
     "duration": 0.283182,
     "end_time": "2022-05-15T12:35:15.599916",
     "exception": false,
     "start_time": "2022-05-15T12:35:15.316734",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from bisect import bisect\n",
    "\n",
    "\n",
    "def count_inversions(a):\n",
    "    inversions = 0\n",
    "    sorted_so_far = []\n",
    "    for i, u in enumerate(a):\n",
    "        j = bisect(sorted_so_far, u)\n",
    "        inversions += i - j\n",
    "        sorted_so_far.insert(j, u)\n",
    "    return inversions\n",
    "\n",
    "\n",
    "def kendall_tau(ground_truth, predictions):\n",
    "    total_inversions = 0\n",
    "    total_2max = 0  # twice the maximum possible inversions across all instances\n",
    "    for gt, pred in zip(ground_truth, predictions):\n",
    "        ranks = [gt.index(x) for x in pred]  # rank predicted order in terms of ground truth\n",
    "        total_inversions += count_inversions(ranks)\n",
    "        n = len(gt)\n",
    "        total_2max += n * (n - 1)\n",
    "    return 1 - 4 * total_inversions / total_2max"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cca19ee1",
   "metadata": {
    "papermill": {
     "duration": 0.276114,
     "end_time": "2022-05-15T12:35:16.152185",
     "exception": false,
     "start_time": "2022-05-15T12:35:15.876071",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's test the metric with a dummy submission created from the ids of the shuffled notebooks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "906138fc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:16.704933Z",
     "iopub.status.busy": "2022-05-15T12:35:16.704467Z",
     "iopub.status.idle": "2022-05-15T12:35:16.805702Z",
     "shell.execute_reply": "2022-05-15T12:35:16.804856Z"
    },
    "papermill": {
     "duration": 0.380042,
     "end_time": "2022-05-15T12:35:16.807574",
     "exception": false,
     "start_time": "2022-05-15T12:35:16.427532",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.42511216883092573"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_dummy = df_valid.reset_index('cell_id').groupby('id')['cell_id'].apply(list)\n",
    "kendall_tau(y_valid, y_dummy)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65460ab5",
   "metadata": {
    "papermill": {
     "duration": 0.274292,
     "end_time": "2022-05-15T12:35:17.358708",
     "exception": false,
     "start_time": "2022-05-15T12:35:17.084416",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Comparing this to the score on the predictions, we can see that our model was indeed able to improve the cell ordering somewhat."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "ea86bceb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:17.907014Z",
     "iopub.status.busy": "2022-05-15T12:35:17.906399Z",
     "iopub.status.idle": "2022-05-15T12:35:17.970923Z",
     "shell.execute_reply": "2022-05-15T12:35:17.970016Z"
    },
    "papermill": {
     "duration": 0.340574,
     "end_time": "2022-05-15T12:35:17.972803",
     "exception": false,
     "start_time": "2022-05-15T12:35:17.632229",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6325826824249605"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kendall_tau(y_valid, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee13ccef",
   "metadata": {
    "papermill": {
     "duration": 0.282067,
     "end_time": "2022-05-15T12:35:18.533499",
     "exception": false,
     "start_time": "2022-05-15T12:35:18.251432",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Submission #"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec926f74",
   "metadata": {
    "papermill": {
     "duration": 0.300952,
     "end_time": "2022-05-15T12:35:19.135979",
     "exception": false,
     "start_time": "2022-05-15T12:35:18.835027",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "To create a submission for this competition, we'll apply our model to the notebooks in the test set. Note that this is a **Code Competition**, which means that the test data we see here is only a small sample. When we submit our notebook for scoring, this example data will be replaced with the full test set of about 20,000 notebooks."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48eb3130",
   "metadata": {
    "papermill": {
     "duration": 0.314016,
     "end_time": "2022-05-15T12:35:19.738982",
     "exception": false,
     "start_time": "2022-05-15T12:35:19.424966",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "First we load the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "bc9ae577",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:20.319403Z",
     "iopub.status.busy": "2022-05-15T12:35:20.318793Z",
     "iopub.status.idle": "2022-05-15T12:35:20.372298Z",
     "shell.execute_reply": "2022-05-15T12:35:20.371625Z"
    },
    "papermill": {
     "duration": 0.349178,
     "end_time": "2022-05-15T12:35:20.374378",
     "exception": false,
     "start_time": "2022-05-15T12:35:20.025200",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Test NBs: 100%|██████████| 4/4 [00:00<00:00, 115.66it/s]\n"
     ]
    }
   ],
   "source": [
    "paths_test = list((data_dir / 'test').glob('*.json'))\n",
    "notebooks_test = [\n",
    "    read_notebook(path) for path in tqdm(paths_test, desc='Test NBs')\n",
    "]\n",
    "df_test = (\n",
    "    pd.concat(notebooks_test)\n",
    "    .set_index('id', append=True)\n",
    "    .swaplevel()\n",
    "    .sort_index(level='id', sort_remaining=False)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7995a864",
   "metadata": {
    "papermill": {
     "duration": 0.298713,
     "end_time": "2022-05-15T12:35:20.954673",
     "exception": false,
     "start_time": "2022-05-15T12:35:20.655960",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Then create the tf-idf and code cell features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "6197aa16",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:21.526879Z",
     "iopub.status.busy": "2022-05-15T12:35:21.526342Z",
     "iopub.status.idle": "2022-05-15T12:35:21.541493Z",
     "shell.execute_reply": "2022-05-15T12:35:21.540860Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "papermill": {
     "duration": 0.305538,
     "end_time": "2022-05-15T12:35:21.543489",
     "exception": false,
     "start_time": "2022-05-15T12:35:21.237951",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_test = tfidf.transform(df_test['source'].astype(str))\n",
    "X_test = sparse.hstack((\n",
    "    X_test,\n",
    "    np.where(\n",
    "        df_test['cell_type'] == 'code',\n",
    "        df_test.groupby(['id', 'cell_type']).cumcount().to_numpy() + 1,\n",
    "        0,\n",
    "    ).reshape(-1, 1)\n",
    "))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8673514f",
   "metadata": {
    "papermill": {
     "duration": 0.271968,
     "end_time": "2022-05-15T12:35:22.088270",
     "exception": false,
     "start_time": "2022-05-15T12:35:21.816302",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "And then create predictions on the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "6d983b3b",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:22.633831Z",
     "iopub.status.busy": "2022-05-15T12:35:22.633379Z",
     "iopub.status.idle": "2022-05-15T12:35:22.653713Z",
     "shell.execute_reply": "2022-05-15T12:35:22.652879Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "papermill": {
     "duration": 0.296197,
     "end_time": "2022-05-15T12:35:22.655856",
     "exception": false,
     "start_time": "2022-05-15T12:35:22.359659",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id\n",
       "0009d135ece78d    [ddfd239c, c6cd22db, 1372ae9b, 8cb8d28a, 7f388a41, 0a226b6a, 90ed07ab, 2843a25a, 06dbf8cf, f9893819, e25aa9bd, ba55e...\n",
       "0010483c12ba9b                       [54c7cab3, fe66203e, 7844d5f8, 5ce8863c, 4a32c095, 7f270e34, 02a0be6d, 4a0777c4, 865ad516, 4703bb6d]\n",
       "0010a919d60e4f    [aafc3d23, b7578789, 80e077ec, b190ebb4, ed415c3c, bbff12d4, 322850af, 8ce62db4, c069ed33, 23607d04, 868c4eae, 80433...\n",
       "0028856e09c5b7                                                                                   [012c9d02, eb293dfc, 3ae7ece3, d22526d1]\n",
       "Name: cell_id, dtype: object"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_infer = pd.DataFrame({'rank': model.predict(X_test)}, index=df_test.index)\n",
    "y_infer = y_infer.sort_values(['id', 'rank']).reset_index('cell_id').groupby('id')['cell_id'].apply(list)\n",
    "y_infer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04b0482f",
   "metadata": {
    "papermill": {
     "duration": 0.272973,
     "end_time": "2022-05-15T12:35:23.205056",
     "exception": false,
     "start_time": "2022-05-15T12:35:22.932083",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The `sample_submission.csv` file shows what a correctly formatted submission must look like. We'll just use it as a visual check, but you might like to directly modify the values of sample submission instead. (This would help prevent failed submissions due to missing notebook ids or incorrectly named columns, for instance.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "d0e5ae48",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:23.756696Z",
     "iopub.status.busy": "2022-05-15T12:35:23.756164Z",
     "iopub.status.idle": "2022-05-15T12:35:23.771646Z",
     "shell.execute_reply": "2022-05-15T12:35:23.770841Z"
    },
    "papermill": {
     "duration": 0.29342,
     "end_time": "2022-05-15T12:35:23.773911",
     "exception": false,
     "start_time": "2022-05-15T12:35:23.480491",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id\n",
       "0009d135ece78d       ddfd239c c6cd22db 1372ae9b 90ed07ab 7f388a41 2843a25a 06dbf8cf f9893819 ba55e576 39e937ec e25aa9bd 0a226b6a 8cb8d28a\n",
       "0010483c12ba9b                                  54c7cab3 fe66203e 7844d5f8 5ce8863c 4a0777c4 4703bb6d 4a32c095 865ad516 02a0be6d 7f270e34\n",
       "0010a919d60e4f    aafc3d23 80e077ec b190ebb4 ed415c3c 322850af c069ed33 868c4eae 80433cf3 bd8fbd76 0e2529e8 1345b8b2 cdae286f 4907b9ef...\n",
       "0028856e09c5b7                                                                                        012c9d02 d22526d1 3ae7ece3 eb293dfc\n",
       "Name: cell_order, dtype: object"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_sample = pd.read_csv(data_dir / 'sample_submission.csv', index_col='id', squeeze=True)\n",
    "y_sample"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da3c9a27",
   "metadata": {
    "papermill": {
     "duration": 0.272357,
     "end_time": "2022-05-15T12:35:24.320761",
     "exception": false,
     "start_time": "2022-05-15T12:35:24.048404",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We can see that a correctly formatted submission needs the index named `id` and the column of cell orders named `cell_order`. Moreover, we need to convert the list of cell ids into a space-delimited string of cell ids."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "7ff78e46",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:24.870657Z",
     "iopub.status.busy": "2022-05-15T12:35:24.870340Z",
     "iopub.status.idle": "2022-05-15T12:35:24.878295Z",
     "shell.execute_reply": "2022-05-15T12:35:24.877343Z"
    },
    "papermill": {
     "duration": 0.287525,
     "end_time": "2022-05-15T12:35:24.880265",
     "exception": false,
     "start_time": "2022-05-15T12:35:24.592740",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id\n",
       "0009d135ece78d       ddfd239c c6cd22db 1372ae9b 8cb8d28a 7f388a41 0a226b6a 90ed07ab 2843a25a 06dbf8cf f9893819 e25aa9bd ba55e576 39e937ec\n",
       "0010483c12ba9b                                  54c7cab3 fe66203e 7844d5f8 5ce8863c 4a32c095 7f270e34 02a0be6d 4a0777c4 865ad516 4703bb6d\n",
       "0010a919d60e4f    aafc3d23 b7578789 80e077ec b190ebb4 ed415c3c bbff12d4 322850af 8ce62db4 c069ed33 23607d04 868c4eae 80433cf3 bac960d3...\n",
       "0028856e09c5b7                                                                                        012c9d02 eb293dfc 3ae7ece3 d22526d1\n",
       "Name: cell_order, dtype: object"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_submit = (\n",
    "    y_infer\n",
    "    .apply(' '.join)  # list of ids -> string of ids\n",
    "    .rename_axis('id')\n",
    "    .rename('cell_order')\n",
    ")\n",
    "y_submit"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52ecc2eb",
   "metadata": {
    "papermill": {
     "duration": 0.275327,
     "end_time": "2022-05-15T12:35:25.430876",
     "exception": false,
     "start_time": "2022-05-15T12:35:25.155549",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "And finally we'll write out the formatted submissions to a file `submission.csv`. When we submit our notebook, it will be rerun on the full test data to create the submission file that's actually scored."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "a71e1e47",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-15T12:35:25.980697Z",
     "iopub.status.busy": "2022-05-15T12:35:25.980079Z",
     "iopub.status.idle": "2022-05-15T12:35:25.987960Z",
     "shell.execute_reply": "2022-05-15T12:35:25.987416Z"
    },
    "papermill": {
     "duration": 0.287047,
     "end_time": "2022-05-15T12:35:25.989859",
     "exception": false,
     "start_time": "2022-05-15T12:35:25.702812",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "y_submit.to_csv('submission.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "229795cb",
   "metadata": {
    "papermill": {
     "duration": 0.27294,
     "end_time": "2022-05-15T12:35:26.534557",
     "exception": false,
     "start_time": "2022-05-15T12:35:26.261617",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<h1 style=\"color:#fc3f51\"> Conclusion </h1>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4aaa3063",
   "metadata": {
    "papermill": {
     "duration": 0.279413,
     "end_time": "2022-05-15T12:35:27.086375",
     "exception": false,
     "start_time": "2022-05-15T12:35:26.806962",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The readjustment of the model parameters improved the score (0.588 to 0.592), but a good vectorization of the data improves much better (0.592 to 0.603)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f216860b",
   "metadata": {
    "papermill": {
     "duration": 0.273596,
     "end_time": "2022-05-15T12:35:27.634712",
     "exception": false,
     "start_time": "2022-05-15T12:35:27.361116",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<h1 style=\"color:#fc3f51\"> What next ? </h1>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72dc7563",
   "metadata": {
    "papermill": {
     "duration": 0.276071,
     "end_time": "2022-05-15T12:35:28.182708",
     "exception": false,
     "start_time": "2022-05-15T12:35:27.906637",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "- Improve TfidfVectorizer params for better score\n",
    "- Research other Feature Engineering\n",
    "- Other Model (like LGBMRanker)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a3d2eba",
   "metadata": {
    "papermill": {
     "duration": 0.274078,
     "end_time": "2022-05-15T12:35:28.729596",
     "exception": false,
     "start_time": "2022-05-15T12:35:28.455518",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<center>\n",
    "    <h1 style=\"color:#3c3f51\"> Thanks for reading 🙂 </h1>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 197.549121,
   "end_time": "2022-05-15T12:35:30.428005",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2022-05-15T12:32:12.878884",
   "version": "2.3.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}