pandas/selection.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Selection\r\n",
    "\r\n",
    "After we create a `DataFrame`, we have several ways to select data from either **row-wise** or **column-wise**, either through **indexing** or **slicing**, and either by **label's name** or **position**.\r\n",
    "\r\n",
    "I have concluded a table for selecting the data.\r\n",
    "\r\n",
    "|                     | **Single Column**                                | **Multiple Columns**                                       | **Continuous Columns**              | **All Columns**  |\r\n",
    "| ------------------- | ------------------------------------------------ | ---------------------------------------------------------- | ----------------------------------- | ---------------- |\r\n",
    "| **Single Row**      | `df.loc[row, column]` or <br> `df.at[row, column]` | `df.loc[row, [column, column]]`                            | `df.loc[row, column:column]`        | `df.loc[row]`    |\r\n",
    "| **Multiple Rows**   | `df.loc[[row, row], column]`                     | `df.loc[[row, row], [column, column]]`                     | `df.loc[[row, row], column:column]` | `df[[row, row]]` |\r\n",
    "| **Continuous Rows** | `df.loc[row:row, column]`                        | `df.loc[row:row, [column, column]]`                        | `df.loc[row:row, column:column]`    | `df[row:row]`    |\r\n",
    "| **All Rows**        | `df[column]`                                     | `df[[column, column]]` or <br> `df.loc[:, [column, column]]` | `df.loc[:, column:column]`          | `df`             |\r\n",
    "\r\n",
    "- `df.iloc` is same as `df.loc` but using position.\r\n",
    "- `df.iat` is same as `df.at` but using position."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\r\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>col1</th>\n      <th>col2</th>\n      <th>col3</th>\n      <th>col4</th>\n      <th>col5</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>a</th>\n      <td>0</td>\n      <td>1</td>\n      <td>2</td>\n      <td>3</td>\n      <td>4</td>\n    </tr>\n    <tr>\n      <th>b</th>\n      <td>5</td>\n      <td>6</td>\n      <td>7</td>\n      <td>8</td>\n      <td>9</td>\n    </tr>\n    <tr>\n      <th>c</th>\n      <td>10</td>\n      <td>11</td>\n      <td>12</td>\n      <td>13</td>\n      <td>14</td>\n    </tr>\n    <tr>\n      <th>d</th>\n      <td>15</td>\n      <td>16</td>\n      <td>17</td>\n      <td>18</td>\n      <td>19</td>\n    </tr>\n    <tr>\n      <th>e</th>\n      <td>20</td>\n      <td>21</td>\n      <td>22</td>\n      <td>23</td>\n      <td>24</td>\n    </tr>\n    <tr>\n      <th>f</th>\n      <td>25</td>\n      <td>26</td>\n      <td>27</td>\n      <td>28</td>\n      <td>29</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "   col1  col2  col3  col4  col5\na     0     1     2     3     4\nb     5     6     7     8     9\nc    10    11    12    13    14\nd    15    16    17    18    19\ne    20    21    22    23    24\nf    25    26    27    28    29"
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\r\n",
    "    np.arange(30).reshape(6, 5),\r\n",
    "    index=list(\"abcdef\"),\r\n",
    "    columns=[\"col1\", \"col2\", \"col3\", \"col4\", \"col5\"]\r\n",
    "    )\r\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use the `DataFrame` above to demonstrate the techniques of `selection`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\r\n",
    "\r\n",
    "## Getting Directly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>col1</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>a</th>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>b</th>\n      <td>5</td>\n    </tr>\n    <tr>\n      <th>c</th>\n      <td>10</td>\n    </tr>\n    <tr>\n      <th>d</th>\n      <td>15</td>\n    </tr>\n    <tr>\n      <th>e</th>\n      <td>20</td>\n    </tr>\n    <tr>\n      <th>f</th>\n      <td>25</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "   col1\na     0\nb     5\nc    10\nd    15\ne    20\nf    25"
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"col1\"]  # same as `df.col1`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>col1</th>\n      <th>col2</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>a</th>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>b</th>\n      <td>5</td>\n      <td>6</td>\n    </tr>\n    <tr>\n      <th>c</th>\n      <td>10</td>\n      <td>11</td>\n    </tr>\n    <tr>\n      <th>d</th>\n      <td>15</td>\n      <td>16</td>\n    </tr>\n    <tr>\n      <th>e</th>\n      <td>20</td>\n      <td>21</td>\n    </tr>\n    <tr>\n      <th>f</th>\n      <td>25</td>\n      <td>26</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "   col1  col2\na     0     1\nb     5     6\nc    10    11\nd    15    16\ne    20    21\nf    25    26"
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[[\"col1\", \"col2\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>col1</th>\n      <th>col2</th>\n      <th>col3</th>\n      <th>col4</th>\n      <th>col5</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>a</th>\n      <td>0</td>\n      <td>1</td>\n      <td>2</td>\n      <td>3</td>\n      <td>4</td>\n    </tr>\n    <tr>\n      <th>b</th>\n      <td>5</td>\n      <td>6</td>\n      <td>7</td>\n      <td>8</td>\n      <td>9</td>\n    </tr>\n    <tr>\n      <th>c</th>\n      <td>10</td>\n      <td>11</td>\n      <td>12</td>\n      <td>13</td>\n      <td>14</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "   col1  col2  col3  col4  col5\na     0     1     2     3     4\nb     5     6     7     8     9\nc    10    11    12    13    14"
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[0:3]  # same as `df[\"a\":\"c\"]`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\r\n",
    "\r\n",
    "## Selection with `loc` and `at` method (select by label)\r\n",
    "\r\n",
    "The first element in `loc` is the parameter of `row`, and the second one is of `column`.\r\n",
    "\r\n",
    "There are three implementation for a parameter:\r\n",
    "\r\n",
    "- single element (e.g., `\"a\"`)\r\n",
    "- list (e.g., `[\"a\", \"c\", \"e\"]`)\r\n",
    "- slicing (e.g., `\"a\":\"e\"`)\r\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>col1</th>\n      <th>col2</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>a</th>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>b</th>\n      <td>5</td>\n      <td>6</td>\n    </tr>\n    <tr>\n      <th>c</th>\n      <td>10</td>\n      <td>11</td>\n    </tr>\n    <tr>\n      <th>d</th>\n      <td>15</td>\n      <td>16</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "   col1  col2\na     0     1\nb     5     6\nc    10    11\nd    15    16"
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[\"a\":\"d\", [\"col1\", \"col2\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": "4"
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[\"a\", \"col5\"]\r\n",
    "df.at[\"a\", \"col5\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\r\n",
    "\r\n",
    "## Selection with `iloc` and `iat` method (select by position)\r\n",
    "\r\n",
    "`iloc` is almost the same as `loc` method but using positions as the index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>col1</th>\n      <th>col4</th>\n      <th>col5</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>a</th>\n      <td>0</td>\n      <td>3</td>\n      <td>4</td>\n    </tr>\n    <tr>\n      <th>b</th>\n      <td>5</td>\n      <td>8</td>\n      <td>9</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "   col1  col4  col5\na     0     3     4\nb     5     8     9"
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.iloc[0:2, [0, 3, 4]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": "9"
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.iloc[1, 4]\r\n",
    "df.iat[1, 4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\r\n",
    "\r\n",
    "## Boolean indexing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>col1</th>\n      <th>col2</th>\n      <th>col3</th>\n      <th>col4</th>\n      <th>col5</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>e</th>\n      <td>20</td>\n      <td>21</td>\n      <td>22</td>\n      <td>23</td>\n      <td>24</td>\n    </tr>\n    <tr>\n      <th>f</th>\n      <td>25</td>\n      <td>26</td>\n      <td>27</td>\n      <td>28</td>\n      <td>29</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "   col1  col2  col3  col4  col5\ne    20    21    22    23    24\nf    25    26    27    28    29"
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[(df[\"col1\"] > 18)]  # Using a single column's values to select data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>col1</th>\n      <th>col2</th>\n      <th>col3</th>\n      <th>col4</th>\n      <th>col5</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>a</th>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n    </tr>\n    <tr>\n      <th>b</th>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>7.0</td>\n      <td>8.0</td>\n      <td>9.0</td>\n    </tr>\n    <tr>\n      <th>c</th>\n      <td>10.0</td>\n      <td>11.0</td>\n      <td>12.0</td>\n      <td>13.0</td>\n      <td>14.0</td>\n    </tr>\n    <tr>\n      <th>d</th>\n      <td>15.0</td>\n      <td>16.0</td>\n      <td>17.0</td>\n      <td>18.0</td>\n      <td>19.0</td>\n    </tr>\n    <tr>\n      <th>e</th>\n      <td>20.0</td>\n      <td>21.0</td>\n      <td>22.0</td>\n      <td>23.0</td>\n      <td>24.0</td>\n    </tr>\n    <tr>\n      <th>f</th>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "   col1  col2  col3  col4  col5\na   NaN   NaN   NaN   NaN   NaN\nb   NaN   NaN   7.0   8.0   9.0\nc  10.0  11.0  12.0  13.0  14.0\nd  15.0  16.0  17.0  18.0  19.0\ne  20.0  21.0  22.0  23.0  24.0\nf   NaN   NaN   NaN   NaN   NaN"
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[(df > 6) & (df < 25)]  # Selecting values from a DataFrame where a boolean condition is met."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>col1</th>\n      <th>col2</th>\n      <th>col3</th>\n      <th>col4</th>\n      <th>col5</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>a</th>\n      <td>0</td>\n      <td>1</td>\n      <td>2</td>\n      <td>3</td>\n      <td>4</td>\n    </tr>\n    <tr>\n      <th>c</th>\n      <td>10</td>\n      <td>11</td>\n      <td>12</td>\n      <td>13</td>\n      <td>14</td>\n    </tr>\n    <tr>\n      <th>d</th>\n      <td>15</td>\n      <td>16</td>\n      <td>17</td>\n      <td>18</td>\n      <td>19</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "   col1  col2  col3  col4  col5\na     0     1     2     3     4\nc    10    11    12    13    14\nd    15    16    17    18    19"
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df[\"col1\"].isin([10, 15, 0])]  # Using the isin() method for filtering."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\r\n",
    "\r\n",
    "# Reference\r\n",
    "\r\n",
    "- https://pandas.pydata.org/pandas-docs/stable/user_guide/10min.html#selection\r\n",
    "- https://pandas.pydata.org/pandas-docs/stable/user_guide/dsintro.html#column-selection-addition-deletion"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.7.7 64-bit",
   "metadata": {
    "interpreter": {
     "hash": "1005f71a2f2c9f7da6acb75e9bc0247674183f6cab85bd2a339bd069fc0cd207"
    }
   },
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7-final"
  },
  "orig_nbformat": 2
 },
 "nbformat": 4,
 "nbformat_minor": 2
}