DB0201EN-Week4-2-2-PeerAssign-v5-py.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "    <img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/Logos/organization_logo/organization_logo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\"  />\n",
    "</center>\n",
    "\n",
    "<h1 align=center><font size = 5>Assignment: Notebook for Peer Assignment</font></h1>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "Using this Python notebook you will:\n",
    "\n",
    "1.  Understand 3 Chicago datasets  \n",
    "2.  Load the 3 datasets into 3 tables in a Db2 database\n",
    "3.  Execute SQL queries to answer assignment questions \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Understand the datasets\n",
    "\n",
    "To complete the assignment problems in this notebook you will be using three datasets that are available on the city of Chicago's Data Portal:\n",
    "\n",
    "1.  <a href=\"https://data.cityofchicago.org/Health-Human-Services/Census-Data-Selected-socioeconomic-indicators-in-C/kn9c-c2s2\">Socioeconomic Indicators in Chicago</a>\n",
    "2.  <a href=\"https://data.cityofchicago.org/Education/Chicago-Public-Schools-Progress-Report-Cards-2011-/9xs2-f89t\">Chicago Public Schools</a>\n",
    "3.  <a href=\"https://data.cityofchicago.org/Public-Safety/Crimes-2001-to-present/ijzp-q8t2\">Chicago Crime Data</a>\n",
    "\n",
    "### 1. Socioeconomic Indicators in Chicago\n",
    "\n",
    "This dataset contains a selection of six socioeconomic indicators of public health significance and a “hardship index,” for each Chicago community area, for the years 2008 – 2012.\n",
    "\n",
    "For this assignment you will use a snapshot of this dataset which can be downloaded from:<a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_SKO/data/Census_Data_-_Selected_socioeconomic_indicators_in_Chicago__2008___2012-v2.csv\" target=\"_blank\"> Census Data </a>\n",
    "\n",
    "A detailed description of this dataset and the original dataset can be obtained from the Chicago Data Portal at:\n",
    "[https://data.cityofchicago.org/Health-Human-Services/Census-Data-Selected-socioeconomic-indicators-in-C/kn9c-c2s2](https://data.cityofchicago.org/Health-Human-Services/Census-Data-Selected-socioeconomic-indicators-in-C/kn9c-c2s2?cm_mmc=Email_Newsletter-_-Developer_Ed%2BTech-_-WW_WW-_-SkillsNetwork-Courses-IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork-20127838&cm_mmca1=000026UJ&cm_mmca2=10006555&cm_mmca3=M12345678&cvosrc=email.Newsletter.M12345678&cvo_campaign=000026UJ&cm_mmc=Email_Newsletter-_-Developer_Ed%2BTech-_-WW_WW-_-SkillsNetwork-Courses-IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork-20127838&cm_mmca1=000026UJ&cm_mmca2=10006555&cm_mmca3=M12345678&cvosrc=email.Newsletter.M12345678&cvo_campaign=000026UJ)\n",
    "\n",
    "### 2. Chicago Public Schools\n",
    "\n",
    "This dataset shows all school level performance data used to create CPS School Report Cards for the 2011-2012 school year. This dataset is provided by the city of Chicago's Data Portal.\n",
    "\n",
    "For this assignment you will use a snapshot of this dataset which can be downloaded from: <a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_SKO/data/Chicago_Public_Schools_-_Progress_Report_Cards__2011-2012-v3.csv\" target=\"_blank\"> Chicago Public School </a>\n",
    "\n",
    "A detailed description of this dataset and the original dataset can be obtained from the Chicago Data Portal at:\n",
    "[https://data.cityofchicago.org/Education/Chicago-Public-Schools-Progress-Report-Cards-2011-/9xs2-f89t](https://data.cityofchicago.org/Education/Chicago-Public-Schools-Progress-Report-Cards-2011-/9xs2-f89t?cm_mmc=Email_Newsletter-_-Developer_Ed%2BTech-_-WW_WW-_-SkillsNetwork-Courses-IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork-20127838&cm_mmca1=000026UJ&cm_mmca2=10006555&cm_mmca3=M12345678&cvosrc=email.Newsletter.M12345678&cvo_campaign=000026UJ)\n",
    "\n",
    "### 3. Chicago Crime Data\n",
    "\n",
    "This dataset reflects reported incidents of crime (with the exception of murders where data exists for each victim) that occurred in the City of Chicago from 2001 to present, minus the most recent seven days. \n",
    "\n",
    "This dataset is quite large - over 1.5GB in size with over 6.5 million rows. For the purposes of this assignment we will use a much smaller sample of this dataset which can be downloaded from:<a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_SKO/data/Chicago_Crime_Data-v2.csv\" target=\"_blank\"> Chicago Crime Data </a>\n",
    "\n",
    "A detailed description of this dataset and the original dataset can be obtained from the Chicago Data Portal at:\n",
    "[https://data.cityofchicago.org/Public-Safety/Crimes-2001-to-present/ijzp-q8t2](https://data.cityofchicago.org/Public-Safety/Crimes-2001-to-present/ijzp-q8t2?cm_mmc=Email_Newsletter-_-Developer_Ed%2BTech-_-WW_WW-_-SkillsNetwork-Courses-IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork-20127838&cm_mmca1=000026UJ&cm_mmca2=10006555&cm_mmca3=M12345678&cvosrc=email.Newsletter.M12345678&cvo_campaign=000026UJ)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the datasets\n",
    "\n",
    "In many cases the dataset to be analyzed is available as a .CSV (comma separated values) file, perhaps on the internet. Click on the links below to download and save the datasets (.CSV files):\n",
    "\n",
    "1.  **CENSUS_DATA:** <a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera/data/Census_Data_-_Selected_socioeconomic_indicators_in_Chicago__2008___2012-v2.csv\" target=\"_blank\">Census Dataset</a>\n",
    "\n",
    "2.  **CHICAGO_PUBLIC_SCHOOLS**  <a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera/data/Chicago_Public_Schools_-_Progress_Report_Cards__2011-2012-v3.csv\" target=\"_blank\"> Chicago Public School</a>\n",
    "\n",
    "3.  **CHICAGO_CRIME_DATA:** <a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera/data/Chicago_Crime_Data-v2.csv\" target=\"_blank\"> Chicago Crime Data </a>\n",
    "\n",
    "**NOTE:** Ensure you have downloaded the datasets using the links above instead of directly from the Chicago Data Portal. The versions linked here are subsets of the original datasets and have some of the column names modified to be more database friendly which will make it easier to complete this assignment.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Store the datasets in database tables\n",
    "\n",
    "To analyze the data using SQL, it first needs to be stored in the database.\n",
    "\n",
    "While it is easier to read the dataset into a Pandas dataframe and then PERSIST it into the database as we saw in Week 3 Lab 3, it results in mapping to default datatypes which may not be optimal for SQL querying. For example a long textual field may map to a CLOB instead of a VARCHAR. \n",
    "\n",
    "Therefore, **it is highly recommended to manually load the table using the database console LOAD tool, as indicated in Week 2 Lab 1 Part II**. The only difference with that lab is that in Step 5 of the instructions you will need to click on create \"(+) New Table\" and specify the name of the table you want to create and then click \"Next\". \n",
    "\n",
    "<img src = \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_edX/images/LOADschool.jpg\">\n",
    "\n",
    "##### Now open the Db2 console, open the LOAD tool, Select / Drag the .CSV file for the first dataset, Next create a New Table, and then follow the steps on-screen instructions to load the data. Name the new tables as folows:\n",
    "\n",
    "1.  **CENSUS_DATA**\n",
    "2.  **CHICAGO_PUBLIC_SCHOOLS**\n",
    "3.  **CHICAGO_CRIME_DATA**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Connect to the database\n",
    "\n",
    "Let us first load the SQL extension and establish a connection with the database\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext sql"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next cell enter your db2 connection string. Recall you created Service Credentials for your Db2 instance in first lab in Week 3. From the **uri** field of your Db2 service credentials copy everything after db2:// (except the double quote at the end) and paste it in the cell below after ibm_db_sa://\n",
    "\n",
    "<img src =\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_edX/images/URI.jpg\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Connected: vnw20824@BLUDB'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Remember the connection string is of the format:\n",
    "# %sql ibm_db_sa://my-username:my-password@my-hostname:my-port/my-db-name\n",
    "# Enter the connection string for your Db2 on Cloud database instance below\n",
    "%sql ibm_db_sa://vnw20824:g3ncxv0q%5Exf7n3sf@dashdb-txn-sbox-yp-dal09-14.services.dal.bluemix.net:50000/BLUDB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problems\n",
    "\n",
    "Now write and execute SQL queries to solve assignment problems\n",
    "\n",
    "### Problem 1\n",
    "\n",
    "##### Find the total number of crimes recorded in the CRIME table\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://vnw20824:***@dashdb-txn-sbox-yp-dal09-14.services.dal.bluemix.net:50000/BLUDB\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>1</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>533</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "[(Decimal('533'),)]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Rows in Crime table\n",
    "%sql select COUNT(*) from CRIME_DATA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 2\n",
    "\n",
    "##### Retrieve first 10 rows from the CRIME table\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://vnw20824:***@dashdb-txn-sbox-yp-dal09-14.services.dal.bluemix.net:50000/BLUDB\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>id</th>\n",
       "            <th>case_number</th>\n",
       "            <th>DATE</th>\n",
       "            <th>block</th>\n",
       "            <th>iucr</th>\n",
       "            <th>primary_type</th>\n",
       "            <th>description</th>\n",
       "            <th>location_description</th>\n",
       "            <th>arrest</th>\n",
       "            <th>domestic</th>\n",
       "            <th>beat</th>\n",
       "            <th>district</th>\n",
       "            <th>ward</th>\n",
       "            <th>community_area_number</th>\n",
       "            <th>fbicode</th>\n",
       "            <th>x_coordinate</th>\n",
       "            <th>y_coordinate</th>\n",
       "            <th>YEAR</th>\n",
       "            <th>updatedon</th>\n",
       "            <th>latitude</th>\n",
       "            <th>longitude</th>\n",
       "            <th>location</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>21149</td>\n",
       "            <td>HW519443</td>\n",
       "            <td>11/03/2013 07:27:00 PM</td>\n",
       "            <td>044XX S RICHMOND ST</td>\n",
       "            <td>110</td>\n",
       "            <td>HOMICIDE</td>\n",
       "            <td>FIRST DEGREE MURDER</td>\n",
       "            <td>HOUSE</td>\n",
       "            <td>TRUE</td>\n",
       "            <td>TRUE</td>\n",
       "            <td>922</td>\n",
       "            <td>9</td>\n",
       "            <td>14</td>\n",
       "            <td>58</td>\n",
       "            <td>01A</td>\n",
       "            <td>1157439</td>\n",
       "            <td>1875086</td>\n",
       "            <td>2013</td>\n",
       "            <td>08/05/2016 03:48:24 PM</td>\n",
       "            <td>41</td>\n",
       "            <td>-87</td>\n",
       "            <td>(41.812995227, -87.698028592)</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>23469</td>\n",
       "            <td>JA359626</td>\n",
       "            <td>07/23/2017 09:25:00 AM</td>\n",
       "            <td>015XX E 82ND ST</td>\n",
       "            <td>110</td>\n",
       "            <td>HOMICIDE</td>\n",
       "            <td>FIRST DEGREE MURDER</td>\n",
       "            <td>STREET</td>\n",
       "            <td>FALSE</td>\n",
       "            <td>FALSE</td>\n",
       "            <td>411</td>\n",
       "            <td>4</td>\n",
       "            <td>8</td>\n",
       "            <td>45</td>\n",
       "            <td>01A</td>\n",
       "            <td>1188090</td>\n",
       "            <td>1850923</td>\n",
       "            <td>2017</td>\n",
       "            <td>07/30/2017 03:51:44 PM</td>\n",
       "            <td>41</td>\n",
       "            <td>-87</td>\n",
       "            <td>(41.746013191, -87.58637073)</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>1326195</td>\n",
       "            <td>G021609</td>\n",
       "            <td>01/11/2001 02:30:41 AM</td>\n",
       "            <td>087XX S ESCANABA AV</td>\n",
       "            <td>9901</td>\n",
       "            <td>DOMESTIC VIOLENCE</td>\n",
       "            <td>DOMESTIC VIOLENCE</td>\n",
       "            <td>APARTMENT</td>\n",
       "            <td>TRUE</td>\n",
       "            <td>TRUE</td>\n",
       "            <td>423</td>\n",
       "            <td>4</td>\n",
       "            <td>None</td>\n",
       "            <td>None</td>\n",
       "            <td>08B</td>\n",
       "            <td>1196869</td>\n",
       "            <td>1847416</td>\n",
       "            <td>2001</td>\n",
       "            <td>08/17/2015 03:03:40 PM</td>\n",
       "            <td>41</td>\n",
       "            <td>-87</td>\n",
       "            <td>(41.73617608, -87.554319607)</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>1340847</td>\n",
       "            <td>G040244</td>\n",
       "            <td>01/19/2001 06:39:03 PM</td>\n",
       "            <td>063XX N NAGLE AV</td>\n",
       "            <td>820</td>\n",
       "            <td>THEFT</td>\n",
       "            <td>$500 AND UNDER</td>\n",
       "            <td>GROCERY FOOD STORE</td>\n",
       "            <td>TRUE</td>\n",
       "            <td>FALSE</td>\n",
       "            <td>1611</td>\n",
       "            <td>16</td>\n",
       "            <td>None</td>\n",
       "            <td>None</td>\n",
       "            <td>6</td>\n",
       "            <td>1132586</td>\n",
       "            <td>1941599</td>\n",
       "            <td>2001</td>\n",
       "            <td>08/17/2015 03:03:40 PM</td>\n",
       "            <td>41</td>\n",
       "            <td>-87</td>\n",
       "            <td>(41.99598354, -87.787639887)</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>1353618</td>\n",
       "            <td>G056330</td>\n",
       "            <td>01/27/2001 04:20:00 PM</td>\n",
       "            <td>078XX S SAWYER AV</td>\n",
       "            <td>460</td>\n",
       "            <td>BATTERY</td>\n",
       "            <td>SIMPLE</td>\n",
       "            <td>RESIDENCE PORCH/HALLWAY</td>\n",
       "            <td>TRUE</td>\n",
       "            <td>TRUE</td>\n",
       "            <td>835</td>\n",
       "            <td>8</td>\n",
       "            <td>None</td>\n",
       "            <td>None</td>\n",
       "            <td>08B</td>\n",
       "            <td>1156032</td>\n",
       "            <td>1852572</td>\n",
       "            <td>2001</td>\n",
       "            <td>08/17/2015 03:03:40 PM</td>\n",
       "            <td>41</td>\n",
       "            <td>-87</td>\n",
       "            <td>(41.751241937, -87.703794164)</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>1363954</td>\n",
       "            <td>G070193</td>\n",
       "            <td>02/03/2001 03:00:00 AM</td>\n",
       "            <td>004XX W WRIGHTWOOD AV</td>\n",
       "            <td>460</td>\n",
       "            <td>BATTERY</td>\n",
       "            <td>SIMPLE</td>\n",
       "            <td>RESIDENCE</td>\n",
       "            <td>FALSE</td>\n",
       "            <td>FALSE</td>\n",
       "            <td>2333</td>\n",
       "            <td>19</td>\n",
       "            <td>None</td>\n",
       "            <td>None</td>\n",
       "            <td>08B</td>\n",
       "            <td>1172852</td>\n",
       "            <td>1918278</td>\n",
       "            <td>2001</td>\n",
       "            <td>08/17/2015 03:03:40 PM</td>\n",
       "            <td>41</td>\n",
       "            <td>-87</td>\n",
       "            <td>(41.93119046, -87.640214004)</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>1367327</td>\n",
       "            <td>G057394</td>\n",
       "            <td>01/28/2001 07:10:00 AM</td>\n",
       "            <td>046XX S CICERO AV</td>\n",
       "            <td>1513</td>\n",
       "            <td>PROSTITUTION</td>\n",
       "            <td>SOLICIT FOR BUSINESS</td>\n",
       "            <td>STREET</td>\n",
       "            <td>TRUE</td>\n",
       "            <td>FALSE</td>\n",
       "            <td>814</td>\n",
       "            <td>8</td>\n",
       "            <td>None</td>\n",
       "            <td>None</td>\n",
       "            <td>16</td>\n",
       "            <td>1145110</td>\n",
       "            <td>1873073</td>\n",
       "            <td>2001</td>\n",
       "            <td>08/17/2015 03:03:40 PM</td>\n",
       "            <td>41</td>\n",
       "            <td>-87</td>\n",
       "            <td>(41.807712461, -87.743303038)</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>1414626</td>\n",
       "            <td>G134016</td>\n",
       "            <td>03/01/2001 11:00:00 PM</td>\n",
       "            <td>055XX S NOTTINGHAM AV</td>\n",
       "            <td>1310</td>\n",
       "            <td>CRIMINAL DAMAGE</td>\n",
       "            <td>TO PROPERTY</td>\n",
       "            <td>RESIDENCE</td>\n",
       "            <td>FALSE</td>\n",
       "            <td>FALSE</td>\n",
       "            <td>811</td>\n",
       "            <td>8</td>\n",
       "            <td>None</td>\n",
       "            <td>None</td>\n",
       "            <td>14</td>\n",
       "            <td>1130022</td>\n",
       "            <td>1866716</td>\n",
       "            <td>2001</td>\n",
       "            <td>08/17/2015 03:03:40 PM</td>\n",
       "            <td>41</td>\n",
       "            <td>-87</td>\n",
       "            <td>(41.790538595, -87.79878798)</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>1419496</td>\n",
       "            <td>G140454</td>\n",
       "            <td>03/11/2001 04:44:05 PM</td>\n",
       "            <td>077XX S SOUTH SHORE DR</td>\n",
       "            <td>460</td>\n",
       "            <td>BATTERY</td>\n",
       "            <td>SIMPLE</td>\n",
       "            <td>APARTMENT</td>\n",
       "            <td>FALSE</td>\n",
       "            <td>FALSE</td>\n",
       "            <td>421</td>\n",
       "            <td>4</td>\n",
       "            <td>None</td>\n",
       "            <td>None</td>\n",
       "            <td>08B</td>\n",
       "            <td>1197205</td>\n",
       "            <td>1854743</td>\n",
       "            <td>2001</td>\n",
       "            <td>08/17/2015 03:03:40 PM</td>\n",
       "            <td>41</td>\n",
       "            <td>-87</td>\n",
       "            <td>(41.756273565, -87.552845167)</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>1427912</td>\n",
       "            <td>G122095</td>\n",
       "            <td>03/02/2001 04:20:00 PM</td>\n",
       "            <td>039XX N ASHLAND AV</td>\n",
       "            <td>1505</td>\n",
       "            <td>PROSTITUTION</td>\n",
       "            <td>CALL OPERATION</td>\n",
       "            <td>RESIDENCE</td>\n",
       "            <td>TRUE</td>\n",
       "            <td>FALSE</td>\n",
       "            <td>1923</td>\n",
       "            <td>19</td>\n",
       "            <td>None</td>\n",
       "            <td>None</td>\n",
       "            <td>16</td>\n",
       "            <td>1164982</td>\n",
       "            <td>1926580</td>\n",
       "            <td>2001</td>\n",
       "            <td>08/17/2015 03:03:40 PM</td>\n",
       "            <td>41</td>\n",
       "            <td>-87</td>\n",
       "            <td>(41.954142513, -87.668898147)</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "[(21149, 'HW519443', '11/03/2013 07:27:00 PM', '044XX S RICHMOND ST', '110', 'HOMICIDE', 'FIRST DEGREE MURDER', 'HOUSE', 'TRUE', 'TRUE', 922, 9, 14, 58, '01A', 1157439, 1875086, 2013, '08/05/2016 03:48:24 PM', Decimal('41'), Decimal('-87'), '(41.812995227, -87.698028592)'),\n",
       " (23469, 'JA359626', '07/23/2017 09:25:00 AM', '015XX E 82ND ST', '110', 'HOMICIDE', 'FIRST DEGREE MURDER', 'STREET', 'FALSE', 'FALSE', 411, 4, 8, 45, '01A', 1188090, 1850923, 2017, '07/30/2017 03:51:44 PM', Decimal('41'), Decimal('-87'), '(41.746013191, -87.58637073)'),\n",
       " (1326195, 'G021609', '01/11/2001 02:30:41 AM', '087XX S ESCANABA AV', '9901', 'DOMESTIC VIOLENCE', 'DOMESTIC VIOLENCE', 'APARTMENT', 'TRUE', 'TRUE', 423, 4, None, None, '08B', 1196869, 1847416, 2001, '08/17/2015 03:03:40 PM', Decimal('41'), Decimal('-87'), '(41.73617608, -87.554319607)'),\n",
       " (1340847, 'G040244', '01/19/2001 06:39:03 PM', '063XX N NAGLE AV', '820', 'THEFT', '$500 AND UNDER', 'GROCERY FOOD STORE', 'TRUE', 'FALSE', 1611, 16, None, None, '6', 1132586, 1941599, 2001, '08/17/2015 03:03:40 PM', Decimal('41'), Decimal('-87'), '(41.99598354, -87.787639887)'),\n",
       " (1353618, 'G056330', '01/27/2001 04:20:00 PM', '078XX S SAWYER AV', '460', 'BATTERY', 'SIMPLE', 'RESIDENCE PORCH/HALLWAY', 'TRUE', 'TRUE', 835, 8, None, None, '08B', 1156032, 1852572, 2001, '08/17/2015 03:03:40 PM', Decimal('41'), Decimal('-87'), '(41.751241937, -87.703794164)'),\n",
       " (1363954, 'G070193', '02/03/2001 03:00:00 AM', '004XX W WRIGHTWOOD AV', '460', 'BATTERY', 'SIMPLE', 'RESIDENCE', 'FALSE', 'FALSE', 2333, 19, None, None, '08B', 1172852, 1918278, 2001, '08/17/2015 03:03:40 PM', Decimal('41'), Decimal('-87'), '(41.93119046, -87.640214004)'),\n",
       " (1367327, 'G057394', '01/28/2001 07:10:00 AM', '046XX S CICERO AV', '1513', 'PROSTITUTION', 'SOLICIT FOR BUSINESS', 'STREET', 'TRUE', 'FALSE', 814, 8, None, None, '16', 1145110, 1873073, 2001, '08/17/2015 03:03:40 PM', Decimal('41'), Decimal('-87'), '(41.807712461, -87.743303038)'),\n",
       " (1414626, 'G134016', '03/01/2001 11:00:00 PM', '055XX S NOTTINGHAM AV', '1310', 'CRIMINAL DAMAGE', 'TO PROPERTY', 'RESIDENCE', 'FALSE', 'FALSE', 811, 8, None, None, '14', 1130022, 1866716, 2001, '08/17/2015 03:03:40 PM', Decimal('41'), Decimal('-87'), '(41.790538595, -87.79878798)'),\n",
       " (1419496, 'G140454', '03/11/2001 04:44:05 PM', '077XX S SOUTH SHORE DR', '460', 'BATTERY', 'SIMPLE', 'APARTMENT', 'FALSE', 'FALSE', 421, 4, None, None, '08B', 1197205, 1854743, 2001, '08/17/2015 03:03:40 PM', Decimal('41'), Decimal('-87'), '(41.756273565, -87.552845167)'),\n",
       " (1427912, 'G122095', '03/02/2001 04:20:00 PM', '039XX N ASHLAND AV', '1505', 'PROSTITUTION', 'CALL OPERATION', 'RESIDENCE', 'TRUE', 'FALSE', 1923, 19, None, None, '16', 1164982, 1926580, 2001, '08/17/2015 03:03:40 PM', Decimal('41'), Decimal('-87'), '(41.954142513, -87.668898147)')]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select * from CRIME_DATA LIMIT 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 3\n",
    "\n",
    "##### How many crimes involve an arrest?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://vnw20824:***@dashdb-txn-sbox-yp-dal09-14.services.dal.bluemix.net:50000/BLUDB\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>1</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>163</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "[(Decimal('163'),)]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select COUNT(*) from CRIME_DATA where ARREST = 'TRUE'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 4\n",
    "\n",
    "##### Which unique types of crimes have been recorded at GAS STATION locations?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://vnw20824:***@dashdb-txn-sbox-yp-dal09-14.services.dal.bluemix.net:50000/BLUDB\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>primary_type</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>CRIMINAL TRESPASS</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>NARCOTICS</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>ROBBERY</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>THEFT</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "[('CRIMINAL TRESPASS',), ('NARCOTICS',), ('ROBBERY',), ('THEFT',)]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select DISTINCT(PRIMARY_TYPE) from CRIME_DATA where LOCATION_DESCRIPTION = 'GAS STATION'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hint: Which column lists types of crimes e.g. THEFT?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 5\n",
    "\n",
    "##### In the CENUS_DATA table list all Community Areas whose names start with the letter ‘B’.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://vnw20824:***@dashdb-txn-sbox-yp-dal09-14.services.dal.bluemix.net:50000/BLUDB\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>community_area_name</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>Belmont Cragin</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>Burnside</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>Brighton Park</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>Bridgeport</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>Beverly</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "[('Belmont Cragin',),\n",
       " ('Burnside',),\n",
       " ('Brighton Park',),\n",
       " ('Bridgeport',),\n",
       " ('Beverly',)]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select COMMUNITY_AREA_NAME from CENSUS_DATA where COMMUNITY_AREA_NAME LIKE 'B%'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 6\n",
    "\n",
    "##### Which schools in Community Areas 10 to 15 are healthy school certified?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://vnw20824:***@dashdb-txn-sbox-yp-dal09-14.services.dal.bluemix.net:50000/BLUDB\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>name_of_school</th>\n",
       "            <th>community_area_number</th>\n",
       "            <th>healthy_school_certified</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>Rufus M Hitch Elementary School</td>\n",
       "            <td>10</td>\n",
       "            <td>Yes</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "[('Rufus M Hitch Elementary School', 10, 'Yes')]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%sql select S.NAME_OF_SCHOOL,S.COMMUNITY_AREA_NUMBER,S.HEALTHY_SCHOOL_CERTIFIED from CHICAGO_SCHOOLS S, CENSUS_DATA C where S.COMMUNITY_AREA_NUMBER = C.COMMUNITY_AREA_NUMBER and\n",
    "C.COMMUNITY_AREA_NUMBER BETWEEN 10 AND 15 AND S.healthy_school_certified = 'Yes'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 7\n",
    "\n",
    "##### What is the average school Safety Score?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://vnw20824:***@dashdb-txn-sbox-yp-dal09-14.services.dal.bluemix.net:50000/BLUDB\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>average_score</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>49.504873</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "[(Decimal('49.504873'),)]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select AVG(SAFETY_SCORE) AS AVERAGE_SCORE from CHICAGO_SCHOOLS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 8\n",
    "\n",
    "##### List the top 5 Community Areas by average College Enrollment [number of students]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://vnw20824:***@dashdb-txn-sbox-yp-dal09-14.services.dal.bluemix.net:50000/BLUDB\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>community_area_name</th>\n",
       "            <th>avg_college_enrollment</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>ARCHER HEIGHTS</td>\n",
       "            <td>2411.500000</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>MONTCLARE</td>\n",
       "            <td>1317.000000</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>WEST ELSDON</td>\n",
       "            <td>1233.333333</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>BRIGHTON PARK</td>\n",
       "            <td>1205.875000</td>\n",
       "        </tr>\n",
       "        <tr>\n",
       "            <td>BELMONT CRAGIN</td>\n",
       "            <td>1198.833333</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "[('ARCHER HEIGHTS', Decimal('2411.500000')),\n",
       " ('MONTCLARE', Decimal('1317.000000')),\n",
       " ('WEST ELSDON', Decimal('1233.333333')),\n",
       " ('BRIGHTON PARK', Decimal('1205.875000')),\n",
       " ('BELMONT CRAGIN', Decimal('1198.833333'))]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%sql select COMMUNITY_AREA_NAME,AVG(COLLEGE_ENROLLMENT) AS AVG_COLLEGE_ENROLLMENT  \n",
    "FROM CHICAGO_SCHOOLS group by COMMUNITY_AREA_NAME order by AVG_COLLEGE_ENROLLMENT desc limit 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 9\n",
    "\n",
    "##### Use a sub-query to determine which Community Area has the least value for school Safety Score?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://vnw20824:***@dashdb-txn-sbox-yp-dal09-14.services.dal.bluemix.net:50000/BLUDB\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>community_area_name</th>\n",
       "            <th>safety_score</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>WASHINGTON PARK</td>\n",
       "            <td>1</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "[('WASHINGTON PARK', 1)]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select COMMUNITY_AREA_NAME, SAFETY_SCORE from CHICAGO_SCHOOLS where SAFETY_SCORE = (select min(SAFETY_SCORE) FROM CHICAGO_SCHOOLS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 10\n",
    "\n",
    "##### [Without using an explicit JOIN operator] Find the Per Capita Income of the Community Area which has a school Safety Score of 1.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://vnw20824:***@dashdb-txn-sbox-yp-dal09-14.services.dal.bluemix.net:50000/BLUDB\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>community_area_name</th>\n",
       "            <th>per_capita_income</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>Washington Park</td>\n",
       "            <td>13785</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "[('Washington Park', 13785)]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%sql select COMMUNITY_AREA_NAME, PER_CAPITA_INCOME from CENSUS_DATA where COMMUNITY_AREA_NUMBER = \n",
    "(select COMMUNITY_AREA_NUMBER from CHICAGO_SCHOOLS where SAFETY_SCORE = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright © 2020 [cognitiveclass.ai](cognitiveclass.ai?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license?cm_mmc=Email_Newsletter-_-Developer_Ed%2BTech-_-WW_WW-_-SkillsNetwork-Courses-IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork-20127838&cm_mmca1=000026UJ&cm_mmca2=10006555&cm_mmca3=M12345678&cvosrc=email.Newsletter.M12345678&cvo_campaign=000026UJ&cm_mmc=Email_Newsletter-_-Developer_Ed%2BTech-_-WW_WW-_-SkillsNetwork-Courses-IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork-20127838&cm_mmca1=000026UJ&cm_mmca2=10006555&cm_mmca3=M12345678&cvosrc=email.Newsletter.M12345678&cvo_campaign=000026UJ).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Author(s)\n",
    "\n",
    "<h4> Rav Ahuja </h4>\n",
    "\n",
    "## Change log\n",
    "\n",
    "| Date       | Version | Changed by    | Change Description                 |\n",
    "| ---------- | ------- | ------------- | ---------------------------------- |\n",
    "| 2020-09-05 | 2.0     | Malika Singla | Moved lab to course repo in GitLab |\n",
    "\n",
    "## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python",
   "language": "python",
   "name": "conda-env-python-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}