notebooks/0_Prerequisite/mnist_dataset_intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MNIST Dataset Introduction\n",
    "\n",
    "Most examples are using MNIST dataset of handwritten digits. The dataset contains 60,000 examples for training and 10,000 examples for testing. The digits have been size-normalized and centered in a fixed-size image (28x28 pixels) with values from 0 to 1. For simplicity, each image has been flatten and converted to a 1-D numpy array of 784 features (28*28).\n",
    "\n",
    "## Overview\n",
    "\n",
    "![MNIST Digits](http://neuralnetworksanddeeplearning.com/images/mnist_100_digits.png)\n",
    "\n",
    "## Usage\n",
    "In our examples, we are using TensorFlow [input_data.py](https://github.com/tensorflow/tensorflow/blob/r0.7/tensorflow/examples/tutorials/mnist/input_data.py) script to load that dataset.\n",
    "It is quite useful for managing our data, and handle:\n",
    "\n",
    "- Dataset downloading\n",
    "\n",
    "- Loading the entire dataset into numpy array: \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install tensorflow --user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import MNIST\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "import os\n",
    "data_path = \"./dataset/mnist_dataset_intro/\"\n",
    "try:\n",
    "    os.makedirs(data_path)\n",
    "except FileExistsError:\n",
    "    pass\n",
    "mnist = input_data.read_data_sets(data_path, one_hot=True)\n",
    "\n",
    "# Load data\n",
    "X_train = mnist.train.images\n",
    "Y_train = mnist.train.labels\n",
    "X_test = mnist.test.images\n",
    "Y_test = mnist.test.labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- A `next_batch` function that can iterate over the whole dataset and return only the desired fraction of the dataset samples (in order to save memory and avoid to load the entire dataset)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the next 64 images array and labels\n",
    "batch_X, batch_Y = mnist.train.next_batch(64)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Link: http://yann.lecun.com/exdb/mnist/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "PyCharm (jupyterconverter)",
   "language": "python",
   "name": "pycharm-3058efaf"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "source": [],
    "metadata": {
     "collapsed": false
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}