17_Estimator_API_CN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow Tutorial #17\n",
    "# Estimator API\n",
    "\n",
    "by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n",
    "/ [GitHub](https://github.com/Hvass-Labs/TensorFlow-Tutorials) / [Videos on YouTube](https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ)\n",
    "\n",
    "中文翻译[ZhouGeorge](https://github.com/ZhouGeorge)/[GitHub](https://github.com/ZhouGeorge/TensorFlow-Tutorials)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction介绍\n",
    "\n",
    "在软件开发中高级API是相当重要的，因为它们可以为完成复杂的任务提供简单的描述。这样可以更简单地去完成和理解你的源码并且出错的风险很低。\n",
    "\n",
    "在教程 #03中我们见过如何用不同的构筑API在Tensorflow中创建神经网络。然而，为了训练模型和在新的数据上使用它们需要添加很多的代码。 Estimator是另一个高级的API可以以完成其中的大部分工作，尽管它到底有多易用是有待讨论的。\n",
    "\n",
    "使用 Estimator API 包括以下几个步骤：\n",
    "1. 给 Estimator 定义输入数据的函数。\n",
    "2. 或者使用一个已经存在的Estimator（例如 一个深度神经网络），它也可以称为预制（pre-made）或封装（Canned）的Estimator。或者创建一个你自己的Estimator，你需要定义优化器，评价指标，等等。\n",
    "3. 利用步骤1.中定义的数据集训练Estimator。\n",
    "4. 在步骤1.定义的测试机上评价Estimator的表现。\n",
    "5. 用训练好的Estimator去预测其他的数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/magnus/anaconda3/envs/tf-gpu/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6\n",
      "  return f(*args, **kwds)\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "开发环境Python 3.6（Anaconda）和TensorFlow version：\n",
    "\n",
    "译者注：Estimator需要1.3.0以上版本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.4.0'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 加载数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MNIST 数据集大学12MB，如果指定路径下不存在它会被自动下载。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting data/MNIST/train-images-idx3-ubyte.gz\n",
      "Extracting data/MNIST/train-labels-idx1-ubyte.gz\n",
      "Extracting data/MNIST/t10k-images-idx3-ubyte.gz\n",
      "Extracting data/MNIST/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "data = input_data.read_data_sets('data/MNIST/', one_hot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MNIST数据集已经被加载了，它包含70000张图像和对应的标签（图像的分类）。数据集被划为3个子集。我们在这个教程中只需要用到训练集和测试集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of:\n",
      "- Training-set:\t\t55000\n",
      "- Test-set:\t\t10000\n",
      "- Validation-set:\t5000\n"
     ]
    }
   ],
   "source": [
    "print(\"Size of:\")\n",
    "print(\"- Training-set:\\t\\t{}\".format(len(data.train.labels)))\n",
    "print(\"- Test-set:\\t\\t{}\".format(len(data.test.labels)))\n",
    "print(\"- Validation-set:\\t{}\".format(len(data.validation.labels)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "类别标签是独热编码，意味着每个标签是有10个元素的向量，除了一个元素外其余都是0，这个元素的索引就是对应图像的类数。我们也需要类别数的整数形式，所以我们就通过下面的计算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data.train.cls = np.argmax(data.train.labels, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data.test.cls = np.argmax(data.test.labels, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面是独热编码的标签举例："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.],\n",
       "       [ 0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.],\n",
       "       [ 0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.],\n",
       "       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.train.labels[0:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面是对应的类别数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([7, 3, 4, 6, 1, 8, 1, 0, 9, 8])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.train.cls[0:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据的尺寸"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据的尺寸在下面多个地方被用到。它们被定义了一次，所以我们可以在下面的源代码中使用这些变量来代替数字。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# We know that MNIST images are 28 pixels in each dimension.\n",
    "img_size = 28\n",
    "\n",
    "# Images are stored in one-dimensional arrays of this length.\n",
    "img_size_flat = img_size * img_size\n",
    "\n",
    "# Tuple with height and width of images used to reshape arrays.\n",
    "img_shape = (img_size, img_size)\n",
    "\n",
    "# Number of colour channels for the images: 1 channel for gray-scale.\n",
    "num_channels = 1\n",
    "\n",
    "# Number of classes, one class for each of 10 digits.\n",
    "num_classes = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 画图的辅助函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在3x3的网格中画9张图，包括在每张图下面显示正确的类别和预测的类别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_images(images, cls_true, cls_pred=None):\n",
    "    assert len(images) == len(cls_true) == 9\n",
    "    \n",
    "    # Create figure with 3x3 sub-plots.\n",
    "    fig, axes = plt.subplots(3, 3)\n",
    "    fig.subplots_adjust(hspace=0.3, wspace=0.3)\n",
    "\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Plot image.\n",
    "        ax.imshow(images[i].reshape(img_shape), cmap='binary')\n",
    "\n",
    "        # Show true and predicted classes.\n",
    "        if cls_pred is None:\n",
    "            xlabel = \"True: {0}\".format(cls_true[i])\n",
    "        else:\n",
    "            xlabel = \"True: {0}, Pred: {1}\".format(cls_true[i], cls_pred[i])\n",
    "\n",
    "        # Show the classes as the label on the x-axis.\n",
    "        ax.set_xlabel(xlabel)\n",
    "        \n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "    \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 画一些图像检验数据是否正确"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUMAAAD5CAYAAAC9FVegAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHihJREFUeJzt3XmUFNXZx/HvA0LYVQQFFWdOwAVCFBWDu0aBKCogccG4\nEGM0osEtAaNx1xglKBzRE7YD4QQNigKCUVFAEV8EJIIi4wYiCsRlhLggIsJ9/5i5XdUzPXtXVU/7\n+5zjmequ6qpnvPSdp27dxZxziIj80DVIOgARkVygylBEBFWGIiKAKkMREUCVoYgIoMpQRARQZSgi\nAqgyFBEBVBmKiACwS00ObtOmjSssLIwolNzzwQcfUFxcbEnHESeVcf5TGWdWo8qwsLCQZcuW1T6q\neqZ79+5JhxA7lXH+UxlnpttkERFUGYqIAKoMRUQAVYYiIoAqQxERoIZPk0Vqa8SIEQBs3boVgDfe\neAOAxx9/vNyxgwcPBuCoo44C4MILL4wjRPmBU2YoIoIyQ4nYueeeC8C0adMy7jcr3xd2zJgxAMyd\nOxeAE044AYD99tsvihAlQe+++y4ABx54IAAPPPAAAEOGDIk9FmWGIiIoM5QI+GwQKs4IDzroIABO\nOeUUAN5///3UvlmzZgGwevVqAKZMmQLAjTfemP1gJVHLly8HoEGDkrxsn332SSwWZYYiIigzlCzy\n411nzJhRbl/Xrl2BIOtr06YNAC1atADgu+++Sx3bo0cPAF5//XUAPv/884gilqStWLECCP4dDBgw\nILFYlBmKiBBDZuj7kY0fPx6AvffeO7WvSZMmAJx//vkAtGvXDoBOnTpFHZZE4L///S8AzrnUez4j\nnDNnDgDt27fP+FnfDxHgrbfeStt3+umnZzVOSd7KlSsBGD16NAAXXXRRkuEAygxFRIAYMsOhQ4cC\nJRMsVsT3K2vVqhUAXbp0ycq1O3ToAMCwYcOAH+bcdXE644wzgOApMEDLli0BaN26daWfffTRR1Pb\n4fZDyU/vvPMOAFu2bAHSeyAkRZmhiAiqDEVEgBhukydMmAAE3STCt8BFRUVA0PHyxRdfBGDx4sVA\nMPzqww8/rPD8jRo1AoKuGr4RP3wef7us2+R4FBQUVPvYv/3tb0AwLCvMd7HxPyV/DB8+HChZggBy\n47upzFBEhBgyw5NPPjntZ5gfiuVt3rwZCDJF/9fi1VdfrfD8P/rRj4BgoLcf5gWwadMmADp27Fir\n2CU6Tz31FAC33HILANu2bUvt22uvvQC45557AGjWrFnM0UkUwg9R/Xfaf2+bN2+eREhplBmKiJBj\nw/F23313AE466aS09zNllWU98cQTQJBdAhx88MEADBw4MFshSpb4oXvhjNDz3Sz81F2SHxYsWFDu\nvbZt2yYQSWbKDEVEyLHMsDY+/fRTAK644gogfSiYb4+qqsOvxKd///5AMDzPGzRoUGr7rrvuijUm\niYdf6iHMD4jIBcoMRUTIg8zwoYceAoIMcbfddkvt80+qJHm+/+eiRYuAoK3QtxnddNNNqWP9dE6S\nH1555RUAJk2alHrv0EMPBaBXr16JxJSJMkMREepxZvjyyy8DQV8078knn0xt++mjJHl+0s7i4uK0\n9/30beoLmr/mzZsHpPf08H2M/TR+uUCZoYgIqgxFRIB6fJv89NNPA8Hcdz179gTgqKOOSiwmKc+v\neeKHWHonnngiAHfccUfcIUnM/CQtYWeffXYCkVROmaGICPUwM9y6dSsAzz77LBBM1HD77bcDwZRe\nkpzwanZ33303UH726m7dugHqRpPPPv74YwAWLlwIpE+icuaZZyYSU2WUGYqIUA8zQz8ZqG+DOvXU\nUwE4+uijE4tJ0t13332p7aVLl6bt88Px1FaY//7xj38A8MknnwDBdzVXKTMUEaGeZIZ+IlCAO++8\nE4Bdd90VgJtvvjmRmKRi999/f4X7/PBJtRXmv3Xr1qW99lP05SplhiIi5Hhm6J9KXnXVVan3vv/+\newD69OkDqF9hfePLtDpP/X3274/dvn07AF988UW5Y/1Qr5EjR2Y8V8OGDVPb9957L6DlBKI2e/bs\ntNenn356QpFUjzJDERFUGYqIADl6m7xjxw4gmNli7dq1qX2dOnUCggcpUr/4dWmq45xzzgGgffv2\nQNBFY+rUqXWKwa++F55DUbLHd7L25VVfKDMUESFHM8M1a9YAwQpqYb7bhua/y13+4RbAzJkza32e\nxx57rMpj/MOVBg3S/6737dsXCNbeDjv22GNrHZNUbcaMGUDwsNPPap3rqx0qMxQRIccyQ99Js3fv\n3mnvjxgxIrWd64/nBaZPn57aHj58OFB+ogavqKgIqLwd8JJLLgGgoKCg3L5f/vKXAHTu3Ll2wUrW\nfPPNNwA888wzae/76brC3ZtykTJDERFyLDMcO3YsUH4YT7itwcxijUnqprrr4j7yyCMRRyJR8+23\nfoXKfv36AXD11VcnFlNNKDMUESFHMkPfL+nBBx9MOBIRqS2fGfp1kusbZYYiIuRIZujXQP7qq6/S\n3vejTTTdk4hETZmhiAiqDEVEgBy5TS7Lr5w2b948AFq3bp1kOCLyA6DMUESEHMkMb7jhhrSfIiJx\nU2YoIgKYc676B5t9Bqyr8sD8UeCca5t0EHFSGec/lXFmNaoMRUTylW6TRURQZSgiAkT8NNnM9gDm\nlb5sB+wAPit9/TPnXOYZP+t2zS5AeD6ojsANzjnNAhGBhMq4AJgM7Ak44O8q3+gkUcal150M9AE2\nOOe6RXGNtOvF1WZoZrcBXzvnRpR530rj2BnBNRsBG4DDnHPrs31+SRdXGZvZ3sCezrkVZtYKWA6c\n6px7Nxvnl4rF+T02sxOArcC4OCrDRG6TzayTmRWZ2cPAKqCDmf0vtH+gmU0o3d7LzKab2TIzW2pm\nR9bgUr2At1QRxi/KMnbObXTOrSjd/hJ4G9gnut9GMon6e+ycWwBsiuwXKCPJNsODgJHOuS6UZG8V\neQAY7pzrDpwD+P+5PcxsTBXXGAj8KxvBSq1EXsZm9mOgK/BqdkKWGorjexyLJEegrHHOlV8LtLye\nwIGh6f53N7OmzrklwJKKPmRmTYDTgOvqHKnUVtRl3Ap4AhjinPu6ztFKbURaxnFKsjLcEtreCYQX\nN2kS2jZq10h7GrDEOVdcy/ik7iIrYzNrDEwHJjnnZtUpSqmLqL/HscmJrjWlja6bzWx/M2sAnBna\nPRe40r8ws+o2pJ6HbpFzRjbLuLSx/h/ACufcAxGEK7UQ0fc4NjlRGZa6HpgDLALCDzyuBI4xszfM\nrAi4FCpvazCzlsDPgZnRhiw1lK0yPoGSP3a9zGxF6X+/iDh2qZ5sfo+nAQuBLma23sx+HWXgGo4n\nIkJuZYYiIolRZSgigipDERFAlaGICKDKUEQEqGGn6zZt2rjCwsKIQsk9H3zwAcXFxVb1kflDZZz/\nVMaZ1agyLCwsZNmy6oy8yQ/du3dPOoTYqYzzn8o4M90mi4igylBEBFBlKCICqDIUEQFUGYqIAKoM\nRUSAZCd3rdCWLSXzRQ4dOhSAMWOCGX78Y/Jp06YBUFBQEHN0IpKPlBmKiJCjmeHGjRsBGD9+PAAN\nGzZM7fOdRWfPng3A73//+5ijk9p47bXXABgwYABQMiqgtp577rnUdufOnQHo0KFD7YOTxPjvcd++\nfQEYPXo0AIMHD04dE/7+R0mZoYgIOZYZfvbZZwAMGjQo4Ugk2+bMmQPAtm3b6nyuWbOC9Z8mTpwI\nwNSpU+t8XonP559/DqRngABDhgwB4JJLLkm917Rp01hiUmYoIkKOZIYPPFCywNnMmSXrN736atXr\ngS9cuBAAv4bLIYccAsDxxx8fRYhSS99//z0ATz/9dNbOGR54f//99wNBD4TmzZtn7ToSnZdeegmA\nDRvS150/77zzAGjSpEm5z0RNmaGICDmSGV5zzTVAzZ4aTZ8+Pe3nfvvtB8Bjjz2WOubwww/PVohS\nSy+88AIAixYtAuD666+v8zk3bdqU2l61ahUA33zzDaDMMJeF24vvuuuujMdceOGFAJQsjR0vZYYi\nIqgyFBEBEr5N7tOnDxA8BNmxY0eVn2nTpg0Q3A6tW7cOgLVr1wJwxBFHpI7duXNn9oKValu5cmVq\ne+DAgQB06tQJgBtvvLHO5w93rZH644033kht+0743i67lFRFp556aqwxhSkzFBEhgcxwwYIFqe23\n334bCBpLK3qAcvnll6e2e/fuDcCuu+4KwPz58wH4y1/+Uu5zf//734HyHTslWuGy8A82pkyZAkCL\nFi1qfV7/4CT8byiJhnapHf+wM5NevXrFGElmygxFRIgxM/QD830bEkBxcXHGY303mbPOOguAW2+9\nNbWvWbNmacf6KbzGjh1b7pzDhg0D4NtvvwWCSR0aNWpUu19CKvX4448D6R2sfVthuC23tnx3jHA2\neOKJJwKw22671fn8Eq1wRu81btwYgLvvvjvucMpRZigiQoyZ4fbt24GKs0EIhtI9+uijQPDkuDI+\nM/RPKa+77rrUPj9Ey2eIfpqgjh071ih2qR4/4a7//w7Zaa/1dxWPPPIIEDx5BLjpppsAZfu5zHe4\nf+WVV8rt83d63bp1izWmTJQZioiQI8PxfHvSpEmTgOplhGX5rO/hhx9Ovbd06dIsRCdV+eKLLwBY\nvHhxuX1XXHFFnc8/btw4IJjirUuXLql9J510Up3PL9GqbOKVXOrpocxQRIQEMsNMo0yWLFlS5/P6\nUSzhUSdlR7b4p9K+z5tkhx+Av379eiCYhilb1qxZk/a6a9euWT2/RCtTZuif/mfjziFblBmKiKDK\nUEQEiPE22a99HNVKV36VreXLl6feKzvM7/bbb4/k2j90LVu2BILuEeGJGvwQutatW9f4vJ9++ikQ\ndNnxjjnmmFrFKfF6+eWXgaBLVJgfTrvvvvvGGlNllBmKiBBjZvjUU09l9Xy+m0VRURFQ+XAe31VH\nHXOj4Vcv80Pv/LA8gNNOOw1I7wyfyZtvvpna9g9M/PRsZSdjaNBAf8PrA78Cnn+QGZYLEzOUpX9V\nIiLkSKfr2vDTRD300EMVHlNYWAjA5MmTgWACCInGbbfdBqRnAv6OIDxBRyZt27ZNbftMsKKhmxdf\nfHFdwpSYlG3rDU+mcdlll8UdTpWUGYqIUA8zQ79UgJ8YtjJ+2NZxxx0XaUxSonPnzkD6CoX+6X7Z\njtNl+enawgYNGgSU7yTv2yglN/nO92WfIoefHGdjSrdsU2YoIkKMmWFliz4988wzaa8vvfRSADZu\n3Fjheaoz3Xu2n2BLzR166KFpP2vixz/+ccb3w/0Yf/rTn9YuMImMn7Kr7FPkfv36JRFOtSkzFBFB\nlaGICBDjbbKft8zPOh3mO+aWHaqXaeiev82uzkp6Ur/526yyt1u6Nc5tvrO15wc9XHPNNUmEU23K\nDEVEiDEzHDBgAADDhw9PvVfZeihV8X9tfHeO8ePHA9C+fftan1Nyi39IprWR65c5c+akve7QoQMQ\nTM6Qq5QZiogQY2boV7HzK98BzJw5E4BRo0bV+Hx//vOfgWAtZMk/fr1rT52tc5tfAXP16tVp7zdp\n0gTI/YlSlBmKiJDAcDy/NnJ4u3fv3kCwCpqfqPWMM84A4He/+13qM/7JYniFNMlPfrVEP8D/lltu\nSTIcqYKfWs0PtVu1ahUA+++/f2Ix1YQyQxERcmSihlNOOSXtpwgEGca1114LaI3kXOf7/vrp9Xwv\ngMMOOyyxmGpCmaGICDmSGYpk4tuOpX7Ze++9AZg4cWLCkdSMMkMREVQZiogAqgxFRABVhiIigCpD\nERFAlaGICACWabX7Cg82+wxYF104OafAOde26sPyh8o4/6mMM6tRZSgikq90mywigipDERFAlaGI\nCBDx2GQz2wOYV/qyHbAD+Kz09c+cc99FdN0+wEigITDWOfe3KK4jyZVx6bV3AV4D3nfO9Y/qOj90\nCX6PJwN9gA3OuW5RXCPtenE9QDGz24CvnXMjyrxvpXHszNJ1GgHvAD8HPgaWAb90zr2bjfNLxeIq\n49B5hwHdgGaqDOMRZxmb2QnAVmBcHJVhIrfJZtbJzIrM7GFgFdDBzP4X2j/QzCaUbu9lZtPNbJmZ\nLTWzI6s4/ZHAW865dc65bcBjQL+ofhfJLOIyxswKgF7ApKh+B6lc1GXsnFsAbIrsFygjyTbDg4CR\nzrkuwIZKjnsAGO6c6w6cA/j/uT3MbEyG4/cBPgq9Xl/6nsQvqjIGGAUMBdQ3LFlRlnGskpzPcI1z\nblk1jusJHBhaO3d3M2vqnFsCLIksOsmGSMrYzPoDHznnVphZz+yFK7WQN9/jJCvDLaHtnUB4pfAm\noW2jZo20G4AOodf7UvlfLIlOVGV8NDDAzPqWnqeVmU12zg2qU7RSG1GVcexyomtNaaPrZjPb38wa\nAGeGds8FrvQvzKyqhtTFQBczKzCzH1GSks/KdsxSM9ksY+fcMOfcvs65QuAC4DlVhMnL8vc4djlR\nGZa6HpgDLKKknc+7EjjGzN4wsyLgUqi4rcE5tx24CngeKAKmOOfeiTp4qZaslLHktKyVsZlNAxZS\nktysN7NfRxm4xiaLiJBbmaGISGJUGYqIoMpQRARQZSgiAtSwn2GbNm1cYWFhRKHkng8++IDi4mKr\n+sj8oTLOfyrjzGpUGRYWFrJsWXU6m+eH7t27Jx1C7FTG+U9lnJluk0VEUGUoIgKoMhQRAVQZiogA\nqgxFRABVhiIigCpDEREg2cldRUQA2Lx5MwAffvhhhccUFBQAMHLkSAC6du0KwAEHHADAIYccUqcY\nlBmKiJBwZvjpp58CcM455wBw9NFHA3DZZZcBJT3ls+GLL74A4KWXXgLglFNOAaBRo0ZZOb+I1MxT\nTz0FwOzZswF48cUXAXjvvfcq/MyBBx4IlAyvA9i2bVva/p0767ZKqTJDERESyAx92wDAT37yEyDI\n3Pbaay8g+xnhYYcdBkBxcTFAalzm/vvvn5XrSPV9+eWXAPzpT38CYNWqVQDMnTs3dYwy9vywZs0a\nAB566CEAxo0bl9q3detWAGoy0/4770S7eocyQxERYswMfVbm2wcBPv/8cwCuvLJk0azRo0dn9Zp3\n3XUXAGvXrgWCv0zKCOM3ZcoUAG666Sag/FNDnzEC7LHHHvEFJpFZv75kPahRo0bV6TwHHXQQEDw9\njooyQxERYswMX3vtNSB4ahR2yy23ZO06b775Zmp7xIgRAJx5Zsnyreeee27WriPV47ODa6+9Fgju\nEMzS59ocMmRIavvBBx8EoHXr1nGEKLXgyxGCzO/YY48Fgt4ajRs3BmDXXXcFoEWLFqnPfP311wD8\n4he/AIKsr0ePHgAceuihqWObNm0KQPPmzbP8W6RTZigigipDEREghttk37H6iSeeKLdv4sSJALRt\n27bO1/G3x7169Sq3b8CAAQC0bNmyzteRmvFNFf5hWUWmTp2a2n7mmWeA4GGLv4X2t12SnC1btgDp\n37PXX38dgJkzZ6Yde9RRRwGwfPlyIL3LnH+Atu+++wLQoEHyeVnyEYiI5IDIM8M//OEPQNC1wneA\nBjj77LOzdp2XX34ZgI8//jj13sUXXwzABRdckLXrSNXWrVuX2p40aVLaPj+Y3newf/7558t93neW\n91nl+eefD0C7du2yH6xUy3fffQfAr371KyDIBgFuvPFGAHr27Jnxs5kGUey3335ZjrDulBmKiBBD\nZui7UPif++yzT2pfXdqA/HCeu+++GwiG/IS7bPg2SYnXihUrUtu+M/Xxxx8PwIIFCwD49ttvAXjk\nkUcA+Otf/5r6zOrVq4Egy+/Xrx8QtCWqy018fBcY/z3zEyuE2/mHDh0KQLNmzWKOLruUGYqIkMBE\nDX7qHoDevXsDsNtuuwEwePDgKj/vO237n4sXL07bn812SKmd8NRKPlP3na69Jk2aAPCb3/wGgMcf\nfzy1zw/w94P4fcahp8nx80+I77nnHiCYYHXhwoWpY3yn6vpOmaGICDFkhldffTUA8+fPB2Djxo2p\nfb79yGcATz75ZJXn88eWHc7VsWNHIGjbkOT861//Kvfev//9bwD69++f8TN+WrVMjjzySCB9OJfE\nY9GiRWmv/TA53z8wnygzFBEhhszw8MMPB2DlypVA+pPGZ599FoDhw4cDsOeeewIwaNCgCs934YUX\nAnDwwQenve+XDPAZoiTnvPPOS237bP/VV18F4O233waCfw8zZswA0if99W3I/j0/9Zov+y5dukQW\nu6QLt+VC8ET/9ttvT73Xt29fIH1yhfpImaGICKoMRUQAsJqsQdC9e3dXWUN3HN5//30guB3u1q0b\nAM899xyQnUkfvO7du7Ns2TKr+sj8kY0y3rRpU2rbl5MfYlfRA7DwwH/fgf70008H4N133wWCVRPH\njBlTp/jCVMaVKztoIpOGDRsCcPnllwPBnIQfffQRAJ06dQKCNY/C/Bo4flKHKB7MVLeMlRmKiJDw\nusm1cccddwDBXyr/8CWbGaHUTXi43LRp0wA466yzgPIZ4lVXXQXAvffem/qM75Dtp17zQ/XmzJkD\nBJ2yQQ/MovbHP/4RgPvuu6/CY3bs2AEEGb3/WRP+4emJJ54IpE/pFhdlhiIi1JPM0GcXAJMnTwag\nVatWgFZSy3V+WiffRcNPzOC7z/hM32eDYTfffDMAb731FhB00/GfgeDfg0TDD8Pzq1r66dS2b9+e\nOsavc+MzxNrwk0D773p4JTw/yW/UlBmKiFBPMkPf0TPstNNOA9Ini5Xc5TPEiiYAzcSviuZXNfSZ\n4QsvvJA6xj+51rRe0fBPio844gggeLIfNm/ePCDIFm+77TYAli5dWuPr+bbk//znPzX+bF0pMxQR\noR5mhn7tVP+US/Kfb6+aNWsWkP6k0a+xnM21t6VmTj755LTXfsitzwwbNWoEBMtwAFx66aUAjBw5\nEgjakpOkzFBEBFWGIiJAjt8m+2FX4RXv/KpqenDyw+HX1B02bBiQvj6vb6wfOHAgAAcccEC8wUk5\nfgZ7v2qef7DiZx8CeO+994BgxvqywmslxUWZoYgI9SQzDA8S79OnT9oxX331FRDMfZeL67FKdvhJ\nOe68887Ue/5B2g033AAE63P7bjkSv86dOwNBl6hHH3203DHh7lEAu+xSUhX5LnPh4ZlxUWYoIkKO\nZ4aZ+L8gPgPwj+b98B0Nz8p/F110UWp77NixAEyfPh0I2qLKzoQu8fFZ+ahRo4Dg7i3ckfqTTz4B\noLCwEAjK1LcBJ0GZoYgI9TAzHD9+PAATJkwA4Le//S0QDOqX/Beerm3u3LlAsJ6vn1ggFzrx/tD5\nnh9+rfR//vOfqX2vvPIKEGSCfgqvJCkzFBEhxzPD0aNHA3Drrbem3jv++OMBGDx4MAC77747AI0b\nN445OskFvveAXzbAD9krKioCtJJeLvGrG5bdzhXKDEVEyPHM8LjjjgNg/vz5CUciuc5PHnvIIYcA\nsHr1akCZoVSfMkMREVQZiogAOX6bLFJdfk2ctWvXJhyJ1FfKDEVEUGUoIgKoMhQRAcD8alTVOtjs\nM2BddOHknALnXNuqD8sfKuP8pzLOrEaVoYhIvtJtsogIqgxFRICI+xma2R7AvNKX7YAdwGelr3/m\nnPsuwmvvArwGvO+c6x/VdX7okipjM7sOuKT05Rjn3OgoriOJlvF6YHPp9bY553pEcZ3U9eJqMzSz\n24CvnXMjyrxvpXHszPL1hgHdgGaqDOMRVxmbWTdgMnAk8D3wHPAb55x6XEcszu9xaWXY1Tn3v2yd\nszKJ3CabWSczKzKzh4FVQAcz+19o/0Azm1C6vZeZTTezZWa21MyOrMb5C4BewKSofgepXMRl3BlY\n7Jzb6pzbDrwEnBnV7yKZRf09jluSbYYHASOdc12ADZUc9wAw3DnXHTgH8P9ze5jZmAo+MwoYCuhR\nebKiKuOVwAlm1trMmgOnAh2yG7pUU5TfYwfMN7P/mNklFRyTNUmOTV7jnFtWjeN6AgeGlgvd3cya\nOueWAEvKHmxm/YGPnHMrzKxn9sKVWoikjJ1zb5rZ/cBc4GtgOSXtShK/SMq41JHOuQ1m1g543sze\ncs4tykLMGSVZGW4Jbe8ELPS6SWjbqFkj7dHAADPrW3qeVmY22Tk3qE7RSm1EVcY458YB4wDMbDiw\nug5xSu1FWcYbSn9+bGZPAj8DIqsMc6JrTWmj62Yz29/MGpDe/jMXuNK/KG08r+xcw5xz+zrnCoEL\ngOdUESYvm2VcesyepT8Lgb7A1GzGKzWXzTI2sxZm1qJ0uzklzwDezH7UgZyoDEtdD8yhpOZfH3r/\nSuAYM3vDzIqAS6HKtgbJTdks45mlx84ELnfOfRlh3FJ92Srj9sD/mdnrwFJghnNubpSBazieiAi5\nlRmKiCRGlaGICKoMRUQAVYYiIoAqQxERQJWhiAigylBEBFBlKCICwP8D3P5bzM0W5d8AAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f00292a1978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get the first images from the test-set.\n",
    "images = data.test.images[0:9]\n",
    "\n",
    "# Get the true classes for those images.\n",
    "cls_true = data.test.cls[0:9]\n",
    "\n",
    "# Plot the images and labels using our helper-function above.\n",
    "plot_images(images=images, cls_true=cls_true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Estimator 的输入函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "相较于直接给Estimator直接提供原始数据，我们必须提供一个函数来返回这些数据再输入。这允许在数据源上有更大的灵活性，例如可以对数据进行随机的清洗和迭代。\n",
    "\n",
    "注意到我们将利用`DNNClassifier`来创建一个Estimator，它默认类别数为整形，所以我们要用`data.train.cls` 来代替`data.train.labels`（独热数组）。\n",
    "\n",
    "这个函数为了能更好的控制数据的读取包含 `batch_size`, `queue_capacity` 和 `num_threads`参数。在这个例子中我们直接从内存中读取数据的numpy数组，所以在这里不需要这些。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "    x={\"x\": np.array(data.train.images)},\n",
    "    y=np.array(data.train.cls),\n",
    "    num_epochs=None,\n",
    "    shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这实际上返回一个函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.estimator.inputs.numpy_io.numpy_input_fn.<locals>.input_fn>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_input_fn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "调用这个函数返回一个TensorFlow的返回输入和输出数据的操作的元组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "({'x': <tf.Tensor 'random_shuffle_queue_DequeueMany:1' shape=(128, 784) dtype=float32>},\n",
       " <tf.Tensor 'random_shuffle_queue_DequeueMany:2' shape=(128,) dtype=int64>)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_input_fn()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "相同的，我们也需要需要创建一个用于读取测试集的函数。注意到我们只想要处理这些图像1次所以`num_epochs=1`并且我们不希望图像的顺序被打乱所以 `shuffle=False`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "    x={\"x\": np.array(data.test.images)},\n",
    "    y=np.array(data.test.cls),\n",
    "    num_epochs=1,\n",
    "    shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "还需要一个输入函数来预测新数据的类。作为一个例子我们只取了了测试集中的一部分图像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "some_images = data.test.images[0:9]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "predict_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "    x={\"x\": some_images},\n",
    "    num_epochs=1,\n",
    "    shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这个输入函数中正确的类别数实际上没有被用到，因为在预测阶段这是不需要的。然而，正确的类别数在我们之后画图中需要用到。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "some_images_cls = data.test.cls[0:9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  预制/封装 Estimator\n",
    "\n",
    "当我们使用预制Estimator时，我们需要为数据指定输入特性。在这个例子中，我们希望从我们的data-set中输入图像，它们是给定形状的数字数组。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_x = tf.feature_column.numeric_column(\"x\", shape=img_shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "你可以有几个输入特性，然后将它们组合在一个列表中："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "feature_columns = [feature_x]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这个例子中我们想要用3个隐藏层的DNN，分别有512.256，128单元。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "num_hidden_units = [512, 256, 128]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用`DNNClassifier`构筑我们的神经网络。我们也可以指定激活函数和其他不同的参数（见文档）。这里我们只指定分类的类别数量和checkpoints保存的地址。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using default config.\n",
      "INFO:tensorflow:Using config: {'_model_dir': './checkpoints_tutorial17-1/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': None, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f0028f03198>, '_task_type': 'worker', '_task_id': 0, '_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n"
     ]
    }
   ],
   "source": [
    "model = tf.estimator.DNNClassifier(feature_columns=feature_columns,\n",
    "                                   hidden_units=num_hidden_units,\n",
    "                                   activation_fn=tf.nn.relu,\n",
    "                                   n_classes=num_classes,\n",
    "                                   model_dir=\"./checkpoints_tutorial17-1/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练\n",
    "\n",
    "我们现在可以给定的迭代次数训练模型。这将自动加载并保存checkpoints，这样我们就可以在以后继续训练。\n",
    "\n",
    "注意到`INFO:tensorflow:`在每一行都被打印，这有阻碍快速阅读进展。它应该被单独的打印一行。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Saving checkpoints for 1 into ./checkpoints_tutorial17-1/model.ckpt.\n",
      "INFO:tensorflow:loss = 300.688, step = 1\n",
      "INFO:tensorflow:global_step/sec: 370.039\n",
      "INFO:tensorflow:loss = 26.462, step = 101 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 521.366\n",
      "INFO:tensorflow:loss = 22.0528, step = 201 (0.191 sec)\n",
      "INFO:tensorflow:global_step/sec: 549.886\n",
      "INFO:tensorflow:loss = 32.07, step = 301 (0.182 sec)\n",
      "INFO:tensorflow:global_step/sec: 548.856\n",
      "INFO:tensorflow:loss = 13.8037, step = 401 (0.182 sec)\n",
      "INFO:tensorflow:global_step/sec: 516.064\n",
      "INFO:tensorflow:loss = 23.2653, step = 501 (0.194 sec)\n",
      "INFO:tensorflow:global_step/sec: 552.268\n",
      "INFO:tensorflow:loss = 17.7141, step = 601 (0.180 sec)\n",
      "INFO:tensorflow:global_step/sec: 529.426\n",
      "INFO:tensorflow:loss = 25.7157, step = 701 (0.189 sec)\n",
      "INFO:tensorflow:global_step/sec: 513.375\n",
      "INFO:tensorflow:loss = 5.08285, step = 801 (0.195 sec)\n",
      "INFO:tensorflow:global_step/sec: 536.319\n",
      "INFO:tensorflow:loss = 10.3937, step = 901 (0.187 sec)\n",
      "INFO:tensorflow:global_step/sec: 534.847\n",
      "INFO:tensorflow:loss = 3.12976, step = 1001 (0.187 sec)\n",
      "INFO:tensorflow:global_step/sec: 540.827\n",
      "INFO:tensorflow:loss = 5.54126, step = 1101 (0.185 sec)\n",
      "INFO:tensorflow:global_step/sec: 483.467\n",
      "INFO:tensorflow:loss = 10.2708, step = 1201 (0.209 sec)\n",
      "INFO:tensorflow:global_step/sec: 527.042\n",
      "INFO:tensorflow:loss = 7.62363, step = 1301 (0.187 sec)\n",
      "INFO:tensorflow:global_step/sec: 557.67\n",
      "INFO:tensorflow:loss = 2.30585, step = 1401 (0.180 sec)\n",
      "INFO:tensorflow:global_step/sec: 547.406\n",
      "INFO:tensorflow:loss = 7.69151, step = 1501 (0.182 sec)\n",
      "INFO:tensorflow:global_step/sec: 557.682\n",
      "INFO:tensorflow:loss = 10.7881, step = 1601 (0.179 sec)\n",
      "INFO:tensorflow:global_step/sec: 547.859\n",
      "INFO:tensorflow:loss = 7.09411, step = 1701 (0.184 sec)\n",
      "INFO:tensorflow:global_step/sec: 544.495\n",
      "INFO:tensorflow:loss = 2.6387, step = 1801 (0.182 sec)\n",
      "INFO:tensorflow:global_step/sec: 549.648\n",
      "INFO:tensorflow:loss = 0.772691, step = 1901 (0.182 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 2000 into ./checkpoints_tutorial17-1/model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 7.35222.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.estimator.canned.dnn.DNNClassifier at 0x7f0028f03a90>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.train(input_fn=train_input_fn, steps=2000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 评估\n",
    "\n",
    "一旦训练好模型，我们可以在测试集上评估它的性能。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Starting evaluation at 2017-11-17-12:07:56\n",
      "INFO:tensorflow:Restoring parameters from ./checkpoints_tutorial17-1/model.ckpt-2000\n",
      "INFO:tensorflow:Finished evaluation at 2017-11-17-12:07:56\n",
      "INFO:tensorflow:Saving dict for global step 2000: accuracy = 0.9727, average_loss = 0.0934177, global_step = 2000, loss = 11.825\n"
     ]
    }
   ],
   "source": [
    "result = model.evaluate(input_fn=test_input_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'accuracy': 0.9727,\n",
       " 'average_loss': 0.093417682,\n",
       " 'global_step': 2000,\n",
       " 'loss': 11.825023}"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification accuracy: 97.27%\n"
     ]
    }
   ],
   "source": [
    "print(\"Classification accuracy: {0:.2%}\".format(result[\"accuracy\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 预测\n",
    "\n",
    "训练好的模型也可以拿来预测新的数据。\n",
    "\n",
    "注意到在对新数据预测时，TensorFlow图被再次创建并且加载checkpoint一次。如果模型非常大，这开销会是非常大的。\n",
    "\n",
    "不知道为什么Estimator被设计成这种方式，可能是为了总是能加载最近的一个Checkpoint，它也可以很容易地分布在多台计算机上。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = model.predict(input_fn=predict_input_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./checkpoints_tutorial17-1/model.ckpt-2000\n"
     ]
    }
   ],
   "source": [
    "cls = [p['classes'] for p in predictions]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([7, 2, 1, 0, 4, 1, 4, 9, 5])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cls_pred = np.array(cls, dtype='int').squeeze()\n",
    "cls_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe8VNW5//HPg2IQVBCxU86NWECjmGCvV5EoFpQYxViI\nsUQxlngDSYwSTJAYbPwsV1FfoglWEBGJimJXQISASrGAokEvIqJGiWJ7fn/MXjN7Tp19ph++79eL\n19kzs8uj68w6z1p77bXM3RERkdy0KncAIiLVRJWmiEgCqjRFRBJQpSkikoAqTRGRBFRpiogkoEpT\nRCQBVZoiIgmo0hQRSWDdfA7u1KmT19TUFCiU6jBnzpyV7r5pueMoFZVxy6cyTiavSrOmpobZs2fn\nc4qqY2bvlDuGUlIZt3wq42TUPBcRSUCVpohIAqo0RUQSUKUpIpKAKk0RkQTyunsu0lxXXnklAF98\n8QUAr7zyCgATJkyos+/ZZ58NwF577QXAySefXIoQReqlTFNEJAFlmlJSxx9/PADjx4+v93Mzq/Pe\nTTfdBMC0adMAOOCAAwDo2rVrMUKUMnrjjTcA2H777QG49tprATj33HPLFlNtyjRFRBJQpilFF7JL\naDjD3GGHHQA49NBDAXjrrbfSn02ePBmAxYsXAzBu3DgALrroosIHK2U1d+5cAFq1SuVzW2+9dTnD\nqZcyTRGRBJRpStGE55kfeOCBOp/ttNNOQCaL7NSpEwAbbLABAF999VV63z322AOAl19+GYCPPvqo\nSBFLuc2bNw/I/B4MGDCgnOHUS5mmiEgCJc80wzi8W265BYCtttoq/VmbNm0AOPHEEwHYYostAOje\nvXspQ5QC+b//+z8A3D39Xsgwp06dCsCWW25Z77FhHCfAokWLsj474ogjChqnlN+rr74KwHXXXQfA\nKaecUs5wGqVMU0QkgZJnmkOGDAFg6dKlDe4TxuVttNFGAPTs2bMg1+7SpQsAQ4cOBaB3794FOa/U\n78gjjwQyd70BNtxwQwA6duzY6LH33ntvejvevykt0+uvvw7A6tWrgewRF5VGmaaISAKqNEVEEih5\n8/zWW28FMsNH4k3vhQsXApkBrk8//TQAM2fOBDKPzb377rsNnr9169ZAZghLuBkRP09opqt5Xhrd\nunXLed8rrrgCyDxOFxeGHoWf0nKMGjUKSC29AZX93VSmKSKSQMkzzYMPPjjrZ1x4hC74+OOPgUzm\nGf76vPTSSw2e/3vf+x6QeeA/PJ4HsGrVKgC22WabZsUuxTNlyhQAhg0bBsCaNWvSn22++eYAXH75\n5QC0bdu2xNFJMcRvBofvdPjetmvXrhwh5USZpohIAhX9GOXGG28MwEEHHZT1fn1Zam33338/kMlW\nAXbeeWcABg4cWKgQpUDCI5fxDDMIw0/ClHDSMjzzzDN13tt008pfbl6ZpohIAhWdaTbHihUrABg8\neDCQ/Qhf6C9ramC1lM7RRx8NZB6rDAYNGpTeHjFiREljktIIS5zEhQdPKpkyTRGRBFpcpnnDDTcA\nmYyzQ4cO6c/CnTkpvzB+dvr06UCmLzP0aV188cXpfcM0YdIyzJgxA4CxY8em39t1110BOOSQQ8oS\nUxLKNEVEEmgxmebzzz8PZMbyBQ8++GB6O0xLJuUXJpdduXJl1vthWkCNpW25nnjiCSB7ZEsYox2m\nh6xkyjRFRBJQpSkikkCLaZ4//PDDQGbuxT59+gCw1157lS0mqSusCRQejQ0OPPBAAP70pz+VOiQp\nsTBZT9xPf/rTMkTSPMo0RUQSqPpM84svvgDg0UcfBTITdlx66aVAZqo4KZ/46pEjR44E6s7G3qtX\nL0DDi1qy5cuXA/Dcc88B2ZPpHHPMMWWJqTmUaYqIJFD1mWaYtDb0kR122GEA7L333mWLSbJdddVV\n6e1Zs2ZlfRYeo1RfZst3++23A/DBBx8Ame9qtVGmKSKSQFVmmmHCWoA///nPALRv3x6ASy65pCwx\nScOuvvrqBj8Lj72qL7Ple+edd7Jeh6kfq40yTRGRBKoq0wx3Yc8777z0e9988w0A/fr1AzQus9qE\nMs1llENoTYR9v/76awA+/fTTOvuGR/Suueaaes+1zjrrpLf/+te/AlpGo9geeuihrNdHHHFEmSLJ\njzJNEZEEVGmKiCRQFc3zb7/9FsjMhPL222+nP+vevTuQuSEk1SWs25SL4447DoAtt9wSyAxdueee\ne/KKIax2GZ/DUwonDGYP5VXtlGmKiCRQFZnmkiVLgMyKhXFhOIvmX6xc4SYdwKRJk5p9nvvuu6/J\nfcJNolatsvOBo446CoDevXvXOWbfffdtdkzStAceeADI3LQNs7RX6+qiyjRFRBKo6EwzDIbt27dv\n1vtXXnllertahy2sTSZOnJjeHjVqFFB3wo5g4cKFQOP9lKeddhoA3bp1q/PZT37yEwB69OjRvGCl\nYP7zn/8A8Mgjj2S9H6aBiw/7qibKNEVEEqjoTHPMmDFA3cev4n0hZlbSmCQ/ua5rfddddxU5Eim2\n0L8cVoTt378/AOeff37ZYioEZZoiIglUZKYZxnVdf/31ZY5ERJorZJphnfOWQpmmiEgCFZlphjXM\nP/vss6z3w9M/mkZMRMpFmaaISAKqNEVEEqjI5nltYaXCJ554AoCOHTuWMxwRWYsp0xQRSaAiM83f\n//73WT9FRCqFMk0RkQTM3Zt/sNmHwDtN7tiydHP3TcsdRKmojFs+lXEyeVWaIiJrGzXPRUQSUKUp\nIpJAo5WmmW1iZvOif8vN7L3Y6/WKEZCZ9YxdY56ZfWZmv2rimNPN7MNo/0Vm9os8YxhnZkc3sc/v\nYjEuMLNvzKx9PtcthzKVcTcze9rMFkb/7xot3+iYcpTxKWb2qpm9YmYvmNkP8rlmuZSjjKPr3hHK\nLMf9y1HGO5rZDDNbY2YX5HRid8/pHzAc+E097xvQKtfzJPkHtAZWAJ2b2O90YHS0vQWwEuhUa591\nE1x3HHB0gv2PAR4rxv+DUv4rVRkDWwG9ou2NgCXAdpVWxsA+QIdo+0jghXKXUbWUcXTOA4DdgXk5\n7l+OMt4c6A1cDlyQy3mb1Tw3s+5RlnAnsADoYmafxD4faGa3Rtubm9lEM5ttZrPMbM8ElzoEWOTu\ny3I9wN2XA0uBrmY2wsz+ZmYvALeb2bpmdnUUxytmdnoUYysz+18ze83MHgc6JYgR4ATg7oTHVLRi\nlrG7v+/u86LtfwOvAVvnGlupytjdX3D38N88E+ica4zVoNjfY3d/BljVnNhKWMYfuPts4JtcY8tn\ncPsOwCnuPtvMGjvPtcAod59pZjXAFGAnM9sDONXdz2rk2IEkrIzMrDvQDXgrFuf+7v6lmQ0GVrj7\n7mb2PWCmmT0G7An8F9CTVBa0ELgpOt9lpDKMhxu43gZAH+CMJHFWiaKXsZl9H9gJeCnXoEpdxpHT\ngEca+bxaleJ7nFiZyjgn+VSaS6Iauil9gO0tsyzFxma2vru/CLzY0EFm1gY4HLgwx3hONLMDgTXA\n6e7+SXTNB939y2ifvkAPMxsYvW4PbAvsD9zt7t8By8zs6XBSd/9DE9ftDzzj7p/mGGc1KXYZbwTc\nD5zr7p/ncJ2ylLGZ9QFOBlriWr9FLeNmKNf3OGf5VJqrY9vfkeoTCdrEtg3Y3d3rX36wYYcDL7r7\nyhz3v9Pd6+vIjcdpwGB3fyK+g5kdkzC2uIHA3/M4vpIVrYwtdQNiIjDW3SfneFjJy9jMegFjgB+7\n+8fNOUeFK/b3OKlyfY9zVpAhR1HN/rGZbWtmrUjdGAmmAeeEF9EvYS7q9BOa2flmlk8zYCowODRD\nzGx7M1sfeBY4PuoT2ZpUB3aTzGxjYG/goTxiqgqFLGNLpQ63k7pBcG2tzyqmjKNm6ATgZ+6+OI+Y\nqkKRvsd1VFIZN0chx2n+ltR/zHQgfuPmHGCfqMN2IVHfn5ntYWY31XciM9sQ+G9gUq2PegAf5RHj\nGOBNYJ6ZzQduJJVtTwDeJdUHMhZIL2piZpeZWb8GzvcT4BF3/yKPmKpJocr4AFJ/FA+xzNCXH0ef\nVVIZDwc6AmOiGAvZDK1UhfwejweeA3qa2TIz+3n0UcWUsZl1NrNlwHnA8CjOto1dvKoeozSzfwD9\n3T3nO11SXVTGLV+1l3FVVZoiIuWmxyhFRBJQpSkikoAqTRGRBFRpiogkkNcaQZ06dfKampoChVId\n5syZs9LXolm9VcYtn8o4mbwqzZqaGmbPzuUJrJbDzNaqZQFUxi2fyjgZNc9FRBJQpSkikoAqTRGR\nBFRpiogkoEpTRCQBVZoiIgnkNeSoVFavTs0/OmTIEABuuikzE1Xv3r0BGD9+PADdunUrcXQisjZR\npikikkBVZJrvv/8+ALfccgsA66yzTvqzMCj3oYdSk6f/6ldNLqEtFeCf//wnAAMGDABg6dKlzT7X\nY489lt7u0aMHAF26dGl+cFI24Xt81FFHAXDdddcBcPbZZ6f3iX//y0GZpohIAhWdaX744YcADBo0\nqMyRSKFNnToVgDVr1uR9rsmTM+uy3XbbbQDcc889eZ9XSuejj1KrX8QzSoBzzz0XgNNOOy393vrr\nr1+6wOqhTFNEJIGKzDSvvTa1QOGkSal11V566aUmj3nuuecACMt37LLLLgDsv//+xQhRmumbb1LL\nwjz88MMFO2cYQQFw9dVXA5kRF+3atSvYdaR4nn32WQDee++9rPdPOOEEANq0aVPnmHJRpikikkBF\nZpoXXJBaKz7JXbKJEydm/ezatSsA9913X3qfH/3oR4UKUZrpqaeeAmD69OkA/Pa3v837nKtWrUpv\nL1iwAID//Oc/gDLNShbvzx4xYkS9+5x88skAmFlJYsqFMk0RkQRUaYqIJFBRzfN+/foBmZs53377\nbZPHdOrUCcg0w955JzUh89tvvw3Abrvtlt73u+++K1ywkrNXX301vT1w4EAAunfvDsBFF12U9/nj\nQ46kerzyyivp7fCwQ7Duuqmq6bDDDitpTLlQpikikkDZM81nnnkmvf3aa68BmU7fhm4EnXXWWent\nvn37AtC+fXsAnnzySQAuu+yyOsfdeOONQN0BtFJc8bIIN2jGjRsHwAYbbNDs84YbQPHfoUq6YSCN\nCzdt63PIIYeUMJJklGmKiCRQtkwzTNAQ+rgAVq5cWe++YfjQscceC8Af//jH9Gdt27bN2jdMDTdm\nzJg65xw6dCgAX375JZCZ3KN169bN+4+QRk2YMAHIHsge+jLjfc3NFYapxLPLAw88EIAOHTrkfX4p\nrngLIVhvvfUAGDlyZKnDyZkyTRGRBMqWaX799ddAw9klZB6BvPfee4HMnfLGhEwz3JW98MIL05+F\nR+tCxhmmn9pmm20SxS65CRNDh//vUJj+5NBKueuuu4DMnVaAiy++GFDroZKFBxtmzJhR57PQcuzV\nq1dJY0pCmaaISAJlv3ten9DfNXbsWCC3DLO2kEXeeeed6fdmzZpVgOikKZ9++ikAM2fOrPPZ4MGD\n8z7/zTffDGSmDuzZs2f6s4MOOijv80txNTYBTzWMbFGmKSKSQNkzzfqe+nnxxRfzPm94qij+FFDt\nJ43CXfgwZlAKI0zEsGzZMiAzvVehLFmyJOv1TjvtVNDzS3HVl2mG0Q6FaIkUmzJNEZEEVGmKiCRQ\ntuZ5WLu8WCvLhVXt5s6dm36v9uOZl156aVGuvbbbcMMNgcywkfiEHeHRx44dOyY+74oVK4DMUKZg\nn332aVacUlrPP/88kBkqFhceg+7cuXNJY2oOZZoiIgmULdOcMmVKQc8Xhp8sXLgQaPwxrDCESQOg\niyOsFhgemQyPUwIcfvjhQPZDB/WZP39+ejvc+AnT/tWelKNVK/3trwZhxclwQzaukifoqE2/bSIi\nCZR9yFGhhOnHbrjhhgb3qampAeCOO+4AMhOBSHEMHz4cyM4sQgsjPlFLfTbddNP0dsgsG3rk9tRT\nT80nTCmR2n3R8UlVzjzzzFKH02zKNEVEEqj6TDMskREmMG5MeNxuv/32K2pMktKjRw8ge0XQMJqh\n9gD12sI0gHGDBg0C6j6MEPpQpTKFhxxq3zWP3ykvxFSBpaJMU0QkgbJlmo0tnvbII49kvT7jjDMA\neP/99xs8Ty7LHBT6jr0kt+uuu2b9TOL73/9+ve/Hx4H+4Ac/aF5gUjRhKrjad8379+9fjnDypkxT\nRCQBVZoiIgmUrXke5s0Ls6jHhQHQtR+xrO+Ry9C8z2XlSqluoXlXu5mnJnllC4Pag/BwyQUXXFCO\ncPKmTFNEJIGyZZoDBgwAYNSoUen3GlsvqCnhr1cY5nLLLbcAsOWWWzb7nFJZws0+rW1eXaZOnZr1\nukuXLkBmko5qo0xTRCSBsmWaYdXIsNIkwKRJkwAYPXp04vP94Q9/ADJrmUvLE9arDzSovbKFFWcX\nL16c9X6bNm2A6p0wR5mmiEgCZX+MMqxtHt/u27cvkFl1MEwofOSRRwLwy1/+Mn1MuJMaX5FQWqaw\nOmmY6GHYsGHlDEeaEKbsC49ILliwAIBtt922bDEVgjJNEZEEyp5p1ufQQw/N+ikCmYzl17/+NaA1\nzitdGDsdpm0Mox5++MMfli2mQlCmKSKSQEVmmiL1CX3bUl222morAG677bYyR1IYyjRFRBJQpSki\nkoAqTRGRBFRpiogkoEpTRCQBVZoiIglY7QldEx1s9iHwTuHCqQrd3H3TpndrGVTGLZ/KOJm8Kk0R\nkbWNmuciIgmo0hQRSUCVpohIAo1Wmma2iZnNi/4tN7P3Yq/XK1ZQZtbPzF43s8VmNiSH/UfEYnvV\nzA7P8/rPm1mvJvZpY2YTohhnmFnXfK5ZLuUq4+ja65rZK2Y2KYd9S17GsX2PNzPPdf9KU8bv8R1m\n9qGZzctx/9PD/ma2yMx+kef1x5nZ0U3ss2P0/V1jZjktj9nohB3u/hHQKzr5cOBzd7+y1kWN1A2l\n73K5YFPMrDVwPfDfwHJgtpk96O5vNHHoFe4+2sx2Ap4ys808dpfLzNZ1928KEWPkTGC5u3c3s5OA\nvwAnFvD8JVGOMo65EJgPtM1x/1KXMWa2ETAYmF3I85ZSGcv4NuAG4OYEx9zp7heY2RbAfDOb7O7p\nFReLUMYrgXOBY3M9oFnNczPrbmYLzexOYAHQxcw+iX0+0MxujbY3N7OJZjbbzGaZ2Z5NnH5PYJG7\nv+Pua4D7gP65xubu8wEDNo7+0txoZrOAkWa2gZndHsUx18yOjGJsa2bjo79u9wNtcrhUf+COaPs+\n4Me5xlgNilzGmFk34BBgbNLYSljGACOjf2uSxlnpil3G7v4MsKo5sbn7cmAp0DVqZfzNzF4Abo9a\nKFdHcbxiZqdHMbYys/81s9fM7HGgUw7X+cDdZwM5V8T59GnuAFzj7j2B9xrZ71pglLv3Bo4DQiHs\nYWY31bP/1sC/Yq+XRe/lxMz2Br5091BYWwJ7uvtQYBjwqLvvDhwEXGVmbYBfAR+7ew9gBLBr7Hxj\nG2iWpeN096+A1WbWIdc4q0SxyhhgNDAESDzmrVRlbGa7AZu5+9Tan7UgxSzjZjOz7kA34K1YnAe7\n+0mkWnkrojLeDTjHUt1jxwL/BfQETgX2jp3vMjPrV4jY8plPc0lUQzelD7C9Zdaq3tjM1nf3F4EX\n87h+bUPM7OfAZ8DxsffHx5ocfYHDzOx30es2QFdgf2AUgLvPNbMF4WB3P7WAMVabopRx1M/0L3ef\nZ2Z9EsRTsjI2s1bAVVRhl0tClfY9PtHMDiSV2Z/u7p9E13zQ3cNypH2BHmY2MHrdHtiWVBnfHf0u\nLDOzp8NJ3f0PhQown0pzdWz7O1LNpSDe9DFg9ygby8V7QJfY6840/hcwuMLd61v7Nx6nAUe7+5L4\nDrFfhCRCnMst1Znezt0/aeKYalOsMt4bGGBmR0Xn2cjM7nD3QU0cV8oy7kAqY3kuOnYL4GEzO9zd\n5yY9WQUrVhk3153uXt8NmdplPNjdn4jvYGbHFDWySEGGHEU1+8dmtm30Fzoe/DTgnPCigaZu3Eyg\np5l1M7PvkWoKTI6OHRX6qJppKqlO3xBLaKI9C/wsem8XYMcczjUZCF/y44DH8oir4hWyjN19qLt3\ndvca4CTgsVBhVkoZu/sqd+/k7jVRnLOBfi2swsxS4O9xg8zsfDM7q/mRMhUYbGbrRufb3szWJ1XG\nx0d9m1sDB+RxjQYVcpzmb0n9x0wn1Q8ZnAPsE3XYLgTOgIb7Qtz9a+A84HFgITDO3V+PPt6Z1B31\n5roUaGepISsLgOHR+9cDm5jZIuASIP3FaKRP82ZgSzNbTKq/7KI84qoWBSnjJlRSGa+NClbGZjYe\neI5UErQs6loB6AF8lEeMY4A3gXlmNh+4kVSreQLwLql6YywwIxZLvX2aZtbZzJaRqnOGR3E2Opqj\nap49t1Qb6RF31xKVLZTKeO1gZv8A+hd6eFipVE2lKSJSCfQYpYhIAqo0RUQSUKUpIpJAPuM06dSp\nk9fU1BQolOowZ86clWvTrN4q45ZPZZxMXpVmTU0Ns2dX7TwGzWJma9WyACrjlk9lnIya5yIiCajS\nFBFJQJWmiEgCqjRFRBJQpSkikoAqTRGRBFRpiogkkNc4TRGRYvj4448BePfddxvcp1u3bgBcc801\nAOy0004AbLfddgDssssuRYlNmaaISAIVlWmuWLECgOOOOw6AvfdOrYt05plnAqknFwrh008/BeDZ\nZ58F4NBDU9M3tm7duiDnF5FkpkyZAsBDDz0EwNNPPw3Am2++2eAx22+/PQBLly4FYM2a7AVDv/uu\n0CtOpyjTFBFJoOyZZui7ANhxx9SyLSET3HzzzYHCZ5g//OEPAVi5MrUGfXjudtttty3IdSR3//73\nvwH43e9Si0cuWJBaJHLatGnpfdQCaBmWLEmtdXfDDTcAcPPNN6c/++KLLwBIMin666+/3vRORaBM\nU0QkgbJlmiHLC/2XAB99lFpr6ZxzUoveXXfddQW95ogRIwB4++23gcxfOmWYpTdu3DgALr74YqDu\nXdKQgQJssskmpQtMimbZstQ6baNH17cKc+522GEHIHO3vNSUaYqIJFC2TPOf//wnkLlLFjds2LCC\nXWf+/Pnp7SuvvBKAY45JLed8/PHHF+w6kpuQbfz6178GMi2O1EKUGeeem166nOuvvx6Ajh07liJE\naYZQjpDJJPfdd18gMzplvfXWA6B9+/YAbLDBBuljPv/8cwB+/OMfA5ksco899gBg1113Te+7/vrr\nA9CuXbsC/1fkRpmmiEgCqjRFRBIoefM8DGC///7763x22223AbDppvkvzxKa5YccckidzwYMGADA\nhhtumPd1JJnQRRJu+jXknnvuSW8/8sgjQOamUWi6h+aelM/q1auB7O/Zyy+/DMCkSZOy9t1rr70A\nmDt3LpA9lDDcCOzcuTMArVpVbj5XuZGJiFSgkmea//M//wNkhpyEgeYAP/3pTwt2neeffx6A5cuX\np9879dRTATjppJMKdh1p2jvvZNawGjt2bNZnYVKF8CDD448/Xuf48FBCyFJPPPFEALbYYovCBys5\n+eqrrwD42c9+BmSyS4CLLroIgD59+tR7bH0Pq3Tt2rXAERaPMk0RkQRKnmmGoSXh59Zbb53+LJ8+\nqvAY1siRI4HMo1rxoSyhz1RKa968eentMGh9//33B+CZZ54B4MsvvwTgrrvuAuAvf/lL+pjFixcD\nmVZD//79gUxfp4YilU4YGhS+Z2GCjfh9iCFDhgDQtm3bEkdXGso0RUQSKPuEHWFKKIC+ffsC0KFD\nBwDOPvvsJo8Pg+PDz5kzZ2Z9Xsh+Umme+JRdIfMPg9uDNm3aAPCLX/wCgAkTJqQ/CxM9hMkcQgaj\nu+elF+6IX3755UBmIuDnnnsuvU8YvN5SKdMUEUmg5Jnm+eefD8CTTz4JwPvvv5/+LPRvhYziwQcf\nbPJ8Yd/aj+Fts802QKbvRcrn7rvvrvPeP/7xDwCOPvroeo8J0/XVZ8899wSyH8OT0pg+fXrW6/B4\nYxhfuTZQpikikkDJM80f/ehHALz66qtA9p3VRx99FIBRo0YBsNlmmwEwaNCgBs938sknA7Dzzjtn\nvR+WyggZp5TPCSeckN4OrYeXXnoJgNdeew3I/D488MADQPbk1KGPO7wXpvQLZd+zZ8+ixS7Z4n3N\nkBnBcOmll6bfO+qoo4DsSTZaEmWaIiIJqNIUEUnAkqzJUVvv3r29sQ77UnjrrbeATDO8V69eADz2\n2GNAYSb/iDOzOe7eu6AnrWCFKONVq1alt0M5hUcjG7qRF58AIjyocMQRRwDwxhtvAJlVSm+66aa8\n4qtNZdyw2g+n1GedddYB4KyzzgIyc2L+61//AqB79+5AZk2wuLBGVJjco1g3mPIpY2WaIiIJlH1w\ne77+9Kc/AZm/fOEmUqEzTGm++GOO48ePB+DYY48F6mac5513HgB//etf08eEge9hSr/wiOXUqVOB\nzOB30I2/YvvNb34DwFVXXdXgPt9++y2QaSGEn0mEm8AHHnggkD1VYLkp0xQRSaAqM82QrQDccccd\nAGy00UaAVi6sdGG6sDB0JUzQEYYVhZZDyC7jLrnkEgAWLVoEZIYvhWMg8/sgxREenwyryIZp+r7+\n+uv0PmEdqJBxNkeYrDx81+MrT4bJqMtFmaaISAJVmWmGAbVxhx9+OJA9qbFUrpBxNjRRbX3CKoRh\nFdGQaT711FPpfcKdek0XVxzhzvhuu+0GZEYyxD3xxBNAJvscPnw4ALNmzUp8vdDXPWfOnMTHFosy\nTRGRBKo+0wxrH4e7etLyhf60yZMnA9l3VsMa6cOGDSt9YALAwQcfnPU6PCodMs3WrVsDmeVnAM44\n4wwArrnmGiDT112JlGmKiCSgSlNEJIGqap6Hx+XiK0yGVQx1A2jtEdbEHjp0KJC9vna46TBw4EAA\ntttuu9IGJ3WEFRnCKpXhBlGYrQrgzTffBDIrMNQWX0us3JRpiogkUJWZZnyygH79+mXt89lnnwGZ\nuReraT1lSSZMzvLnP/85/V64Ifj73/8egHHjxgGZ4UpSej169AAyQ8XuvffeOvvEh40BrLtuqmoK\nQwnjj9XY76NRAAAG3klEQVSWmzJNEZEEqirTrE/4ixQyijBkITx2pcfqWr5TTjklvT1mzBgAJk6c\nCGT6ymrP7C+lE7L80aNHA5nWYHzA+gcffABATU0NkCnT0EddSZRpiogkUPWZ5i233ALArbfeCsDp\np58OZCZ3kJYvPg3gtGnTgMx63GGCiUoeLL22CCNdpkyZAsDf//739GczZswAMpllmBquEinTFBFJ\noKoyzeuuuw6AP/7xj+n39t9/fwDOPvtsADbeeGMA1ltvvRJHJ5UgjJYIy2WERy0XLlwIaOXKShJW\nE629XemUaYqIJFBVmeZ+++0HwJNPPlnmSKTShUmOd9llFwAWL14MKNOU/CnTFBFJQJWmiEgCVdU8\nF8lVWDPq7bffLnMk0tIo0xQRSUCVpohIAqo0RUQSsLDaW7MONvsQeKdw4VSFbu6+adO7tQwq45ZP\nZZxMXpWmiMjaRs1zEZEEVGmKiCTQaKVpZpuY2bzo33Izey/2uqgzYpjZumb2iplNymHfEbHYXjWz\nw/O89vNm1ivHfY83M891/0pTrjI2swvNbEH079wc9j/dzD6M4lpkZr/I8/rjzOzoJvbpaGaTo9/D\nF82sKp/BLGMZL4u+j/PM7MUc9i9HGfcxs09j/z/+0NR5Gx3c7u4fAb2ikw8HPnf3K2td1Ej1jX7X\n1MUSuhCYD7TNcf8r3H20me0EPGVmm3msw9bM1nX3bwoZoJltBAwGZhfyvKVUjjKO/sAMAnoD3wCP\nmdkUd29qJPqd7n6BmW0BzDezye6+MnbeQpfxJcCL7n6Ume0I/D/gkAKevyTK/D3ez90/SbB/qcsY\n4Cl3b7RyjWtW89zMupvZQjO7E1gAdDGzT2KfDzSzW6Ptzc1sopnNNrNZZrZnDufvRuqXc2zS2Nx9\nPmDAxtFfmhvNbBYw0sw2MLPbozjmmtmR0fXamtn46K/b/UCbHC83Mvq3Jmmcla7IZdwDmOnuX7j7\n18CzwDG5xubuy4GlQNeolfE3M3sBuD1qoVwdxfGKmZ0exdjKzP7XzF4zs8eBTjlcqifwZHTNBcB2\nZrZJrnFWumJ/j/NRwjJOLJ8+zR2Aa9y9J/BeI/tdC4xy997AcUAohD3M7KYGjhkNDAES39o3s72B\nL919VfTWlsCe7j4UGAY86u67AwcBV5lZG+BXwMfu3gMYAewaO99Yq6fpbWa7AZu5+9SkMVaRYpXx\nq8ABlmr+tgMOA7rkGpSZdQe6AW/F4jzY3U8CzgRWRGW8G3COmXUFjgX+i1RFeCqwd+x8l5lZ9rKm\nKS8DA6J99gI6R/9akmJ+jx140szmmNlpSYIqYRkD7GtmL5vZw5ZDF0w+z54vcfdcmqV9gO0ts+zu\nxma2vru/CNTp54j6IP7l7vPMrE+CeIaY2c+Bz4DjY++PjzU5+gKHmdnvotdtgK7A/sAoAHefa2YL\nwsHufmo9MbYCrgJOTBBfNSpKGbv7fDO7GpgGfA7MBb7N4TonmtmBpDL70939k+iaD7r7l9E+fYEe\nZjYwet0e2JZUGd8d/S4sM7OnY/E01I91GXCtmc0jVYG+nGOc1aQoZRzZ093fi5raj5vZInef3sR1\nSl3GLwE17v551PKcSKqCblA+lebq2PZ3pJrEQbx5a8Du7v5VjufdGxhgZkdF59nIzO5w90FNHHeF\nu49uIk4Djnb3JfEdYr8IuepA6q/Zc9GxWwAPm9nh7j436ckqWLHKGHe/GbgZwMxGAYtzOOxOd7+g\niTgNGOzuT8R3MLOcm/+xGD8l1fca/lAuBVraDCDFLOP3op/LzexBYHegqUqzHGUcth+KuvM6NNYP\nW5AhR1HN/rGZbRv9csWDnwacE17U19Stda6h7t7Z3WuAk4DHQoVpZqNCP2QzTQXSd2rNLDTDnwV+\nFr23C7BjEzGucvdO7l4TxTkb6NfCKswshSzjaJ/Nop81wFHAPdHr883srDxCnQoMNrN1o/Ntb2br\nkyrj46N+r62BA3KIsYOZtY5e/hKY5u6rGzummhWyjC11/2CDaLsdqXsU86PXlVTGW8S29wS+aerG\nVSHHaf6W1H/MdGBZ7P1zgH2iDtuFwBlRgI31hTRkZ2B5HjFeCrSz1DCIBcDw6P3rgU3MbBGpO6bp\nyq+hPs21VCHLeFK07yTgLHf/d/R+D+CjPGIcA7wJzDOz+cCNpFpUE4B3gYWkbjDOCAc00t/1A2Ch\nmb0OHExqREdLV6gy3hJ4wcxeBmYBD7j7tOizSirjgZYa9jYPuIbsrr16Vc1jlJZqBz/i7oeWOxYp\nHjP7B9C/CMNKpEJUexlXTaUpIlIJ9BiliEgCqjRFRBJQpSkikoAqTRGRBFRpiogkoEpTRCQBVZoi\nIgn8f6S1hmwDdPmXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0026103a20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_images(images=some_images,\n",
    "            cls_true=some_images_cls,\n",
    "            cls_pred=cls_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 新的Estimator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果你不使用内置的Estimator，那么你可以自己创建一个任意TensorFlow 模型。为了完成这个，你首先需要创建如下的几个函数：\n",
    "1. TensorFlow 模型，例如一个卷积神经网络\n",
    "2. 模型的输出。\n",
    "3. 用于模型优化的损失函数。\n",
    "4. 优化的方法。\n",
    "5. 性能指标。\n",
    "\n",
    "这个Estimator可以运行三种模式：训练，评估，或预测。这些代码大体相似，但是预测模式下我们不需要设置损失函数和优化器。\n",
    "\n",
    "这是Estimator API的另一个方面，它被设计的糟糕，就像我们过去用结构体来做ANSI C编程一样。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def model_fn(features, labels, mode, params):\n",
    "    # Args:\n",
    "    #\n",
    "    # features: This is the x-arg from the input_fn.\n",
    "    # labels:   This is the y-arg from the input_fn,\n",
    "    #           see e.g. train_input_fn for these two.\n",
    "    # mode:     Either TRAIN, EVAL, or PREDICT\n",
    "    # params:   User-defined hyper-parameters, e.g. learning-rate.\n",
    "    \n",
    "    # Reference to the tensor named \"x\" in the input-function.\n",
    "    x = features[\"x\"]\n",
    "\n",
    "    # The convolutional layers expect 4-rank tensors\n",
    "    # but x is a 2-rank tensor, so reshape it.\n",
    "    net = tf.reshape(x, [-1, img_size, img_size, num_channels])    \n",
    "\n",
    "    # First convolutional layer.\n",
    "    net = tf.layers.conv2d(inputs=net, name='layer_conv1',\n",
    "                           filters=16, kernel_size=5,\n",
    "                           padding='same', activation=tf.nn.relu)\n",
    "    net = tf.layers.max_pooling2d(inputs=net, pool_size=2, strides=2)\n",
    "\n",
    "    # Second convolutional layer.\n",
    "    net = tf.layers.conv2d(inputs=net, name='layer_conv2',\n",
    "                           filters=36, kernel_size=5,\n",
    "                           padding='same', activation=tf.nn.relu)\n",
    "    net = tf.layers.max_pooling2d(inputs=net, pool_size=2, strides=2)    \n",
    "\n",
    "    # Flatten to a 2-rank tensor.\n",
    "    net = tf.contrib.layers.flatten(net)\n",
    "    # Eventually this should be replaced with:\n",
    "    # net = tf.layers.flatten(net)\n",
    "\n",
    "    # First fully-connected / dense layer.\n",
    "    # This uses the ReLU activation function.\n",
    "    net = tf.layers.dense(inputs=net, name='layer_fc1',\n",
    "                          units=128, activation=tf.nn.relu)    \n",
    "\n",
    "    # Second fully-connected / dense layer.\n",
    "    # This is the last layer so it does not use an activation function.\n",
    "    net = tf.layers.dense(inputs=net, name='layer_fc2',\n",
    "                          units=10)\n",
    "\n",
    "    # Logits output of the neural network.\n",
    "    logits = net\n",
    "\n",
    "    # Softmax output of the neural network.\n",
    "    y_pred = tf.nn.softmax(logits=logits)\n",
    "    \n",
    "    # Classification output of the neural network.\n",
    "    y_pred_cls = tf.argmax(y_pred, axis=1)\n",
    "\n",
    "    if mode == tf.estimator.ModeKeys.PREDICT:\n",
    "        # If the estimator is supposed to be in prediction-mode（如果是在预测模式下，预测的类别是神经网络的输出。优化等等是不需要的。\n",
    "        # then use the predicted class-number that is output by\n",
    "        # the neural network. Optimization etc. is not needed.\n",
    "        spec = tf.estimator.EstimatorSpec(mode=mode,\n",
    "                                          predictions=y_pred_cls)\n",
    "    else:\n",
    "        # Otherwise the estimator is supposed to be in either\n",
    "        # training or evaluation-mode. Note that the loss-function（注意损失函数在评估模式下也是需要的）\n",
    "        # is also required in Evaluation mode.\n",
    "        \n",
    "        # Define the loss-function to be optimized, by first（定义用于优化的损失函数，首先计算网络输出和真实标签的交叉熵）\n",
    "        # calculating the cross-entropy between the output of\n",
    "        # the neural network and the true labels for the input data.\n",
    "        # This gives the cross-entropy for each image in the batch.\n",
    "        cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=labels,\n",
    "                                                                       logits=logits)\n",
    "\n",
    "        # Reduce the cross-entropy batch-tensor to a single number\n",
    "        # which can be used in optimization of the neural network.\n",
    "        loss = tf.reduce_mean(cross_entropy)\n",
    "\n",
    "        # Define the optimizer for improving the neural network.\n",
    "        optimizer = tf.train.AdamOptimizer(learning_rate=params[\"learning_rate\"])\n",
    "\n",
    "        # Get the TensorFlow op for doing a single optimization step.\n",
    "        train_op = optimizer.minimize(\n",
    "            loss=loss, global_step=tf.train.get_global_step())\n",
    "\n",
    "        # Define the evaluation metrics,\n",
    "        # in this case the classification accuracy.\n",
    "        metrics = \\\n",
    "        {\n",
    "            \"accuracy\": tf.metrics.accuracy(labels, y_pred_cls)\n",
    "        }\n",
    "\n",
    "        # Wrap all of this in an EstimatorSpec.（封装成Estimator的规格）\n",
    "        spec = tf.estimator.EstimatorSpec(\n",
    "            mode=mode,\n",
    "            loss=loss,\n",
    "            train_op=train_op,\n",
    "            eval_metric_ops=metrics)\n",
    "        \n",
    "    return spec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建Estimator的一个实例\n",
    "\n",
    "我们可以指定超参数，例如优化器的学习率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\"learning_rate\": 1e-4}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以创建这个新的Estimator的一个实例。\n",
    "\n",
    "注意我们没有提供feature-columns，因为它会当`model_fn()`被调用时，从data-functions自动推断出来。\n",
    "\n",
    "在TensorFlow文档中写的不清楚，为什么在上面使用`DNNClassifier`必须指定feature-columns，而这里不需要。 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using default config.\n",
      "INFO:tensorflow:Using config: {'_model_dir': './checkpoints_tutorial17-2/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': None, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f00290fac88>, '_task_type': 'worker', '_task_id': 0, '_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n"
     ]
    }
   ],
   "source": [
    "model = tf.estimator.Estimator(model_fn=model_fn,\n",
    "                               params=params,\n",
    "                               model_dir=\"./checkpoints_tutorial17-2/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练\n",
    "\n",
    "现在我们新的Estimator已经被创建，我们可以训练它。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Saving checkpoints for 1 into ./checkpoints_tutorial17-2/model.ckpt.\n",
      "INFO:tensorflow:loss = 2.33444, step = 1\n",
      "INFO:tensorflow:global_step/sec: 190.454\n",
      "INFO:tensorflow:loss = 0.810317, step = 101 (0.527 sec)\n",
      "INFO:tensorflow:global_step/sec: 198.129\n",
      "INFO:tensorflow:loss = 0.349305, step = 201 (0.504 sec)\n",
      "INFO:tensorflow:global_step/sec: 184.116\n",
      "INFO:tensorflow:loss = 0.288062, step = 301 (0.543 sec)\n",
      "INFO:tensorflow:global_step/sec: 195.138\n",
      "INFO:tensorflow:loss = 0.0948148, step = 401 (0.512 sec)\n",
      "INFO:tensorflow:global_step/sec: 199.116\n",
      "INFO:tensorflow:loss = 0.203272, step = 501 (0.502 sec)\n",
      "INFO:tensorflow:global_step/sec: 190.777\n",
      "INFO:tensorflow:loss = 0.22347, step = 601 (0.524 sec)\n",
      "INFO:tensorflow:global_step/sec: 198.669\n",
      "INFO:tensorflow:loss = 0.161297, step = 701 (0.505 sec)\n",
      "INFO:tensorflow:global_step/sec: 192.277\n",
      "INFO:tensorflow:loss = 0.154663, step = 801 (0.518 sec)\n",
      "INFO:tensorflow:global_step/sec: 158.865\n",
      "INFO:tensorflow:loss = 0.136487, step = 901 (0.634 sec)\n",
      "INFO:tensorflow:global_step/sec: 121.05\n",
      "INFO:tensorflow:loss = 0.144933, step = 1001 (0.826 sec)\n",
      "INFO:tensorflow:global_step/sec: 118.257\n",
      "INFO:tensorflow:loss = 0.103951, step = 1101 (0.848 sec)\n",
      "INFO:tensorflow:global_step/sec: 118.136\n",
      "INFO:tensorflow:loss = 0.133236, step = 1201 (0.845 sec)\n",
      "INFO:tensorflow:global_step/sec: 112.046\n",
      "INFO:tensorflow:loss = 0.060983, step = 1301 (0.896 sec)\n",
      "INFO:tensorflow:global_step/sec: 99.9212\n",
      "INFO:tensorflow:loss = 0.0838628, step = 1401 (0.997 sec)\n",
      "INFO:tensorflow:global_step/sec: 115.121\n",
      "INFO:tensorflow:loss = 0.118691, step = 1501 (0.868 sec)\n",
      "INFO:tensorflow:global_step/sec: 96.8269\n",
      "INFO:tensorflow:loss = 0.179758, step = 1601 (1.038 sec)\n",
      "INFO:tensorflow:global_step/sec: 99.8103\n",
      "INFO:tensorflow:loss = 0.0996531, step = 1701 (0.998 sec)\n",
      "INFO:tensorflow:global_step/sec: 128.677\n",
      "INFO:tensorflow:loss = 0.097964, step = 1801 (0.775 sec)\n",
      "INFO:tensorflow:global_step/sec: 124.224\n",
      "INFO:tensorflow:loss = 0.086759, step = 1901 (0.806 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 2000 into ./checkpoints_tutorial17-2/model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 0.0712585.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.estimator.estimator.Estimator at 0x7f0026148f98>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.train(input_fn=train_input_fn, steps=2000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 评估\n",
    "\n",
    "模型一旦被训练好，我们可以在测试集上去评估它的性能。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Starting evaluation at 2017-11-17-12:08:18\n",
      "INFO:tensorflow:Restoring parameters from ./checkpoints_tutorial17-2/model.ckpt-2000\n",
      "INFO:tensorflow:Finished evaluation at 2017-11-17-12:08:18\n",
      "INFO:tensorflow:Saving dict for global step 2000: accuracy = 0.9761, global_step = 2000, loss = 0.0760049\n"
     ]
    }
   ],
   "source": [
    "result = model.evaluate(input_fn=test_input_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'accuracy': 0.97610003, 'global_step': 2000, 'loss': 0.076004863}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification accuracy: 97.61%\n"
     ]
    }
   ],
   "source": [
    "print(\"Classification accuracy: {0:.2%}\".format(result[\"accuracy\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 预测\n",
    "\n",
    "模型也可以用来在新的数据上进行预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "predictions = model.predict(input_fn=predict_input_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./checkpoints_tutorial17-2/model.ckpt-2000\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([7, 2, 1, 0, 4, 1, 4, 9, 5])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cls_pred = np.array(list(predictions))\n",
    "cls_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe8VNW5//HPg2IQVBCxU86NWECjmGCvV5EoFpQYxViI\nsUQxlngDSYwSTJAYbPwsV1FfoglWEBGJimJXQISASrGAokEvIqJGiWJ7fn/MXjN7Tp19ph++79eL\n19kzs8uj68w6z1p77bXM3RERkdy0KncAIiLVRJWmiEgCqjRFRBJQpSkikoAqTRGRBFRpiogkoEpT\nRCQBVZoiIgmo0hQRSWDdfA7u1KmT19TUFCiU6jBnzpyV7r5pueMoFZVxy6cyTiavSrOmpobZs2fn\nc4qqY2bvlDuGUlIZt3wq42TUPBcRSUCVpohIAqo0RUQSUKUpIpKAKk0RkQTyunsu0lxXXnklAF98\n8QUAr7zyCgATJkyos+/ZZ58NwF577QXAySefXIoQReqlTFNEJAFlmlJSxx9/PADjx4+v93Mzq/Pe\nTTfdBMC0adMAOOCAAwDo2rVrMUKUMnrjjTcA2H777QG49tprATj33HPLFlNtyjRFRBJQpilFF7JL\naDjD3GGHHQA49NBDAXjrrbfSn02ePBmAxYsXAzBu3DgALrroosIHK2U1d+5cAFq1SuVzW2+9dTnD\nqZcyTRGRBJRpStGE55kfeOCBOp/ttNNOQCaL7NSpEwAbbLABAF999VV63z322AOAl19+GYCPPvqo\nSBFLuc2bNw/I/B4MGDCgnOHUS5mmiEgCJc80wzi8W265BYCtttoq/VmbNm0AOPHEEwHYYostAOje\nvXspQ5QC+b//+z8A3D39Xsgwp06dCsCWW25Z77FhHCfAokWLsj474ogjChqnlN+rr74KwHXXXQfA\nKaecUs5wGqVMU0QkgZJnmkOGDAFg6dKlDe4TxuVttNFGAPTs2bMg1+7SpQsAQ4cOBaB3794FOa/U\n78gjjwQyd70BNtxwQwA6duzY6LH33ntvejvevykt0+uvvw7A6tWrgewRF5VGmaaISAKqNEVEEih5\n8/zWW28FMsNH4k3vhQsXApkBrk8//TQAM2fOBDKPzb377rsNnr9169ZAZghLuBkRP09opqt5Xhrd\nunXLed8rrrgCyDxOFxeGHoWf0nKMGjUKSC29AZX93VSmKSKSQMkzzYMPPjjrZ1x4hC74+OOPgUzm\nGf76vPTSSw2e/3vf+x6QeeA/PJ4HsGrVKgC22WabZsUuxTNlyhQAhg0bBsCaNWvSn22++eYAXH75\n5QC0bdu2xNFJMcRvBofvdPjetmvXrhwh5USZpohIAhX9GOXGG28MwEEHHZT1fn1Zam33338/kMlW\nAXbeeWcABg4cWKgQpUDCI5fxDDMIw0/ClHDSMjzzzDN13tt008pfbl6ZpohIAhWdaTbHihUrABg8\neDCQ/Qhf6C9ramC1lM7RRx8NZB6rDAYNGpTeHjFiREljktIIS5zEhQdPKpkyTRGRBFpcpnnDDTcA\nmYyzQ4cO6c/CnTkpvzB+dvr06UCmLzP0aV188cXpfcM0YdIyzJgxA4CxY8em39t1110BOOSQQ8oS\nUxLKNEVEEmgxmebzzz8PZMbyBQ8++GB6O0xLJuUXJpdduXJl1vthWkCNpW25nnjiCSB7ZEsYox2m\nh6xkyjRFRBJQpSkikkCLaZ4//PDDQGbuxT59+gCw1157lS0mqSusCRQejQ0OPPBAAP70pz+VOiQp\nsTBZT9xPf/rTMkTSPMo0RUQSqPpM84svvgDg0UcfBTITdlx66aVAZqo4KZ/46pEjR44E6s7G3qtX\nL0DDi1qy5cuXA/Dcc88B2ZPpHHPMMWWJqTmUaYqIJFD1mWaYtDb0kR122GEA7L333mWLSbJdddVV\n6e1Zs2ZlfRYeo1RfZst3++23A/DBBx8Ame9qtVGmKSKSQFVmmmHCWoA///nPALRv3x6ASy65pCwx\nScOuvvrqBj8Lj72qL7Ple+edd7Jeh6kfq40yTRGRBKoq0wx3Yc8777z0e9988w0A/fr1AzQus9qE\nMs1llENoTYR9v/76awA+/fTTOvuGR/Suueaaes+1zjrrpLf/+te/AlpGo9geeuihrNdHHHFEmSLJ\njzJNEZEEVGmKiCRQFc3zb7/9FsjMhPL222+nP+vevTuQuSEk1SWs25SL4447DoAtt9wSyAxdueee\ne/KKIax2GZ/DUwonDGYP5VXtlGmKiCRQFZnmkiVLgMyKhXFhOIvmX6xc4SYdwKRJk5p9nvvuu6/J\nfcJNolatsvOBo446CoDevXvXOWbfffdtdkzStAceeADI3LQNs7RX6+qiyjRFRBKo6EwzDIbt27dv\n1vtXXnllertahy2sTSZOnJjeHjVqFFB3wo5g4cKFQOP9lKeddhoA3bp1q/PZT37yEwB69OjRvGCl\nYP7zn/8A8Mgjj2S9H6aBiw/7qibKNEVEEqjoTHPMmDFA3cev4n0hZlbSmCQ/ua5rfddddxU5Eim2\n0L8cVoTt378/AOeff37ZYioEZZoiIglUZKYZxnVdf/31ZY5ERJorZJphnfOWQpmmiEgCFZlphjXM\nP/vss6z3w9M/mkZMRMpFmaaISAKqNEVEEqjI5nltYaXCJ554AoCOHTuWMxwRWYsp0xQRSaAiM83f\n//73WT9FRCqFMk0RkQTM3Zt/sNmHwDtN7tiydHP3TcsdRKmojFs+lXEyeVWaIiJrGzXPRUQSUKUp\nIpJAo5WmmW1iZvOif8vN7L3Y6/WKEZCZ9YxdY56ZfWZmv2rimNPN7MNo/0Vm9os8YxhnZkc3sc/v\nYjEuMLNvzKx9PtcthzKVcTcze9rMFkb/7xot3+iYcpTxKWb2qpm9YmYvmNkP8rlmuZSjjKPr3hHK\nLMf9y1HGO5rZDDNbY2YX5HRid8/pHzAc+E097xvQKtfzJPkHtAZWAJ2b2O90YHS0vQWwEuhUa591\nE1x3HHB0gv2PAR4rxv+DUv4rVRkDWwG9ou2NgCXAdpVWxsA+QIdo+0jghXKXUbWUcXTOA4DdgXk5\n7l+OMt4c6A1cDlyQy3mb1Tw3s+5RlnAnsADoYmafxD4faGa3Rtubm9lEM5ttZrPMbM8ElzoEWOTu\ny3I9wN2XA0uBrmY2wsz+ZmYvALeb2bpmdnUUxytmdnoUYysz+18ze83MHgc6JYgR4ATg7oTHVLRi\nlrG7v+/u86LtfwOvAVvnGlupytjdX3D38N88E+ica4zVoNjfY3d/BljVnNhKWMYfuPts4JtcY8tn\ncPsOwCnuPtvMGjvPtcAod59pZjXAFGAnM9sDONXdz2rk2IEkrIzMrDvQDXgrFuf+7v6lmQ0GVrj7\n7mb2PWCmmT0G7An8F9CTVBa0ELgpOt9lpDKMhxu43gZAH+CMJHFWiaKXsZl9H9gJeCnXoEpdxpHT\ngEca+bxaleJ7nFiZyjgn+VSaS6Iauil9gO0tsyzFxma2vru/CLzY0EFm1gY4HLgwx3hONLMDgTXA\n6e7+SXTNB939y2ifvkAPMxsYvW4PbAvsD9zt7t8By8zs6XBSd/9DE9ftDzzj7p/mGGc1KXYZbwTc\nD5zr7p/ncJ2ylLGZ9QFOBlriWr9FLeNmKNf3OGf5VJqrY9vfkeoTCdrEtg3Y3d3rX36wYYcDL7r7\nyhz3v9Pd6+vIjcdpwGB3fyK+g5kdkzC2uIHA3/M4vpIVrYwtdQNiIjDW3SfneFjJy9jMegFjgB+7\n+8fNOUeFK/b3OKlyfY9zVpAhR1HN/rGZbWtmrUjdGAmmAeeEF9EvYS7q9BOa2flmlk8zYCowODRD\nzGx7M1sfeBY4PuoT2ZpUB3aTzGxjYG/goTxiqgqFLGNLpQ63k7pBcG2tzyqmjKNm6ATgZ+6+OI+Y\nqkKRvsd1VFIZN0chx2n+ltR/zHQgfuPmHGCfqMN2IVHfn5ntYWY31XciM9sQ+G9gUq2PegAf5RHj\nGOBNYJ6ZzQduJJVtTwDeJdUHMhZIL2piZpeZWb8GzvcT4BF3/yKPmKpJocr4AFJ/FA+xzNCXH0ef\nVVIZDwc6AmOiGAvZDK1UhfwejweeA3qa2TIz+3n0UcWUsZl1NrNlwHnA8CjOto1dvKoeozSzfwD9\n3T3nO11SXVTGLV+1l3FVVZoiIuWmxyhFRBJQpSkikoAqTRGRBFRpiogkkNcaQZ06dfKampoChVId\n5syZs9LXolm9VcYtn8o4mbwqzZqaGmbPzuUJrJbDzNaqZQFUxi2fyjgZNc9FRBJQpSkikoAqTRGR\nBFRpiogkoEpTRCQBVZoiIgnkNeSoVFavTs0/OmTIEABuuikzE1Xv3r0BGD9+PADdunUrcXQisjZR\npikikkBVZJrvv/8+ALfccgsA66yzTvqzMCj3oYdSk6f/6ldNLqEtFeCf//wnAAMGDABg6dKlzT7X\nY489lt7u0aMHAF26dGl+cFI24Xt81FFHAXDdddcBcPbZZ6f3iX//y0GZpohIAhWdaX744YcADBo0\nqMyRSKFNnToVgDVr1uR9rsmTM+uy3XbbbQDcc889eZ9XSuejj1KrX8QzSoBzzz0XgNNOOy393vrr\nr1+6wOqhTFNEJIGKzDSvvTa1QOGkSal11V566aUmj3nuuecACMt37LLLLgDsv//+xQhRmumbb1LL\nwjz88MMFO2cYQQFw9dVXA5kRF+3atSvYdaR4nn32WQDee++9rPdPOOEEANq0aVPnmHJRpikikkBF\nZpoXXJBaKz7JXbKJEydm/ezatSsA9913X3qfH/3oR4UKUZrpqaeeAmD69OkA/Pa3v837nKtWrUpv\nL1iwAID//Oc/gDLNShbvzx4xYkS9+5x88skAmFlJYsqFMk0RkQRUaYqIJFBRzfN+/foBmZs53377\nbZPHdOrUCcg0w955JzUh89tvvw3Abrvtlt73u+++K1ywkrNXX301vT1w4EAAunfvDsBFF12U9/nj\nQ46kerzyyivp7fCwQ7Duuqmq6bDDDitpTLlQpikikkDZM81nnnkmvf3aa68BmU7fhm4EnXXWWent\nvn37AtC+fXsAnnzySQAuu+yyOsfdeOONQN0BtFJc8bIIN2jGjRsHwAYbbNDs84YbQPHfoUq6YSCN\nCzdt63PIIYeUMJJklGmKiCRQtkwzTNAQ+rgAVq5cWe++YfjQscceC8Af//jH9Gdt27bN2jdMDTdm\nzJg65xw6dCgAX375JZCZ3KN169bN+4+QRk2YMAHIHsge+jLjfc3NFYapxLPLAw88EIAOHTrkfX4p\nrngLIVhvvfUAGDlyZKnDyZkyTRGRBMqWaX799ddAw9klZB6BvPfee4HMnfLGhEwz3JW98MIL05+F\nR+tCxhmmn9pmm20SxS65CRNDh//vUJj+5NBKueuuu4DMnVaAiy++GFDroZKFBxtmzJhR57PQcuzV\nq1dJY0pCmaaISAJlv3ten9DfNXbsWCC3DLO2kEXeeeed6fdmzZpVgOikKZ9++ikAM2fOrPPZ4MGD\n8z7/zTffDGSmDuzZs2f6s4MOOijv80txNTYBTzWMbFGmKSKSQNkzzfqe+nnxxRfzPm94qij+FFDt\nJ43CXfgwZlAKI0zEsGzZMiAzvVehLFmyJOv1TjvtVNDzS3HVl2mG0Q6FaIkUmzJNEZEEVGmKiCRQ\ntuZ5WLu8WCvLhVXt5s6dm36v9uOZl156aVGuvbbbcMMNgcywkfiEHeHRx44dOyY+74oVK4DMUKZg\nn332aVacUlrPP/88kBkqFhceg+7cuXNJY2oOZZoiIgmULdOcMmVKQc8Xhp8sXLgQaPwxrDCESQOg\niyOsFhgemQyPUwIcfvjhQPZDB/WZP39+ejvc+AnT/tWelKNVK/3trwZhxclwQzaukifoqE2/bSIi\nCZR9yFGhhOnHbrjhhgb3qampAeCOO+4AMhOBSHEMHz4cyM4sQgsjPlFLfTbddNP0dsgsG3rk9tRT\nT80nTCmR2n3R8UlVzjzzzFKH02zKNEVEEqj6TDMskREmMG5MeNxuv/32K2pMktKjRw8ge0XQMJqh\n9gD12sI0gHGDBg0C6j6MEPpQpTKFhxxq3zWP3ykvxFSBpaJMU0QkgbJlmo0tnvbII49kvT7jjDMA\neP/99xs8Ty7LHBT6jr0kt+uuu2b9TOL73/9+ve/Hx4H+4Ac/aF5gUjRhKrjad8379+9fjnDypkxT\nRCQBVZoiIgmUrXke5s0Ls6jHhQHQtR+xrO+Ry9C8z2XlSqluoXlXu5mnJnllC4Pag/BwyQUXXFCO\ncPKmTFNEJIGyZZoDBgwAYNSoUen3GlsvqCnhr1cY5nLLLbcAsOWWWzb7nFJZws0+rW1eXaZOnZr1\nukuXLkBmko5qo0xTRCSBsmWaYdXIsNIkwKRJkwAYPXp04vP94Q9/ADJrmUvLE9arDzSovbKFFWcX\nL16c9X6bNm2A6p0wR5mmiEgCZX+MMqxtHt/u27cvkFl1MEwofOSRRwLwy1/+Mn1MuJMaX5FQWqaw\nOmmY6GHYsGHlDEeaEKbsC49ILliwAIBtt922bDEVgjJNEZEEyp5p1ufQQw/N+ikCmYzl17/+NaA1\nzitdGDsdpm0Mox5++MMfli2mQlCmKSKSQEVmmiL1CX3bUl222morAG677bYyR1IYyjRFRBJQpSki\nkoAqTRGRBFRpiogkoEpTRCQBVZoiIglY7QldEx1s9iHwTuHCqQrd3H3TpndrGVTGLZ/KOJm8Kk0R\nkbWNmuciIgmo0hQRSUCVpohIAo1Wmma2iZnNi/4tN7P3Yq/XK1ZQZtbPzF43s8VmNiSH/UfEYnvV\nzA7P8/rPm1mvJvZpY2YTohhnmFnXfK5ZLuUq4+ja65rZK2Y2KYd9S17GsX2PNzPPdf9KU8bv8R1m\n9qGZzctx/9PD/ma2yMx+kef1x5nZ0U3ss2P0/V1jZjktj9nohB3u/hHQKzr5cOBzd7+y1kWN1A2l\n73K5YFPMrDVwPfDfwHJgtpk96O5vNHHoFe4+2sx2Ap4ys808dpfLzNZ1928KEWPkTGC5u3c3s5OA\nvwAnFvD8JVGOMo65EJgPtM1x/1KXMWa2ETAYmF3I85ZSGcv4NuAG4OYEx9zp7heY2RbAfDOb7O7p\nFReLUMYrgXOBY3M9oFnNczPrbmYLzexOYAHQxcw+iX0+0MxujbY3N7OJZjbbzGaZ2Z5NnH5PYJG7\nv+Pua4D7gP65xubu8wEDNo7+0txoZrOAkWa2gZndHsUx18yOjGJsa2bjo79u9wNtcrhUf+COaPs+\n4Me5xlgNilzGmFk34BBgbNLYSljGACOjf2uSxlnpil3G7v4MsKo5sbn7cmAp0DVqZfzNzF4Abo9a\nKFdHcbxiZqdHMbYys/81s9fM7HGgUw7X+cDdZwM5V8T59GnuAFzj7j2B9xrZ71pglLv3Bo4DQiHs\nYWY31bP/1sC/Yq+XRe/lxMz2Br5091BYWwJ7uvtQYBjwqLvvDhwEXGVmbYBfAR+7ew9gBLBr7Hxj\nG2iWpeN096+A1WbWIdc4q0SxyhhgNDAESDzmrVRlbGa7AZu5+9Tan7UgxSzjZjOz7kA34K1YnAe7\n+0mkWnkrojLeDTjHUt1jxwL/BfQETgX2jp3vMjPrV4jY8plPc0lUQzelD7C9Zdaq3tjM1nf3F4EX\n87h+bUPM7OfAZ8DxsffHx5ocfYHDzOx30es2QFdgf2AUgLvPNbMF4WB3P7WAMVabopRx1M/0L3ef\nZ2Z9EsRTsjI2s1bAVVRhl0tClfY9PtHMDiSV2Z/u7p9E13zQ3cNypH2BHmY2MHrdHtiWVBnfHf0u\nLDOzp8NJ3f0PhQown0pzdWz7O1LNpSDe9DFg9ygby8V7QJfY6840/hcwuMLd61v7Nx6nAUe7+5L4\nDrFfhCRCnMst1Znezt0/aeKYalOsMt4bGGBmR0Xn2cjM7nD3QU0cV8oy7kAqY3kuOnYL4GEzO9zd\n5yY9WQUrVhk3153uXt8NmdplPNjdn4jvYGbHFDWySEGGHEU1+8dmtm30Fzoe/DTgnPCigaZu3Eyg\np5l1M7PvkWoKTI6OHRX6qJppKqlO3xBLaKI9C/wsem8XYMcczjUZCF/y44DH8oir4hWyjN19qLt3\ndvca4CTgsVBhVkoZu/sqd+/k7jVRnLOBfi2swsxS4O9xg8zsfDM7q/mRMhUYbGbrRufb3szWJ1XG\nx0d9m1sDB+RxjQYVcpzmb0n9x0wn1Q8ZnAPsE3XYLgTOgIb7Qtz9a+A84HFgITDO3V+PPt6Z1B31\n5roUaGepISsLgOHR+9cDm5jZIuASIP3FaKRP82ZgSzNbTKq/7KI84qoWBSnjJlRSGa+NClbGZjYe\neI5UErQs6loB6AF8lEeMY4A3gXlmNh+4kVSreQLwLql6YywwIxZLvX2aZtbZzJaRqnOGR3E2Opqj\nap49t1Qb6RF31xKVLZTKeO1gZv8A+hd6eFipVE2lKSJSCfQYpYhIAqo0RUQSUKUpIpJAPuM06dSp\nk9fU1BQolOowZ86clWvTrN4q45ZPZZxMXpVmTU0Ns2dX7TwGzWJma9WyACrjlk9lnIya5yIiCajS\nFBFJQJWmiEgCqjRFRBJQpSkikoAqTRGRBFRpiogkkNc4TRGRYvj4448BePfddxvcp1u3bgBcc801\nAOy0004AbLfddgDssssuRYlNmaaISAIVlWmuWLECgOOOOw6AvfdOrYt05plnAqknFwrh008/BeDZ\nZ58F4NBDU9M3tm7duiDnF5FkpkyZAsBDDz0EwNNPPw3Am2++2eAx22+/PQBLly4FYM2a7AVDv/uu\n0CtOpyjTFBFJoOyZZui7ANhxx9SyLSET3HzzzYHCZ5g//OEPAVi5MrUGfXjudtttty3IdSR3//73\nvwH43e9Si0cuWJBaJHLatGnpfdQCaBmWLEmtdXfDDTcAcPPNN6c/++KLLwBIMin666+/3vRORaBM\nU0QkgbJlmiHLC/2XAB99lFpr6ZxzUoveXXfddQW95ogRIwB4++23gcxfOmWYpTdu3DgALr74YqDu\nXdKQgQJssskmpQtMimbZstQ6baNH17cKc+522GEHIHO3vNSUaYqIJFC2TPOf//wnkLlLFjds2LCC\nXWf+/Pnp7SuvvBKAY45JLed8/PHHF+w6kpuQbfz6178GMi2O1EKUGeeem166nOuvvx6Ajh07liJE\naYZQjpDJJPfdd18gMzplvfXWA6B9+/YAbLDBBuljPv/8cwB+/OMfA5ksco899gBg1113Te+7/vrr\nA9CuXbsC/1fkRpmmiEgCqjRFRBIoefM8DGC///7763x22223AbDppvkvzxKa5YccckidzwYMGADA\nhhtumPd1JJnQRRJu+jXknnvuSW8/8sgjQOamUWi6h+aelM/q1auB7O/Zyy+/DMCkSZOy9t1rr70A\nmDt3LpA9lDDcCOzcuTMArVpVbj5XuZGJiFSgkmea//M//wNkhpyEgeYAP/3pTwt2neeffx6A5cuX\np9879dRTATjppJMKdh1p2jvvZNawGjt2bNZnYVKF8CDD448/Xuf48FBCyFJPPPFEALbYYovCBys5\n+eqrrwD42c9+BmSyS4CLLroIgD59+tR7bH0Pq3Tt2rXAERaPMk0RkQRKnmmGoSXh59Zbb53+LJ8+\nqvAY1siRI4HMo1rxoSyhz1RKa968eentMGh9//33B+CZZ54B4MsvvwTgrrvuAuAvf/lL+pjFixcD\nmVZD//79gUxfp4YilU4YGhS+Z2GCjfh9iCFDhgDQtm3bEkdXGso0RUQSKPuEHWFKKIC+ffsC0KFD\nBwDOPvvsJo8Pg+PDz5kzZ2Z9Xsh+Umme+JRdIfMPg9uDNm3aAPCLX/wCgAkTJqQ/CxM9hMkcQgaj\nu+elF+6IX3755UBmIuDnnnsuvU8YvN5SKdMUEUmg5Jnm+eefD8CTTz4JwPvvv5/+LPRvhYziwQcf\nbPJ8Yd/aj+Fts802QKbvRcrn7rvvrvPeP/7xDwCOPvroeo8J0/XVZ8899wSyH8OT0pg+fXrW6/B4\nYxhfuTZQpikikkDJM80f/ehHALz66qtA9p3VRx99FIBRo0YBsNlmmwEwaNCgBs938sknA7Dzzjtn\nvR+WyggZp5TPCSeckN4OrYeXXnoJgNdeew3I/D488MADQPbk1KGPO7wXpvQLZd+zZ8+ixS7Z4n3N\nkBnBcOmll6bfO+qoo4DsSTZaEmWaIiIJqNIUEUnAkqzJUVvv3r29sQ77UnjrrbeATDO8V69eADz2\n2GNAYSb/iDOzOe7eu6AnrWCFKONVq1alt0M5hUcjG7qRF58AIjyocMQRRwDwxhtvAJlVSm+66aa8\n4qtNZdyw2g+n1GedddYB4KyzzgIyc2L+61//AqB79+5AZk2wuLBGVJjco1g3mPIpY2WaIiIJlH1w\ne77+9Kc/AZm/fOEmUqEzTGm++GOO48ePB+DYY48F6mac5513HgB//etf08eEge9hSr/wiOXUqVOB\nzOB30I2/YvvNb34DwFVXXdXgPt9++y2QaSGEn0mEm8AHHnggkD1VYLkp0xQRSaAqM82QrQDccccd\nAGy00UaAVi6sdGG6sDB0JUzQEYYVhZZDyC7jLrnkEgAWLVoEZIYvhWMg8/sgxREenwyryIZp+r7+\n+uv0PmEdqJBxNkeYrDx81+MrT4bJqMtFmaaISAJVmWmGAbVxhx9+OJA9qbFUrpBxNjRRbX3CKoRh\nFdGQaT711FPpfcKdek0XVxzhzvhuu+0GZEYyxD3xxBNAJvscPnw4ALNmzUp8vdDXPWfOnMTHFosy\nTRGRBKo+0wxrH4e7etLyhf60yZMnA9l3VsMa6cOGDSt9YALAwQcfnPU6PCodMs3WrVsDmeVnAM44\n4wwArrnmGiDT112JlGmKiCSgSlNEJIGqap6Hx+XiK0yGVQx1A2jtEdbEHjp0KJC9vna46TBw4EAA\ntttuu9IGJ3WEFRnCKpXhBlGYrQrgzTffBDIrMNQWX0us3JRpiogkUJWZZnyygH79+mXt89lnnwGZ\nuReraT1lSSZMzvLnP/85/V64Ifj73/8egHHjxgGZ4UpSej169AAyQ8XuvffeOvvEh40BrLtuqmoK\nQwnjj9XY76NRAAAG3klEQVSWmzJNEZEEqirTrE/4ixQyijBkITx2pcfqWr5TTjklvT1mzBgAJk6c\nCGT6ymrP7C+lE7L80aNHA5nWYHzA+gcffABATU0NkCnT0EddSZRpiogkUPWZ5i233ALArbfeCsDp\np58OZCZ3kJYvPg3gtGnTgMx63GGCiUoeLL22CCNdpkyZAsDf//739GczZswAMpllmBquEinTFBFJ\noKoyzeuuuw6AP/7xj+n39t9/fwDOPvtsADbeeGMA1ltvvRJHJ5UgjJYIy2WERy0XLlwIaOXKShJW\nE629XemUaYqIJFBVmeZ+++0HwJNPPlnmSKTShUmOd9llFwAWL14MKNOU/CnTFBFJQJWmiEgCVdU8\nF8lVWDPq7bffLnMk0tIo0xQRSUCVpohIAqo0RUQSsLDaW7MONvsQeKdw4VSFbu6+adO7tQwq45ZP\nZZxMXpWmiMjaRs1zEZEEVGmKiCTQaKVpZpuY2bzo33Izey/2uqgzYpjZumb2iplNymHfEbHYXjWz\nw/O89vNm1ivHfY83M891/0pTrjI2swvNbEH079wc9j/dzD6M4lpkZr/I8/rjzOzoJvbpaGaTo9/D\nF82sKp/BLGMZL4u+j/PM7MUc9i9HGfcxs09j/z/+0NR5Gx3c7u4fAb2ikw8HPnf3K2td1Ej1jX7X\n1MUSuhCYD7TNcf8r3H20me0EPGVmm3msw9bM1nX3bwoZoJltBAwGZhfyvKVUjjKO/sAMAnoD3wCP\nmdkUd29qJPqd7n6BmW0BzDezye6+MnbeQpfxJcCL7n6Ume0I/D/gkAKevyTK/D3ez90/SbB/qcsY\n4Cl3b7RyjWtW89zMupvZQjO7E1gAdDGzT2KfDzSzW6Ptzc1sopnNNrNZZrZnDufvRuqXc2zS2Nx9\nPmDAxtFfmhvNbBYw0sw2MLPbozjmmtmR0fXamtn46K/b/UCbHC83Mvq3Jmmcla7IZdwDmOnuX7j7\n18CzwDG5xubuy4GlQNeolfE3M3sBuD1qoVwdxfGKmZ0exdjKzP7XzF4zs8eBTjlcqifwZHTNBcB2\nZrZJrnFWumJ/j/NRwjJOLJ8+zR2Aa9y9J/BeI/tdC4xy997AcUAohD3M7KYGjhkNDAES39o3s72B\nL919VfTWlsCe7j4UGAY86u67AwcBV5lZG+BXwMfu3gMYAewaO99Yq6fpbWa7AZu5+9SkMVaRYpXx\nq8ABlmr+tgMOA7rkGpSZdQe6AW/F4jzY3U8CzgRWRGW8G3COmXUFjgX+i1RFeCqwd+x8l5lZ9rKm\nKS8DA6J99gI6R/9akmJ+jx140szmmNlpSYIqYRkD7GtmL5vZw5ZDF0w+z54vcfdcmqV9gO0ts+zu\nxma2vru/CNTp54j6IP7l7vPMrE+CeIaY2c+Bz4DjY++PjzU5+gKHmdnvotdtgK7A/sAoAHefa2YL\nwsHufmo9MbYCrgJOTBBfNSpKGbv7fDO7GpgGfA7MBb7N4TonmtmBpDL70939k+iaD7r7l9E+fYEe\nZjYwet0e2JZUGd8d/S4sM7OnY/E01I91GXCtmc0jVYG+nGOc1aQoZRzZ093fi5raj5vZInef3sR1\nSl3GLwE17v551PKcSKqCblA+lebq2PZ3pJrEQbx5a8Du7v5VjufdGxhgZkdF59nIzO5w90FNHHeF\nu49uIk4Djnb3JfEdYr8IuepA6q/Zc9GxWwAPm9nh7j436ckqWLHKGHe/GbgZwMxGAYtzOOxOd7+g\niTgNGOzuT8R3MLOcm/+xGD8l1fca/lAuBVraDCDFLOP3op/LzexBYHegqUqzHGUcth+KuvM6NNYP\nW5AhR1HN/rGZbRv9csWDnwacE17U19Stda6h7t7Z3WuAk4DHQoVpZqNCP2QzTQXSd2rNLDTDnwV+\nFr23C7BjEzGucvdO7l4TxTkb6NfCKswshSzjaJ/Nop81wFHAPdHr883srDxCnQoMNrN1o/Ntb2br\nkyrj46N+r62BA3KIsYOZtY5e/hKY5u6rGzummhWyjC11/2CDaLsdqXsU86PXlVTGW8S29wS+aerG\nVSHHaf6W1H/MdGBZ7P1zgH2iDtuFwBlRgI31hTRkZ2B5HjFeCrSz1DCIBcDw6P3rgU3MbBGpO6bp\nyq+hPs21VCHLeFK07yTgLHf/d/R+D+CjPGIcA7wJzDOz+cCNpFpUE4B3gYWkbjDOCAc00t/1A2Ch\nmb0OHExqREdLV6gy3hJ4wcxeBmYBD7j7tOizSirjgZYa9jYPuIbsrr16Vc1jlJZqBz/i7oeWOxYp\nHjP7B9C/CMNKpEJUexlXTaUpIlIJ9BiliEgCqjRFRBJQpSkikoAqTRGRBFRpiogkoEpTRCQBVZoi\nIgn8f6S1hmwDdPmXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0026645d68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_images(images=some_images,\n",
    "            cls_true=some_images_cls,\n",
    "            cls_pred=cls_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结\n",
    "\n",
    "这个教程展示给我们如何在TensorFlow中去使用Estimator API。它被认为可以使训练和使用模型变得更容易，但它似乎有几个设计问题：\n",
    "\n",
    "* Estimator API是复杂的，不一致的并且有些让人困惑。\n",
    "* 错误报告相当长并且经常难以理解。\n",
    "* 当你想用一个训练好的模型去预测新的数据，每一次 TensorFlow的图都会被再次创建并且加载checkpoint一次。有一些模型非常大所以这可能会会是一个很大的开销。一个更好的方法是，如果检查点在磁盘上发生了变化，则只重新载入模型。\n",
    "* 目前还不清楚如何获得训练好的模型，例如画出神经网络的权重。\n",
    "\n",
    "似乎Estimator API可以更简单，更容易使用。对于小的项目你可能发现它太复杂了并且困惑付出的努力是否值得，但。如果你有一个非常大的数据集，并且如果你在许多机器上进行训练，那么Estiator API是有用的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 练习\n",
    "\n",
    "下面使一些可能会让你提升TensorFlow技能的一些建议练习。为了学习如何更合适地使用TensorFlow，实践经验是很重要的。\n",
    "\n",
    "在你对这个Notebook进行修改之前，可能需要先备份一下。\n",
    "\n",
    "* 对每个模型运行另10000训练的迭代。\n",
    "* 在优化之前和经过1000，2000，10000次迭代后，打印测试集上印分类的准确率。\n",
    "* 在Estimator里改变神经网络的结构。你需要删除checkpoint文件吗？为什么？\n",
    "* 改变input-functions的batch-size。\n",
    "* 在之前的教程中我们画出被错误分类的图像。请在这也完成。\n",
    "* 改变Estimator，用独热编码标签代替整数类别。\n",
    "* 改变input-functions去加载图像文件来代替numpy数组。\n",
    "* 你能找到一个方法来画出神经网络每一层的权重和输出吗？\n",
    "* 列出5个你喜欢和不喜欢Estimator API的点。你有什么改进的建议？也与你应该给开发者一些建议？\n",
    "* 向朋友解释程序是如何工作的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## License (MIT)\n",
    "\n",
    "Copyright (c) 2016-2017 by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n",
    "\n",
    "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n",
    "\n",
    "The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n",
    "\n",
    "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}