19_Hyper-Parameters_CN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow Tutorial #19\n",
    "# 超参数优化\n",
    "\n",
    "by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)/[GitHub中文](https://github.com/Hvass-Labs/TensorFlow-Tutorials-Chinese)\n",
    "/ [GitHub](https://github.com/Hvass-Labs/TensorFlow-Tutorials) / [Videos on YouTube](https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ)\n",
    "\n",
    "中文翻译[ZhouGeorge](https://github.com/ZhouGeorge)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 介绍\n",
    "\n",
    "在利用Tensorflow构筑和训练神经网络时你有很多参数需要选择。它们通常被成为超参数Hyper-Parameters。比如，神经网络有多少层是一个超参数，一个层里有多少个节点是另一个超参数，我们选取什么激活函数也是一个超参数，等等。优化方法中也有许多超参数需要选择，例如学习率。\n",
    "\n",
    "一个搜索超参数的方法是手动微调，你设置一个参数然后看它的表现，然后你再修改另一个参数看性能是否提高了。你试着建立一种直觉，关于哪些工作有效并据此指导你的参数搜索。这不仅对人类研究者来说是非常耗时的，而且最优的参数通常与人类的直觉相反，你不会发现它们！\n",
    "\n",
    "另一个搜索超参数的方法是将每个参数的有效范围划分为均匀间隔的值，然后让计算机尝试所有参数值的组合。这被称为网格搜索（ Grid Search）。尽管它全部由计算机执行，它变的相当耗时因为当你增加更多的超参数时，参数组合的数量是指数级增加的。这个问题被称为维度灾难。举一个例子，你有4个超参数需要调整，每个参数有10个有效值，它们一共有10^4个组合。如果你再增加一个超参数，那么组合的数量将达到10^5个。\n",
    "\n",
    "还有一个搜索超参数的方法是随机搜索。代替系统地尝试每一种网络搜索中的超参数组合，现在我们随机地尝试一些参数组合。这就像大海捞针一样，并且当超参数增加时，通过随机采样的方式找到合适的超参数组和的可能性会降到0.\n",
    "\n",
    "这个教程中用了一聪明的方法去搜索超参数，即贝叶斯优化。你需要熟悉Tensorflow，Keras和卷积神经网络，参见教程 #01, #02和 #03-C。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 流程图\n",
    "\n",
    "\n",
    "超参数优化的问题在于评估一组参数的性能开销相当大。因为我们首先要建立相应的神经网络，然后我们还要训练它，最后我们还要在测试集上测试它的性能。在这个教程中，我们用MNIST数据，所以训练可以被很快地完成，但是如果在更复杂的问题上即使用一台非常快的电脑训练可能需要花费几小时，几点天或甚至是几周。所以我们需要一个优化方法使超参数的搜索尽可能有效，只有在绝对必要的时候才评估模型实际的性能。\n",
    "\n",
    "贝叶斯优化的想法是构建另一个模型关于超参数的搜索空间。有一种模型被称为高斯过程。它给我们提供了一个关于性能是如何随超参数变化而变化的估计。当我们评估一组超参数的真实性能时，我们知道它的性能是什么-除了一些噪音。我们可以通过贝叶斯优化器得到在搜索空间中关于超参数选择的新的建议，它可能是我们还没有探索过的，或者是贝叶斯优化器认为这个超参数可以带来很大的提升。然后我们重复这个过程多次，直到贝叶斯优化器建立好关于性能是如何随超参数变变化的模型，那么我们就可以选出最好的参数。\n",
    "\n",
    "算法的流程图大致如下：\n",
    "\n",
    "![Flowchart](images/19_flowchart_bayesian_optimization.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/magnus/anaconda3/envs/tf-test/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6\n",
      "  return f(*args, **kwds)\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import math"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们需要引如Keras的一些模块。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# from tf.keras.models import Sequential  # This does not work!\n",
    "from tensorflow.python.keras import backend as K\n",
    "from tensorflow.python.keras.models import Sequential\n",
    "from tensorflow.python.keras.layers import InputLayer, Input\n",
    "from tensorflow.python.keras.layers import Reshape, MaxPooling2D\n",
    "from tensorflow.python.keras.layers import Conv2D, Dense, Flatten\n",
    "from tensorflow.python.keras.callbacks import TensorBoard\n",
    "from tensorflow.python.keras.optimizers import Adam\n",
    "from tensorflow.python.keras.models import load_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**注意：** 我们用Keras保存和加载模型，所以你必须安装 [h5py](http://docs.h5py.org/en/latest/build.html#install)。你也必须安装 [scikit-optimize](https://scikit-optimize.github.io/)用于超参数的优化。\n",
    "\n",
    "您应该能够在终端中运行下列命令来安装它们：\n",
    "\n",
    "`pip install h5py scikit-optimize`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**注意：** 这个笔记本需要 `scikit-optimize`中的绘制函数，而在撰写本文时，它还没有被合并到官方版本中。如果这个笔记本不能通过上面的指令安装 `scikit-optimize`的版本，你可能需要通过运行以下命令从开发分支中来安装指定的`scikit-optimize`版本 ：\n",
    "`pip install git+git://github.com/Hvass-Labs/scikit-optimize.git@dd7433da068b5a2509ef4ea4e5195458393e6555`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import skopt\n",
    "from skopt import gp_minimize, forest_minimize\n",
    "from skopt.space import Real, Categorical, Integer\n",
    "from skopt.plots import plot_convergence\n",
    "from skopt.plots import plot_objective, plot_evaluations\n",
    "from skopt.plots import plot_histogram, plot_objective_2D\n",
    "from skopt.utils import use_named_args"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "开发环境Python3.6以及各个包的版本："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.4.0'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.0.8-tf'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.keras.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.4'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "skopt.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 超参数\n",
    "\n",
    "在本教程中，我们希望找到一个超参数，它使一个简单的卷积神经网络在为手写的数字进行分类时表现得最好。\n",
    "\n",
    "对于这个示范，我们想要找到以下超参数：\n",
    "\n",
    "* 优化器的学习率。\n",
    "* 全连接层的数目。\n",
    "* 全连接层的节点数目。\n",
    "* 在所有层中‘sigmoid’还是‘relu’激活函数的选择。\n",
    "\n",
    "我们将用Python的包`scikit-optimize` (or `skopt`)来找到这些超参数的最好的选择。在我们开始搜索超参数之前，我们首先需要为这些超参数定义有效的搜索范围或者搜索维度。\n",
    "\n",
    "下面是学习率的收缩维度。它是一个实数（浮点），下界是`1e-6`，上界是`1e-2`。但是，我们用了对数变换来代替直接在这两个边界中搜索，所以我们从搜索数字`k`到搜索`1ek`，这样边界就变成了-6和-2。这比搜索整个指数范围要好。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dim_learning_rate = Real(low=1e-6, high=1e-2, prior='log-uniform',\n",
    "                         name='learning_rate')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面是神经网络全连接层数目的搜索维度。这是一个整数，在这个网络中我们希望至少有1个全连接层，最多有5个全连接层"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dim_num_dense_layers = Integer(low=1, high=5, name='num_dense_layers')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面是每个全连接层节点数目的搜索维度，它也是一个整数并且我们希望在这个网络中它最少有5个节点，最大是512个节点。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dim_num_dense_nodes = Integer(low=5, high=512, name='num_dense_nodes')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面是激活函数的搜索维度。这是一个组合或分类参数，可以是“relu”或“sigmoid”。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dim_activation = Categorical(categories=['relu', 'sigmoid'],\n",
    "                             name='activation')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后我们将所有这些搜索维度合并到一个列表中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dimensions = [dim_learning_rate,\n",
    "              dim_num_dense_layers,\n",
    "              dim_num_dense_nodes,\n",
    "              dim_activation]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用我们通过手工调优找到的一个不错的选择来开始搜索超参数是很有帮助的。但是我们将使用以下参数，这些参数表现得不太好是为了更好地展示超参数优化是有效的：学习率1e-5，单个16个节点的全连接层和relu激活函数\n",
    "\n",
    "注意这些超参数被打包到一个列表中。这就是“skopt”在内部对超参数的工作方式。因此，您需要确保维度的顺序与上面“维度”中给出的顺序一致。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "default_parameters = [1e-5, 1, 16, 'relu']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### log-dir-name 辅助函数\n",
    "\n",
    "我们将记录所有参数组合的训练过程，它们可以用TensorBoard可视化并比较。这些可以通过设置一个公共的上级地址，然后对应每组超参数创建恰当名字的子地址来完成。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def log_dir_name(learning_rate, num_dense_layers,\n",
    "                 num_dense_nodes, activation):\n",
    "\n",
    "    # The dir-name for the TensorBoard log-dir.\n",
    "    s = \"./19_logs/lr_{0:.0e}_layers_{1}_nodes_{2}_{3}/\"\n",
    "\n",
    "    # Insert all the hyper-parameters in the dir-name.\n",
    "    log_dir = s.format(learning_rate,\n",
    "                       num_dense_layers,\n",
    "                       num_dense_nodes,\n",
    "                       activation)\n",
    "\n",
    "    return log_dir"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 加载数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MNIST数据集大约12MB，如果没在文件夹中找到就会自动下载。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting data/MNIST/train-images-idx3-ubyte.gz\n",
      "Extracting data/MNIST/train-labels-idx1-ubyte.gz\n",
      "Extracting data/MNIST/t10k-images-idx3-ubyte.gz\n",
      "Extracting data/MNIST/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "data = input_data.read_data_sets('data/MNIST/', one_hot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在已经载入了MNIST数据集，它由70,000张图像和对应的标签（比如图像的类别）组成。数据集分成三份互相独立的子集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of:\n",
      "- Training-set:\t\t55000\n",
      "- Test-set:\t\t10000\n",
      "- Validation-set:\t5000\n"
     ]
    }
   ],
   "source": [
    "print(\"Size of:\")\n",
    "print(\"- Training-set:\\t\\t{}\".format(len(data.train.labels)))\n",
    "print(\"- Test-set:\\t\\t{}\".format(len(data.test.labels)))\n",
    "print(\"- Validation-set:\\t{}\".format(len(data.validation.labels)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "类型标签使用One-Hot编码，这意外每个标签是长为10的向量，除了一个元素之外，其他的都为零。这个元素的索引就是类别的数字，即相应图片中画的数字。我们也需要测试数据集类别数字的整型值，用下面的方法来计算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data.test.cls = np.argmax(data.test.labels, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们将验证集中的性能作为一个指示（选择在之前没有见过的数据上表现的最好的超参数）。\n",
    "\n",
    "Keras API验证集要是一个元组。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "validation_data = (data.validation.images, data.validation.labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据维度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在下面的源码中，有很多地方用到了数据维度。它们只在一个地方定义，因此我们可以在代码中使用这些名字而不是直接写数字。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# We know that MNIST images are 28 pixels in each dimension.\n",
    "img_size = 28\n",
    "\n",
    "# Images are stored in one-dimensional arrays of this length.\n",
    "img_size_flat = img_size * img_size\n",
    "\n",
    "# Tuple with height and width of images used to reshape arrays.\n",
    "# This is used for plotting the images.\n",
    "img_shape = (img_size, img_size)\n",
    "\n",
    "# Tuple with height, width and depth used to reshape arrays.\n",
    "# This is used for reshaping in Keras.\n",
    "img_shape_full = (img_size, img_size, 1)\n",
    "\n",
    "# Number of colour channels for the images: 1 channel for gray-scale.\n",
    "num_channels = 1\n",
    "\n",
    "# Number of classes, one class for each of 10 digits.\n",
    "num_classes = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用来绘制图片的帮助函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个函数用来在3x3的栅格中画9张图像，然后在每张图像下面写出真实类别和预测类别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_images(images, cls_true, cls_pred=None):\n",
    "    assert len(images) == len(cls_true) == 9\n",
    "    \n",
    "    # Create figure with 3x3 sub-plots.\n",
    "    fig, axes = plt.subplots(3, 3)\n",
    "    fig.subplots_adjust(hspace=0.3, wspace=0.3)\n",
    "\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Plot image.\n",
    "        ax.imshow(images[i].reshape(img_shape), cmap='binary')\n",
    "\n",
    "        # Show true and predicted classes.\n",
    "        if cls_pred is None:\n",
    "            xlabel = \"True: {0}\".format(cls_true[i])\n",
    "        else:\n",
    "            xlabel = \"True: {0}, Pred: {1}\".format(cls_true[i], cls_pred[i])\n",
    "\n",
    "        # Show the classes as the label on the x-axis.\n",
    "        ax.set_xlabel(xlabel)\n",
    "        \n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "    \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 绘制几张图像来看看数据是否正确"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUMAAAD5CAYAAAC9FVegAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHihJREFUeJzt3XmUFNXZx/HvA0LYVQQFFWdOwAVCFBWDu0aBKCogccG4\nEGM0osEtAaNx1xglKBzRE7YD4QQNigKCUVFAEV8EJIIi4wYiCsRlhLggIsJ9/5i5XdUzPXtXVU/7\n+5zjmequ6qpnvPSdp27dxZxziIj80DVIOgARkVygylBEBFWGIiKAKkMREUCVoYgIoMpQRARQZSgi\nAqgyFBEBVBmKiACwS00ObtOmjSssLIwolNzzwQcfUFxcbEnHESeVcf5TGWdWo8qwsLCQZcuW1T6q\neqZ79+5JhxA7lXH+UxlnpttkERFUGYqIAKoMRUQAVYYiIoAqQxERoIZPk0Vqa8SIEQBs3boVgDfe\neAOAxx9/vNyxgwcPBuCoo44C4MILL4wjRPmBU2YoIoIyQ4nYueeeC8C0adMy7jcr3xd2zJgxAMyd\nOxeAE044AYD99tsvihAlQe+++y4ABx54IAAPPPAAAEOGDIk9FmWGIiIoM5QI+GwQKs4IDzroIABO\nOeUUAN5///3UvlmzZgGwevVqAKZMmQLAjTfemP1gJVHLly8HoEGDkrxsn332SSwWZYYiIigzlCzy\n411nzJhRbl/Xrl2BIOtr06YNAC1atADgu+++Sx3bo0cPAF5//XUAPv/884gilqStWLECCP4dDBgw\nILFYlBmKiBBDZuj7kY0fPx6AvffeO7WvSZMmAJx//vkAtGvXDoBOnTpFHZZE4L///S8AzrnUez4j\nnDNnDgDt27fP+FnfDxHgrbfeStt3+umnZzVOSd7KlSsBGD16NAAXXXRRkuEAygxFRIAYMsOhQ4cC\nJRMsVsT3K2vVqhUAXbp0ycq1O3ToAMCwYcOAH+bcdXE644wzgOApMEDLli0BaN26daWfffTRR1Pb\n4fZDyU/vvPMOAFu2bAHSeyAkRZmhiAiqDEVEgBhukydMmAAE3STCt8BFRUVA0PHyxRdfBGDx4sVA\nMPzqww8/rPD8jRo1AoKuGr4RP3wef7us2+R4FBQUVPvYv/3tb0AwLCvMd7HxPyV/DB8+HChZggBy\n47upzFBEhBgyw5NPPjntZ5gfiuVt3rwZCDJF/9fi1VdfrfD8P/rRj4BgoLcf5gWwadMmADp27Fir\n2CU6Tz31FAC33HILANu2bUvt22uvvQC45557AGjWrFnM0UkUwg9R/Xfaf2+bN2+eREhplBmKiJBj\nw/F23313AE466aS09zNllWU98cQTQJBdAhx88MEADBw4MFshSpb4oXvhjNDz3Sz81F2SHxYsWFDu\nvbZt2yYQSWbKDEVEyLHMsDY+/fRTAK644gogfSiYb4+qqsOvxKd///5AMDzPGzRoUGr7rrvuijUm\niYdf6iHMD4jIBcoMRUTIg8zwoYceAoIMcbfddkvt80+qJHm+/+eiRYuAoK3QtxnddNNNqWP9dE6S\nH1555RUAJk2alHrv0EMPBaBXr16JxJSJMkMREepxZvjyyy8DQV8078knn0xt++mjJHl+0s7i4uK0\n9/30beoLmr/mzZsHpPf08H2M/TR+uUCZoYgIqgxFRIB6fJv89NNPA8Hcdz179gTgqKOOSiwmKc+v\neeKHWHonnngiAHfccUfcIUnM/CQtYWeffXYCkVROmaGICPUwM9y6dSsAzz77LBBM1HD77bcDwZRe\nkpzwanZ33303UH726m7dugHqRpPPPv74YwAWLlwIpE+icuaZZyYSU2WUGYqIUA8zQz8ZqG+DOvXU\nUwE4+uijE4tJ0t13332p7aVLl6bt88Px1FaY//7xj38A8MknnwDBdzVXKTMUEaGeZIZ+IlCAO++8\nE4Bdd90VgJtvvjmRmKRi999/f4X7/PBJtRXmv3Xr1qW99lP05SplhiIi5Hhm6J9KXnXVVan3vv/+\newD69OkDqF9hfePLtDpP/X3274/dvn07AF988UW5Y/1Qr5EjR2Y8V8OGDVPb9957L6DlBKI2e/bs\ntNenn356QpFUjzJDERFUGYqIADl6m7xjxw4gmNli7dq1qX2dOnUCggcpUr/4dWmq45xzzgGgffv2\nQNBFY+rUqXWKwa++F55DUbLHd7L25VVfKDMUESFHM8M1a9YAwQpqYb7bhua/y13+4RbAzJkza32e\nxx57rMpj/MOVBg3S/6737dsXCNbeDjv22GNrHZNUbcaMGUDwsNPPap3rqx0qMxQRIccyQ99Js3fv\n3mnvjxgxIrWd64/nBaZPn57aHj58OFB+ogavqKgIqLwd8JJLLgGgoKCg3L5f/vKXAHTu3Ll2wUrW\nfPPNNwA888wzae/76brC3ZtykTJDERFyLDMcO3YsUH4YT7itwcxijUnqprrr4j7yyCMRRyJR8+23\nfoXKfv36AXD11VcnFlNNKDMUESFHMkPfL+nBBx9MOBIRqS2fGfp1kusbZYYiIuRIZujXQP7qq6/S\n3vejTTTdk4hETZmhiAiqDEVEgBy5TS7Lr5w2b948AFq3bp1kOCLyA6DMUESEHMkMb7jhhrSfIiJx\nU2YoIgKYc676B5t9Bqyr8sD8UeCca5t0EHFSGec/lXFmNaoMRUTylW6TRURQZSgiAkT8NNnM9gDm\nlb5sB+wAPit9/TPnXOYZP+t2zS5AeD6ojsANzjnNAhGBhMq4AJgM7Ak44O8q3+gkUcal150M9AE2\nOOe6RXGNtOvF1WZoZrcBXzvnRpR530rj2BnBNRsBG4DDnHPrs31+SRdXGZvZ3sCezrkVZtYKWA6c\n6px7Nxvnl4rF+T02sxOArcC4OCrDRG6TzayTmRWZ2cPAKqCDmf0vtH+gmU0o3d7LzKab2TIzW2pm\nR9bgUr2At1QRxi/KMnbObXTOrSjd/hJ4G9gnut9GMon6e+ycWwBsiuwXKCPJNsODgJHOuS6UZG8V\neQAY7pzrDpwD+P+5PcxsTBXXGAj8KxvBSq1EXsZm9mOgK/BqdkKWGorjexyLJEegrHHOlV8LtLye\nwIGh6f53N7OmzrklwJKKPmRmTYDTgOvqHKnUVtRl3Ap4AhjinPu6ztFKbURaxnFKsjLcEtreCYQX\nN2kS2jZq10h7GrDEOVdcy/ik7iIrYzNrDEwHJjnnZtUpSqmLqL/HscmJrjWlja6bzWx/M2sAnBna\nPRe40r8ws+o2pJ6HbpFzRjbLuLSx/h/ACufcAxGEK7UQ0fc4NjlRGZa6HpgDLALCDzyuBI4xszfM\nrAi4FCpvazCzlsDPgZnRhiw1lK0yPoGSP3a9zGxF6X+/iDh2qZ5sfo+nAQuBLma23sx+HWXgGo4n\nIkJuZYYiIolRZSgigipDERFAlaGICKDKUEQEqGGn6zZt2rjCwsKIQsk9H3zwAcXFxVb1kflDZZz/\nVMaZ1agyLCwsZNmy6oy8yQ/du3dPOoTYqYzzn8o4M90mi4igylBEBFBlKCICqDIUEQFUGYqIAKoM\nRUSAZCd3rdCWLSXzRQ4dOhSAMWOCGX78Y/Jp06YBUFBQEHN0IpKPlBmKiJCjmeHGjRsBGD9+PAAN\nGzZM7fOdRWfPng3A73//+5ijk9p47bXXABgwYABQMiqgtp577rnUdufOnQHo0KFD7YOTxPjvcd++\nfQEYPXo0AIMHD04dE/7+R0mZoYgIOZYZfvbZZwAMGjQo4Ugk2+bMmQPAtm3b6nyuWbOC9Z8mTpwI\nwNSpU+t8XonP559/DqRngABDhgwB4JJLLkm917Rp01hiUmYoIkKOZIYPPFCywNnMmSXrN736atXr\ngS9cuBAAv4bLIYccAsDxxx8fRYhSS99//z0ATz/9dNbOGR54f//99wNBD4TmzZtn7ToSnZdeegmA\nDRvS150/77zzAGjSpEm5z0RNmaGICDmSGV5zzTVAzZ4aTZ8+Pe3nfvvtB8Bjjz2WOubwww/PVohS\nSy+88AIAixYtAuD666+v8zk3bdqU2l61ahUA33zzDaDMMJeF24vvuuuujMdceOGFAJQsjR0vZYYi\nIqgyFBEBEr5N7tOnDxA8BNmxY0eVn2nTpg0Q3A6tW7cOgLVr1wJwxBFHpI7duXNn9oKValu5cmVq\ne+DAgQB06tQJgBtvvLHO5w93rZH644033kht+0743i67lFRFp556aqwxhSkzFBEhgcxwwYIFqe23\n334bCBpLK3qAcvnll6e2e/fuDcCuu+4KwPz58wH4y1/+Uu5zf//734HyHTslWuGy8A82pkyZAkCL\nFi1qfV7/4CT8byiJhnapHf+wM5NevXrFGElmygxFRIgxM/QD830bEkBxcXHGY303mbPOOguAW2+9\nNbWvWbNmacf6KbzGjh1b7pzDhg0D4NtvvwWCSR0aNWpUu19CKvX4448D6R2sfVthuC23tnx3jHA2\neOKJJwKw22671fn8Eq1wRu81btwYgLvvvjvucMpRZigiQoyZ4fbt24GKs0EIhtI9+uijQPDkuDI+\nM/RPKa+77rrUPj9Ey2eIfpqgjh071ih2qR4/4a7//w7Zaa/1dxWPPPIIEDx5BLjpppsAZfu5zHe4\nf+WVV8rt83d63bp1izWmTJQZioiQI8PxfHvSpEmTgOplhGX5rO/hhx9Ovbd06dIsRCdV+eKLLwBY\nvHhxuX1XXHFFnc8/btw4IJjirUuXLql9J510Up3PL9GqbOKVXOrpocxQRIQEMsNMo0yWLFlS5/P6\nUSzhUSdlR7b4p9K+z5tkhx+Av379eiCYhilb1qxZk/a6a9euWT2/RCtTZuif/mfjziFblBmKiKDK\nUEQEiPE22a99HNVKV36VreXLl6feKzvM7/bbb4/k2j90LVu2BILuEeGJGvwQutatW9f4vJ9++ikQ\ndNnxjjnmmFrFKfF6+eWXgaBLVJgfTrvvvvvGGlNllBmKiBBjZvjUU09l9Xy+m0VRURFQ+XAe31VH\nHXOj4Vcv80Pv/LA8gNNOOw1I7wyfyZtvvpna9g9M/PRsZSdjaNBAf8PrA78Cnn+QGZYLEzOUpX9V\nIiLkSKfr2vDTRD300EMVHlNYWAjA5MmTgWACCInGbbfdBqRnAv6OIDxBRyZt27ZNbftMsKKhmxdf\nfHFdwpSYlG3rDU+mcdlll8UdTpWUGYqIUA8zQ79UgJ8YtjJ+2NZxxx0XaUxSonPnzkD6CoX+6X7Z\njtNl+enawgYNGgSU7yTv2yglN/nO92WfIoefHGdjSrdsU2YoIkKMmWFliz4988wzaa8vvfRSADZu\n3Fjheaoz3Xu2n2BLzR166KFpP2vixz/+ccb3w/0Yf/rTn9YuMImMn7Kr7FPkfv36JRFOtSkzFBFB\nlaGICBDjbbKft8zPOh3mO+aWHaqXaeiev82uzkp6Ur/526yyt1u6Nc5tvrO15wc9XHPNNUmEU23K\nDEVEiDEzHDBgAADDhw9PvVfZeihV8X9tfHeO8ePHA9C+fftan1Nyi39IprWR65c5c+akve7QoQMQ\nTM6Qq5QZiogQY2boV7HzK98BzJw5E4BRo0bV+Hx//vOfgWAtZMk/fr1rT52tc5tfAXP16tVp7zdp\n0gTI/YlSlBmKiJDAcDy/NnJ4u3fv3kCwCpqfqPWMM84A4He/+13qM/7JYniFNMlPfrVEP8D/lltu\nSTIcqYKfWs0PtVu1ahUA+++/f2Ix1YQyQxERcmSihlNOOSXtpwgEGca1114LaI3kXOf7/vrp9Xwv\ngMMOOyyxmGpCmaGICDmSGYpk4tuOpX7Ze++9AZg4cWLCkdSMMkMREVQZiogAqgxFRABVhiIigCpD\nERFAlaGICACWabX7Cg82+wxYF104OafAOde26sPyh8o4/6mMM6tRZSgikq90mywigipDERFAlaGI\nCBDx2GQz2wOYV/qyHbAD+Kz09c+cc99FdN0+wEigITDWOfe3KK4jyZVx6bV3AV4D3nfO9Y/qOj90\nCX6PJwN9gA3OuW5RXCPtenE9QDGz24CvnXMjyrxvpXHszNJ1GgHvAD8HPgaWAb90zr2bjfNLxeIq\n49B5hwHdgGaqDOMRZxmb2QnAVmBcHJVhIrfJZtbJzIrM7GFgFdDBzP4X2j/QzCaUbu9lZtPNbJmZ\nLTWzI6s4/ZHAW865dc65bcBjQL+ofhfJLOIyxswKgF7ApKh+B6lc1GXsnFsAbIrsFygjyTbDg4CR\nzrkuwIZKjnsAGO6c6w6cA/j/uT3MbEyG4/cBPgq9Xl/6nsQvqjIGGAUMBdQ3LFlRlnGskpzPcI1z\nblk1jusJHBhaO3d3M2vqnFsCLIksOsmGSMrYzPoDHznnVphZz+yFK7WQN9/jJCvDLaHtnUB4pfAm\noW2jZo20G4AOodf7UvlfLIlOVGV8NDDAzPqWnqeVmU12zg2qU7RSG1GVcexyomtNaaPrZjPb38wa\nAGeGds8FrvQvzKyqhtTFQBczKzCzH1GSks/KdsxSM9ksY+fcMOfcvs65QuAC4DlVhMnL8vc4djlR\nGZa6HpgDLKKknc+7EjjGzN4wsyLgUqi4rcE5tx24CngeKAKmOOfeiTp4qZaslLHktKyVsZlNAxZS\nktysN7NfRxm4xiaLiJBbmaGISGJUGYqIoMpQRARQZSgiAtSwn2GbNm1cYWFhRKHkng8++IDi4mKr\n+sj8oTLOfyrjzGpUGRYWFrJsWXU6m+eH7t27Jx1C7FTG+U9lnJluk0VEUGUoIgKoMhQRAVQZiogA\nqgxFRABVhiIigCpDEREg2cldRUQA2Lx5MwAffvhhhccUFBQAMHLkSAC6du0KwAEHHADAIYccUqcY\nlBmKiJBwZvjpp58CcM455wBw9NFHA3DZZZcBJT3ls+GLL74A4KWXXgLglFNOAaBRo0ZZOb+I1MxT\nTz0FwOzZswF48cUXAXjvvfcq/MyBBx4IlAyvA9i2bVva/p0767ZKqTJDERESyAx92wDAT37yEyDI\n3Pbaay8g+xnhYYcdBkBxcTFAalzm/vvvn5XrSPV9+eWXAPzpT38CYNWqVQDMnTs3dYwy9vywZs0a\nAB566CEAxo0bl9q3detWAGoy0/4770S7eocyQxERYswMfVbm2wcBPv/8cwCuvLJk0azRo0dn9Zp3\n3XUXAGvXrgWCv0zKCOM3ZcoUAG666Sag/FNDnzEC7LHHHvEFJpFZv75kPahRo0bV6TwHHXQQEDw9\njooyQxERYswMX3vtNSB4ahR2yy23ZO06b775Zmp7xIgRAJx5Zsnyreeee27WriPV47ODa6+9Fgju\nEMzS59ocMmRIavvBBx8EoHXr1nGEKLXgyxGCzO/YY48Fgt4ajRs3BmDXXXcFoEWLFqnPfP311wD8\n4he/AIKsr0ePHgAceuihqWObNm0KQPPmzbP8W6RTZigigipDEREghttk37H6iSeeKLdv4sSJALRt\n27bO1/G3x7169Sq3b8CAAQC0bNmyzteRmvFNFf5hWUWmTp2a2n7mmWeA4GGLv4X2t12SnC1btgDp\n37PXX38dgJkzZ6Yde9RRRwGwfPlyIL3LnH+Atu+++wLQoEHyeVnyEYiI5IDIM8M//OEPQNC1wneA\nBjj77LOzdp2XX34ZgI8//jj13sUXXwzABRdckLXrSNXWrVuX2p40aVLaPj+Y3newf/7558t93neW\n91nl+eefD0C7du2yH6xUy3fffQfAr371KyDIBgFuvPFGAHr27Jnxs5kGUey3335ZjrDulBmKiBBD\nZui7UPif++yzT2pfXdqA/HCeu+++GwiG/IS7bPg2SYnXihUrUtu+M/Xxxx8PwIIFCwD49ttvAXjk\nkUcA+Otf/5r6zOrVq4Egy+/Xrx8QtCWqy018fBcY/z3zEyuE2/mHDh0KQLNmzWKOLruUGYqIkMBE\nDX7qHoDevXsDsNtuuwEwePDgKj/vO237n4sXL07bn812SKmd8NRKPlP3na69Jk2aAPCb3/wGgMcf\nfzy1zw/w94P4fcahp8nx80+I77nnHiCYYHXhwoWpY3yn6vpOmaGICDFkhldffTUA8+fPB2Djxo2p\nfb79yGcATz75ZJXn88eWHc7VsWNHIGjbkOT861//Kvfev//9bwD69++f8TN+WrVMjjzySCB9OJfE\nY9GiRWmv/TA53z8wnygzFBEhhszw8MMPB2DlypVA+pPGZ599FoDhw4cDsOeeewIwaNCgCs934YUX\nAnDwwQenve+XDPAZoiTnvPPOS237bP/VV18F4O233waCfw8zZswA0if99W3I/j0/9Zov+y5dukQW\nu6QLt+VC8ET/9ttvT73Xt29fIH1yhfpImaGICKoMRUQAsJqsQdC9e3dXWUN3HN5//30guB3u1q0b\nAM899xyQnUkfvO7du7Ns2TKr+sj8kY0y3rRpU2rbl5MfYlfRA7DwwH/fgf70008H4N133wWCVRPH\njBlTp/jCVMaVKztoIpOGDRsCcPnllwPBnIQfffQRAJ06dQKCNY/C/Bo4flKHKB7MVLeMlRmKiJDw\nusm1cccddwDBXyr/8CWbGaHUTXi43LRp0wA466yzgPIZ4lVXXQXAvffem/qM75Dtp17zQ/XmzJkD\nBJ2yQQ/MovbHP/4RgPvuu6/CY3bs2AEEGb3/WRP+4emJJ54IpE/pFhdlhiIi1JPM0GcXAJMnTwag\nVatWgFZSy3V+WiffRcNPzOC7z/hM32eDYTfffDMAb731FhB00/GfgeDfg0TDD8Pzq1r66dS2b9+e\nOsavc+MzxNrwk0D773p4JTw/yW/UlBmKiFBPMkPf0TPstNNOA9Ini5Xc5TPEiiYAzcSviuZXNfSZ\n4QsvvJA6xj+51rRe0fBPio844gggeLIfNm/ePCDIFm+77TYAli5dWuPr+bbk//znPzX+bF0pMxQR\noR5mhn7tVP+US/Kfb6+aNWsWkP6k0a+xnM21t6VmTj755LTXfsitzwwbNWoEBMtwAFx66aUAjBw5\nEgjakpOkzFBEBFWGIiJAjt8m+2FX4RXv/KpqenDyw+HX1B02bBiQvj6vb6wfOHAgAAcccEC8wUk5\nfgZ7v2qef7DiZx8CeO+994BgxvqywmslxUWZoYgI9SQzDA8S79OnT9oxX331FRDMfZeL67FKdvhJ\nOe68887Ue/5B2g033AAE63P7bjkSv86dOwNBl6hHH3203DHh7lEAu+xSUhX5LnPh4ZlxUWYoIkKO\nZ4aZ+L8gPgPwj+b98B0Nz8p/F110UWp77NixAEyfPh0I2qLKzoQu8fFZ+ahRo4Dg7i3ckfqTTz4B\noLCwEAjK1LcBJ0GZoYgI9TAzHD9+PAATJkwA4Le//S0QDOqX/Beerm3u3LlAsJ6vn1ggFzrx/tD5\nnh9+rfR//vOfqX2vvPIKEGSCfgqvJCkzFBEhxzPD0aNHA3Drrbem3jv++OMBGDx4MAC77747AI0b\nN445OskFvveAXzbAD9krKioCtJJeLvGrG5bdzhXKDEVEyPHM8LjjjgNg/vz5CUciuc5PHnvIIYcA\nsHr1akCZoVSfMkMREVQZiogAOX6bLFJdfk2ctWvXJhyJ1FfKDEVEUGUoIgKoMhQRAcD8alTVOtjs\nM2BddOHknALnXNuqD8sfKuP8pzLOrEaVoYhIvtJtsogIqgxFRICI+xma2R7AvNKX7YAdwGelr3/m\nnPsuwmvvArwGvO+c6x/VdX7okipjM7sOuKT05Rjn3OgoriOJlvF6YHPp9bY553pEcZ3U9eJqMzSz\n24CvnXMjyrxvpXHszPL1hgHdgGaqDOMRVxmbWTdgMnAk8D3wHPAb55x6XEcszu9xaWXY1Tn3v2yd\nszKJ3CabWSczKzKzh4FVQAcz+19o/0Azm1C6vZeZTTezZWa21MyOrMb5C4BewKSofgepXMRl3BlY\n7Jzb6pzbDrwEnBnV7yKZRf09jluSbYYHASOdc12ADZUc9wAw3DnXHTgH8P9ze5jZmAo+MwoYCuhR\nebKiKuOVwAlm1trMmgOnAh2yG7pUU5TfYwfMN7P/mNklFRyTNUmOTV7jnFtWjeN6AgeGlgvd3cya\nOueWAEvKHmxm/YGPnHMrzKxn9sKVWoikjJ1zb5rZ/cBc4GtgOSXtShK/SMq41JHOuQ1m1g543sze\ncs4tykLMGSVZGW4Jbe8ELPS6SWjbqFkj7dHAADPrW3qeVmY22Tk3qE7RSm1EVcY458YB4wDMbDiw\nug5xSu1FWcYbSn9+bGZPAj8DIqsMc6JrTWmj62Yz29/MGpDe/jMXuNK/KG08r+xcw5xz+zrnCoEL\ngOdUESYvm2VcesyepT8Lgb7A1GzGKzWXzTI2sxZm1qJ0uzklzwDezH7UgZyoDEtdD8yhpOZfH3r/\nSuAYM3vDzIqAS6HKtgbJTdks45mlx84ELnfOfRlh3FJ92Srj9sD/mdnrwFJghnNubpSBazieiAi5\nlRmKiCRGlaGICKoMRUQAVYYiIoAqQxERQJWhiAigylBEBFBlKCICwP8D3P5bzM0W5d8AAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc289fca7f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get the first images from the test-set.\n",
    "images = data.test.images[0:9]\n",
    "\n",
    "# Get the true classes for those images.\n",
    "cls_true = data.test.cls[0:9]\n",
    "\n",
    "# Plot the images and labels using our helper-function above.\n",
    "plot_images(images=images, cls_true=cls_true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  绘制错误案例的辅助函数\n",
    "\n",
    "用于绘制测试集中图像被错误分类的案例。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_example_errors(cls_pred):\n",
    "    # cls_pred is an array of the predicted class-number for\n",
    "    # all images in the test-set.\n",
    "\n",
    "    # Boolean array whether the predicted class is incorrect.\n",
    "    incorrect = (cls_pred != data.test.cls)\n",
    "\n",
    "    # Get the images from the test-set that have been\n",
    "    # incorrectly classified.\n",
    "    images = data.test.images[incorrect]\n",
    "    \n",
    "    # Get the predicted classes for those images.\n",
    "    cls_pred = cls_pred[incorrect]\n",
    "\n",
    "    # Get the true classes for those images.\n",
    "    cls_true = data.test.cls[incorrect]\n",
    "    \n",
    "    # Plot the first 9 images.\n",
    "    plot_images(images=images[0:9],\n",
    "                cls_true=cls_true[0:9],\n",
    "                cls_pred=cls_pred[0:9])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 超参数优化\n",
    "\n",
    "完成超参数优化需要完成几个步骤。\n",
    "\n",
    "### 创建模型\n",
    "\n",
    "我们首先需要一个函数取一组超参数并且用这些参数创建对应的卷积神经网络。\n",
    "\n",
    "我们用Keras搭建神经网络，更多细节参见教程 #03-C。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def create_model(learning_rate, num_dense_layers,\n",
    "                 num_dense_nodes, activation):\n",
    "    \"\"\"\n",
    "    Hyper-parameters:\n",
    "    learning_rate:     Learning-rate for the optimizer.\n",
    "    num_dense_layers:  Number of dense layers.\n",
    "    num_dense_nodes:   Number of nodes in each dense layer.\n",
    "    activation:        Activation function for all layers.\n",
    "    \"\"\"\n",
    "    \n",
    "    # Start construction of a Keras Sequential model.\n",
    "    model = Sequential()\n",
    "\n",
    "    # Add an input layer which is similar to a feed_dict in TensorFlow.\n",
    "    # Note that the input-shape must be a tuple containing the image-size.\n",
    "    model.add(InputLayer(input_shape=(img_size_flat,)))\n",
    "\n",
    "    # The input from MNIST is a flattened array with 784 elements,\n",
    "    # but the convolutional layers expect images with shape (28, 28, 1)\n",
    "    model.add(Reshape(img_shape_full))\n",
    "\n",
    "    # First convolutional layer.\n",
    "    # There are many hyper-parameters in this layer, but we only\n",
    "    # want to optimize the activation-function in this example.\n",
    "    model.add(Conv2D(kernel_size=5, strides=1, filters=16, padding='same',\n",
    "                     activation=activation, name='layer_conv1'))\n",
    "    model.add(MaxPooling2D(pool_size=2, strides=2))\n",
    "\n",
    "    # Second convolutional layer.\n",
    "    # Again, we only want to optimize the activation-function here.\n",
    "    model.add(Conv2D(kernel_size=5, strides=1, filters=36, padding='same',\n",
    "                     activation=activation, name='layer_conv2'))\n",
    "    model.add(MaxPooling2D(pool_size=2, strides=2))\n",
    "\n",
    "    # Flatten the 4-rank output of the convolutional layers\n",
    "    # to 2-rank that can be input to a fully-connected / dense layer.\n",
    "    model.add(Flatten())\n",
    "\n",
    "    # Add fully-connected / dense layers.\n",
    "    # The number of layers is a hyper-parameter we want to optimize.\n",
    "    for i in range(num_dense_layers):\n",
    "        # Name of the layer. This is not really necessary\n",
    "        # because Keras should give them unique names.\n",
    "        name = 'layer_dense_{0}'.format(i+1)\n",
    "\n",
    "        # Add the dense / fully-connected layer to the model.\n",
    "        # This has two hyper-parameters we want to optimize:\n",
    "        # The number of nodes and the activation function.\n",
    "        model.add(Dense(num_dense_nodes,\n",
    "                        activation=activation,\n",
    "                        name=name))\n",
    "\n",
    "    # Last fully-connected / dense layer with softmax-activation\n",
    "    # for use in classification.\n",
    "    model.add(Dense(num_classes, activation='softmax'))\n",
    "    \n",
    "    # Use the Adam method for training the network.\n",
    "    # We want to find the best learning-rate for the Adam method.\n",
    "    optimizer = Adam(lr=learning_rate)\n",
    "    \n",
    "    # In Keras we need to compile the model so it can be trained.\n",
    "    model.compile(optimizer=optimizer,\n",
    "                  loss='categorical_crossentropy',\n",
    "                  metrics=['accuracy'])\n",
    "    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练和评估模型\n",
    "\n",
    "具有最优参数的神经网络被保存到硬盘中，它在之后可以被加载。下面是模型的文件名。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "path_best_model = '19_best_model.keras'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这是保存到磁盘的模型的分类精度。这是全局变量，会随着超参数的优化而进行更新。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "best_accuracy = 0.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个函数是用于在给定超参数下创建并训练神经网络，然后在验证集上评估它的性能。然后这个函数返回所谓的适当值（又名：客观值），是在验证集上的负分类精度。这是负值，因为`skopt`通过执行最小化而不是最大化。\n",
    "\n",
    "注意函数修饰符`@use_named_args`，包装了fitness函数，所以它能通过一个是列表的参数被调用，比如`fitness(x=[1e-4, 3, 256, 'relu'])`。这是`skopt`内部的调用样式。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "@use_named_args(dimensions=dimensions)\n",
    "def fitness(learning_rate, num_dense_layers,\n",
    "            num_dense_nodes, activation):\n",
    "    \"\"\"\n",
    "    Hyper-parameters:\n",
    "    learning_rate:     Learning-rate for the optimizer.\n",
    "    num_dense_layers:  Number of dense layers.\n",
    "    num_dense_nodes:   Number of nodes in each dense layer.\n",
    "    activation:        Activation function for all layers.\n",
    "    \"\"\"\n",
    "\n",
    "    # Print the hyper-parameters.\n",
    "    print('learning rate: {0:.1e}'.format(learning_rate))\n",
    "    print('num_dense_layers:', num_dense_layers)\n",
    "    print('num_dense_nodes:', num_dense_nodes)\n",
    "    print('activation:', activation)\n",
    "    print()\n",
    "    \n",
    "    # Create the neural network with these hyper-parameters.\n",
    "    model = create_model(learning_rate=learning_rate,\n",
    "                         num_dense_layers=num_dense_layers,\n",
    "                         num_dense_nodes=num_dense_nodes,\n",
    "                         activation=activation)\n",
    "\n",
    "    # Dir-name for the TensorBoard log-files.\n",
    "    log_dir = log_dir_name(learning_rate, num_dense_layers,\n",
    "                           num_dense_nodes, activation)\n",
    "    \n",
    "    # Create a callback-function for Keras which will be\n",
    "    # run after each epoch has ended during training.\n",
    "    # This saves the log-files for TensorBoard.\n",
    "    # Note that there are complications when histogram_freq=1.\n",
    "    # It might give strange errors and it also does not properly\n",
    "    # support Keras data-generators for the validation-set.\n",
    "    callback_log = TensorBoard(\n",
    "        log_dir=log_dir,\n",
    "        histogram_freq=0,\n",
    "        batch_size=32,\n",
    "        write_graph=True,\n",
    "        write_grads=False,\n",
    "        write_images=False)\n",
    "   \n",
    "    # Use Keras to train the model.\n",
    "    history = model.fit(x=data.train.images,\n",
    "                        y=data.train.labels,\n",
    "                        epochs=3,\n",
    "                        batch_size=128,\n",
    "                        validation_data=validation_data,\n",
    "                        callbacks=[callback_log])\n",
    "\n",
    "    # Get the classification accuracy on the validation-set\n",
    "    # after the last training-epoch.\n",
    "    accuracy = history.history['val_acc'][-1]\n",
    "\n",
    "    # Print the classification accuracy.\n",
    "    print()\n",
    "    print(\"Accuracy: {0:.2%}\".format(accuracy))\n",
    "    print()\n",
    "\n",
    "    # Save the model if it improves on the best-found performance.\n",
    "    # We use the global keyword so we update the variable outside\n",
    "    # of this function.\n",
    "    global best_accuracy\n",
    "\n",
    "    # If the classification accuracy of the saved model is improved ...\n",
    "    if accuracy > best_accuracy:\n",
    "        # Save the new model to harddisk.\n",
    "        model.save(path_best_model)\n",
    "        \n",
    "        # Update the classification accuracy.\n",
    "        best_accuracy = accuracy\n",
    "\n",
    "    # Delete the Keras model with these hyper-parameters from memory.\n",
    "    del model\n",
    "    \n",
    "    # Clear the Keras session, otherwise it will keep adding new\n",
    "    # models to the same TensorFlow graph each time we create\n",
    "    # a model with a different set of hyper-parameters.\n",
    "    K.clear_session()\n",
    "    \n",
    "    # NOTE: Scikit-optimize does minimization so it tries to\n",
    "    # find a set of hyper-parameters with the LOWEST fitness-value.\n",
    "    # Because we are interested in the HIGHEST classification\n",
    "    # accuracy, we need to negate this number so it can be minimized.\n",
    "    return -accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 试运行\n",
    "\n",
    "在我们运行超参数优化之前，让我们先检查当传入默认的超参数时上面的函数是否有效。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "learning rate: 1.0e-05\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 16\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.2525 - acc: 0.1995 - val_loss: 2.1754 - val_acc: 0.3578\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s - loss: 2.0279 - acc: 0.4612 - val_loss: 1.8432 - val_acc: 0.5558\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 4s - loss: 1.6227 - acc: 0.5998 - val_loss: 1.3877 - val_acc: 0.6654\n",
      "\n",
      "Accuracy: 66.54%\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "-0.66539999999999999"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fitness(x=default_parameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 运行超参数优化\n",
    "\n",
    "现在我们已经准备好运行scikit-optimize中的贝叶斯优化来优化真实的超参数。注意，先要用我们通过手工调优发现`default_parameters`来调用`fitness()`作为开始，这将帮助优化器更快地找到更好的超参数。\n",
    "\n",
    "这里有很多的参数可以进行实验，包括调用`fitness()`函数的次数，这里我们设置成40。但是 `fitness()`的评估花销是很大的，所以不应该运行太多次，特别是很大的网络结构和数据集\n",
    "\n",
    "你也可以实验贝叶斯优化器的内部模型的不同的采集函数，它决定了如何找到一组新的超参数 。\n",
    "或者你也可以尝试用另一种贝叶斯优化器，比如随机森林。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "learning rate: 1.0e-05\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 16\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 2s - loss: 2.2287 - acc: 0.1868 - val_loss: 2.1264 - val_acc: 0.3182\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s - loss: 1.9607 - acc: 0.4438 - val_loss: 1.7713 - val_acc: 0.5082\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 5s - loss: 1.5763 - acc: 0.5579 - val_loss: 1.3832 - val_acc: 0.6166\n",
      "\n",
      "Accuracy: 61.66%\n",
      "\n",
      "learning rate: 6.1e-04\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 474\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 1.3354 - acc: 0.5258 - val_loss: 0.3002 - val_acc: 0.9112\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 0.2336 - acc: 0.9269 - val_loss: 0.1626 - val_acc: 0.9538\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.1403 - acc: 0.9563 - val_loss: 0.1113 - val_acc: 0.9692\n",
      "\n",
      "Accuracy: 96.92%\n",
      "\n",
      "learning rate: 6.1e-06\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 333\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.1702 - acc: 0.5067 - val_loss: 1.9186 - val_acc: 0.6892\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 1.4878 - acc: 0.7480 - val_loss: 1.0546 - val_acc: 0.7940\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.8226 - acc: 0.8264 - val_loss: 0.6324 - val_acc: 0.8514\n",
      "\n",
      "Accuracy: 85.14%\n",
      "\n",
      "learning rate: 1.7e-04\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 252\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.3075 - acc: 0.1058 - val_loss: 2.2968 - val_acc: 0.1126\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 1.5272 - acc: 0.4944 - val_loss: 0.8210 - val_acc: 0.7386\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.6595 - acc: 0.7967 - val_loss: 0.4940 - val_acc: 0.8544\n",
      "\n",
      "Accuracy: 85.44%\n",
      "\n",
      "learning rate: 7.3e-03\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 166\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.1821 - acc: 0.9431 - val_loss: 0.0705 - val_acc: 0.9808\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.0605 - acc: 0.9829 - val_loss: 0.0678 - val_acc: 0.9848\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.0549 - acc: 0.9855 - val_loss: 0.0736 - val_acc: 0.9846\n",
      "\n",
      "Accuracy: 98.46%\n",
      "\n",
      "learning rate: 6.1e-05\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 209\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 2s - loss: 2.3187 - acc: 0.1073 - val_loss: 2.3030 - val_acc: 0.0924\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s - loss: 2.3016 - acc: 0.1121 - val_loss: 2.2993 - val_acc: 0.1126\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 5s - loss: 2.2858 - acc: 0.1573 - val_loss: 2.2243 - val_acc: 0.2898\n",
      "\n",
      "Accuracy: 28.98%\n",
      "\n",
      "learning rate: 1.8e-04\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 453\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.3601 - acc: 0.8920 - val_loss: 0.1234 - val_acc: 0.9640\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.0850 - acc: 0.9741 - val_loss: 0.0576 - val_acc: 0.9830\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.0566 - acc: 0.9824 - val_loss: 0.0535 - val_acc: 0.9856\n",
      "\n",
      "Accuracy: 98.56%\n",
      "\n",
      "learning rate: 5.5e-06\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 186\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.3129 - acc: 0.1039 - val_loss: 2.3025 - val_acc: 0.1100\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 2.3016 - acc: 0.1106 - val_loss: 2.3010 - val_acc: 0.1126\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 2.3013 - acc: 0.1123 - val_loss: 2.3011 - val_acc: 0.1126\n",
      "\n",
      "Accuracy: 11.26%\n",
      "\n",
      "learning rate: 3.1e-05\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 427\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.3132 - acc: 0.1070 - val_loss: 2.3007 - val_acc: 0.1126\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 2.3029 - acc: 0.1080 - val_loss: 2.3020 - val_acc: 0.1126\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 2.3021 - acc: 0.1093 - val_loss: 2.3016 - val_acc: 0.1126\n",
      "\n",
      "Accuracy: 11.26%\n",
      "\n",
      "learning rate: 1.4e-04\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 29\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.8474 - acc: 0.7524 - val_loss: 0.2954 - val_acc: 0.9190\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.2392 - acc: 0.9315 - val_loss: 0.1741 - val_acc: 0.9512\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 5s - loss: 0.1643 - acc: 0.9517 - val_loss: 0.1346 - val_acc: 0.9612\n",
      "\n",
      "Accuracy: 96.12%\n",
      "\n",
      "learning rate: 3.7e-04\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 338\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.1610 - acc: 0.1844 - val_loss: 1.0813 - val_acc: 0.6678\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 0.5982 - acc: 0.8131 - val_loss: 0.3252 - val_acc: 0.9100\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.2712 - acc: 0.9201 - val_loss: 0.1858 - val_acc: 0.9468\n",
      "\n",
      "Accuracy: 94.68%\n",
      "\n",
      "learning rate: 1.7e-06\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.2568 - acc: 0.3895 - val_loss: 2.1984 - val_acc: 0.6048\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 2.0854 - acc: 0.6719 - val_loss: 1.9276 - val_acc: 0.7052\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 1.7106 - acc: 0.7158 - val_loss: 1.4589 - val_acc: 0.7290\n",
      "\n",
      "Accuracy: 72.90%\n",
      "\n",
      "learning rate: 1.4e-03\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 62\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.2396 - acc: 0.9249 - val_loss: 0.0643 - val_acc: 0.9822\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.0587 - acc: 0.9819 - val_loss: 0.0536 - val_acc: 0.9838\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s - loss: 0.0427 - acc: 0.9867 - val_loss: 0.0480 - val_acc: 0.9842\n",
      "\n",
      "Accuracy: 98.42%\n",
      "\n",
      "learning rate: 2.7e-03\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 364\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 1.3014 - acc: 0.5223 - val_loss: 0.2531 - val_acc: 0.9232\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.1956 - acc: 0.9386 - val_loss: 0.1221 - val_acc: 0.9650\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 5s - loss: 0.1138 - acc: 0.9646 - val_loss: 0.0846 - val_acc: 0.9758\n",
      "\n",
      "Accuracy: 97.58%\n",
      "\n",
      "learning rate: 5.6e-04\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 13\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.9357 - acc: 0.6775 - val_loss: 0.3024 - val_acc: 0.9184\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.2416 - acc: 0.9338 - val_loss: 0.1749 - val_acc: 0.9520\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.1685 - acc: 0.9525 - val_loss: 0.1541 - val_acc: 0.9570\n",
      "\n",
      "Accuracy: 95.70%\n",
      "\n",
      "learning rate: 1.0e-02\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 352\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.3316 - acc: 0.1049 - val_loss: 2.3019 - val_acc: 0.1070\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.3024 - acc: 0.1090 - val_loss: 2.3017 - val_acc: 0.1126\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 8s - loss: 2.3020 - acc: 0.1104 - val_loss: 2.3014 - val_acc: 0.1126\n",
      "\n",
      "Accuracy: 11.26%\n",
      "\n",
      "learning rate: 1.5e-03\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 5\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 2s - loss: 1.7072 - acc: 0.4784 - val_loss: 1.2153 - val_acc: 0.6980\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.9949 - acc: 0.7914 - val_loss: 0.7749 - val_acc: 0.8564\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s - loss: 0.6663 - acc: 0.8663 - val_loss: 0.5469 - val_acc: 0.9014\n",
      "\n",
      "Accuracy: 90.14%\n",
      "\n",
      "learning rate: 1.0e-03\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 496\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.1843 - acc: 0.9426 - val_loss: 0.0483 - val_acc: 0.9852\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 5s - loss: 0.0506 - acc: 0.9840 - val_loss: 0.0471 - val_acc: 0.9856\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.0347 - acc: 0.9889 - val_loss: 0.0451 - val_acc: 0.9856\n",
      "\n",
      "Accuracy: 98.56%\n",
      "\n",
      "learning rate: 3.7e-03\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 4s - loss: 0.2060 - acc: 0.9377 - val_loss: 0.0739 - val_acc: 0.9832\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 5s - loss: 0.0781 - acc: 0.9814 - val_loss: 0.0765 - val_acc: 0.9842\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 8s - loss: 0.0908 - acc: 0.9818 - val_loss: 0.1368 - val_acc: 0.9766\n",
      "\n",
      "Accuracy: 97.66%\n",
      "\n",
      "learning rate: 1.0e-02\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.3199 - acc: 0.1105 - val_loss: 2.3015 - val_acc: 0.1126\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 2.3020 - acc: 0.1104 - val_loss: 2.3011 - val_acc: 0.1126\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 8s - loss: 2.3018 - acc: 0.1110 - val_loss: 2.3013 - val_acc: 0.1126\n",
      "\n",
      "Accuracy: 11.26%\n",
      "\n",
      "learning rate: 1.9e-04\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 418\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.3595 - acc: 0.8999 - val_loss: 0.0888 - val_acc: 0.9732\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.0868 - acc: 0.9738 - val_loss: 0.0686 - val_acc: 0.9782\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 8s - loss: 0.0584 - acc: 0.9821 - val_loss: 0.0478 - val_acc: 0.9850\n",
      "\n",
      "Accuracy: 98.50%\n",
      "\n",
      "learning rate: 2.4e-03\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 144\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.1906 - acc: 0.9390 - val_loss: 0.0576 - val_acc: 0.9834\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.0550 - acc: 0.9840 - val_loss: 0.0402 - val_acc: 0.9890\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.0380 - acc: 0.9885 - val_loss: 0.0459 - val_acc: 0.9880\n",
      "\n",
      "Accuracy: 98.80%\n",
      "\n",
      "learning rate: 6.8e-03\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 105\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.1552 - acc: 0.9507 - val_loss: 0.0498 - val_acc: 0.9860\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.0485 - acc: 0.9853 - val_loss: 0.0534 - val_acc: 0.9836\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s - loss: 0.0417 - acc: 0.9875 - val_loss: 0.0496 - val_acc: 0.9852\n",
      "\n",
      "Accuracy: 98.52%\n",
      "\n",
      "learning rate: 2.5e-04\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 435\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.3258 - acc: 0.9131 - val_loss: 0.1024 - val_acc: 0.9676\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 0.0856 - acc: 0.9742 - val_loss: 0.0603 - val_acc: 0.9812\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.0601 - acc: 0.9819 - val_loss: 0.0477 - val_acc: 0.9868\n",
      "\n",
      "Accuracy: 98.68%\n",
      "\n",
      "learning rate: 2.5e-06\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 409\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.2504 - acc: 0.3689 - val_loss: 2.1796 - val_acc: 0.5498\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.0835 - acc: 0.6384 - val_loss: 1.9688 - val_acc: 0.6812\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s - loss: 1.8409 - acc: 0.7098 - val_loss: 1.6977 - val_acc: 0.7404\n",
      "\n",
      "Accuracy: 74.04%\n",
      "\n",
      "learning rate: 4.4e-03\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 311\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.1504 - acc: 0.9523 - val_loss: 0.0746 - val_acc: 0.9800\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 0.0559 - acc: 0.9842 - val_loss: 0.0751 - val_acc: 0.9812\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.0431 - acc: 0.9884 - val_loss: 0.0500 - val_acc: 0.9870\n",
      "\n",
      "Accuracy: 98.70%\n",
      "\n",
      "learning rate: 2.1e-03\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 436\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.1884 - acc: 0.9418 - val_loss: 0.0664 - val_acc: 0.9840\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 0.0598 - acc: 0.9837 - val_loss: 0.0454 - val_acc: 0.9880\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 8s - loss: 0.0435 - acc: 0.9887 - val_loss: 0.0553 - val_acc: 0.9864\n",
      "\n",
      "Accuracy: 98.64%\n",
      "\n",
      "learning rate: 1.9e-04\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 441\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.3664 - acc: 0.8989 - val_loss: 0.1076 - val_acc: 0.9698\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 0.0872 - acc: 0.9736 - val_loss: 0.0626 - val_acc: 0.9816\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.0583 - acc: 0.9824 - val_loss: 0.0504 - val_acc: 0.9856\n",
      "\n",
      "Accuracy: 98.56%\n",
      "\n",
      "learning rate: 1.7e-03\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 512\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 1.2528 - acc: 0.5598 - val_loss: 0.2764 - val_acc: 0.9186\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.2010 - acc: 0.9369 - val_loss: 0.1251 - val_acc: 0.9592\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s - loss: 0.1203 - acc: 0.9629 - val_loss: 0.0916 - val_acc: 0.9734\n",
      "\n",
      "Accuracy: 97.34%\n",
      "\n",
      "learning rate: 1.5e-03\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 285\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.1378 - acc: 0.1588 - val_loss: 1.2723 - val_acc: 0.4116\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 0.5670 - acc: 0.7991 - val_loss: 0.2616 - val_acc: 0.9266\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 8s - loss: 0.1877 - acc: 0.9460 - val_loss: 0.1365 - val_acc: 0.9618\n",
      "\n",
      "Accuracy: 96.18%\n",
      "\n",
      "learning rate: 3.3e-04\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 5\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.3570 - acc: 0.0907 - val_loss: 2.3175 - val_acc: 0.0868\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.3074 - acc: 0.0952 - val_loss: 2.3029 - val_acc: 0.1126\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 8s - loss: 2.3019 - acc: 0.1123 - val_loss: 2.3013 - val_acc: 0.1126\n",
      "\n",
      "Accuracy: 11.26%\n",
      "\n",
      "learning rate: 2.3e-04\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 512\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 2.1591 - acc: 0.1861 - val_loss: 1.0381 - val_acc: 0.6422\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 5s - loss: 0.6686 - acc: 0.7868 - val_loss: 0.4403 - val_acc: 0.8662\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 8s - loss: 0.3814 - acc: 0.8831 - val_loss: 0.2920 - val_acc: 0.9090\n",
      "\n",
      "Accuracy: 90.90%\n",
      "\n",
      "learning rate: 2.6e-03\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 126\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 2s - loss: 1.1633 - acc: 0.5922 - val_loss: 0.1928 - val_acc: 0.9422\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.1490 - acc: 0.9550 - val_loss: 0.0859 - val_acc: 0.9778\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 5s - loss: 0.0885 - acc: 0.9729 - val_loss: 0.0735 - val_acc: 0.9786\n",
      "\n",
      "Accuracy: 97.86%\n",
      "\n",
      "learning rate: 5.7e-04\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 246\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 6s - loss: 0.2579 - acc: 0.9261 - val_loss: 0.0748 - val_acc: 0.9782\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 6s - loss: 0.0691 - acc: 0.9787 - val_loss: 0.0502 - val_acc: 0.9858\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s - loss: 0.0465 - acc: 0.9854 - val_loss: 0.0423 - val_acc: 0.9880\n",
      "\n",
      "Accuracy: 98.80%\n",
      "\n",
      "learning rate: 2.4e-04\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 164\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.4321 - acc: 0.8849 - val_loss: 0.1429 - val_acc: 0.9608\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.1163 - acc: 0.9654 - val_loss: 0.0821 - val_acc: 0.9766\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 4s - loss: 0.0794 - acc: 0.9762 - val_loss: 0.0679 - val_acc: 0.9796\n",
      "\n",
      "Accuracy: 97.96%\n",
      "\n",
      "learning rate: 1.0e-06\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 5\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 2s - loss: 2.3000 - acc: 0.1046 - val_loss: 2.2987 - val_acc: 0.1122\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s - loss: 2.2981 - acc: 0.1124 - val_loss: 2.2965 - val_acc: 0.1224\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 4s - loss: 2.2959 - acc: 0.1221 - val_loss: 2.2941 - val_acc: 0.1290\n",
      "\n",
      "Accuracy: 12.90%\n",
      "\n",
      "learning rate: 1.3e-05\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 1.6243 - acc: 0.6472 - val_loss: 0.7587 - val_acc: 0.8260\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 0.5184 - acc: 0.8724 - val_loss: 0.3656 - val_acc: 0.9038\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s - loss: 0.3292 - acc: 0.9091 - val_loss: 0.2724 - val_acc: 0.9268\n",
      "\n",
      "Accuracy: 92.68%\n",
      "\n",
      "learning rate: 7.6e-05\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 241\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.7636 - acc: 0.8233 - val_loss: 0.2393 - val_acc: 0.9368\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s - loss: 0.1961 - acc: 0.9448 - val_loss: 0.1449 - val_acc: 0.9612\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s - loss: 0.1309 - acc: 0.9617 - val_loss: 0.1068 - val_acc: 0.9688\n",
      "\n",
      "Accuracy: 96.88%\n",
      "\n",
      "learning rate: 2.0e-03\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.1668 - acc: 0.9474 - val_loss: 0.0605 - val_acc: 0.9832\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s - loss: 0.0548 - acc: 0.9845 - val_loss: 0.0419 - val_acc: 0.9902\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 8s - loss: 0.0408 - acc: 0.9890 - val_loss: 0.0596 - val_acc: 0.9844\n",
      "\n",
      "Accuracy: 98.44%\n",
      "\n",
      "learning rate: 2.2e-03\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 326\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s - loss: 1.1358 - acc: 0.5865 - val_loss: 0.2104 - val_acc: 0.9356\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s - loss: 0.1576 - acc: 0.9505 - val_loss: 0.0999 - val_acc: 0.9712\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s - loss: 0.1011 - acc: 0.9679 - val_loss: 0.0856 - val_acc: 0.9726\n",
      "\n",
      "Accuracy: 97.26%\n",
      "\n"
     ]
    }
   ],
   "source": [
    "search_result = gp_minimize(func=fitness,\n",
    "                            dimensions=dimensions,\n",
    "                            acq_func='EI', # Expected Improvement.\n",
    "                            n_calls=40,\n",
    "                            x0=default_parameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 优化过程\n",
    "\n",
    "超参数的优化过程可以被很容易的画出。最佳的适度值是在y轴上绘制的，记住这是验证集上负分类精度。\n",
    "\n",
    "注意在发现实质性改进之前必须尝试很少的超参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7fc2830202e8>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAEYCAYAAACQgLsAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYXFWZ7/HvL7fuQAMhCTRRkHCJCgoT6chluKUhYLyd\nAKKoqJkRDOCNOR48MMIo45ERBm/jERVEJCoSvIM6eAKxIzKCTgKBBJCbEAViIiExdEI6t/f8sVd1\nqjtV3VXVVV3V6d/nefbTe++1dtXbBV1v1lp7r6WIwMzMrJpG1DsAMzPb+Ti5mJlZ1Tm5mJlZ1Tm5\nmJlZ1Tm5mJlZ1Tm5mJlZ1Tm5mFlJJE2WFJJG1TsWa3xOLrZTkPRuSYskdUpaIel2ScfVO67hStLl\nkr5b7zisfpxcbMiT9DHgS8C/Aa3AK4CvArPqGVc+/2vfhhsnFxvSJO0BfBr4UET8OCLWR8TmiPhZ\nRHw81WmS9CVJz6XtS5KaUtl0Sc9I+l+SVqVWzz+msqMk/UXSyLz3O13Sg2l/hKRLJD0pabWk70sa\nn8pyXUjnSPoT8Kt0/n2Slqf6/yLpaUkzyni92ZL+JOl5SZfmxTVS0ifStS9KWixpv1T2akl3SHpB\n0qOS3tHH57lQ0mcl/V7SOkm35mIoUPdlkm5Lr/uEpA+k8zOBTwBnpZbkAxX9x7UhzcnFhrpjgGbg\nJ33UuRQ4GpgK/B1wJHBZXvk+wB7Ay4FzgGsk7RkRvwPWAyfl1X038L20/xHgNOBE4GXAGuCaXu99\nInAI8AZJh5K1qM4GJuW9Z04pr3cc8CrgZOCTkg5J5z8GvAt4E7A78H5gg6RdgTtSzHsD7wS+mmIp\n5n3p+knAFuDLRerNA55JsZ4J/JukkyLil2StyFsioiUi/q6P97KdVUR48zZkN7Iv6r/0U+dJ4E15\nx28Ank7704GXgFF55auAo9P+Z4Ab0v5uZMlm/3T8CHBy3nWTgM3AKGAyEMCBeeWfBG7OO94F2ATM\nKOP19s0r/z3wzrT/KDCrwO9+FvCbXueuBT5V5LNaCFyZd3xoinFkXgyjgP2ArcBueXU/C9yY9i8H\nvlvv/z+81W9zP7ANdauBiZJGRcSWInVeBizPO16eznW/Rq9rNwAtaf97wG8lXQCcAdwXEbnX2h/4\niaRtedduJRv3yflzrzi6jyNig6TVeeWlvN5fisS5H1kS7W1/4ChJa/POjQK+U6BuoZiXA6OBib3q\nvAx4ISJe7FV3Wh+va8OIu8VsqLsH6CLrTirmObIv2ZxXpHP9ioiHyb4030jPLjHIvoTfGBHj8rbm\niHg2/yXy9lcA++YOJI0FJpT5esX8GTioyPlf93rNloi4oI/X2i9v/xVkrafne9V5DhgvabdedXOx\nerr1Yc7JxYa0iPgbWXfTNZJOk7SLpNGS3ijp31O1m4HLJO0laWKqX85tst8DLgROAH6Qd/7rwBWS\n9gdIr9/XHWo/BN4q6e8ljSHrOtIAXi/f9cD/kTRFmcMlTQB+DrxS0nvT5zJa0uvzxmoKeY+kQyXt\nQnazxA8jYmt+hYj4M/Bb4LOSmiUdTjZelftcVwKTJfk7Zpjyf3gb8iLi82QD2pcBfyX71/qHgZ+m\nKp8BFgEPAkuB+9K5Ut1MNsj+q4jI/xf8fwC3AfMlvQjcCxzVR5wPkQ3azyNrxXSSje90VfJ6vXwB\n+D4wH1gHfBMYm7qtTiUbyH+OrFvtKqCpj9f6DnBjqtsMfLRIvXeRjcM8R3ZDxaci4s5UlkvCqyXd\nV+LvYDsRRbj1alYPklqAtcCUiHiq3vFAdisy2UD89fWOxYY2t1zMBpGkt6auu12Bz5G1pJ6ub1Rm\n1efkYja4ZpF1Iz0HTCG7ldjdB7bTqXu3WHr69xayvtungXdExJoC9V5BNmi5H9mdKG+KiKclHUDW\nhz0BWAy8NyI2DU70ZmZWSCO0XC4BFkTEFGBBOi7k28DVEXEI2RPWq9L5q4AvRsTBZE80n1PjeM3M\nrB+N0HJ5FJgeESskTQIWRsSretU5FLguIo7rdV5kdwftExFbJB0DXB4Rb+jvfSdOnBiTJ08uWLZ+\n/Xp23XXXyn6hGnNslXFslXFsldmZY1u8ePHzEbFXf/Ua4Qn91ohYkfb/Qs+nkXNeCayV9GPgAOBO\nshbOnsDavKern6HnXE09SJoDzAFobW3lc5/7XMF6nZ2dtLS0FCyrN8dWGcdWGcdWmZ05tvb29uX9\n12Jw5hYjSwbLCmyzyJJDft01Ba4/E/gbcCBZQvwRWffXROCJvHr7ActKiamtrS2K6ejoKFpWb46t\nMo6tMo6tMjtzbMCiaJS5xSJiRrEySSslTYrt3WKrClR7BlgSEX9M1/yUbJbbG4BxefNK7cv26SfM\nzKxOGmFA/zZgdtqfDdxaoM5/kyWRXD/fScDDKYt2kLVs+rrezMwGUSMklyuBUyQ9DsxIx0iaJul6\ngMjmNboIWCBpKdl8TN9I118MfEzSE2S3I39zkOM3M7Ne6j6gHxGryRY+6n1+EXBu3vEdwOEF6v2R\n7NZkMzNrEHVPLkPJ/Lse5tqb7mbV6nXsPWF3zjv7OE49oa8F/czMhicnlxLNv+thrvr6fLq6srue\nVz6/jqu+Ph/ACcbMrJdGGHMZEq696e7uxJLT1bWFa2+6u04RmZk1LieXEq1ava6s82Zmw5mTS4n2\nnrB7WefNzIYzJ5cSnXf2cTQ19RyiamoaxXlnH1fkCjOz4csD+iXKDdr/21d+yZat25iw56586H0n\nejDfzKwAt1zKcOoJh/LKA7N5Na/437OcWMzMinByKVNz6hrbuHFznSMxM2tcTi5lam4aDcDGLicX\nM7NinFzKNLY5l1y29FPTzGz4cnIpU1Ou5eJuMTOzopxcyjQ2l1w2ObmYmRXj5FKm3JjLS265mJkV\n5eRSpu67xTygb2ZWlJNLmZqbfbeYmVl/nFzK1DzGd4uZmfXHyaVM3S0Xj7mYmRVV9+QiabykOyQ9\nnn7uWaTeKyTNl/SIpIclTU7nb5T0lKQlaZtay3jH+iFKM7N+1T25AJcACyJiCrAgHRfybeDqiDgE\nOBJYlVf28YiYmrYltQw2NzPyS04uZmZFNUJymQXMTftzgdN6V5B0KDAqIu4AiIjOiNgweCFu55aL\nmVn/FBH1DUBaGxHj0r6ANbnjvDqnAecCm4ADgDuBSyJiq6QbgWOALlLLJyK6irzXHGAOQGtra9u8\nefMKxtTZ2UlLS0vBsmdXredrtzzCy/bahQ++c/BnRe4rtnpzbJVxbJVxbJUZaGzt7e2LI2JavxUj\nouYbWTJYVmCbBaztVXdNgevPBP4GHEi2Bs2PgHNS2SRAQBNZy+eTpcTU1tYWxXR0dBQte+rPz8ex\nZ1wd7/rwN4vWqaW+Yqs3x1YZx1YZx1aZgcYGLIoSvmMHZbGwiJhRrEzSSkmTImKFpEn0HEvJeQZY\nEhF/TNf8FDga+GZErEh1uiR9C7ioyuH34FmRzcz61whjLrcBs9P+bODWAnX+Gxgnaa90fBLwMEBK\nSLkutdPIWkQ1M9YPUZqZ9asRksuVwCmSHgdmpGMkTZN0PUBEbCVrkSyQtJSsG+wb6fqb0rmlwETg\nM7UMtnmMp38xM+vPoHSL9SUiVgMnFzi/iGwQP3d8B3B4gXon1TTAXsaMGYUEmzZvZevWbYwc2Qj5\n2cyssfibsUySusddujZ5Chgzs0KcXCrgaffNzPrm5FIB3zFmZtY3J5cKeE0XM7O+OblUIDczsucX\nMzMrzMmlArn5xbq8pouZWUFOLhXwgL6ZWd+cXCrgAX0zs745uVSg2VPAmJn1ycmlAp4Cxsysb04u\nFei+W8xjLmZmBTm5VGD73WJOLmZmhTi5VKCpyc+5mJn1xcmlAmO77xbzcy5mZoU4uVTAd4uZmfXN\nyaUC3XOLeUDfzKwgJ5cKNHvMxcysT3VPLpLGS7pD0uPp554F6rRLWpK3bZR0Wio7QNLvJD0h6RZJ\nY2odc7PnFjMz61PdkwtwCbAgIqYAC9JxDxHRERFTI2IqcBKwAZifiq8CvhgRBwNrgHNqHfBYt1zM\nzPrUCMllFjA37c8FTuun/pnA7RGxQZLIks0Py7h+wLoH9D3mYmZWUCMkl9aIWJH2/wK09lP/ncDN\naX8CsDYicv1TzwAvr36IPXVP/7LJycXMrBBFRO3fRLoT2KdA0aXA3IgYl1d3TUTsMO6SyiYBDwIv\ni4jNkiYC96YuMSTtR9aqeW2R6+cAcwBaW1vb5s2bVzDezs5OWlpaiv4+61/azGevf4CxzSO59AOv\nK1qvFvqLrZ4cW2UcW2UcW2UGGlt7e/viiJjWb8WIqOsGPApMSvuTgEf7qHshcF3esYDngVHp+Bjg\n/5Xyvm1tbVFMR0dH0bKIiI0bN8WxZ1wd7Wd9oc96tdBfbPXk2Crj2Crj2Coz0NiARVHCd2wjdIvd\nBsxO+7OBW/uo+y62d4mRftEOsnGYUq6vijFjRiHBps1b2bp1W63fzsxsyGmE5HIlcIqkx4EZ6RhJ\n0yRdn6skaTKwH/DrXtdfDHxM0hNkYzDfrHXAkrbfjrzJtyObmfU2qt4BRMRq4OQC5xcB5+YdP02B\nwfqI+CNwZA1DLKi5aTQvbdzMxq7N7DK25o/WmJkNKY3QchmSclPAeE0XM7MdOblUqLnJk1eamRXj\n5FKh7TMje8zFzKy3kpOLpLdL2i3tXybpx5KOqF1oja15jFsuZmbFlNNy+ZeIeFHScWR3dX0T+Fpt\nwmp8Y1PLxWMuZmY7Kie5bE0/30z2IOMvgGF7m1RTmgKmy1PAmJntoJzk8qyk68jm9vpPSU1lXr9T\nccvFzKy4cpLD24HbgVMiYi2wJ3BRTaIaAny3mJlZcf0+RCnpRSA3u6WAyGa6z/aB3WsWXQNzcjEz\nK67f5BIRuw1GIEON13QxMytu2I6ZDFSzV6M0MyuqnG4xFSiOiBim3WLpbjE/RGlmtgN3i1VorFsu\nZmZFlTUrsqQ9gSlAc+5cRNxV7aCGAo+5mJkVV3JykXQu2UqQ+wJLgKOBe4CTahNaY+ue/sUPUZqZ\n7aCcAf0LgdcDyyOiHXgdsLYmUQ0B21suHnMxM+utnOSyMSI2Akhqiog/AK+qTViNz2MuZmbFlTPm\n8oykccBPgTskrQGW1yasxtfUfbeYk4uZWW8lJ5eIOD3tXi6pA9gD+OVAA5A0HrgFmAw8DbwjItb0\nqtMOfDHv1KuBd0bETyXdCJwI/C2V/UNELBloXP1xy8XMrLiKHqKMiF9HxG0RsakKMVwCLIiIKcCC\ndNz7/ToiYmpETCW7gWADMD+vysdz5YORWACaUnLxcy5mZjsqZ7GwualbLHe8p6QbqhDDLGBu2p8L\nnNZP/TOB2yNiQxXeu2LdsyK75WJmtgNFRP+1AEn3R8Tr+jtXdgDS2ogYl/YFrMkdF6n/K+ALEfHz\ndHwjcAzQRWr5RERXkWvnAHMAWltb2+bNm1fwPTo7O2lpaekz7ojgX76yGIBPf6iNESMKTWBQfaXE\nVi+OrTKOrTKOrTIDja29vX1xREzrt2JElLQBDwB75h2PB5aWeO2dwLIC2yxgba+6a/p4nUnAX4HR\nvc4JaCJr+XyylJja2tqimI6OjqJl+U5+1xfj2DOujvUbukqqXw2lxlYPjq0yjq0yjq0yA40NWBQl\nfMeWc7fY54F7JP0gHb8duKKUCyNiRrEySSslTYqIFZImAav6eKl3AD+JiO6+qIhYkXa7JH2LQVxj\nprlpNBu7trCxazO7jB22i3Kame2g5DGXiPg2cAawMm1nRMR3qhDDbcDstD8buLWPuu8Cbs4/kRJS\nrkvtNLIW0aDwapRmZoWVNbdYRDwMPFzlGK4Evi/pHLLnZt4BIGkacH5EnJuOJwP7Ab/udf1NkvYi\n6xpbApxf5fiKakpTwHR5Chgzsx7KSi61EBGrgZMLnF8EnJt3/DTw8gL16ja32faWi29HNjPL58XC\nBsBLHZuZFVbOrMgnAWeTTVa5DHgQWBZFbvsdDnILhjm5mJn1VE632A3APwGjgcPJBs9fAxxcg7iG\nBLdczMwKKye5LI+In6b9H/RZc5jw3WJmZoWVM+Zyl6T/mW75NfLnF3NyMTPLV07L5VDgMOBiSYvJ\nbvtdEhHDthXjmZHNzAorZ8r9twFIGsv2RHMUw7iLLLemy0Z3i5mZ9VD2cy4R8RKwOG3DWq7lsnGT\nn3MxM8vn51wGoDkN6LvlYmbWk5PLADSPybVcnFzMzPKVlFyU2a/WwQw1brmYmRVWUnJJc/j/Z41j\nGXJ8t5iZWWHldIvdJ+n1NYtkCMrdLdbV5QF9M7N85dwtdhTwHklPA+vJpriPiDi8FoENBd1P6Lvl\nYmbWQznJ5Q01i2KI6p5bzGMuZmY9lNMt9ifgeGB2RCwHAmitSVRDhCeuNDMrrJzk8lXgGLKlhgFe\nBK6pekRDyPbk4jEXM7N8ZY25RMQRku4HiIg1ksbUKK4hwWMuZmaFldNy2SxpJFl3GGnd+m3VCELS\neEl3SHo8/dyzSL1/l/SQpEckfTk3Q7OkNklLJT2Rf77WxozOcvOmTVvYti0G4y3NzIaEcpLLl4Gf\nAHtLugK4G/hsleK4BFgQEVOABem4B0l/DxxLtlDZa4HXAyem4q8BHwCmpG1mleLq04gR8mqUZmYF\nlDMr8k1pqv2TyW5DPi0iHqlSHLOA6Wl/LrAQuLh3CEAzMCa9/2hgpaRJwO4RcS+ApG+TrZJ5e5Vi\n61Nz02g2dm1hY9dmdhk7rHsJzcy6KXv4voSK0lURcXF/5yoKQlobEePSvoA1ueNe9T4HnEuWXL4S\nEZdKmgZcGREzUp3jgYsj4i0Frp8DzAFobW1tmzdvXsF4Ojs7aWlpKSn2z934IGtf3MTH3ncY4/do\nKumagSgntsHm2Crj2Crj2Coz0Nja29sXR8S0fitGREkbcF+Bcw+Wcf2dwLIC2yxgba+6awpcfzDw\nC6AlbfeQ3Ro9Dbgzr97xwM/7i6etrS2K6ejoKFrW29kfvSGOPePqeHL5qpKvGYhyYhtsjq0yjq0y\njq0yA40NWBQlfOf32y0m6QLgg8CBkh7MK9oN+K9+s9f2JDajj/dYKWlSRKxI3VyrClQ7Hbg3IjrT\nNbeT3Rr9HWDfvHr7As+WGtdA5cZcXtro25HNzHJKGdB/E/AWYCTw1rytLSLeU6U4bgNmp/3ZwK0F\n6vwJOFHSKEmjyQbzH4mIFcA6SUenLrX3Fbm+JnIzI3d5QN/MrFspyeUgYDPwKLCO7OHJFyG7hbhK\ncVwJnCLpcWBGOkbSNEnXpzo/BJ4ElgIPAA9ExM9S2QeB64EnUp1BGcwHz4xsZlZIKXeLfZ3s9uAD\nyJY2zn+GJIADBxpERKwmuwut9/lFZAP4RMRW4Lwi1y8iuz150DV5Chgzsx3023KJiC9HxCHAtyLi\nwIg4IG8bcGIZ6sY6uZiZ7aCc51wuSE/OTyF73iR3/q5aBDZUeDVKM7MdlZxcJJ0LXEh2N9YS4Giy\n24FPqk1oQ0PzmPSE/ibfLWZmllPO9C8Xkk25sjwi2oHXAWtrEtUQ4paLmdmOykkuGyNiI4Ckpoj4\nA/Cq2oQ1dPhuMTOzHZUz5f4zksYBPwXukLQGWF6bsIYO3y1mZrajcgb0T0+7l0vqAPYAflmTqIYQ\n3y1mZrajclou3SLi19UOZKhqyk257zEXM7Nu5Yy5WAG51Si91LGZ2XZOLgPU7G4xM7MdlJ1cJO2a\nljs2ticX3y1mZrZdv8lF0ghJ75b0C0mrgD8AKyQ9LOlqSQfXPszGlUsunhXZzGy7UlouHWQzI/8z\nsE9E7BcRewPHAfcCV0mq1tT7Q05uzOUlD+ibmXUr5W6xGRGxwzdnRLwA/Aj4UVpfZVjKLRbmAX0z\ns+1KmRV5M4Ck/0iLcRWtMxx5QN/MbEflDOi/CNwmaVcASW+QVPIyxzurpjFpzGXTFrZtizpHY2bW\nGMp5Qv8ySe8GFkraBHQCl9QssiFixAjRNGYUXZu2sLFrM7uMHVPvkMzM6q7kloukk4EPAOuBicBH\nI+I3Aw1A0nhJd0h6PP3cs0i9f5f0kKRHJH0510UnaaGkRyUtSdveA42pXLlB/a5N7hozM4PyusUu\nBf4lIqYDZwK3SKrGWi6XAAsiYgrZcso7tIYk/T1wLHA42XLGrwdOzKtydkRMTduqKsRUlu5nXXzH\nmJkZUEZyiYiTIuLutL8UeCPwmSrEMAuYm/bnAqcVenuy1S/HAE3AaGBlFd67KrbfMebkYmYGoIi+\nB6ElKYpUkjQ2Il7qq06/AUhrI2Jc7r2ANbnjXvU+B5wLCPhKRFyazi8EJgBbyW6N/kwf8c4B5gC0\ntra2zZs3r2BMnZ2dtLS0lPw7fO2Wh3l21QbOf/ur2Xef0q+rRLmxDSbHVhnHVhnHVpmBxtbe3r44\nIqb1WzEi+tyAhcBHgFf0Oj+GbInjucA/9PMadwLLCmyzgLW96q4pcP3BwC+AlrTdAxyfyl6efu4G\nzAfe19/vFBG0tbVFMR0dHUXLCvnQZTfHsWdcHYsfXF7WdZUoN7bB5Ngq49gq49gqM9DYgEVRwnds\nKXeLzQTeD9ws6QCypY2bgZHpy/xLEXF/PwlsRrEySSslTYqIFZImAYXGTE4H7o2IznTN7cAxwG8i\n4tn0Hi9K+h5wJPDtEn6vqul+1sUD+mZmQGljLldFxFeBU4D9gZOBIyJi/4j4QH+JpQS3AbPT/mzg\n1gJ1/gScKGlUmg3gROCRdDwRIJ1/C1mLaFA1ewoYM7MeSkkuJ6Sfv4mIzRGxIiLWVjGGK4FTJD0O\nzEjHSJom6fpU54fAk8BS4AHggYj4Gdng/v+T9CCwBHgW+EYVYyuJB/TNzHoqpVtsgaR7gH0kvZ/s\ny31ZRHRVI4CIWE3WGup9fhHZAD4RsRU4r0Cd9UBbNeIYiO1TwHh+MTMzKCG5RMRFkg4imx35AOB/\nAK9JT+kvi4izahxjwxvr+cXMzHooafqXiHhS0oyIeCx3TlIL2QONw153y8VjLmZmQBlziwHL09xi\nk3tdd29VIxqCcgP6brmYmWXKSS63An8DFgNVGW/ZWXipYzOznspJLvtGxMyaRTKE+W4xM7Oeypm4\n8reSDqtZJEOY7xYzM+upnJbLccA/SHqKrFtMQETE4TWJbAjJTbnvAX0zs0w5yeWNNYtiiGse4wF9\nM7N85axEubyWgQxlvlvMzKynfsdcJN2dfr4oaV36mdvW1T7ExucBfTOznkp5Qv+49HO32oczNHkl\nSjOznkruFpM0DfgEvR6i9ID+9gH9rk2+W8zMDMob0L8J+DjZzMTbahPO0NQ0xi0XM7N85SSXv0bE\nbTWLZAjLdYt1bdrCtm3BiBGqc0RmZvVVTnL5VFpfZQF5079ExI+rHtUQM2KEaBoziq5NW+jatJmx\nzWPqHZKZWV2Vk1z+EXg1MJrt3WIBDPvkAlnrpWvTFjZ2ObmYmZWTXF4fEa+qWSRDXHPTKP72oqeA\nMTOD8ucWO7RmkQxxuTvGPKhvZlZecjkaWCLpUUkPSlqa1q4fEEnjJd0h6fH0c88i9a6StCxtZ+Wd\nP0DS7yQ9IekWSXXpk2ryapRmZt3KSS4zgSnAqcBbgbeknwN1CbAgIqaQ3SxwSe8Kkt4MHAFMBY4C\nLpK0eyq+CvhiRBwMrAHOqUJMZfNSx2Zm25WcXCJieaGtCjHMAuam/bnAaQXqHArcFRFbImI98CAw\nU5KAk4Af9nN9zTU7uZiZdVNE1DcAaW1EjEv7AtbkjvPqnAp8CjgF2AX4PXANWTK5N7VakLQfcHtE\nvLbIe80B5gC0tra2zZs3r2BMnZ2dtLS0lPV73PyfT/LQk2s4a+aBHDZlfFnXlqOS2AaLY6uMY6uM\nY6vMQGNrb29fHBHT+qtXzt1iFZN0J7BPgaJL8w8iIiTtkO0iYr6k1wO/Bf4K3ANsLTeOiLgOuA5g\n2rRpMX369IL1Fi5cSLGyYu5euoGHnlzDQQe9kunTC+a2qqgktsHi2Crj2Crj2CozWLENSnKJiBnF\nyiStlDQpIlZImgSsKvIaVwBXpGu+BzwGrAbGSRoVEVuAfYFnq/4LlCC3pstL7hYzMytrQL9WbgNm\np/3ZwK29K0gaKWlC2j8cOByYH1mfXgdwZl/XDwav6WJmtl0jJJcrgVMkPQ7MSMdImpamm4FsVoDf\nSHqYrFvrPamlAnAx8DFJTwATgG8OavSJ7xYzM9tuULrF+hIRq4GTC5xfBJyb9jeS3TFW6Po/AkfW\nMsZSNOUWDPNDlGZmDdFy2SnkWi4eczEzc3KpmtyYS5fnFjMzc3Kplma3XMzMujm5VEn3E/oeczEz\nc3Kplu67xTY5uZiZOblUSZNbLmZm3ZxcqmRsc7oV2WMuZmZOLtWyfVZk3y1mZubkUiWect/MbDsn\nlypxcjEz287JpUqaxuTGXLawbVt918gxM6s3J5cqGTlyBGNSguny7chmNsw5uVTRWA/qm5kBTi5V\n1T0zssddzGyYc3Kpou6Zkf0gpZkNc04uVdQ9M7LHXMxsmHNyqaLmMW65mJmBk0tVNXsKGDMzoAGS\ni6Txku6Q9Hj6uWeReldJWpa2s/LO3yjpKUlL0jZ18KLvyXeLmZll6p5cgEuABRExBViQjnuQ9Gbg\nCGAqcBRwkaTd86p8PCKmpm3JYARdiGdGNjPLNEJymQXMTftzgdMK1DkUuCsitkTEeuBBYOYgxVey\nsV6N0swMAEXUd6oSSWsjYlzaF7Amd5xX51TgU8ApwC7A74FrIuLzkm4EjgG6SC2fiOgq8l5zgDkA\nra2tbfPmzSsYU2dnJy0tLWX/Lrff/Wf+6/6VvOHYfTn+iH3Kvr4UlcY2GBxbZRxbZRxbZQYaW3t7\n++KImNZvxYio+QbcCSwrsM0C1vaqu6bIa1wKLAHuAG4C/imdnwQIaCJr+XyylJja2tqimI6OjqJl\nffnG934Tx55xdVw/7+6Kri9FpbENBsdWGcdWGcdWmYHGBiyKEr5jR1WcvsoQETOKlUlaKWlSRKyQ\nNAlYVeQ1rgCuSNd8D3gsnV+RqnRJ+hZwUVWDL0PuORePuZjZcNcIYy63AbPT/mzg1t4VJI2UNCHt\nHw4cDszF32AmAAANfUlEQVRPx5PST5GN1ywbhJgL6p52f5PvFjOz4W1QWi79uBL4vqRzgOXAOwAk\nTQPOj4hzgdHAb7L8wTrgPRGR+wa/SdJeZF1jS4DzBzn+bs2+W8zMDGiA5BIRq4GTC5xfBJyb9jeS\n3TFW6PqTahpgGcY2+24xMzNojG6xnUZTmv6ly8nFzIa5urdcdiYPPfYcAPfc9xRvO+86zjv7OE49\nYXuDa/5dD3PtTXezavU69p6we4/yvsrMzIYaJ5cqmX/Xw9zys0XdxyufX8dVX5tP5/ouph/zShbe\n8xjXfPvXdKXB/vxyYMeyr88HcIIxsyHJyaVKrr3pbjZt3trjXNemLXzh+gV84foFBa/JlRcs69rC\ntTfd7eRiZkOSx1yqZNXqdUXL9txjl6q/pplZI3NyqZK9J+xe8HzrxN352Q0fpHVi8fJiZbu3NFct\nPjOzweTkUiXnnX0cTU09exmbmkZx3tnH9VteqAxg3Ysb+fXvHq9d0GZmNeLkUiWnnnAoF59/Kq0T\nd0fKWiQXn39q95hJX+WFyo4/8iAC+NQXfsa99z9V31/OzKxMHtCvolyiqKS8d1lE8H9vXMj3f76Y\nT/z7rXz+srfxutfsV/WYzcxqwcmlQUniI/8wnZc2buJndy7lY5/+Abu1NPPC2g203vxYWc/QmJkN\nNieXBiaJi+acwlN/Xs2yR5/jhbUbgB2fg5l/18Nc9fX5dHUVf06mv+QzkAc8c+Urn19XUeLbWWMr\nNfZKYzNrZHVfLKxepk2bFosWLSpYtnDhQqZPnz64AfXhbeddy8rnX9zh/IgRYvy4XXlh7Xq2bdvx\nv2PTmFHMnP4ann/hRX53/9Ns2bqtu2z0qBG8+eTDOOxVL2fpo8/yiwVL2bxlx3KgaFl/19a6vJFj\nq0XsTWNGcfEFp1blHwz9lfdIfBMHN+k6tvrEVipJJS0W5uRSQKMll+PP/BzD9D+T9TJq5AhOOGoK\nW7dt47eL/sjmLdsf3G0aM4oPz57ePSPEV+Yu7J71oZxyoOJra13u2KoYW9OoHjcdlcrJpR9DKbm8\n7bzrWPn8jg9U7jW+heuuPJs5l9zEX1/o3KF8j93Gcu47j+Xz37iz6GufesIhzL/rkYri6u/aWpc3\ncmy1jN2sWlon7s6Prp1T1jWlJhffijwEFHtG5oL3nsBeE3bjgveeULD8wve3c/rMqX0+wPnJC99c\n0QOepVxb6/JGjq1WsY8ftyuXfuSNBctyxu0+dkDltXxtx1af8mJqOQuIk8sQkP8cDJT3DA1U/wHP\nUq+tdXkjx1ar2D88+0TeOP01fSann3/rQwMqr+VrO7bGiq3YzCLVMPLyyy+v2Ys3suuuu+7yOXMK\nNweffvppJk+ePLgB9eOg/ffirLe2cWDrJi7+yNs4aP+9Cpa//x1/z1lvbetRftD+ezFpr935w5Mr\n2fBSF60Td+fC97d3J5++ysu5dv2G8l57Z46tnNgriW3PPcZy75Kn2Jp3k0autXrQ/nsNqHza4a+o\n2Ws7tsaMrRz/+q//uuLyyy+/rr96dR9zkfR24HLgEODItAJloXozgf8ARgLXR8SV6fwBwDxgArAY\neG9EbOrvfYfSmEs+x1aZnTE23/Xk2Br5bjEioq4bWVJ5FbAQmFakzkjgSeBAYAzwAHBoKvs+8M60\n/3XgglLet62tLYrp6OgoWlZvjq0yjq0yjq0yO3NswKIo4Tu27mMuEfFIRDzaT7UjgSci4o+RtUrm\nAbMkCTgJ+GGqNxc4rXbRmplZKereLZYjaSFwURToFpN0JjAzIs5Nx+8FjiLrTrs3Ig5O5/cDbo+I\n1xZ5jznAHIDW1ta2efPmFYyls7OTlpaWgf5KNeHYKuPYKuPYKrMzx9be3l5St9igTP8i6U5gnwJF\nl0bErYMRA0BEXAdcB9mYS7F+7p2xf34wOLbKOLbKOLbKDFZsg5JcImLGAF/iWSB/SuB907nVwDhJ\noyJiS955MzOro7qPuZTov4Epkg6QNAZ4J3BbGlzqAM5M9WYDg9YSMjOzwuo+5iLpdOD/AnsBa4El\nEfEGSS8ju+X4Tanem4Avkd05dkNEXJHOH0g2wD8euB94T0R0lfC+fwWWFymeCDw/oF+sdhxbZRxb\nZRxbZXbm2PaPiH4fjql7cmlEkhaVMmBVD46tMo6tMo6tMo5t6HSLmZnZEOLkYmZmVefkUli/8+bU\nkWOrjGOrjGOrzLCPzWMuZmZWdW65mJlZ1Tm5mJlZ1Tm59CJppqRHJT0h6ZJ6x5NP0tOSlkpaIqnw\negGDF8sNklZJWpZ3brykOyQ9nn7u2UCxXS7p2fTZLUnPTdUjtv0kdUh6WNJDki5M5+v+2fURW90/\nO0nNkn4v6YEU27+m8wdI+l36e70lPWTdKLHdKOmpvM9t6mDHlhfjSEn3S/p5Oq755+bkkkfSSOAa\n4I3AocC7JJW/4EFttUfE1Aa4h/5GYGavc5cACyJiCrAgHdfDjewYG8AX02c3NSL+c5BjytkC/K+I\nOBQ4GvhQ+n+sET67YrFB/T+7LuCkiPg7YCowU9LRwFUptoOBNcA5DRQbwMfzPrcldYgt50Lgkbzj\nmn9uTi49FZzav84xNaSIuAt4odfpWWTLHkAdlz8oEltDiIgVEXFf2n+R7A/+5TTAZ9dHbHWXlhLp\nTIej0xY0wJIbfcTWECTtC7wZuD4dD8pSJU4uPb0c+HPe8TM0yB9XEsB8SYvT8gGNpjUiVqT9vwCt\n9QymgA9LejB1m9Wlyy6fpMnA64Df0WCfXa/YoAE+u9S1swRYBdxBtoDg2jRpLdTx77V3bBGR+9yu\nSJ/bFyU11SM2smmz/jeQW+N4AoPwuTm5DC3HRcQRZN12H5J0Qr0DKiZNKtow/3oDvgYcRNZtsQL4\nfD2DkdQC/Aj4p4hYl19W78+uQGwN8dlFxNaImEo2+/mRwKvrEUchvWOT9Frgn8lifD3Z3IcXD3Zc\nkt4CrIqIxYP93k4uPRWb2r8hRMSz6ecq4Cdkf2CNZKWkSQDp56o6x9MtIlamL4BtwDeo42cnaTTZ\nl/dNEfHjdLohPrtCsTXSZ5fiWUs2G/oxpCU3UlHd/17zYpuZuhkjTaT7LerzuR0L/A9JT5N1858E\n/AeD8Lk5ufRUcGr/OscEgKRdJe2W2wdOBZb1fdWgu41s2QNosOUPcl/cyenU6bNL/d3fBB6JiC/k\nFdX9sysWWyN8dpL2kjQu7Y8FTiEbE6r7khtFYvtD3j8WRDamMeifW0T8c0TsGxGTyb7PfhURZzMY\nn1tEeMvbgDcBj5H1515a73jy4joQeCBtD9U7NuBmsi6SzWR9tueQ9eUuAB4H7gTGN1Bs3wGWAg+S\nfZFPqlNsx5F1eT0ILEnbmxrhs+sjtrp/dsDhZEtqPEj2Jf3JdP5A4PfAE8APgKYGiu1X6XNbBnwX\naKnH/3N5cU4Hfj5Yn5unfzEzs6pzt5iZmVWdk4uZmVWdk4uZmVWdk4uZmVWdk4uZmVWdk4uZmVWd\nk4uZmVWdk4sNG5JC0ufzji+SdHkVXndy/toxtSTpo5IekXTTAF+ns9C+WbU4udhw0gWcIWlivQPJ\np0ypf4sfBE6JbAoPs4bl5GLDyRbgOuB/5p/s3fLItWjS+T+kFQUfk3STpBmS/kvZipH5ExGOSuWP\nSPqhpF3Sa70nrVK4RNK1aUG63Hs+KunbZNOD7Ncrpo9JWpa2f0rnvk42bcftknr8Dqn8fWl69wck\nfSed+2laouGh/pZpSPPX/SJdv0zSWQXq/FjSZyTdJelPkmb09Zo2fDm52HBzDXC2pD1KrH8w2RTz\nr07bu8nm4LoI+ERevVcBX42IQ4B1wAclHQKcBRwb2XTsW4H8FseUdM1rImJ57qSkNuAfgaPIVoT8\ngKTXRcT5wHNkq5F+MT9ISa8BLmP7iogXpqL3R0QbMA34qKQJffyuM4HnIuLvIuK1wC8L1DmMbC2Q\nE9J7uAVlBTm52LAS2fok3wY+WuIlT0XE0simm3+IbCniIJuQcHJevT9HxH+l/e+SJaCTgTbgv9NC\nUieTtTxylkfEvQXe8zjgJxGxPrIVDn8MHN9PnCcBP4iI59PvmVuJ86OSHgDuJWsdTenjNZYCp0i6\nStLxEfG3/MLUGtsDyCW20cDafuKyYWpU/1XMdjpfAu4jW2MDsu6y/H9oNeftd+Xtb8s73kbPv5/e\nM8AGIGBuRPxzkTjWlxFz2SRNB2YAx0TEBkkL6fm79RARj0k6gmwm5M9IWhARn86rciiwOCK2puPD\nabxlH6xBuOViw076V/33yabiB1gJ7C1pQlqK9i0VvOwrJB2T9t8N3E02hf6ZkvYGkDRe0v4lvNZv\ngNMk7ZLW7jk9nevLr4C357q9JI0na2WsSYnl1WRdbEVJehmwISK+C1wNHNGrymFk0/DnHE42zbzZ\nDtxyseHq88CHASJis6RPk61v8Szwhwpe71GypadvAB4Gvpa+1C8D5qe7wTYDHwKW9/E6RMR9km5M\n8QBcHxH393PNQ5KuAH4taSvZ+iLnAedLeiTFV6gLLt9hwNWStqVYLyhQ/ru849filosV4fVczMys\n6twtZmZmVefkYmZmVefkYmZmVefkYmZmVefkYmZmVefkYmZmVefkYmZmVff/AeWG+IOGdYFxAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc285dc5cf8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_convergence(search_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 最优超参数\n",
    "\n",
    "贝叶斯优化器所发现的最好的超参数被打包成一个列表，因为这是它在内部使用的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.0023584457378584664, 4, 144, 'relu']"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search_result.x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以转换这些参数到一个在搜索空间维度中专有名称的地址\n",
    "\n",
    "首先，我们需要对搜索空间对象的引用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "space = search_result.space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后我们可以用它创建一个地址，超参数在搜索空间维度里具有的专有的名称。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'activation': 'relu',\n",
       " 'learning_rate': 0.0023584457378584664,\n",
       " 'num_dense_layers': 4,\n",
       " 'num_dense_nodes': 144}"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "space.point_to_dict(search_result.x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面这些超参数对应的适度值。这是一个负值，因为贝叶斯优化器执行最小化，所以我们不得不将分类进度最大化问题取反。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.98799999999999999"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search_result.fun"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们也看到所有被贝叶斯优化器尝试过的超参数和它们对应的适度值（负的分类精度）。对它们进行排序，以便首先显示最高的分类准确度。\n",
    "\n",
    "看起来’relu‘激活总的表现要比’sigmoid‘好。我们需要画出这些结果，否则，很难看出哪种参数选择模式是好的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(-0.98799999999999999, [0.00057102338020535671, 1, 246, 'relu']),\n",
       " (-0.98799999999999999, [0.0023584457378584664, 4, 144, 'relu']),\n",
       " (-0.98699999999999999, [0.0043924439217142824, 3, 311, 'relu']),\n",
       " (-0.98680000000000001, [0.00025070302453255417, 2, 435, 'relu']),\n",
       " (-0.98640000000000005, [0.0020904801989242469, 5, 436, 'relu']),\n",
       " (-0.98560000000000003, [0.00017567744133971055, 4, 453, 'relu']),\n",
       " (-0.98560000000000003, [0.00018871091218374878, 3, 441, 'relu']),\n",
       " (-0.98560000000000003, [0.0010013922052631494, 3, 496, 'relu']),\n",
       " (-0.98519999999999996, [0.006752254693985822, 2, 105, 'relu']),\n",
       " (-0.98499999999999999, [0.0001905308801138268, 4, 418, 'relu']),\n",
       " (-0.98460000000000003, [0.0073224617473678331, 3, 166, 'relu']),\n",
       " (-0.98440000000000005, [0.0020143982003767271, 4, 512, 'relu']),\n",
       " (-0.98419999999999996, [0.0014193250864683331, 2, 62, 'relu']),\n",
       " (-0.97960000000000003, [0.00023735076383216567, 1, 164, 'relu']),\n",
       " (-0.97860000000000003, [0.0026064900033469073, 1, 126, 'sigmoid']),\n",
       " (-0.97660000000000002, [0.0037123587226393501, 5, 512, 'relu']),\n",
       " (-0.9758, [0.0027230837381696737, 2, 364, 'sigmoid']),\n",
       " (-0.97340000000000004, [0.0016597651372777609, 1, 512, 'sigmoid']),\n",
       " (-0.97260000000000002, [0.0022460993827137423, 2, 326, 'sigmoid']),\n",
       " (-0.96919999999999995, [0.00060563429543890952, 2, 474, 'sigmoid']),\n",
       " (-0.96879999999999999, [7.5808558985641429e-05, 1, 241, 'relu']),\n",
       " (-0.96179999999999999, [0.0014963322170155162, 5, 285, 'sigmoid']),\n",
       " (-0.96120000000000005, [0.00013559943302194881, 2, 29, 'relu']),\n",
       " (-0.95699999999999996, [0.00056441093780360571, 5, 13, 'relu']),\n",
       " (-0.94679999999999997, [0.00036704404112128516, 4, 338, 'sigmoid']),\n",
       " (-0.92679999999999996, [1.3066947342663859e-05, 2, 512, 'relu']),\n",
       " (-0.90900000000000003, [0.00023277413216549582, 4, 512, 'sigmoid']),\n",
       " (-0.90139999999999998, [0.001544493082361837, 1, 5, 'sigmoid']),\n",
       " (-0.85440000000000005, [0.00016937303683800523, 4, 252, 'sigmoid']),\n",
       " (-0.85140000000000005, [6.1458838378363633e-06, 2, 333, 'relu']),\n",
       " (-0.74039999999999995, [2.4847514577863683e-06, 1, 409, 'relu']),\n",
       " (-0.72899999999999998, [1.7068698743151031e-06, 4, 512, 'relu']),\n",
       " (-0.61660000000000004, [1e-05, 1, 16, 'relu']),\n",
       " (-0.2898, [6.1011365846453456e-05, 2, 209, 'sigmoid']),\n",
       " (-0.129, [9.9999999999999995e-07, 2, 5, 'relu']),\n",
       " (-0.11260000000000001, [5.4599879082087208e-06, 4, 186, 'sigmoid']),\n",
       " (-0.11260000000000001, [3.1218037895598157e-05, 3, 427, 'sigmoid']),\n",
       " (-0.11260000000000001, [0.00033099542158994725, 5, 5, 'sigmoid']),\n",
       " (-0.11260000000000001, [0.01, 5, 352, 'sigmoid']),\n",
       " (-0.11260000000000001, [0.01, 5, 512, 'relu'])]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sorted(zip(search_result.func_vals, search_result.x_iters))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 绘制\n",
    "\n",
    "在`skopt`里有一些可利用的绘制函数。举个例子，我们可以画出`activation`参数的直方图，它显示了在超参数优化期间样本的分布。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEetJREFUeJzt3XuwXWV9xvHvI2ChwihMMgxa4qGKWrwhBIqiCF46qLTq\ngNi0VazUeMVLtVPG1opFK1SUmdYKhkIRC4gKeINaHQRRW8QkAgkiqAgqRQjFIhErt1//2OvUQziX\nzUnW3uG838/Mmb3Xu9flR+awn/O+a613paqQJLXrIeMuQJI0XgaBJDXOIJCkxhkEktQ4g0CSGmcQ\nSFLjDAJJapxBIEmNMwgkqXFbjruAYSxatKgmJibGXYYkPaisWrXqlqpaPNd6D4ogmJiYYOXKleMu\nQ5IeVJJcP8x6Dg1JUuMMAklqnEEgSY0zCCSpcQaBJDWutyBIsnOSC5N8J8mVSd7StR+V5IYkl3U/\nL+yrBknS3Pq8fPRu4O1VtTrJdsCqJF/uPju+qo7r8diSpCH1FgRVdSNwY/f+9iRXAY/q63iSpPkZ\nyTmCJBPA04Bvdk1vSnJFklOSbD+KGiRJ0+v9zuIk2wJnA2+tqp8nOQE4Gqju9YPAq6fZbjmwHGDJ\nkiXzPv7EkefNe1stbNcd86JxlyBtFnrtESTZikEInF5V5wBU1U1VdU9V3QucBOw93bZVtaKqllbV\n0sWL55wqQ5I0T31eNRTgZOCqqvrQlPadpqz2UmBtXzVIkubW59DQvsArgDVJLuva3gksS7I7g6Gh\n64DX9liDJGkOfV419HUg03x0fl/HlCQ9cN5ZLEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS\n4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXO\nIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwC\nSWpcb0GQZOckFyb5TpIrk7yla98hyZeTfK973b6vGiRJc+uzR3A38Paq2g3YB3hjkt2AI4ELqmpX\n4IJuWZI0Jr0FQVXdWFWru/e3A1cBjwJeDHysW+1jwEv6qkGSNLeRnCNIMgE8DfgmsGNV3dh99FNg\nx1HUIEmaXu9BkGRb4GzgrVX186mfVVUBNcN2y5OsTLJy3bp1fZcpSc3qNQiSbMUgBE6vqnO65puS\n7NR9vhNw83TbVtWKqlpaVUsXL17cZ5mS1LQ+rxoKcDJwVVV9aMpHnwMO694fBny2rxokSXPbssd9\n7wu8AliT5LKu7Z3AMcAnkxwOXA8c2mMNkqQ59BYEVfV1IDN8/Ny+jitJemC8s1iSGmcQSFLjDAJJ\napxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTG\nGQSS1DiDQJIaZxBIUuPmDIIkxw7TJkl6cBqmR/D8adpesKkLkSSNx4wPr0/yeuANwG8nuWLKR9sB\n3+i7MEnSaMwYBMAZwL8B7weOnNJ+e1Xd2mtVkqSRmTEIquo24DZgWZItgB279bdNsm1V/WhENUqS\nejRbjwCAJG8CjgJuAu7tmgt4Sn9lSZJGZc4gAN4KPL6q/rvvYiRJozfMVUM/ZjBEJElagIbpEVwL\nXJTkPOBXk41V9aHeqpIkjcwwQfCj7ueh3Y8kaQGZMwiq6j2jKESSNB7DXDV0IYOrhO6jqp7TS0WS\npJEaZmjoHVPebw0cDNzdTzmSpFEbZmho1QZN30hyaU/1SJJGbJihoR2mLD4E2BN4eG8VSZJGapih\noVUMzhGEwZDQD4HD59ooySnAQcDNVfWkru0o4DXAum61d1bV+Q+8bEnSpjLM0NAu89z3qcCHgdM2\naD++qo6b5z4lSZvYMENDWwGvB/brmi4CPlpVd822XVVdnGRiI+uTJPVsmCkmTmBwXuAj3c+eXdt8\nvSnJFUlOSbL9RuxHkrQJDHOOYK+qeuqU5a8kuXyexzsBOJrBOYejgQ8Cr55uxSTLgeUAS5Ysmefh\npM3fxJHnjbsEbcauO+ZFvR9jmB7BPUkeM7mQ5LeBe+ZzsKq6qaruqap7gZOAvWdZd0VVLa2qpYsX\nL57P4SRJQximR/AXwIVJrmVw5dCjgT+dz8GS7FRVN3aLLwXWzmc/kqRNZ5irhi5Isivw+K7p6qr6\n1WzbACQ5E9gfWJTkJ8C7gf2T7M5gaOg64LXzrFuStInM9vD6PwFSVR/vvviv6NpfkeSeqjpjth1X\n1bJpmk/eqGolSZvcbOcIjgDOnab9HODt/ZQjSRq12YJgq6pav2FjVf0C2Kq/kiRJozRbEGyT5GEb\nNibZDh9QI0kLxmxBcDLw6SSPnmzo7hT+BI71S9KCMePJ4qo6Lsl64OIk23bN64Fjqmpj7iyWJG1G\nZr18tKpOBE7shoOoqttHUpUkaWSGuaHMAJCkBWyYKSYkSQuYQSBJjZszCJL8ZpJ3JTmpW941yUH9\nlyZJGoVhegT/AvwKeHq3fAPw3t4qkiSN1DBB8Jiq+nvgLoCquoPBLKSSpAVgmCC4M8k2DGYMpXs2\nwZyzj0qSHhyGuXz03cAXgZ2TnA7sC7yqz6IkSaMzzPMIvpxkNbAPgyGht1TVLb1XJkkaidmeR7DH\nBk2TTxZbkmRJVa3uryxJ0qjM1iP44CyfFfCcTVyLJGkMZpt07oBRFiJJGo85zxEk2Rp4A/BMBj2B\nrwEnVtX/9lybJGkEhrlq6DTgduAfu+U/Aj4OvKyvoiRJozNMEDypqnabsnxhku/0VZAkabSGuaFs\ndZJ9JheS/C6wsr+SJEmjNEyPYE/gP5L8qFteAlydZA1QVfWU3qqTJPVumCA4sPcqJEljM8ydxdcn\n2R7Yeer63lAmSQvDMJePHs1gbqEf0E08hzeUSdKCMczQ0KEMpqK+s+9iJEmjN8xVQ2uBR/RdiCRp\nPIbpEbwf+HaStUx5DkFV/UFvVUmSRmaYIPgYcCywBri333IkSaM2TBDcUVX/0HslkqSxGCYIvpbk\n/cDnuO/QkJePStICMEwQPK173WdK25yXjyY5BTgIuLmqntS17QCcBUwA1wGHVtXPHljJkqRNac6r\nhqrqgGl+hrmH4FTuf1fykcAFVbUrcEG3LEkao2F6BCR5EfBEYOvJtqr629m2qaqLk0xs0PxiYP/u\n/ceAi4C/HKpSSVIv5uwRJDkReDlwBIOH178MePQ8j7djVU0++/inwI7z3I8kaRMZ5oayZ1TVK4Gf\nVdV7gKcDj9vYA1dV8espK+4nyfIkK5OsXLdu3cYeTpI0g2GC4Jfd6x1JHgncBew0z+PdlGQngO71\n5plWrKoVVbW0qpYuXrx4noeTJM1lmCD4QpJHAB8AVjO42ueMeR7vc8Bh3fvDgM/Ocz+SpE1kmGmo\nj+7enp3kC8DWVXXbXNslOZPBieFFSX4CvBs4BvhkksOB6xlMaCdJGqMZgyDJXsCPq+qn3fIrgYOB\n65McVVW3zrbjqlo2w0fPnW+xkqRNb7ahoY8CdwIk2Y/BX/OnAbcBK/ovTZI0CrMNDW0x5a/+lwMr\nqupsBkNEl/VfmiRpFGbrEWyRZDIongt8ZcpnQ92IJkna/M32hX4m8NUktzC4hPRrAEkey2B4SJK0\nAMwYBFX1viQXMLhn4EvdDWAw6EUcMYriJEn9m3WIp6oumabtmv7KkSSN2jA3lEmSFjCDQJIaZxBI\nUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1\nziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuO2\nHMdBk1wH3A7cA9xdVUvHUYckaUxB0Dmgqm4Z4/ElSTg0JEnNG1cQFPClJKuSLJ9uhSTLk6xMsnLd\nunUjLk+S2jGuIHhmVe0BvAB4Y5L9NlyhqlZU1dKqWrp48eLRVyhJjRhLEFTVDd3rzcC5wN7jqEOS\nNIYgSPKwJNtNvgd+D1g76jokSQPjuGpoR+DcJJPHP6OqvjiGOiRJjCEIqupa4KmjPq4kaXpePipJ\njTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4\ng0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMI\nJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklq3FiCIMmBSa5O8v0kR46jBknSwMiDIMkWwD8B\nLwB2A5Yl2W3UdUiSBsbRI9gb+H5VXVtVdwKfAF48hjokSYwnCB4F/HjK8k+6NknSGGw57gJmkmQ5\nsLxbXJ/k6nHWs4AsAm4ZdxGbgxw77go0A39Hp9jI39NHD7PSOILgBmDnKcu/1bXdR1WtAFaMqqhW\nJFlZVUvHXYc0E39HR28cQ0PfAnZNskuShwJ/CHxuDHVIkhhDj6Cq7k7yJuDfgS2AU6rqylHXIUka\nGMs5gqo6Hzh/HMeWw23a7Pk7OmKpqnHXIEkaI6eYkKTGGQQNSnJREq/KUK+S/HPfswYkOT/JI6Zp\nPyrJO/o89kKy2d5HoI2TJAyG/u4ddy1qU1X92QiO8cK+j9ECewQLSJKJbjK/04C1wCuS/GeS1Uk+\nlWTbabZZP+X9IUlOHWHJWiCSPCzJeUkuT7I2ycun9jyTHJ7kmiSXJjkpyYe79lOTnJDkkiTXJtk/\nySlJrpr6u5hkWZI13b6PndJ+XZJF3fu/6o7xdeDxo/0XeHAzCBaeXYGPAM8GDgeeV1V7ACuBPx9n\nYVrQDgT+q6qeWlVPAr44+UGSRwLvAvYB9gWesMG22wNPB97G4J6i44EnAk9Osnu3/bHAc4Ddgb2S\nvGTqDpLsyeCepN2BFwJ7bfL/wgXMIFh4rq+qSxj8T7cb8I0klwGHMeTt5tI8rAGen+TYJM+qqtum\nfLY38NWqurWq7gI+tcG2n6/B5YtrgJuqak03pHklMMHgS/2iqlpXVXcDpwP7bbCPZwHnVtUdVfVz\nvEn1AfEcwcLzi+41wJeratkc60+9fnjrfkrSQldV1yTZg8Ff4+9NcsED2PxX3eu9U95PLm8J3LVp\nqtRM7BEsXJcA+yZ5LPz/GO7jplnvpiS/k+QhwEtHWqEWjG745o6q+lfgA8AeUz7+FvDsJNsn2RI4\n+AHu/tJu+0Xd80yWAV/dYJ2LgZck2SbJdsDvz+s/pFH2CBaoqlqX5FXAmUl+o2v+a+CaDVY9EvgC\nsI7BeYT7nVCWhvBk4ANJ7mXwF/zrgeMAquqGJH/H4Av9VuC7wG0z7WhDVXVj9yTDCxn0dM+rqs9u\nsM7qJGcBlwM3MwgfDck7iyX1Lsm2VbW+6xGcy2COsXPHXZcGHBqSNApHdRctrAV+CHxmzPVoCnsE\nktQ4ewSS1DiDQJIaZxBIUuMMAqnTzXPzjCnLr0vyynnu61XdtfWTy73PxCnNlyeLpU6So4D1VXXc\nJtjXRcA7qmrlxu5L6ps9Ai14ST6TZFWSK5Ms79oO7GZlvTzJBUkmgNcBb0tyWZJnTc5pn+QJSS6d\nsr+JJGu693+T5FvdrJgrMnAIsBQ4vdvXNhvMxDnTTJrrk7yvq+mSJDuO7l9JLTMI1IJXV9WeDL6c\n39x9wZ4EHFxVTwVeVlXXAScCx1fV7lX1tcmNq+q7wEOT7NI1vRw4q3v/4araq5txcxvgoKr6NIO7\ntP+429cvJ/c1x0yaDwMu6Wq6GHjNpv+nkO7PIFAL3pzkcgbzL+0MLAcurqofAlTVrUPs45MMAgDu\nGwQHJPlm10N4DoPpk2cz20yadzKY7gNgFYOZN6XeGQRa0JLsDzwPeHr3l/a3gcvmsauzgEO7ifuq\nqr6XZGsGz344pKqezKCXsTEzuN5Vvz5pdw/OBaYRMQi00D0c+FlV3ZHkCQye07A1sN/kUE+SHbp1\nbwe2m24nVfUDBl/O7+LXvYHJL/1buqe/HTJlk5n2NcxMmtJI+ReHFrovAq9LchVwNYPhoXUMhofO\n6abfvhl4PvB54NNJXgwcMc2+zmIwxfIuAFX1P0lOYjB/zk+574yXpwInJvklg6dv0W0z50ya0qh5\n+agkNc6hIUlqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLj/g+27IpfuBANXgAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc2704bb860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plot_histogram(result=search_result,\n",
    "                         dimension_name='activation')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们也可以制作一个landscape-plot是关于显示二个维度搜索空间上估计的适度值，这里采用`learning_rate`和`num_dense_layers`两个变量。\n",
    "\n",
    "贝叶斯优化器的工作原理是建立一个搜索空间的代理模型，然后搜索这个模型而不是真正的搜索空间，因为这样更快。这个图展示了贝叶斯优化器构建的最后一个代理模型，黄色区域比较好，蓝色区域比较糟。黑点显示了优化器对搜索空间的采样，而红星显示了最佳参数。\n",
    "\n",
    "这里应该注意几点。首先，这种搜索空间的代理模型可能不准确。它仅通过调用`fitness（）`函数的40个样本构建，用于训练具有给定超参数选择的神经网络。模拟的landscape可能与它的真实值有很大的不同，尤其是在大搜索空间的区域取很少的样本情况下。其次，由于在神经网络的训练过程中随机产生的噪声，所以每次运行超参数优化时，情况可能会发生变化。第三，这副图显示了改变2个参数`num_dense_layers`和 `learning_rate` 的效果，这也称为部分依赖图，是一种仅在二维中可视化高维空间的方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XucHGWd7/HPd+4zCSRLQiQJxHBJUAkXIQIBTCKGgJIT\ndNWzeHC9rhwVFNfLKrLHXV09rrruusgLEVdfGLyAgrIxB+UmJCABmUACIUCIAQJJcEggCXO/9O/8\n0T0wPZmZTM10TfdMvu/Xq1/d9dRTVb+uqelfP/VU16OIwMzMrFtZsQMwM7PS4sRgZmZ5nBjMzCyP\nE4OZmeVxYjAzszxODGZmlseJwczM8jgxmJlZHicGMzPL48RgZmZ5KoodwFBMnjw5Zs6cWewwzMxG\nlTVr1uyIiIP3VW9UJoaZM2dSX19f7DDMzEYVSc8Mpp5PJZmZWR4nBjMzy+PEYGZmeZwYzMwsjxOD\nmZnlST0xSHpa0iOS1kra61IiZV0uaZOkhyWdmHZMZmbWv5G6XPUtEbGjn3lvA2blHqcA3889m5lZ\nEZTCqaTzgGWRdR8wUdLUYgdlNlotX76ciy++mOXLl5fEeoYraRyFjrtU9sNll13Gsccey2WXXZb6\nthQR6W5Aegp4CQjgBxFxda/5K4B/jYh7ctN3AF+IiH5/wXagDopTKxanGHXxldXWFnX7qq0Z/kqq\nqwdft2bgulFT2f+86r7nZar3bhB3VZf3ms7/bpTpMd1VpR71eryuYq86mVxZV49QussyVZCp7K6f\n/X+L3PTmSz7TZ+z9+dADHxpw/pZVW1j5jyvpau2ioqacc75xGkcsmJ5oGwCbV27l95feS2drF5U1\n5bzzm3OZ/ZapTKhozqs3sTx/ekKvaYCJZb2XaelzmxPL2vcqu+PWVv7+4pdobYHaWvjBlQdxzuK9\nj80DlT1+VtzSxPs/8QLNLUFdrVh25cEsOXvcwG+2hz3R9srrXZnI235Nrbj0P6cxb9EB7MrUsbur\nLluvq47dnXXs6axhT2c2tsaOGho7q2jqqKKlvZLWjkraOirobC8n01EOjeWUdZShDihvF2UdUNae\nfVQ2QXl7UN4O5W1BeXvwzMM38+zmO1+J7Utf+hJf//rXB/2+uklaExFz91VvJE4lnRERWyVNAW6T\n9HhErEq6EkkXAhcC1FBX6BjNUlXW8Wpy6Gnmlf9GVAWqymTrVXZRUdVFdWUnADWVHdRWdTCusp3x\nFe2M7z8/ArDt/m10tXYB0NnaxZb7tg8pMWy5bzudufV0tHaxeXUDs98y8g35e1a10ZrLIy0tcNfK\n1j4TQ7fbV7XQ3JJNvs0twe2rWhIlhoG239oSrLm7iXmLDhhwmcaOgb9URXsZ6mdeeUff5S/+ZUPe\n9PLly4eUGAYr9VNJEbE199wA/AY4uVeVrcBhPaYPzZX1Xs/VETE3IuZWkuCbqNl+ZNop0yivybaK\nKmrKmXHq0D7MZ5w6lYrceipryjli3pSCxZjEGfOrqck1nmtrYeGCgT90F82vpa42+7FbVysWzR9e\ny7vn9mtqxUlvHnqSGY7Jk1+fN7106dJUt5dqi0HSOKAsIl7OvV4MfLVXteXAxZKuI9vpvDsitqcZ\nl9lYNWP+DBZ8bQE7HtjCjFOnDqm1AHDEgumc843TeP7+5zhi3pSitBYA3rq4hv+44q944O42Fi6o\nGbC1ALDk7HEsuzLbclg0v3ZYrYWe2//Dqg5OeXMtx505cGshLUcceTYABxywg6VLl6baWoD0TyW9\nBviNpO5t/Twifi/pYwARcRVwM/B2YBPQDAx8EtXMBjRj/gzmnDn8b/hHLJjOCW+dVICIhueti2t4\n1zmD/+a/5Oxxw04Ivbd/0qIDAdjVVbDVJnbEkWfzhzu+OCLbSjUxRMRm4Pg+yq/q8TqAi9KMw8zM\nBq8ULlc1M7MS4sRgZmZ5nBjMzCyPE4OZmeVxYjAzszxODGZmlseJwczM8jgxmJlZHicGMzPL48Rg\nZmZ5nBjMzCyPE4OZmeVxYjAzszxODGZmlseJwczM8oxIYpBULukhSSv6mPdBSS9IWpt7/N1IxGRm\nZn1LewS3bpcAjwEH9jP/+oi4eIRiMTOzAaTeYpB0KHAu8F9pb8vMzIZvJE4lfRf4ByAzQJ13SXpY\n0g2SDhuBmMzMrB+pJgZJS4CGiFgzQLXfAjMj4jjgNuAn/azrQkn1kuo7aEshWjMzg/RbDKcDSyU9\nDVwHnCnppz0rRMTOiOj+pP8v4KS+VhQRV0fE3IiYW0l1mjGbme3XUk0MEXFpRBwaETOB84E/RMT7\netaRNLXH5FKyndRmZlYkI3VVUh5JXwXqI2I58ClJS4FO4EXgg8WIyczMskYsMUTEXcBduddf7lF+\nKXDpSMVhZmYD8y+fzcwsjxODmZnlcWIwM7M8TgxmZpbHicHMzPI4MZiZWR4nBjMzy+PEYGZmeZwY\nzMwsjxODmZnlcWIwM7M8TgxmZpbHicHMzPI4MZiZWR4nBjMzy+PEYGZmeUYkMUgql/SQpBV9zKuW\ndL2kTZLulzRzJGIys/1EBGW/boSIYkcyaoxUi+ES+h/L+SPASxFxFPAfwDdHKCYz2w/o4XYqL34B\nPdJe7FBGjdSH9pR0KHAu8HXgM31UOQ/459zrG4ArJCki/fTekNnKznieSTqEKWXT095cYg2dW9jR\nuY3JFdOYUjGj2OFYATWtX8/Lf1xNlMH4haegiqBl/ZPUnXAkE+bNzqu7895NPPPQZg49ZRqzzzwk\n0XY2r9zKlvu2M+PUqRyxoPSO8UJacUsTt69qYdH8WpacPQ5e6IIItnz7JY4Etnz7JQ77zmSQ4ODy\nYodb0kZizOfvAv8AHNDP/OnAswAR0SlpNzAJ2JFmUA2ZrTySWU2GLrbFUxzLvJJKDg2dW1jXspIM\nXWzteJLjaxc4OYwRTY+up+HaZdDVBUDrYxuhTNDZRdPd9ZRXvIfq044CYNd9T7L5myvItHWy9eYN\n1FTMZ8b8wR0Hm1du5feX3ktnaxcb/vspzvnGaWM2Oay4pYn3f+IFmluCZdc18t9f6GDRP79ECKYF\nCJh2RwtVb3wWBbTfMo04trrYYZesVE8lSVoCNETEmgKs60JJ9ZLqO2gbdmw743kyZP8xM3SxM54f\n9joLaUfntrz4dnRuK3JEVigtGze+khQAyGSgMzsd7R00r9v0yqzdDz5Fpq0TgK62TrbdP/jjYMt9\n2+lsza63s7WLLfdtL0D0pen2VS00t2RPMjS3BDc+00nHT15Da8Wr334rgNYK6Fj2GieFfUi7j+F0\nYKmkp4HrgDMl/bRXna3AYQCSKoAJwM7eK4qIqyNibkTMrWT4f9RJOoQyss3JMsqZpGRN9LRNrpiW\nF9/kimlFjsgKpXb2bCjvcSqjrAwqstOqqqTu+KNemTXhxMMpq85+tJVXVzDtlMEfBzNOnUpFTXa9\nFTXlzDh1agGiL02L5tdSVysA6mrFovm1ZM6q45nFdQjoJNtqeGZxHZlFdcUMdVRI9VRSRFwKXAog\naSHwuYh4X69qy4EPAKuBdwN/GIn+hSll0zmWeSXbxzClYgbH1y5wH8MYNO6YOUz50Pv77WMY/6bZ\nZD/KYOKps3jdZUtozPUxzJg/+C8wRyyYzjnfOG2/6GNYcvY4ll1Jfh8DcPTaNhA8NKOCN27p5Oh1\n7bgLet9Goo9hL5K+CtRHxHLgR8C1kjYBLwLnj1QcU8qmM4XS/WeZUjHDCWGMGjdnDuPmzCFTmSGq\nAlVlqDvxDZRVdgFdeXUnnXYUMxbMYHxFO9CaaDtHLJg+phNCT0vOHvdKQgCgK8gcUUnXD6ZwzIk1\ndKxppfxbL0FXQLmKF+goMOjEIOlI4LmIaMt9+z8OWBYRuwazfETcBdyVe/3lHuWtwHsGH7KZ2SCU\ni87rXz19FifV5E1b/5L0MdwIdEk6CriabL/Az1OJyszMiiZJYshERCfwTuB7EfF5wOnXzGyMSZIY\nOiS9l2xHcfetLSoLH5KZmRVTksTwIWAe8PWIeErS4cC16YRlZmbFMqjOZ0nlwGURcUF3WUQ8he9r\nZGY25gyqxRARXcBrJVWlHI+ZmRVZkt8xbAb+KGk50NRdGBH/XvCozMysaJIkhj/nHmX0f0M8MzMb\n5QadGCLiKwCS6iKiOb2QzMysmAZ9VZKkeZI2AI/npo+XdGVqkZmZWVEkuVz1u8DZ5O58GhHrgPlp\nBGVmZsWT6LbbEfFsr6KuPiuamdmolaTz+VlJpwEhqZKBx3E2M7NRKkmL4WPARWSH4twKnJCbNjOz\nMSRJiyF6/vLZzMzGpiQthvsk/UrS2yQNapQLSTWS/iRpnaRHJX2ljzoflPSCpLW5x98liMnMzAos\nSYthNrAI+DDwPUm/BK6JiI0DLNMGnBkRjbl+iXsk/S4i7utV7/qIuDhR5GZmlopBtxgi67aIeC/w\nUbK33/6TpJWS5g2wTGNusjL3SH08ZzMzG7okP3CbJOkSSfXA54BPApOBzzLASG6SyiWtBRqA2yLi\n/j6qvUvSw5JukHRYsrdgZmaFlKSPYTVwIPCOiDg3In4dEZ0RUQ9c1d9CEdEVEScAhwInS5rTq8pv\ngZkRcRxwG/CTvtYj6UJJ9ZLqO2hLELaZmSWRpI/h6Ijo8zRQROxzXIaI2CXpTuAcYH2P8p09qv0X\n8K1+lr+a7FjTHKiDfDrKzCwlSRLDZEn/ABwD1HQXRsSZ/S0g6WCgI5cUaoGz6DW4j6SpEbE9N7kU\n/2jOzKyokpxK+hnZG+gdDnwFeBp4YB/LTAXulPRwru5tEbFC0lclLc3V+VTuUtZ1wKeADyaIyczM\nCixJi2FSRPxI0iURsRJYKWnAxBARDwNv7KP8yz1eXwpcmiAOMzNLUZLE0JF73i7pXGAbcFDhQzIz\ns2JKkhi+JmkC2ctTv0f2CqW/TyUqMzMrmiQjuK3IvdwNvCWdcMzMrNj2mRgkfY8Bfq0cEZ8qaERm\nZlZUg2kx1KcehZmZlYx9JoaI6POXyL1J+l5EfHL4IZmZWTElGtpzH04v4LrMzKxICpkYzMxsDHBi\nMDOzPIVMDIMa1c3MzEpb4sQgqa6fWf85zFjMzKwEJBmo5zRJG8jeSA9Jx0u6snt+RFxT+PDMzGyk\nJWkx/AdwNrATICLWAfPTCMrMzIon0amkiHi2V1FXAWMxM7MSkOQmes9KOg0ISZXAJXhQHTOzMSdJ\ni+FjwEXAdGArcEJuul+SaiT9SdK63GA8X+mjTrWk6yVtknS/pJkJYjIzswJLcnfVHcAFCdffBpwZ\nEY25VsY9kn4XEff1qPMR4KWIOErS+WSH/vybhNsxM7MCSXJV0rckHSipUtIdkl6Q9L6Blomsxtxk\nZe7R+06t5wHd92O6AXirJP8mYj/U0PgkGxpuo6HxyWKHMqKa1z7KzmX/TfODG4odSqpW3/4yV/zT\n86y6rbnYoQxb/e27+PlX/8zaO3YOaz1Nj66nYfmNND6+vkCRFUaSU0mLI2IPsITseM9HAZ/f10KS\nyiWtBRrIjvl8f68q04FnASKik+x4D5MSxGVjQEPjk6x9/rds2f0Qa5//7X6THJoffpSdP/w5jXes\nZsf3f0HTmrHZbbf69pf5v5dsY/m1u/jHT+4Y1clh9e0v891PP8WdP9vODz/zBI/+oWFI62lav56G\nn/2U3ff/kW03XsvLG0snOSRJDN2nnc4FfhURuwezUER0RcQJwKHAyZLmJIwRAEkXSqqXVN9B21BW\nYSVsR/PTZKITgEx0sqP56eIGNEJaHttItGdHzY32DloeGZsJcc3dTbS1ZE8WtLYE99/dUuSIhm7N\n3U20t2bfS3trho33Dq3V0PL4RqIj97fv6KDpqScKFuNwJUkMKyQ9DpwE3CHpYKB1sAtHxC7gTuCc\nXrO2AocBSKoAJpD7rUSv5a+OiLkRMbeS6gRh22gwuW4mZcp+9yhTBZPrZhY3oBFS+/rZqKoSAFVV\nUnvsrCJHlI6T3jyO6trsGeKaWnHKm2uLHNHQnfTmcVTVZN9LVU0Zs08b2gmO2tfNRpW5v31lJeMO\nP7pgMQ5Xks7nL0r6FrA7IrokNZHtH+hXLnl0RMQuSbXAWWQ7l3taDnwAWA28G/hDRPQ7YpyNTVPG\nz+KEQ/4HO5qfZnLdTKaMH5sfkL3VHXcMkz76v2h9fCO1c2Yx7qSjGYs/D5q36AC+9J/TWHN3E/MX\nVDL/rP7urFP65i06gE9/93AeuLuFN5w+kZkLp7C7M/l6xs2Zgy54H62PbWT84UdzwBFzoKnw8Q5F\nkt8xALwOmJn7Zt9t2QD1pwI/kVROtnXyy4hYIemrQH1ELAd+BFwraRPwInB+wphsjJgyftZ+kxB6\nqjvhGMad/Prc1NhLCt3mLTqAeYsOYGL56D2N1G3uoonMess0AHYN40827pg5HDjrWMo6gPbCxFYI\ng04Mkq4FjgTW8urRGwyQGCLiYeCNfZR/ucfrVuA9g43DzMzSlaTFMBd4g0/zmJmNbUk6n9cDh6QV\niJmZlYYkLYbJwAZJf4JXrxeNiKUFj8rMzIomSWL457SCMDOz0pHkctWVkl4LzIqI23MjuZWnF5qZ\nmRVDknslfZTsvYx+kCuaDtyURlBmZlY8STqfLwJOB/YARMSTwJQ0gjIzs+JJkhjaIuKVn2DkfuTm\nS1fNzMaYJIlhpaQvAbWSzgJ+Bfw2nbDMzKxYkiSGLwIvAI8A/xu4GfjHNIIyM7PiSXJVUgb4Ye5h\nZmZj1D4Tg6RHGKAvISKOK2hEZmZWVINpMSzJPV+Ue7429/w+3PlsZjbm7DMxRMQzAJLOioied0r9\ngqQHyfY9mJnZGJGk81mSTu8xcVrC5c3MbBRIcq+kjwA/ljQhN70L+HDhQzIzs2Ia9Df+iFgTEccD\nxwPHR8QJEfFg93xJH+i9jKTDJN0paYOkRyVd0kedhZJ2S1qbe3y5dx0zMxs5SYf2JCJ29zPrEuAn\nvco6gc9GxIOSDgDWSLotIjb0qnd3RCzBzMyKrpB9BOpdEBHbu1sVEfEy8BjZm++ZmVmJKmRiGPDS\nVUkzyY7/fH8fs+dJWifpd5KOKWBMZmaWUOJTSQPYq8XwygxpPHAj8OmI2NNr9oPAayOiUdLbyd7K\ne1Yf67gQuBCghrqCBW1mZvkK2WL4Y1+FkirJJoWfRcSve8+PiD0R0Zh7fTNQKWlyH/Wujoi5ETG3\nkuoChm1mZj0NusUgaSLwfmBmz+Ui4lO554v7WEbAj4DHIuLf+1nvIcBfIiIknUw2We1M8B7MzKyA\nkpxKuhm4j+zdVTODXOZ04G+BRyStzZV9CZgBEBFXAe8GPi6pE2gBzo8I32rDzKxIkiSGmoj4TJKV\nR8Q9DND3kKtzBXBFkvWamVl6kvQxXCvpo5KmSjqo+5FaZGZmVhRJWgztwLeBy3j10tQAjih0UGZm\nVjxJEsNngaMiYkdawZiZWfElOZW0CWhOKxAzMysNSVoMTcBaSXcCbd2F3ZermpnZ2JAkMdyUe5iZ\n2Rg26MQQEb3vnGpmZmNQkl8+P0UfN8qLCF+VZGY2hiQ5lTS3x+sa4D2Af8dgZjbGJBnBbWePx9aI\n+C5wboqxmZlZESQ5lXRij8kysi2IQt6228zMSkCSD/bv8GofQyfwNNnTSWZmNoYkSQxvA95F/m23\nzwe+WuCYzMysiJL+jmEX2RHXWtMJx8zMii1JYjg0Is5JLRIzMysJSe6VdK+kY5OsXNJhku6UtEHS\no5Iu6aOOJF0uaZOkh3t1cpuZ2QhL0mI4A/hg7odubWQH4ImIOG6AZTqBz0bEg5IOANZIui0iNvSo\n8zZgVu5xCvD93LOZmRVB0s7nRCJiO7A99/plSY8B04GeieE8YFluOM/7JE2UNDW3bJ9aaaYhs5Up\nZdOThjSqNXRuYUfnNiZXTGNKxYxUt7Nzz1+YXH0YU6pnpradkfTCzsfYuWsTf3XwbCYf/IbUt7dn\n03oan32CcYcfzbg5cwqyzm13P82La55m2inTmDE/vb9/XzbeuZ3NqxuYc8YEjjlzyohs845bW7ln\nVRtnzK/mXefUJl5+xS1N3L6qhUXza1ly9rhBz0tb89pHaX3kSepmH834owtzbBRaknslPTOcDUma\nCbwRuL/XrOnAsz2mn8uV9ZsYOmjjkcxqjmXefpMcGjq3sK5lJRm62NrxJMfXLkglOfTcznOtT3DC\nhEWjPjm8sPMxHtn4SzKZDrY1PMgxx5yfanLYs2k9z624lujsYNe6PzGt4m8Z/7rhfQA0/HEz675y\nO11tnTz52ydZ8LUFI5YcNt65nd98oZ6O1i7W3VTGBf92XOrJ4Y5bW/n7i1+itQVu/GUz464U5yyu\nGfTyK25p4v2feIHmlmDZdY0su5JXEsBA89LWtOYxdv7weqK9g6bVD6AL3seBsxKdoR8RSfoYhkzS\neOBG4NMRsWeI67hQUr2keoAMXeyM5wsZZknb0bmNDF1A9r3v6Nw2AtvpZEfbs/tYovTt3LWJTKYD\ngEymgxdffDLV7TU+/QTRmd1edHbQ9Ocnhr3OHQ9soautE4Cu1i623Z/O378vm1c30NGaPSY6WjNs\nvHdn6tu8Z1UbrS3Z160tcNfKZBdC3r6qheaW7M+umluC21e1DGpe2loeeZJozx0bHR20bNw4YttO\nIvXEIKmSbFL4WUT8uo8qW4HDekwfmivLExFXR8TciJgLUEY5k3RIGiGXpMkV0yijHMi+98kV00Zg\nOxVMrj5sH0uUvkkTj6KsrBKAsrJKDjpoVqrbGz/zaFSR3Z4qKhl35NHDXufkN82gvDrbwC+vKWfa\nKen8/ftyxLwpVNZkj4nKmjJmnzYp9W2eMb+amtzZo5paWLhg8K0FgEXza6mrFQB1tWLR/NpBzUtb\n7bGzUFXu2KispHb27BHbdhLKntpPaeWSgJ8AL0bEp/upcy5wMfB2sp3Ol0fEyQOtt0o18YayN43p\n00hltXsfrCPaxxAF6GOorh583ZqB60ZNZf/zqvuel6l+9Uxpf30MXdVlvZZ5dbqrSj3q9XhdxV51\nMrmyrlwoffUxZCqhqyr7/xa5epnKDFEVqCoDQFllFxVVXVRXZlsHNZUd1FZ1MK6ynT2rNw66j+HA\nisL81Kh7Pf31MUwszx/UcUL53oM8TizLL5tY3vc39Ill7XnTSfoYDtTex89Q+hj2xCtjkLErEz1e\nZ//Au7pqc9N17O6qy5XVsbuzjj2dNezprKGxI5vEGjuraOqooqW9ktaOSto6KuhsL+flP27M62Mo\nbxdlHVDWDpVN2efy9qC8HcrbgvL2oHJP9nj4wx1fHHA/7IukNd1frgesl3JiOAO4G3gEyOSKvwTM\nAIiIq3LJ4wrgHLJDh34oIuoHWu+BOihOrVicWtyloK/EMJJUm+wbWp9KKDF066ou7zWdTmLoWfbK\n8zATw/iKdsZXDu4Dv9CJoduEit4f8uklhvx56nce9J0YhmIkEkNXUyVqF2UdZaiDkkwMqd4ELyLu\nIXtZ60B1ArgozTjMzGzwRqTz2czMRg8nBjMzy+PEYGZmeZwYzMwsjxODmZnlcWIwM7M8TgxmZpbH\nicHMzPI4MZiZWR4nBjMzy+PEYGZmeZwYzMwsjxODmZnlcWIwM7M8TgxmZpYn1cQg6ceSGiSt72f+\nQkm7Ja3NPb6cZjxmZrZvqQ7UA1xDdnS2ZQPUuTsilqQch5mZDVKqLYaIWAW8mOY2zMyssEqhj2Ge\npHWSfifpmGIHY2a2v0v7VNK+PAi8NiIaJb0duAmY1VdFSRcCFwLUUDdyEZpZv3Z11TGxvLnYYYyo\nXZmx//lT1BZDROyJiMbc65uBSkmT+6l7dUTMjYi5lVSPaJxmZvuToiYGSYdIUu71ybl4dhYzJjOz\n/V2qp5Ik/QJYCEyW9BzwT0AlQERcBbwb+LikTqAFOD8iIs2YzMxsYKkmhoh47z7mX0H2clYzMysR\npXBVkpmZlRAnBjMzy+PEYGZmeZwYzMwsjxODmZnlcWIwM7M8TgxmZpbHicHMzPI4MZiZWR4nBjMz\ny+PEYGZmeZwYzMwsjxODmZnlcWIwM7M8TgxmZpbHicHMzPKkmhgk/VhSg6T1/cyXpMslbZL0sKQT\n04zHzMz2Le0WwzXAOQPMfxswK/e4EPh+yvGYmdk+pD205ypJMweoch6wLDfO832SJkqaGhHb04xr\nOBoyW9kZzzNJhzClbHqxw0msoXMLOzq3MbliGlMqZhQ7nJKx8y8beGnHRiZMPZpJU48pdjh5dt67\niWce2syhp0xj9pmHjOi2N965nc2rGzhi3hRmv2XqiG67p9/f2spdK1tZuKAG4JXX5yyuSbSeFbc0\ncfuqFhbNr2XJ2ePSCHVMSDUxDMJ04Nke08/lykoyMTRktvJIZjUZutgWT3Es80ZVcmjo3MK6lpVk\n6GJrx5McX7vAyYFsUnhs7c/JZDp4fms9R8+9oGSSw677nmTzN1eQaetk680bqKmYz4z5I/M327xy\nK7dcWk9HaxfrbtrCO785lzedNWFEtt3THbe28pmLX6KlBX7682YA2tvhF9c384MrDxp0clhxSxPv\n/8QLNLcEy65rZNmVDCk5TCxrZlemLvFyo0pEpPoAZgLr+5m3Ajijx/QdwNx+6l4I1AP1M2bMiGK4\n6KKLAnjlcdFFFxUljqEa7fGnpZT3SzFjK5X90juOocZUKu+nmID6GMTndrGvStoKHNZj+tBc2V4i\n4uqImBsRcw8++OARCa63xYsXU1eX/aZQV1fH4sWLixLHUI32+NNSyvulmLGVyn7pGUdVVRXV1dVD\niqlU3s9ooGwSSXED2T6GFRExp4955wIXA28HTgEuj4iT97XOuXPnRn19fYEjHZzly5dz6623snjx\nYpYuXVqUGIZjtMefllLeL8WMrVT2S884gCHHVCrvp1gkrYmIufusl2ZikPQLYCEwGfgL8E9AJUBE\nXCVJwBVkr1xqBj4UEfv8xC9mYjAzG60GmxjSvirpvfuYH8BFacZgZmbJFLuPwczMSowTg5mZ5XFi\nMDOzPE4MZmaWx4nBzMzyODGYmVme1H/glgZJLwNPpLiJCcDuFJfbV73+5vdVPpiy3tOTgR2DiHOo\nirn/ks4tEcpZAAAGNklEQVRLuv/G8r4baP5gy/fnY2+g+aXyv/vaiNj3rSMGc9+MUnswyPt9DGP9\nV6e53L7q9Te/r/LBlPUxPWb3X9J5SfffWN53A80fbPn+fOwl3X+l+L/b/fCppL79NuXl9lWvv/l9\nlQ+mbKjvZ6iKuf+Sziu1/Teajr2+yvfnY2+g+aPh2HvFaD2VVB+D+Fm39c37b+i874bH+294Rmr/\njdYWw9XFDmCU8/4bOu+74fH+G54R2X+jssVgZmbpGa0tBjMzS4kTg5mZ5XFiMDOzPGMuMUgqk/R1\nSd+T9IFixzPaSFoo6W5JV0laWOx4RhtJ4yTVS1pS7FhGG0mvzx13N0j6eLHjGW0kvUPSDyVdL2lY\n45aWVGKQ9GNJDZLW9yo/R9ITkjZJ+uI+VnMe2bGjO4Dn0oq1FBVo/wXQCNSwH+2/Au07gC8Av0wn\nytJViP0XEY9FxMeA/wmcnma8paZA+++miPgo8DHgb4YVTyldlSRpPtkPpWWRGyNaUjmwETiL7AfV\nA8B7gXLgG71W8eHc46WI+IGkGyLi3SMVf7EVaP/tiIiMpNcA/x4RF4xU/MVUoH13PDCJbFLdEREr\nRib64ivE/ouIBklLgY8D10bEz0cq/mIr1P7LLfcd4GcR8eBQ40l1aM+kImKVpJm9ik8GNkXEZgBJ\n1wHnRcQ3gL2a65KeA9pzk13pRVt6CrH/engJqE4jzlJUoGNvITAOeAPQIunmiMikGXepKNSxFxHL\ngeWS/h+w3ySGAh1/Av4V+N1wkgKUWGLox3Tg2R7TzwGnDFD/18D3JL0ZWJVmYKNEov0n6a+Bs4GJ\nwBXphlbyEu27iLgMQNIHybW8Uo2u9CU99hYCf032C8nNqUY2OiT97PsksAiYIOmoiLhqqBseDYkh\nkYhoBj5S7DhGq4j4NdnkakMUEdcUO4bRKCLuAu4qchijVkRcDlxeiHWVVOdzP7YCh/WYPjRXZoPj\n/Td03nfD4/03PEXbf6MhMTwAzJJ0uKQq4HxgeZFjGk28/4bO+254vP+Gp2j7r6QSg6RfAKuBoyU9\nJ+kjEdEJXAzcAjwG/DIiHi1mnKXK+2/ovO+Gx/tveEpt/5XU5apmZlZ8JdViMDOz4nNiMDOzPE4M\nZmaWx4nBzMzyODGYmVkeJwYzM8vjxGBmZnmcGGxMkdQ4AttYOsixGdLY9jskvaEY27b9h3/gZmOK\npMaIGF+A9ZRHRFFu2z7QtiVdA6yIiBtGNirbn7jFYGOWpM9LekDSw5K+0qP8JklrJD0q6cIe5Y2S\nviNpHTBP0tOSviLpQUmPSHpdrt4HJV2Re32NpMsl3Stps6R358rLJF0p6XFJt0m6uXteP7E+Lemb\nkh4E3iPpo7nY10m6UVKdpNOApcC3Ja2VdGTu8fvc+7m7O0az4XBisDFJ2TFvZ5Ed7OQE4KTcKFmQ\nHe3qJGAu8ClJk3Ll44D7I+L4iLgnV7YjIk4Evg98rp/NTQXOIDt4yr/myv4amEl20J6/BeYNIuyd\nEXFiRFwH/Doi3hQRx5O9T85HIuJesjdR+3xEnBARfwauBj6Zez+fA64cxHbMBjTmxmMwy1mcezyU\nmx5PNlGsIpsM3pkrPyxXvpPsiH839lpP99gUa8h+2PflptygPBtyQ6JCNlH8Klf+vKQ7BxHz9T1e\nz5H0NbIDJo0neyO1PJLGA6cBv8oO3gXsR6PuWXqcGGysEvCNiPhBXmF2lLBFwLyIaJZ0F9kxmgFa\n+zi335Z77qL//5e2Hq/VT53BaOrx+hrgHRGxLjci3MI+6pcBuyLihGFs02wvPpVkY9UtwIdz36qR\nNF3SFGAC8FIuKbwOODWl7f8ReFeur+E19P3BPpADgO2SKoELepS/nJtHROwBnpL0HsiO+Svp+GFH\nbvs9JwYbkyLiVrKDya+W9AhwA9kP1N8DFZIeI9sfcF9KIdxIdozeDcBPgQeB3QmW/z/A/WQTzOM9\nyq8DPi/pIUlHkk0aH8l1mD8KnFeA2G0/58tVzVIiaXxENOY6t/8EnB4Rzxc7LrN9cR+DWXpWSJoI\nVAH/4qRgo4VbDGYjSNJvgMN7FX8hIva66sisWJwYzMwsjzufzcwsjxODmZnlcWIwM7M8TgxmZpbH\nicHMzPL8fy6cRQja3MY6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc270556320>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plot_objective_2D(result=search_result,\n",
    "                        dimension_name1='learning_rate',\n",
    "                        dimension_name2='num_dense_layers',\n",
    "                        levels=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们不能制作一个landscape关于 `activation`超参数的图，因为它是一个分类变量可以是两个字符串中的一个`relu` 或 `sigmoid`。如何编码取决于贝叶斯优化器，例如，它是使用高斯过程还是随机的森林。但是目前还不能使用`skopt`的内置函数来绘制。\n",
    "\n",
    "相反，我们只想使用我们通过名称识别搜索空间的实数和整数值维度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dim_names = ['learning_rate', 'num_dense_nodes', 'num_dense_layers']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后我们可以对这些维度的所有组合做一个模型图。\n",
    "\n",
    "对角线上的图上显示了单个维度对适度值的影响。这是一个所谓的部分依赖图。它显示了近似的适度值随着那个维度的值的变化而变化。\n",
    "\n",
    "在对角线下方的图显示2个维度的部分依赖图。这显示了当我们同时改变两个维度时，近似的适度值是如何变化的。\n",
    "\n",
    "这些部分依赖图只是模型的适应性函数的近似值-只是对真正的适应性函数 `fitness()`的近似。这里可能有一点难理解。举个例子，部分依赖是通过固定“学习速率”一个值来计算的，然后在搜索空间中为剩余的维度取大量的随机样本。然后对所有这些点的估计适度值进行平均。然后，对`learning_rate`的其他值重复该过程，以显示它如何影响平均的适度值。两个维度的部分依赖图的处理也是相同的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAGECAYAAADKqHwZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4VPX1+PH3yUJWSIAECGEJAoKIghIsqFhFbKvi1mpr\nf+5at7rWtmprW+32ra3WpdWquG+tinXFXeqCO0FWERVZZIdAEiAh65zfH/dOmISZyWT2TM7ree6T\nO3fu3HuG0Tnz2UVVMcYYY0KRlugAjDHGdB2WNIwxxoTMkoYxxpiQWdIwxhgTMksaxhhjQmZJwxhj\nTMgsaRhjjAmZJQ0/RGRnHO5xvIhcG+v7BLj3iSIyJhH3NsZ0bWKD+/YkIjtVNT8K10lX1ZZoxBTN\ne4vIQ8AsVX06vlEZY7o6K2l0QER+KSJzRWSRiPze5/hzIjJPRD4TkQt8ju8Ukb+LyEJgsoisEpHf\ni8inIrJYREa7550tIne4+w+JyD9E5AMRWSEiJ7vH00TkXyKyTETeEJGXvc8FiHWViPxVRD4FThGR\n893YF4rIf0UkV0QOBo4HbhKRBSIy3N1edd/PHG+MxhjTniWNIETkO8BI4CBgPDBBRA5znz5XVScA\n5cDlItLXPZ4HfKyq41T1PfdYpaoeCNwF/CLA7UqAQ4HpwI3use8DZcAY4Axgcghhb1XVA1X1CeAZ\nVZ2oquOAz4HzVPUD4AXgl6o6XlW/BmYAl7nv5xfAv0K4jzGmG8pIdABJ7jvuNt99nI+TRN7FSRQn\nuccHu8e3Ai3Af9td5xn37zycRODPc6rqAZaKSH/32KHATPf4RhF5K4SYn/TZHysifwIK3dhfa3+y\niOQDBwMzRcR7OCuE+xhjuiFLGsEJ8BdVvafNQZHDgWnAZFWtE5G3gWz36Xo/bQkN7t8WAv+bN/js\nS4BzQlHrs/8QcKKqLhSRs4HD/ZyfBlSr6vgI7mmM6Saseiq414Bz3V/jiEipiPQDCoAqN2GMBibF\n6P7vAz9w2zb64/9LP5iewAYRyQRO8zm+w30OVd0OrBSRUwDEMS7iyI0xKcmSRhCq+jrwb+BDEVkM\nPI3zZfsqkCEin+O0P3wUoxD+C6wFlgKPAZ8CNZ14/W+Bj3GSzzKf408AvxSR+SIyHCehnOc23n8G\nnBCF2I0xKci63CY5EclX1Z1uQ/snwCGqujHRcRljuidr00h+s0SkEOgB/NEShjEmkayk0QWJyLPA\nsHaHr1HVPXpHGWNMNHWYNEQkF/g5MERVzxeRkcAoVZ0VjwA7q6ioSMvKyhIdhgHmzZtXqarFiY7D\nGBM9oVRPPYgzvsA7sGwdMBNIyqRRVlZGRUVFosMwgIisTnQMxpjoCqX31HBV/RvQBKCqdUQ2jsB0\nFevXJzoCY0ySCSVpNIpIDqAAbhfNhuAvSZxN2+tpbPYkOozUUFqa6AiMMUkmlKRxPc64hMEi8jgw\nG7g6plFFYPOOBn7z3GKsgd8YY6KvwzYNVX3DnTV1Ek611BWqWhnzyMLUr2cWT1WsZWjfPC45YkSi\nwzHGmJTSYUnDnZSvWVVfcntMNYvIibEPLTz9e2VzwviB3PTaF7y40OrkjTEmmkKqnlLV1qkrVLUa\np8oqbCLSx10f4iv3b+8g5/YSkbXetSdC8beT92diWW9+PnMh81ZviyRUY4wxPkJJGv7OiXQk+bXA\nbFUdidNGEmzZ0z/iTEUesqyMdGacUc7AgmzOf2Qeq7fWdvwis6frI/ptYIxJQaEkjQoRucVnhbdb\ncMZtROIE4GF3/2HAb3WXiEwA+gOvd/YGvfN68OA5B+FR5ZyH5lJd1xh2sN3WDTckOgJjTJIJJWlc\nBjTiLO7zJE5320sivG9/Vd3g7m/ESQxtiEga8HcCr3TXoWFFecw4o5y123Zx4aPzrCtuZ9k4DWNM\nO6H0nqolePWRXyLyJjDAz1PXtbu+ioi//rE/BV5W1bU+K8oFutcFwAUAQ4YMafPcQcP68LeT9+fK\nJxdw7TOL+Psp4+joesZVWgrWddkY46PDpCEie+P82i/zPV9VpwZ7napOC3LNTSJSoqobRKQE2Ozn\ntMnAFBH5Kc5SpT1EZKeq7pHAVHUGzjrXlJeX7/Etd+IBpazeWsetb35JWd88Lj9yZLDQjTHGBBBK\ng/ZM4G7gPpzlSqPhBeAsnAWMzgKeb3+CqrauNOcuVVruL2GE6vIjR7B6ay23vPElQ/vmcsJ4G+1s\njDGdFUrSaFbVu6J83xuBp0TkPGA18EMAESkHLlLVn0T5fogIf/nBfqyr3sUvZy5iYGEOE8v6RPs2\nxhiT0kKZGv0GnOqjZ/GZc0pVk3IARHl5uQab5ba6rpHv/+sDquoaeeanhzCsKC+O0XUxIhG1aYjI\nPFUtj2JExpgEC6X31FnAL4EPcLrazgO67Nzjhbk9ePCciYgI5z40l6pa64obkI3TMMa002HSUNVh\nfra94hFcrAztm8eMMyawrtrpitvQHK2mmhQTwTiNml1N0YvDGJM0Qpl7KldEfiMiM9zHI0VkeuxD\ni63ysj7cfMo4Plm1jWueXmSz4voTwTiNU2d8FMVAjDHJIpTqqQdxBvcd7D5eB/wpZhHF0fHjBvLL\n747iuQXrue3NrxIdTvIJcz0NVWXttrooB2OMSQbdfuW+nx4+nFMmDOL22V/x33lrEx1OSqiua2JH\nQ3OiwzDGxEAoXW671Mp9nSUi/Pkkpyvutc8sorR3DpP26pvosLq0b6yUYUzKSrmV+8LRIyONu06f\nwNC+eVz46Dy+3rIz0SF1aWuqLGkYk6pC6T31BvB94GzgPzgjs9+ObVjxV5CTyYNnTyQjTTjnwbls\n3Zkyham4W7NtV6JDMMbESMCkISIHejdgKLABWA8McY+lnMF9crn3rHI2ba/ngkfnUd/UzbvihjlO\nY01VHb1zM6McjDEmGQQrafzd3e4EPsaZEPBed//O2IeWGAcO6c2tPxrPvNVV/GLmQjyebtwVN8xx\nGmu21TG4T250YzHGJIWASUNVj1DVI3BKGAeqarmqTgAOwOl2m7KO2a+Ea48ezaxFG7jljS8THU7i\nhDlOY23VLgb3tqRhTCoKpSF8lKou9j5Q1SXAPrELKTlceNhe/Pigwdzx1nKeqliT6HASI4xxGh6P\nsq5qF4P65MQgIGNMooXS5XaRiNwHPOY+Pg1YFLuQkoOI8IcTxrK2ahe/fmYxgwpzOHhEUaLDSnqb\ndzTQ2OKxkoYxKSqUksY5wGfAFe621D2W8jLT07jztAMZVpTHhY/NY/nmHYkOKel5u9sO6m0lDWNS\nUShdbutV9VZVPcndblXV+ngElwx6ZWfywNkTycpI5+wH51JpXXGDWtuaNKykYUwqCmXCwkNE5A0R\n+VJEVni3eASXLAb3yeW+s8qp3NnA+Y9UWFfcILxjNKykYUxqCqV66n7gFuBQYKLP1q2MH1zIbT86\ngAVrqvn5U92kK24Y4zTWVtVR3DOL7Mz0GARkjEm0UJJGjaq+oqqbVXWrd4t5ZEnoe2MH8Ouj9+Gl\nxRu46fUvEh1O7IUxTmPNtl0MtlKGMSkrlN5Tb4nITcAztF3u9dOYRZXEfjJlGKu21nLX218ztE8u\npx40JNEhxc769TBwYKdesra6jgOH9I5RQMaYRAslaXzL/eu71rMCU8O9qYj0AZ4EyoBVwA9VtcrP\neUOA+4DB7j2PUdVV4d43GkSE3x+/L2urdnHdc0so7Z3DlJHFiQwpdkpLO7VGeHOLh/XV9Rw/zkoa\nxqSqUHpPHeFnCzthuK4FZqvqSJxZc68NcN4jwE2qug9wELA5wvtGRUZ6Gnf8vwMY2S+fnz72KV9u\nsq64ABtq6mnxqI3RMCaFhdJ7qr+I3C8ir7iPx4jIeRHe9wTgYXf/YeBEP/cdA2S4s+yiqjvdBaCS\nQk+3K25Oj3TOeXAum3d0m17IAa2t8vacsqRhTKoKpSH8IeA1wFu5/SVwZYT37a+qG9z9jUB/P+fs\nDVSLyDMiMl9EbhIRv11yROQCEakQkYotW7ZEGFroBhbmcP9ZE9lW28j5D1ewq7F7d8X1DuwbbFOI\nGJOyQkkaRar6FOABUNVmoMNvRxF5U0SW+NlO8D1PVRV3VcB2MoApwC9wuvjuhbOmxx5UdYY7oWJ5\ncXF82xf2G1TAP358AIvW1XDlk/O7R1fcANZuqyNNnGRqjElNoSSNWhHpy+7lXicBNR29SFWnqepY\nP9vzwCYRKXGvV4L/toq1wAJVXeEmqueApFzH46gx/fntsWN47bNN3PjqskSHEz2dHKexpmoXA3pl\nk5keyn9WxpiuKJTeU1cBLwDDReR9oBg4OcL7vgCcBdzo/n3ezzlzgUIRKVbVLTi9tSoivG/MnHNI\nGau31jLj3RUM6ZPL6ZOGJjqkyHVynMaabXUMsnU0jElpofSe+hT4NnAwcCGwr6pGOsvtjcBRIvIV\nMM19jIiUuzPqoqotOFVTs0VkMSA4i0AlJRHht9PHMHV0P65/4TPe/iIpOnpFppPraaypqrOeU8ak\nuA5LGiKSDfwUZxoRBeaIyN2RTFrojig/0s/xCuAnPo/fAPYP9z7xlpGexj9/fACn3P0hl/57PjMv\nmsw+Jb0SHVb4OjFOo76phU3bG6wR3JgUF0rl8yPAvsA/gTvc/UdjGVRXlpeVwQNnTyQ/K4NzH5rL\npu3doyvuumqnu+0Qq54yJqWFkjTGqup5qvqWu52PkzhMAAMKsrn/7HJqdjVx3sNzqWtsTnRIMbdm\nm7e7rSUNY1JZKEnjU7fHFAAi8i2SuEE6Wew7sIA7/t8BLF2/ncv/s4CWFO+K25o0rE3DmJQWStKY\nAHwgIqtEZBXwITBRRBaLSMov+xqJqaP7c/1x+/Lm55v480ufJzqcmFpTtYseGWn065mV6FCMMTEU\nSpfb78U8ihR21sFlrNpaywPvr6SsKJczJ5clOqTQdWKcxjdb6xjcO4e0NIlhQMaYRAuly+1qnFlm\np7r7tUCaqq52H5sO/ObYMUzbpx83vPAZby3rQl1xOzFO45ttddaeYUw3EMqEhdcD1wC/cg/1AB6L\nZVCpJj1NuP3UAxgzsBeX/vtTPlvf4YD65BDiOA1VZc22OoZa0jAm5YXSpnEScDxOCQNVXQ/0jGVQ\nqSgvK4P7z5pIr5xMznuogo01XaArbmlpSKdV1zWxo6HZShrGdAOhJI1G30kFRSQvtiGlrv69snng\n7InsqG/i3IfmUtuQGl1xV7s9p4b2tf80jEl1oSSNp0TkHpx5oM4H3iSJp/NIdvuU9OLO0w7ki007\nuOw/81OiK+7qrbUADO1rJQ1jUl0oDeE3A08D/wVGAb9T1X/GOrBUdvioftxw/L78b9lm/jhraaLD\nidg3W52Sho0GNyb1hTL3VCFQDTwFfKmqXaQVN7mdMWko32yt5d45KxnaN5dzDhmW6JDCtnpbHf17\nZZGd6XeNLGNMCgmYNEQkC7gHZynWFTilkqEi8ixwkao2xifE1PWro/fhm211/GHWUgb1zuWoMf4W\nMEygEMdprN5aS5m1ZxjTLQSrnroOyAQGq+qBqjoeGIKTaH4bj+BSXVqacNuPDmD/0gIu/898Fq9N\nskJciOM0Vm2ts6RhTDcRLGl8HzhfVXd4D7j7P8XphmuiIKdHOveeVU6fvB6c9/Bc1ruzxSaFEMZp\n1DY0s2VHA0OsEdyYbiFY0vCoal37g6q6E/9repsw9evpdMXd1djCuQ/NZUd9U6JDcoQwTmNlpdNz\naq8iK2kY0x0ESxoqIr1FpE/7DfDEK8DuYtSAnvzr9AP5avNOLv33fJpbusY/8Sq3u22ZJQ1juoVg\nSaMAmBdgsxHhMTBlZDF/OnEs73y5hetf+AwNcdW8RFq5xU0a1qZhTLcQsPeUqpbF6qZuaeVJoAxY\nBfxQVav8nPc34Fic5PYGcIV2hW/SCPz4oCGs3lrH3e98zbCiPH4yZa9EhxTUyspaBhZkk9PDutsa\n0x2EMiI8Fq4FZqvqSGC2+7gNETkYOARnjfCxwETg2/EMMlGu/u4ojtlvAH9++XNeXbIx0eEE9XVl\nLcOKrZRhTHeRqKRxAvCwu/8wzliQ9hTIxplVNwun+++muESXYGlpwi0/HM+4QYVc+eR8Fq6pTkwg\nHYzTUFVWbNnJMGvPMKbbSFTS6K+qG9z9jcAeo9pU9UPgLWCDu72mqqm9/J2P7Mx07jurnKL8LM57\nuIK1VXt0ZIu9DsZpbNnZwI76ZkYU58cnHmNMwgVMGv56TbXrQRWUiLwpIkv8bCf4nuc7g267148A\n9gEGAaXAVBGZEuBeF4hIhYhUbNmypaPQuoyi/CweOmciDc0tnPdQBdvj3RW3g3EayzfvBGB4P0sa\nxnQXweaemofzZe5v/U4FgrbQquq0QM+JyCYRKVHVDSJSAvhbzu4k4CN3XAgi8gowGZjj514zgBkA\n5eXlKdVQPqJfT+45fQJnPvAJlzz+KQ+cPZHM9DgVEEtLIUi/g6/dpDHCkoYx3UbAbx9VHaaqe7l/\n22+Rdul5ATjL3T8LeN7POd8A3xaRDBHJxGkE7zbVU74OHlHE/520H3O+quR3zy9Jmq64X23eSc+s\nDAb0yk50KMaYOOlwllsAEekNjMRpmAZAVd+N4L434qzTcR6wGvihe59ynMkQf4IzHftUYDFOyeZV\nVX0xgnt2aT+cOJjV22q5862vKeubx4XfHp7okPhy0w5G9M9HxF9h1BiTikKZGv0nwBU4bQsLgEnA\nhzhf6GFR1a3AkX6OVwA/cfdbgAvDvUcq+vlRo1i9tY6/vLKMwX1yOWa/koTG89WmnUzbJ8lm5jXG\nxFQoleNX4IyRWK2qRwAH4KyvYeIsLU24+ZRxTBjam589uYD53+wxHjJutuxoYGttI3sPsMkBjOlO\nQkka9apaD84aG6q6DGcFP5MA2ZnpzDhjAv17ZXP+IxW8++WW2E1wGGScxhcbncmP97GkYUy3Ekqb\nxlp39b7ngDdEpAqnHcIkSN/8LB48ZyIn3/UBZz7wCSIwojif8YMLGTe4kPGDCxk1oGfkvayCjNNY\ntnE74Ey0aIzpPjpMGqrqXTvjBhF5C2ciw1djGpXp0PDifN65+gg+XV3FwjU1LFxbzexlm5k5by0A\nWRlpTNunP7f+aDw9MsJMHuvXw8CBfp9aun47/Xtl0Tc/K9y3YIzpgoIt99pLVbe3G8i32P2bD2yL\naWSmQ72yMzl8VD8OH9UPcKb1WFu1iwVrqvloxVYe//gbBhZmc92xY8K7QZBxGkvW17DvwIJwQzfG\ndFHBShr/BqbTdpCf79/knn61GxIRBvfJZXCfXI4bNxARuHfOSibt1Zcjo9jLqb6pha+31PK9fQdE\n7ZrGmK4h2OC+6e5f30F+e0VpcJ+Jg98cO4YxJb34+cyFUV1G9rP1NbR4lLGlVtIwprvpsLJbRGaH\ncswkn+zMdO487UCamj1c9p/5NEVpNcD53zg9rscPLozK9YwxXUewCQuz3faMonbLvpbhTCBouoBh\nRXn85Qf7M291FTe//kVUrrlgTTUDC7LpZ9OHGNPtBGvTuBC4EhiI067hnStiO3BHjOMyUXT8uIF8\ntGIr97yzgoPK+oTevuFnnIaqUrGqionDOpzo2BiTgoK1adwOjAD+1G7iwnGqakmji/nd9DHsO7AX\nP3tyAau31ob2Ij/jNNZs28XG7fUcZEnDmG4paJuGO//T9+MUi4mh7Mx07j59AiLCRY99yq7Glo5f\n5Gc9jY9WbAXgoDJLGsZ0R6GM+potIj8Qm8q0yxvcJ5fbTh3Pso3bue7ZxR1PsV66Z9PVO19toV/P\nLPbub2toGNMdhZI0LgRmAg0isl1EdojI9hjHZWLkiFH9uOLIkTwzfx3/mL28U69tbvHw3leVfHvv\nYpsO3ZhuKpRpRGxyoRRz+dSRfLOtjlvf/BJFuXLa3iG97qMV26jZ1cTU0f1iHKExJlklahEmk0Bp\nacJNJ48jTYTb3vyKXU0tXP3d0aSnBS89zFq0nrwe6RxhScOYbishizCZxEtPE/72g/3JykjjnndW\n8OXGHdz2owMoyM30e/72+iZeXLie744dQHZmepyjNcYkC1uEqRtLSxP+dOJY/njiWOZ8Vcmx/5xD\nxSqfeSh9xmn85+NvqG1s4ZyDhyUgUmNMsrBFmLo5EeGMSUN56qLJiMAP7/mQv7zyOfVNLa3jNNZX\n7+Kf/1vOt/cuZr9BNt+UMd1ZKEmj/SJMzxPhIkwicoqIfCYiHhEpD3Le90TkCxFZLiLXRnJPE9yB\nQ3rz8uVT+GH5YO55ZwVTb36bx575gOcXrOP/3fsRHlX+cMK+iQ7TGJNg0mFffd+TRb6NuwiTqjaG\nfVORfQAPcA/wC1Wt8HNOOvAlcBSwFpgL/FhVlwa7dnl5uVZU7HE50wkffr2Vm1//gv/+9BDKrplF\ncc8s7j79QCYM7dyAPhGZp6oBfxQYY7qeYIswZQMX4Uwlshi4X1XficZNVfVz9x7BTjsIWK6qK9xz\nnwBOAIImDRO5ycP78vRFk+Gn8OKlh7JPSU8yIl061hiTEoJ9EzwMlOMkjKOBv8clot1KgTU+j9cS\nYHZdEblARCpEpGLLli1xCS7VeRP6foMKLGEYY1oF63I7RlX3AxCR+4FPOnNhEXkT8Le023Wq+nxn\nrtURVZ0BzACneiqa1zbGGLNbsKTR5N1R1ebOThuhqtPCDcq1Dhjs83iQe8wYY0yCBEsa43zmmBIg\nx30sgKpqrxjHNhcYKSLDcJLFqcD/6+hF8+bN2ykiwVYbKgBqAjz2t+/tY9p+P9aKgMoAzwV7D76P\nI38Pkc0xZV2zjUk1qhr3DTgJp42iAdgEvOYeHwi87HPeMTg9qL7GqdYK5doVHTw/I9Bjf/s41V57\n7Mfh3yjg+wj2HtrHnqzvwTbbbOuaW0hzT0Wbqj4LPOvn+HqcROF9/DLwcpRv/2KQx/72Az2fSMHe\ng+/jZH4PxpguqFPjNLoCEanQFBgbkArvIxXegzGmrVTsSzkj0QFESSq8j1R4D8YYHylX0jDGGBM7\nqVjSMMYYEyOWNIwxxoTMkoYxnSAiZSKyJNFxhEtEbhCRXyQ6DtN1WdIwxhgTMksapktyf/F/LiL3\numuzvC4iOSLytneNFhEpEpFV7v7ZIvKciLwhIqtE5FIRuUpE5ovIRyIScN53EZkgIgtFZCFwic/x\ndBG5SUTmisgiEbnQPX64G8fTIrJMRB4Xdx4eEblRRJa659/sHisWkf+615krIocEieUGEXnAvf4K\nEbnc57mrRGSJu13pc/w6EflSRN7DZ5S+iAwXkVdFZJ6IzBGR0e7xU9xrLBSRdzv50ZhUl+jRhbbZ\nFs4GlAHNwHj38VPA6cDbQLl7rAhY5e6fDSwHegLFONOoXOQ+dytwZZB7LQIOc/dvApa4+xcAv3H3\ns4AKYBhwuHv9QTg/zD4EDgX6Al+wu9diofv338Ch7v4Q4PMgsdwAfODerwjYCmQCE3BmpM4D8oHP\ncJZm9h7PBXq5/wa/cK81Gxjp7n8L+J+7vxgo9Y3RNtu8W0JGhBsTJStVdYG7Pw8nkQTzlqruAHaI\nSA27R8cvBvb39wJ31cpCVfX+4n4UZ6kAgO8A+4vIye7jAmAk0Ah8oqpr3WsscGP7CKgH7heRWcAs\n93XTgDE+k4L2EpF8Vd0Z4H28pKoNQIOIbAb64ySlZ1W11r3nM8AUnKT1rKrWucdfcP/mAwcDM33u\nm+X+fR94SESeAp4JEIPppixpmK6swWe/BcjBKX14q12zg5zv8XnsIbz/FwS4TFVfa3NQ5HA/sWWo\nM1v0QcCRwMnApcBUN95Jqlof4n33uHYYsacB1ao6vv0TqnqRiHwLOBaYJyITVHVrGPcwKcjaNEyq\nWYVTJQPOF3NEVLUaqBaRQ91Dp/k8/RpwsYhkAojI3iKSF+ha7q/7AnXmVPsZMM596nXgMp/z9vgi\nD8Ec4EQRyXVjOMk99q57PEdEegLHue9rO7BSRE5x7ykiMs7dH66qH6vq74AttF2iwHRzVtIwqeZm\n4CkRuQB4KUrXPAd4QEQU5wve6z6caqdP3YbuLcCJQa7TE3jeXUpZgKvc45cDd4rIIpz/J9/FWWo5\nZKr6qYg8xO7F0u5T1fkAIvIksBDYjLPkgNdpwF0i8hucdpEn3PNuEpGRboyz3WPGADaNiDHGmE6w\n6iljjDEhs+opY1wicifQfozE7ar6YAJiOQe4ot3h91X1En/nGxMvKVc9lderpzb3KWh9rB6lac36\n1seZgwciaREtYdrhddMzPRFdOyejifzmPHZm1LY53jPd6Vzj8ShrPt/93Jh9M0iPsMzY4oGlnzXv\ncdx77fbP9967L/WeHs5rm9KQJiGtCWdraGbHro0AqGrk/9iuoqIiLSsri9blTJjmzZtXqarFiY7D\nJEbKlTT6FvdDfn1hm2OVdz9OXcUicsv3p+ii0wK8svOCXbf3gB1hX3e/og1MrZzC/4rmtDn+7d67\nlz6/58plVLxSybHTs7njrsJO36O21kNeXttMc+nF1bw0q56SkjQ2bPDscW3v8+VHFzH6+hNYXFkC\nQNXGnmRtzCR3I+SvayFvTS2LFj3CxsYVnY4rmLKyMioqKqJ6TdN5IrI60TGYxEm5ksaQ4cO1fdIA\n8NQ3kJad5ecVkenouuEkj1CSBsBk/XKPL/5QeL/8/SUcbzLxl1QAZm0ZRnZeOu9UjQqaNGTlOl7b\neu98VT2w0wEGUF5ernFLGuvXw8CB8blXFyMi89RWZOy2uk1DeCwSRiyv25FpucvDShi1tR5emuVU\nc700q57a2rZVad5rBrp2dl566/5+RRs6ul1k9XSJVFqa6AiMSUpRSRoi0ltE/E7D0N1VbeyZ6BDa\nyMtL49jpzkDpY6dnh5V4jDHdV9jfGO4sm73c2UE/Be4VkVuiF1rqSHTiaF+auOOuQpYs6xdWW8i0\n3OUhnafD7Je6Makokp+ZBe5UBN8HHlHVb+FMvGb86Ezi8LYVRMOlF1czdvRmLr24us1xK2EYY8IR\nyTdHhoiUAD9k92ydJol01H5hjDGdFUnS+APOhG1fq+pcEdkL+Co6YaWmeFdTWftFBK6/PtERGJOU\nwh6noapxirs+AAAgAElEQVQzgZk+j1cAP4hGUKmsamPPiMZw1Ne2tOnB1JE77irkrzf77z4bK7WD\n88hbU9vxicnshhsSHYExSSmShvC9RWS2iCxxH+/vzpZpYuSeK5dx2YEfcs+Vyzr1OithhGH9+o7P\nMaYbiuTb5F7gV0ATgKouAk6NRlCpLpxqqqa6JipeqQSg4pXKpG2f2FkaeikoFCJygYhUiEjFli1b\nonrtoGychjF+RZI0clX1k3bH9py8yERFZm4mvQc4cz31HtAj5qWHjpJS+9HpsaKqM1S1XFXLi4tt\nuiNjEi2Sb55KERkOKIC7TnKHQ4RNaHY1Z7Z53FTXRNXGRgCqNja2fqnHosQRqJtuR+oGRD0UY0yS\niSRpXALcA4wWkXXAlcDFUYkqxXjqG/Y41tkqqszcTMqPLgKg/Ogi8vLSwv5yD8a66Rpjggk7aajq\nClWdBhQDo1X1UFVdFbXIUkTl3Y+z9tLfUXn34xFf68LbRvPPTydz4W2jY/blbt10jTHBdLrLrYhc\nFeA4AKqa2KlEFLI27q7aaRjQlLBQPPUN1FUsAqCuYhGe+pPbTHAYTvfb7Lx06mtbyCt2vty9s9VG\n88s9nG66DQOa2vy7d3k2TsMYv8IZp+GtVxkFTARecB8fx+5F7ZNGKF9ksUosadlZ5Jbv37rmRjRm\nxPWuo/GqO615rMZgdPsSho3TMMavTicNVf09gIi8CxyoqjvcxzcAL4VyDRFJByqAdao6XUSGAU8A\nfYF5wBmq2igiWcAjwARgK/CjaFWBeRoaSMtyvsQDJZZoJJOii07bo4QRrvraltZuty/Nqo/7oL1A\neg/YkfBJGaPO1tMwxq9IvnH6A40+jxvdY6G4Avjc5/FfgVtVdQRQBZznHj8PqHKP3+qeF7ENTzzC\n13/8FRueeCToeVkbM/fYwhEsYXTmyzY7L721MTzZ2xtqB+clOoTI2DgNY/yK5FvnEeATEblBRH4P\nfAw81NGLRGQQcCxwn/tYgKnA0+4pDwMnuvsnuI9xnz9SvI0nYfI0NLBzyQIAdi5ZgKdhz55NwUSS\nPKLB2xgezrTmxhgTqUjmnvqziLwCTMEZq3GOqs4P4aW3AVezu22kL1Ctqt6BgWsB78+8UmCNe79m\nEalxz6/0vaCIXABcAFBUVMzVw4P/Smy68194dtWRlpNLZp++IYQcmGZEvlxuenPLHseKyWbyxqnk\nZOyuIutZPbF1/8vKziW7aCr0OCWnqc3ZTHbHk7QUpCN5QtoQSG90/k3efzZhIRpjYiTspOFqwVnS\nUwlhaU8RmQ5sVtV5InJ4hPdupaozgBkAQ/Yarv/4el2Hr/E0NJFWXw9VHZ8bikjbP9r3orqgeR9m\nZHy+x5Kq3pHY5SEuhhQLb9aNAGi7Tnjl7nXCwVkr3BiTeiKZsPAK4HGgCOgHPCYil3XwskOA40Vk\nFU7D91TgdqBQRLwJbBDg/SZfBwx275cBFOA0iEfM2wgeLYmutkqUENYJN8akkEjaNM4DvqWq16vq\n74BJwPnBXqCqv1LVQapahjO54f9U9TTgLeBk97SzgOfd/Rfcx7jP/09VI68PiqHumDhSko3TMMav\nSJKG4FRPebW4x8JxDXCViCzHabO43z1+P9DXPX4VcG2Y14+r7pw4Umb+KRunYYxfkbRpPAh8LCLe\n5s4T2f1l3yFVfRt4291fARzk55x64JQIYkyYrI2ZnWrniHRxpliqrU2O8SBxZeM0jPErkrmnbgHO\nBba52zmqelu0AksFqVDiiMWkiOFq9sSxZtLGaRjjV6Q/HxfgjJ94DtgqIkMiDym1xCpxeHswxVJn\nJ0X0U7KKavHk8w3bOe+huby6ZCNNLTb7rjGJEEnvqcuATcAbwCycKURmRSmulBJq4vA3OtzbpTUR\nIpnxdsnixwEOiGY8xT2zWLK+hosem8fkv/yPG19ZxuqtXXwtcmO6mEjaNK4ARqlqVLrAmuQUzqSI\nLY0N1CxfEPVYBvTK5v1rpvLOl1v4zydruHfOCu5+52sO27uYMyYNZerofqSnRTRhgDGmA5EkjTVA\nTbQCSXWdbRgPVywarUO5nu+khek9sigYMT4miSMjPY0j9+nPkfv0Z2NNPU/M/Yb/fPIN5z9SwaDe\nOZwxaSg/mjiYwtweUb+3MSayOucVwNsi8isRucq7RSuwZNHZuamCiXXDeLwbrYOtE172vTMBQplW\nJmwDCrK5ctrevHfNVP512oGUFubwl1eWMekvs/n1s4v5alMEvdFsnIYxfkVS0vjG3Xq4W8rZ8MQj\n7FyygPyx4yk59cxEhxNU+0braEybHm6ppW4A3ulE4tJanZmexjH7lXDMfiV8vmE7D72/iqfnreXf\nH3/Dt/cu5idThnHoiCI6NdeljdMwxq9Iutz+3t/mfV5E/hmdEBMj0tlwA+motBHuuhTRXqY1mbra\ndsY+Jb3468n78+G1U/n5UXuzdMN2zrj/E46+fQ4zK9bQ4GdySL/Wr49toMZ0UbEcsXVIDK8dc2lZ\nWeSPHQ9A/tjxUZ+rKhbuuKuQJcv6RTxteqzWH4+nvvlZXHbkSN675ghuPmUcAL98ehGH/vUt7nxr\nOTV1HbQv2TgNY/zqZsN8O6fk1DMZ/tu/RL1qKpK2jXeqRgV9PhqN4B2VWqblLqe+dvcvdt9JCxO5\nJrs/WRnpnDxhEK9cMYVHzj2I0QN6ctNrXzD5xtn8/sXPWFtVl+gQjelSIp0aPemIttanB9SZ+ZGS\noYSxuLIk7rPJButqe+nF1bw060PKjy5i9PXBk1iyEBEO27uYw/YuZun67dw7ZwWPfriaRz5czfT9\nSzh/yl6MLS1IdJjGJL1YljSStsN87sbgWzwEK20ky3rb/hKGb9VVxSuVNHVUzRMhEblARCpEpGLL\nli1RueaYgb249UfjeffqIzj3kDLeXLqJ6f98j9Pu+4i3v9hMkk+kbExCRZw0RCQ3wFO3R3rtRElU\nIukKfKuuyo8uIjPXf/KL1my3qjpDVctVtby4uDg6F3UNLMzhumPH8MGvjuTao0ezfPNOzn5wLt+7\nbQ4Ajc1dry3HmFiLZBqRg0VkKbDMfTxORP7lfV5VH4o8vOQRiwTSVSc0vOOuQv756WQuvG10m+PJ\nOktvRwpyMrno28OZc/VU/n7KOETgtkN+zKF//R93vrWcqtrGRIdoTNKIpKRxK/Bd3JX0VHUhcFg0\ngkp2qVoC6Uwvqey89BhGkhg9MtL4gdtofuADtzPKp9H8umcXs3zzzkSHaEzCRVQ9papr2h3qdgtD\np0ri6KrjMmJBRDgsv4lHz/sWr145hRPGlTJz3lqm3fIOZz3wCW99sRlPPKdpNyaJRJI01ojIwYCK\nSKaI/AL4PEpxdSmRlDoCVVG1NMXvl3wqjMuIOnecxugBzmDBD66dylVH7c3nG7ZzzoNzOfKWd7j/\nvZXU7EquLsbGxFokSeMi4BKgFFgHjHcfd1uxLHX4myI9WmtqRHs0ebKN1YiGovwsLj9yJO9dM5Xb\nTx1Pn7we/HHWUib932yueXoRi9ZaCc10D2GP01DVSuC0KMaSEnI3dr7nULxmwA0mnCnQu6MeGWmc\nML6UE8aXsmRdDY99tJrnF6znyYo1jC3txakTh3D8+IH0yu6anRyM6Ugkvaf+JiK93Kqp2SKyRURO\nj2ZwXVWsG8k7GhUernAShnem23gPPkwGY0sLuPEH+/PxdUfyhxP2pcUDv3luCQf9+U1+9uQCPlhe\naW0fJuVEMiL8O6p6tYicBKwCvg+8CzwWjcCMM8jPtxtrU11TwHERJnF6ZWdy5uQyzpg0lMXranhy\n7hpeWLieZ+evo7QwhxPGD+SkA0oZ2T85Bm0aE4lIkob3tccCM1W1plNTT3cDnamq6qiKatVfn2Hh\nnKUMPXIYU/48NeQYYrEoUzC+izF1aWGspyEi7D+okP0HFfLb6WN47bONPDt/Hfe8u4J/vf01+5T0\n4rhxJUzfbyBD+gYaE2tMcovk22SWiCwDJgCzRaQYqI9OWKmjM9VUnnr/06+37GqkZs5SAFbPXhnS\n1B1v1o3glAuaE9aNNtFtNBGLcD2N7Mx0ThhfykPnHMRHvzqS648bQ05mGn979QsOu+ktjv3HHO58\naznLN++waUtMlxJJQ/i1IvI3oEZVW0SkFjgheqGljlBKHN4Fn3LL96foorb9C9JzelAwZQw1bkkj\nWBWVt0dVfW0LFa9UAtFblKlbWb8eBg6MyqWKe2ZxziHDOOeQYazZVserSzby8pIN3PTaF9z02hcM\nK8pj2j79OGJ0PyaW9SEz3T4nk7wineV2NFAmIr7XeSTCa6akYInDd8GnuopFeOpPhnYTrpZd831a\nLp/O+MFb/V6jfffb7Lx0yo8uouKVyqh0o+12SkshBiWAwX1yOf+wvTj/sL3YWFPPG59v4vXPNvLQ\nB6u4d85KemZlcPCIvhy2dzFTRhRbNZZJOmEnDRF5FBgOLGD3SHDFkkaneRd88pY00rJ3T8fu2xie\nnrPnqrrBxmpceNtozvpzizvlx/Kox20iM6AgmzMmDeWMSUPZ2dDM+8sreWvZZt79cguvfbYJgEG9\nczh4eF++NawvBw3rw6DeOZ1bttaYKIukpFEOjFGrkA1ZsNJGyaln4mn4EU1Do1siSMU5olJRflYG\n3913AN/ddwCqyorKWt77qpIPvq7ktc828VTFWgBKCrI5cGhvJgzpzfghhYwp6UV2pn3GJn4iSRpL\ngAFA9+ugH4FgicNZ8Cm0BuR3qka1jpEIxZt1I5iWG9vSxn5FG/yOXDedIyIML85neHE+Zx1chsej\nfLFpB5+s3MbcVduY/001Ly1y/rfLSBNGl/Rk7MAC9i0tYExJL0YN6El+Vsqtr2aSRCT/ZRUBS0Xk\nE6C124+qHh/oBSIyGKf6qj9OVdYMVb1dRPoATwJlOGM+fqiqVeKUw28HjgHqgLNV9dMIYk46noaG\nNqsDJsPo8EilTLfbJJGWJuxT0ot9Snpx1sFlAGyo2cXCNTUsXFvNorXVvPrZRp6Yu3v+0EG9c9i7\nf09G9s93E1Aew4ry6Z2badVbJiKRJI0bwnhNM/BzVf1URHoC80TkDeBsYLaq3igi1wLXAtcARwMj\n3e1bwF3u3y7NW9rw9pjKHzs+6uuQ+xPN0sa03OWt7Snf7v1FzEapJ0wY4zTiqaQgh5KCHL431im2\nqirrqnexbMMOlm3czrKNO1i+eSfvfVVJY8vuCSh7ZmcwtG8uQ/rkMqh3LqWFOQwszKGkIJuSgmx6\n5/YgLc2Sigkski6374jIUGCkqr7pruAXtHJVVTfgVmep6g4R+RxnwsMTgMPd0x4G3sZJGicAj7jt\nJh+JSKGIlLjXCXATyF8XfIb2naWJrwP27TG1c8kCPA0/CrgeuW9jeCLWC++WIhynEW8iwqDeTiKY\nNqZ/6/HmFg9rq3axonInK7bU8s22OlZvrWPZxh28+fnmPVYnzEgT+vXMoqhnFkX5WfTN60Gf/B70\nzetBYU4PCmxGgm4vkt5T5wMXAH1welGVAncDR4b4+jLgAOBjoL9PItiIU32Fe03fNTvWusci+tbs\nKKlA7BNLftXuHlP5Y8fvUUXVvsutibMojtNIpIz0NMqK8igrymNq24UW8XiUytoGNlTXs6FmFxtr\n6tm0o4FN2+up3NnIxpp6lq7fzrbaxjalFdO9RVI9dQlwEM6XPqr6lYj0C+WFIpIP/Be4UlW3+9ax\nqqqKSKd6ZInIBTgJjKKiIs44dnBnXt6hlh6xKa57DvkVeDyQtmePqX7pmfyiYGjr4/Tm3Ykup9Kp\nFupZPbHT9/wyzf+o83AUenYnuqnNztTqk5udX6ItBelcHrU7JUCMxmkkk7Q0oV/PbPr1zGbc4MKA\n56kqtY0t1Oxqoqq2kf3+GscgTdKJJGk0qGqj9wvfHeDX4f9lIpKJkzAeV9Vn3MObvNVOIlICbHaP\nrwN8M8Ag91gbqjoDmAEwtGy4PvpS+wUFw9PS3EB6Rtsqo2iXQAL1pLps1EBurlnd+th34kJv9VRn\nek95VUPU2jV8x4h42zS8vaeqKq0hPFWICPlZGeRnZVBamJPocEyCRTIo4B0R+TWQIyJHATOBF4O9\nwO0NdT/wuare4vPUC8BZ7v5ZwPM+x88UxyScKUviUqG/7JPH+OjF37Dsk7aT9uava2ndYkmaOy7d\nJHPjs2+SM8akjkiSxrXAFmAxcCHwMvCbDl5zCHAGMFVEFrjbMcCNwFEi8hUwzX2Me80VOMOZ7wV+\nGkG8IWtpbmDruoUAbF23kJZm/1U60UgcoU5o6NuF1cZCGGMSJZLeUx6cL/J7O/Ga94BAP6H3aEB3\ne03FfQnZ9Iws+paOY+u6hfQtHbdHFZUvb+JIhh5ZycAG+BmT2jqdNERkMUHaLlR1/4giShKjDzqd\nluZTgiYMX/nrWuKeOJrqmqB3518Xy9HhoUzb3iUk+TgNYxIlnJLGdPevtwTwqPv3dEJoCO9KQk0Y\nXuGWOgJNLRJsdPic6/7H6tkr+froIi68bbTfcyKVts2Dp0/oNZjemAqmjKHsmu/HJKa46WLjNIyJ\nl04nDVVdDSAiR6nqAT5PXSMin+K0dXRrsSp1eAf5texqZPXslQBUvFLJ8F/uFXCNjXB6WHn1+d12\nKu8I3BXTV1NdU2tMNXOW0nL59A5eERrf7tRDhgyJyjVDkiLjNIyJtkgawkVEDvF5cHCE10spnW0k\n78wKf+k5PRh65DCADhdlClfGimbyn68nY2VzSOdn5ma2xlQwZYzfadzDoaozVLVcVcuLi4ujcs2Q\nlJbG717GdCGRjNM4D3hARLxjl6uBcyMPKXXEsp2j8GenMem6bzpMGCHPhutRZNfu2sW8F5yVe/Oe\nr2f7+bsXAtIcgQBzE03581QmXdfEsro4lgiMMXEVdslAVeep6jhgHDBOVcf7zkArImcFfnX3EWm3\n3KyNgZNCVEsYCgX31DJk380MHb2Z3n/fCUDvv+9k6OjNDNl3M71m1AVstfImpliUeowxySPi6iRV\nrVHVGj9PXRHptbubzlRRRWqPFf/SheqrerLxiT40lrT9z6KxJI2NT/ah5mf5kG4zoBrTncWyDcK+\nXVzRHD0ezjoVnRk53jCpBzvPzWtzbNvpuTR8KzptFMaYri2WSSOlut9GKtbTjkRT7qv1eLJh26k5\nNKTB1zft5NKLq0N+fUpM3W7jNIzxy0oacRRK4vBXRRWoXSMWI6/TN7SQVqtseLEvG27oyQQP9AQ+\nnVVPbW03mh7bxmkY41csk8b7Mby2iZUW2PBCX5pGZ5KXl0bZ9Gy+BXz3yCzy8rpRj+r16xMdgTFJ\nKZJFmAqBM3HW9W69jqpe7v69NNLgUlEiphuB0LvetgxqG9sddxVSe7MnpITRVNeUOr2nusF6GsaE\nI5Kfji/jJIzFwDyfzUSoo15U4TSGRyKUhHHPlct4cuojzLnuf3GIyBiTKJEM7stW1auiFkk3Ek5p\nI9A8VN52jUQ2PtfXtlDxSiUAq2evZNJ1KTJpoTFmD5GUNB4VkfNFpERE+ni3qEVmOmVxZUmnG8b3\nGKsRpuy8dMqPLgJiN62JMSY5RFLSaARuAq5jd/daBfaKNKjuoKPSRlqYP9YXV5YkpNRx4W2jg06c\naIxJDZGUNH4OjFDVMlUd5m6WMOIkWLtGoFJHrJeHTamEYeM0jPErkqSxHKiLViDdUWcH/AWbh8of\nW0EvAjZOwxi/IqmeqgUWiMhbQOsi2t4utyZygRZn6gr2K9rAwkQHEQlbT8MYvyJJGs+5W1IRVfLW\n1LY5Vjs4L8DZiZeocRvt1daGNhaj27BxGsb4FXbSUNWHoxlILLVPIl7JnExC4V3JL5hQGsYvvbia\nl2bVc+z0bO64K7SV+owx3VMkI8JX4mdSwq7UGN4Vkkn7Kqpg64aHo7bWw0uznAWXXppVz19DHP1t\njOmeIvl2KAcmutsU4B/AY9EIKtHy1tS2bvEQzxlw2/egystL49jp2QAcOz3bEoYxJqhIqqe2tjt0\nm4jMA34XWUjJxZs4kqn0EU1v1o3gjruWd7qEEa2BgcaYriWS6qkDfR6m4ZQ8ImlYT2qxTh7hNoiH\n0q4RCithtGPjNIzxK5Iv+b+zu02jGVgFnBJpQMkuESWPrG8aaBiStftxu3aNjhJHokaJd2k2TsMY\nvyL5eXk0cD8wG2ftjHXAqdEIqiuIV3vHqlcfYcmMX7HhiUeCnhfvmW9Tnq2nYYxfkSSN54DjgCZg\np7vF55s0SUQ7cbRvEG9pbKBm+QIAdi5ZgKehwd/LWlniiKLS0kRHYExSiqR6apCqfi9qkXQhzS0N\nZKQ71UV5a2pjVlWV3iOLghHjqVm+gPyx40nLyurwNe2rqlp2NZKe0yMm8Rljup9IksYHIrKfqi6O\nWjRdwMKvZ7Kp6jP6996XccOdJpxYJo6y751JS+OPSO+R1Wair2DjNbyJY9Vfn6FmzlIKpoxhv5sm\ntz4f6ip+xhjTXiTVU4cC80TkCxFZJCKLRWRRtAILVywnfmhuaWBT1WcAbKr6jOaW3dVF0aqq8jdm\nI71HxyUMT33bqquWXY3UzFkKQM2cpSxY07fDa9TWekKMMn5E5AIRqRCRii1btiQ6HGO6vUgbwkcC\n38Fp25ju/k0oj6eJhV/PjMm1M9Kz6N97XwD69963tYrKK16N4+1V3v04ay/9HZV3P956bHtNXwqm\njAGgYMqYDquoLr24mrGjN3PpxdUxjbWzVHWGqparanlxcXGiwzGm2xNNsUnZRMT7huYD/n46FwA1\nAR772y9wH/vu7whw7WgqAioDPOcbZxpwgM9z84Ge7I53h/sY9nw/Xv6uEY33N0pVo9Y6LyJbgNXR\nul6UBPucEi1WsQ1VVcvg3ZWqptQGVHTw/IxAj/3tAzP87SfyfQR7D+1jT9b3kCpbMr/HZI7Ntq67\npewI7iBeDPLY336g5xMp2HvwfZzM78EY0wWlYvVUhaqWJzqOSKXC+0iF99CRZH6PyRyb6bpSccKh\nGYkOIEpS4X2kwnvoSDK/x2SOzXRRKVfSMMYYEzupWNIwxhgTI5Y0jDHGhMyShjGm2xORMhFZksD7\nny0idyTq/p1hScMYY1KciERteIUlDWNM0nB/8X8uIveKyGci8rqI5IjI2yJS7p5TJCKr3P2zReQ5\nEXlDRFaJyKUicpWIzBeRj0SkT5B7TRCRhSKyELjE53i6iNwkInPdefUudI8f7sbxtIgsE5HHRUTc\n524UkaXu+Te7x4pF5L/udeaKyCEh/hscJyIfu+/hTRHpLyJpIvKViBS756SJyHL3Hn7vIyI3iMij\nIvI+8KiI7Csin4jIAjfOkWF8RJY0jDFJZyRwp6ruC1QDP+jg/LHA94GJwJ+BOlU9APgQODPI6x4E\nLlPVce2OnwfUqOpE95rni8gw97kDgCuBMcBewCEi0hc4CdhXVfcH/uSeeztwq3udHwD3dfA+vN4D\nJrnv4QngalX1AI8Bp7nnTAMWquqWDu4zBpimqj8GLgJuV9XxOMtzrw0xnja644hwY0xyW6mqC9z9\neUBZB+e/pao7gB0iUsPuWQ8WA/v7e4GIFAKFqvque+hRnElYwZmEdX8ROdl9XICTyBqBT1R1rXuN\nBW5sHwH1wP0iMguY5b5uGjDGLYwA9BKRfFXd2cH7GQQ8KSIlQA9gpXv8AeB54DbgXJykF/A+7v4L\nqrrL3f8QuE5EBgHPqOpXHcThl5U0jDHJxnee/xacH7fN7P6+yg5yvsfnsYfwfhgLTglkvLsNU9XX\nA8Wmqs3AQcDTOLN9v+o+n4ZTYvBepzSEhAHwT+AOVd0PuBD3/arqGmCTiEx17/dKCPdpnXpbVf8N\nHA/sAl52r9NpljSMMV3BKmCCu39ykPNCoqrVQLWIHOoeOs3n6deAi0UkE0BE9haRgKusub/qC1T1\nZeBngLe663XgMp/zxocYXgGwzt0/q91z9+FUU81UVe/iOyHdR0T2Alao6j9wSix+S2Ed6RJJw23g\nWuw24FQkOh5jTNzdjPNFPh9nyvdoOAe4061mEp/j9wFLgU/dbrj3ELzE0hOY5S5C9x5wlXv8cqDc\nbXReitOmEIobgJkiMo89p7Z/Achnd9VUZ+7zQ2CJ+37HAo+EGE8bXWIaEbenRLmqJuu6BcYYE3Nu\nD7JbVXVKomKwhnBjjOkCRORa4GLaVqXFP44uUtJYCVThLAF+j6ra7J3GmJCIyJ1A+zESt6vqg/7O\nj3Es5wBXtDv8vqpe4u/8ZNRVkkapqq4TkX7AGzg9G971ef4C4AKArKysCQOKSyK/aZp0fE4nabBL\nis+TAhkiNKvS5tMRwF3NNk2UNJznm+tbWk/JyE5HwO/xdAnwWSs01Ptf3TUnZ3dcu3btfn1Gdjrq\nVgN7VJw3p27FsEfxtDSxdu1aVIO+6w75fraZmZkT+vfvD0AaGc69AnxOIf9btznu/3DA/0N8Pqc9\nnwv8/1Wa9zmFlobm1uPpWRmtMaQFuGugzzvN3/00A6S53bG2n3VWdtoe7zvgvRXq6zUqn6uvoqIi\nLSsri9blTJjmzZtXqSEs49slkoYvEbkB2KmqN/t7vmzwMB2988jIbzR4QOTXaKehJPBy2XX9M3fv\nFwkXH1jKXZ+uo97nI2wqaia7r9PleljRVvYrWA/AC1d/xBevr2XUdwZx/N8mtZ7f/vjE/BUB73/r\n5Sv48OVqjjvO6c344ov1HHdcNnff1bv1nJMv2MWHL1cz+ZhCDv2/aSyuGQjAysq+1G/NIbMyg+wt\nkFupLPjvH2isq4nql4t3/fcBmcMYl+f2FgzwOYX6b93meJH/UOsD/G/UVNTMz3sP4e9V3+zxXE5R\nXcD7l/Xd1rq/4IZX2PjWVww4YiTjbzi69bj3s/XH3+ft77PNWP4TmkfsOZ7M+1lPPqaQn/1jrz2e\nPyR7Q8B7TzxoE+vXe6L6uZaXl2tFhfVvSTQRmRfKol1JnzTcrm5pqrrD3X8D+IOqvurv/O6YNAAa\n6177qRYAACAASURBVJrpkbtnE5Xv8WBJA2B8yzry8pwOdbW1ntZ9r/frS6ivbSE7L525O/cKmjQA\nPnn05wvd0adR0SujSA/KP5YM8fnS7+JJA6C5rpGM3B5tjgVLGrDn592ZpAG0fo7+BEsaAKWDNkT1\nc7WkEaL162HgwJhdPtSk0RUawvsDz7qjHTOAfwdKGN2Zv4QR7Lg/vkmifcLwCvRFE0Bzx6eETpC2\nCSNFtE8YoejM5+pPJz/H9qL6uZoQlZY6dYQJFtekISLDgbWq2iAih+MMLnnEHWjjl6quYPdgGWOM\nMQkU78F9/wVaRGQEzvrFg4F/xzkGY4wxYYp30vC487ScBPxTVX8JRKGrkzHGmHiId9JoEpEf48yn\n4p0JMvUqqY0xJkXFO2mcA0wG/qyqK9056h+NcwzGGNP1XH99oiMA4tgQLiLpwHWq2joEXlVXAn+N\nVwzGGNNl3XBDoiMA4ljScKfxHSoine9faIwx3d364GN34iXe4zRWAO+LyAu0XRzkljjHYYxJIN/p\nYYYMGZLgaLqIJBmnEe82ja9xGsDTcOag927GmG5EVWeoarmqlhcXdzjdkUkicS1pqOrvAUQkV1UD\nz7NgjDEmKcW1pCEik92VpZa5j8eJyL/iGYMxxpjwxbt66jbgu8BWAFVdCBwW5xiMMcaEKe5rhKvq\nmnaHWvyeaIwxZrfuNk7DtUZEDgZURDJxVrD6PM4xGGNM19Pdxmm4LgIuAUqBdcB497Exxphguuk4\nDfUdEW6SW9o2D54+ca/BTAqFu3ZSnZOf6DCM2a2bjtP4SERmisjRIoEWavZPRNJFZL6IzOr4bBMN\nvX5bk+gQEuZXbz2b6BCMSUrxThp746yjcSbwlYj8n4jsHeJrrf0jjtJXNJP7fD3pK7rfIm1DqzZz\nzBfzGVK1JdGhGJN04po01PGGqv4YOB9nivRPROQdEZkc6HUiMgg4FvC/4HEImrWp869paQz3djHh\nqW/As8vZwFknuj1/x0K7uCK1ntYt42ln7GXOC7taj6XXtoAn8cXjaBP1kNPUQE6js33viwUAHP3F\nfHIaG8htcDbxeFpf42lo6PR9vJ9buML+bF31tYE7KgZ7zg+bP64bi/dyr32B04EzgE3AZcALOA3i\nM4FhAV56G3A1YU45srD2f2xsWsmAzGGMy5sa2mu+eYaNNZ8zoGAfxg35fji3jaqN/3mEnYsXtD5e\n0y+fVzfvZNR3BnH83yYB8MLVH/HF62vbHAuZQv7dteT/Yyfi8/3R6+ad9Lp5J5oOoy5JY+klA6Lx\ndpKKKJy29G3OWfwmGbo7MVz64atc+uGrNKelcecRR/GvI45C2f1Z5JbvD9f8OqR7rLt5Jjs++Iye\nB+9L2V+O7HSMEX22wK2Xr+DDl6uZfEwhP/vHXiE/197wERsA9ut0ACZliMaxYUVEvsRZP+NBVV3b\n7rlrVHWPadJFZDpwjKr+1F1X/BeqOr3dOa2TnxUVFU34v+tubn1OUXa0bG193DO9L0Lw5pQ9XpM9\ngE42wfi/bmbggp0nc/f1PRlCcW4mW+qa8GQA6qFh/bqAr+03uhCAzcuq2xxLS2sbc25a8F+6+WlN\nsMODrmxp81NSM6FlSAY7cnYfrfNkUdfirJ/V+P/bO/f4usoq739/uTTpvbQJvaQtLVKBcilCVKDK\nMIAoaKEvgy8UsYpopx0VqHQUFOctn9FBBQYFK1LFKQUFESpS5KbIRbm3paWlBenbYqEXeqHXhKRJ\nzpo/9j7JSXJuSU52zknX9/M5n7P3s59n7/Xsdc5e+7mt1VhCrLEINYqiRihqDH5Tsy65cKmZVae9\naAYSdXvQ4KEn/HBuG9+WfZLH8Mr2XidSHttP1e4dlMZarGZDUTHvDB1GTVlZeOLWuhhz6HjebWrf\nii0qaTE+xGLUrd/SvDtoQiUUpf899StuOWcsZp3SreorsLLtWMxYv/r95vTxE/uisHyyYwNLkrdo\n9u83Xn+9kTlz5mBmXf9DhFRXV9uSJUtydbrey9y53TrtVlJW/9eojYasgxeUdB1By6QRKAcGAYvM\n7OJk+ceNGW9H7Gv9JteploY9l/OWRv3I1A2l2uEtD7/aCjHr+CpuXbaRutCXW9uWRvnBA6jrYEvj\nwwPWpZVvcvlmAB77xDYuWdPy4Nh9zUBqZg7g2bqWyLwv7zuUlbtHAbB++zDqdvSldHsJ5dug3/ZA\nxS/deWWXjUYig0sq7aSB57ZOHJO85ZPtvW6VXiG+uOQvXPnXlrkWN3x8Cred9c+t8iW2NH7wrW9z\n484N7c7Vt6K1a7XElsbkLFoaxwxuPb2yM7otWftlGg8LenQ70tKI/w6S8YHDNlNXhxuNXki2RiPq\nKbcVkr4JHEVgAAAws5RPcjO7GrgaIKGlkdRgpGJS/9M4yhooUfaRZSeNPY+jmvZTUpwf3bcjpk2n\n/iv/QvnQ4K3wA2P2cXjpBvr0a1HhOT86kf1zG1uldYbP9ROxctgztS+Df/8+fR+to2bmgTH99PS1\nq3i/pJQ/HH48U19fyulrV3IbrY3GiGnTiZ13AU1VxVmft2rOZ4m9fw5FfcuA9zosV1d1O/vmQ5l1\nXRPl/dvLnO5YW/7/2pFUjd68slNCOF1j0yYYNaqnpYh89tSvCZwVjgeuBd4CXo7iwh0xGM1l8sRg\nxCkqL6Oob1n44CHpA6SrBqNocxOqMbY/VEHtDUPY9scKtM8o2tL7vb0cvG8X/RrquWjaFXz/zAuY\ndtFs+jfUc/Ce9lOPi+LdVR0grrfO0lXdpjMK2RiMBPJrhsiBQlVVT0sARN/SGGZmt0u63MyeBp6W\nlLXRMLOngKe6SzgH1GRsW1wBfYPeh8YjStm2uILi92IZShY+xbEYF027nPqS4GVhbcVILpp2Of1i\nNRlKOs6BQ9RGIz66t1nSp4FNwNCIZXDS0DQ6yU+ir4KumLro5YmSzYPa/xTrS/qwe0h+tTgdpyeJ\n2mh8T9Jg4ErgFoJB7dkRy+A4juN0kqgj98WnpeyGNqOLjuM4Tt4TidGQdAuQcqqtmV0WhRyO4zgF\nywEWT8MnYTuO43SFPImnEYnRMLM7sskn6RYz+3p3y+M4jlNwHKDrNDIxuacFcBzHyUvyZJ1GvhkN\nx3EcJ49xo+E4juNkTb4ZjZw5QXMcx3FyT48YDUn9Uhz6SaSCOI7TI0iaIWmJpCXbtnmExEIiUqMh\n6WRJqwmcFiJpkqSfxY+b2YIo5XEcp2cws/lmVm1m1ZWVlT0tTmGQJ+s0om5p3AR8EtgBYGYrgFMi\nlsFxHKfwyJN1GpF3T5nZ222Ser/PbcdxnK6yaVPmPBEQtcPCtyWdDJikUuByYE3EMjiO4xQeVVUQ\nYaTVVETd0pgJfBWoAjYCx4X7KZFULuklSSskvSbp2gjkdBzHcZIQtZfb7cDnOlisHjjNzPaFrZO/\nSXrEzF7IvYSO4zhOOiI1GpJ+BHwPeB94FDgWmG1md6UqY2YG7At3S8NPz7fROkBjDmONx96v73LY\nUIC6muxiQqdif21j0vSm/fVA7wtaFKuv71SI186yv7brsd5TEdd9R34DNTUx+vfvno6JddtquOC2\n57vl3L2J30Je3Keou6fONLM9wGcI4oMfBvx7pkKSiiUtB7YCfzKzF7tVyhyyYsMinlh9PSs2LOry\nubbNv4s3L/4vNt14b5fOc9Nl65g+aQU3Xbau0+V/cvIDLJ/7SKv0LXcvZM28q1n7zMIuyZdvbLl7\nIeuuvZotd0dTr+VzH+EnJz/Ag9/MfWM6rvtZH1+Z9W9g5qydfPDwd5k5a2fO5XEKD1mEAyuSVpnZ\n0ZJ+CdxnZo9KWmFmk7IsPwT4PfB1M1uVkD4DmAFQUVFxwn9954auC9untMunMDP21m1p3h8wcBRS\n8kXvsdKW9FiJqOxXyrbaBmLxl02LUb9pY3OeQRMq6Vea/G0/Ff2K6rGYsX71+81p4yf2RUXBtQcU\nNaQqCsC+WGm78oMmVLI/VkpsP+x/u2V2R7+hVfzbl6YtNbPqDgnZhkTdHjR46Ak/nPvfrTOk0JOV\npn4fSrzXrdJL2qebxajb2nLfy0ZVgYJzW4kxvLgP7zbtb1euqCR1TPU+JWn0FjP2vNmy2O3gI4ZQ\nVJTZUUK/ovp2aaqvwMq2N++31V2cxN8AtP4dxGKwalXL/tFHl/KpT13eZb0mUl1dbUuWePSEjMyd\n263TbiVlpdeoZ089JOl1gu6pWZIq6UDkaTPbJelJ4FPAqoT0+cB8gHFjxtv91/6165KOGdH1cxC0\nNLbsXsOIwUdyxEe/kDJf7fCWh19thZh1fBW3LttIXcK6p02LFlK75FUGnnwUk687nWMGd2wK3ocH\nBG+V9/90Hc8/vIuTzh7C7KmHNh8/qnxz2vLP1o1sVX7EP0/guLlnsX77MOp292XHHb9h38rlDD1k\nEoedMr1DsqUiUbeDSyrb6zaFnupHDkx5zsR73Sq9IvnD+a0/38W+lcsZcMxxjJjWUq+GikauPGgs\nN+7c0K5M34ralNcfN+i9lMcAlv/xEbY8+SaHnzmac844MW3eOHHdJlKy9ss0HvbLVmlx3Q0bWcqO\nzQ3tfgPQ/ncwb95OFi+uY8qUcqZdeFBW8jjdQJ6s04h6IPyqcFxjt5k1SaoBzk1XJjQsDaHB6At8\nAvhhBOLmhEljz+OocEyj/btgx6iccTF9rtgVjmns6PR5Zt98KLOu6/yYxuybD+Uj14zljYaxrdJH\nTJtO6SkXMHBP7xrTGDFtOrHzLohsTOO4uWdx+PeP6pYxjUTdZzum8fNbD+LGG7pvTMPJkjyJpxF1\nSwPgCGCcpMRrp+ssHgncIamYYAzm3oRY4wVBrgbBgZwMggNdGgQHggfa7vbpxX3KKLB5ClkR5SA4\n0G2D4NCi+478Btxg5AF5sk4j6tlTdwIfAJbTshLcSGM0zOxV4EPdL53jOI6TiahbGtXARIty9N1x\nHMfJGVG3OVcBuRlhdhzHcSIn6pZGBbBa0kvQMi5sZudELIfjOI7TCaI2GnMjvp7jOE7vIE/iaUQ9\n5fZpSYcAE8zsz2EEv65N43EcxzkQyJN1GlFH7vsKcB9wW5hUBTwQpQyO4zgFSZ7E04h6IPyrwGRg\nD4CZvQkcHLEMjuM4hUdVVU9LAERvNOrNrNlRT7jAz6ffOo7jFAhRG42nJX0b6CvpE8DvgMURy+A4\njuN0kqiNxlXANmAl8K/Aw8A1EcvgOI7jdJKoZ0/FgF+EH8dxHKfAiMRoSFpJmrELMzs2Cjkcx3EK\nlgNsncZnwu+vht93ht8X4wPhjuM4mcmTdRqRGA0z+weApE+YWaLH2m9JWkYw1uE4zgFCYkTGsWPH\nZsjtAHkTTyPqgXBJmpywc3IPyOA4Tg9jZvPNrNrMqisrKzMXcPJmnUbUvqcuBX4laXC4vwv4UroC\nksYQxNsYTtCVNd/MftKtUjqO4zhJiXr21FJgUtxomFmr2G+SvmBmd7Qp1ghcaWbLJA0Elkr6k5mt\njkZqx3EcJ06PdA2Z2e62BiPk8iR5N5vZsnB7L7CGwGdV8nPnTMqAxqb9XTqe8fyNmSOHx+pT59lf\n25ixfDZ56mqa2qXV1MRafWcinZy5wNpot9EaUubNdF+bGnIra6yu++q+v7YxpQ6z0W0ykuk7kQw6\n9y7lA5ieiBGeDqU9KI0jCP36Yqo8MRpZUfMXJvU/rcvCrNiwiC271zBi8JFMGnteh49n4rUVv2Hb\nu69SOfxYxp/5haR5tty9kH0rlzPgmOMY9vWLWh178Jsv8Mbj73D4maM550cnJi2fmOfDP0vu5uum\ny9bx/MO7mDKlnJ/fehAAM2ftZPHiOkaNKmLTphhTppTz+ZtGpqzLtvl3UbvkVQZ98DiO+Ojns6l+\nh9nbtKNZtytq/sKWhvWM2ND+3q/YsIgtK9dQOfxYjpp0UbvzrH1mIe/9YwVDD5nEYadM77Jc8br3\nqz6WyhkXd/l8icT1B7TTcza6TUZc3yedPYTZNx/a7nhc94m/h8RjePjlAxrlU+RVScvM7PgUxwYA\nTwPfN7NFbY41z8SoqKg44aqrrmJg8TCU3galxUpL2Fu3pXl/YPkIpJbzmVna48nP2fKCZmbs29vi\ntbLf0Krm8rESUdmvlK019dRt3dicp8+YURT1CbeLGtjz5rbmYwcfMYSiotbXj8WMra/vat4fP7Ev\napPHYsb61e837x99dCkAq1a1f4tPLF8bK6O2Kchbv7+Y+vWb29Vl1iUXLjWz6nT3JBOJuh08ePAJ\n3/3udxlYPJS9Te8150m89231MmDgqHZ6S3ffkxFL8WplJcbw4lLeXre+Oa3PmFFQVERRSeo39T4l\nmVsH/Yob2ukPWvScjW5VX4GVbW8tcxt9ty2X7PdQFP5sY7HgdzFnzhzMrPN/rjZUV1fbkiVLcnW6\n3svcud067VZSVv/XgmhpSCoF7gd+3dZgQDATA5gPMGbMGLvh6nldb2mMGZHzlkb9yIGt9lO1NGor\nxKzjq/j5K5t56893tWpplA8L/tDjh+5g/eOLWloaZ6Roadzc8jb6n1OTv43e/9OWlsa0C4M3y3nz\n2rc0xk6d2FL3fYeycncw/W/99mG8fc/93dLSSNStpGbdNrc0ktz7uF5StTRefePupC2N2orkz8G6\nFJN7GioaufKgsfzHPQvbtTT6VtSmrNO4Qe+lPBbnmMGBYYvrD2in50y6LVn7ZRoP+2W79Li+Tzp7\nCLOntm9pPH3r6uaWRvz3EGfevJ0ZZXe6iTxZp5FvLY2fmtnX2qQJuAN4z8yuyHSOQ8aMtyP3nd51\nYcYEocwbm/ZTUtwnZbZMxxNpazQg6HsvKSmjdnhpc1rcaNy6bCN1lcFYQVFZGQ0VjS1Go2IHxwze\nxP7aRvr0S2/743k+PGBdyjx1NU2cPmxrq7Samhj9+xc1fz9b19I99XIbo1G3oy/FG5vot6eMftuD\n39RLd17Z5ZZGIoNKKuzkgVOb9xutgZKxY5LmransQ0lJWdJjtcNLaWqop7i09fHOGo0bd24gVldP\nUXnL+dIajWHZGw1oGbdIpud0uk1lNCDQd3n/5PHPJpdvbtZ5MqpGb34lVY9AZ/CWRpZ08zqNvGxp\nSBoCTAfGJV7bzC4Lv7+WpNhk4PPASknLw7Rvm9nDSa+RS4Eho0HI1mCkLJ/iwZZIUVnqPJkMRrZ5\nkj1A4g+NVA+PtqSTMxe07W4sUWmKnJnva1uD0VUSDUauSae/bHSbjFQGI04GnWc3M8LJLVVVkAcv\n+VF3Tz0MvEDg5TarH56Z/Y3c2wLHcRynE0RtNMrN7BsRX9NxHMfJEVHPt75T0lckjZQ0NP6JWAbH\ncRynk0Td0tgPXA98h5Z1eAa0n8LhOI7j5B1RG40rgcPMbHvGnI7jOE4LeRJPI+ruqbVA6rmIjuM4\nTnLyZJ1G1C2NGmC5pCeBZmc98Sm3juM4TgryJJ5G1EbjgfDjOI7jdIQDcZ1GErfnjuM4TgER9Yrw\n9STxXm5mPnvKcRynAIi6eyrRr0k58FnA12k4juMUCJHOnjKzHQmfjWb2Y+DTUcrgOI7jdJ6ou6cS\nPWMWEbQ88s09u+M4Tv6RJ+s0on5g30jLmEYj8BZBF5XjOI6TjjxZpxH14r6zgNuBJ4BngY3AhRHL\n4DiOU3hs2pQ5TwT0xDqNXcAyoC7iazuO4xQuB+I6DWC0mX2qIwUk/Qr4DLDVzI7uHrEcx3GcbIi6\ne+o5Scd0sMwCoEOGxnEcx+keojYaHwOWSnpD0quSVkp6NV0BM3sGyBxUuYM0WkPnyzbtz6EkLTQ1\n1GfOlEA8dnRX88SpqYlRU9M+oGKytJ7AwjkUXdFdKtLd+1h9x/TSquz79cTe73z5XFBX05TrU3b5\nuSFphqQlkpZs27YtFzI5ERF199RZEV8vKStq/sKWhvWMKB3PpP6ndazshkVs2b2GEYOPZNLY83Im\n09pnFvLeP1Yw9JBJjDrvCxnzL5/7CI8++SaHnzmac350YtI8D37zBd54/J20eeLcdNk6nn94FwBT\nppTz81sPAmDmrJ0sXlzHlCnlfP6mkR2sVW7Z27SDp3ffQ53VdEp3qYjf+0EfPI6xn57e6tiGPy5k\nz9+XM+CY4xgxbXqKMyRn4w2/Y+9zrwEw8OSjqJoT/UTBuF5POnsIs2/uuuOFmbN2Anyoq+cxs/nA\nfIDq6uqe76h3skaWBwMrmZA0Dngo1ZiGpBnAjHD3aGBVmtMdROugT+uAneH2YGB3m+3B4f7uJGVf\nIctY5xkoovUf8RWClfKp4o6kqwMEMu9tc854nsT6pLp+XAaSyNWR+h5uZgM7kL8dbXR7QpvDubj/\nye59LItjcSpIrqdU9zTKZtvBwJgcXr+5TmamLpynFZK2Af/I1fmSkEpHhUZ31+MQM6vMmMvM8v4D\njANWZZl3SYbj81PtJ9smeBtqtx1BnVPWI10d2sqer3XoLZ98rmM+y+b3oXDrcSCuxl6cZj/Zdqrj\nPUm6OiTu53MdHMcpQPK+e0rS3cCpBE2zd4H/Z2a3p8m/xMyqUx0vFHpDPXpDHTKRz3XMZ9mipLfc\nh3ypR963NMxsWgeLzO8WQaKnN9SjN9QhE/lcx3yWLUp6y33Ii3rkfUvDcRzHyR+iXqfhOI7jFDBu\nNBwnDZLGSHpS0mpJr0m6PEwfKulPkt4Mvw8K0yXpZklrwwWsx6e/QpflK5b0iqSHwv3xkl4Mr/9b\nSX3C9LJwf214fFx3ypUPpNJdIdJWzz2JGw3HSU8jcKWZTQROBL4qaSJwFfCEmU0g8Np8VZj/LGBC\n+JkB3NrN8l0OrEnY/yFwk5kdRrAu59Iw/VJgZ5h+U5ivt5NKd4VIWz33GAeU0ZBUJOn7km6RlHnZ\ndZ4iqX/oguEzPS1LZ5E0VdIvwrffM3tanlSY2WYzWxZu7yX441YB5wJ3hNnuAKaG2+cCCy3gBWCI\npG5ZSi9pNEHky1+G+wJOA+5LIVdc3vuA08P8vZY0uiso2uq5pykYoyHpV5K2SlrVJv1ToS+rtZKu\nSlU+5FxgNNAAvNNdsqYiR3UA+BZwb/dImZlc1MPMHjCzrwAzgQu6U95cEXbpfAh4ERhuZpvDQ1uA\n4eF2FfB2QrF36L4H1Y+Bb9KyynsYsMvM4g7HEq/dLFd4fHeY/4Cgje4KjbZ67lHyfsptAguAnwIL\n4wmSioF5wCcI/iAvS3oQKAaua1P+S8DhwHNmdpuk+wi6FaJkAV2vwyRgNVAegbypWEAX62FmW8Pt\na8JyeY2kAcD9wBVmtifxJd3MTFKk0xDDVuZWM1sq6dQor11otNVdT8vTEfJRzwVjNMzsmSSDdx8B\n1prZOgBJ9wDnmtl1BDE4WiHpHSDuojbnrj8zkaM6nAr0ByYC70t62MwifQPJUT0E/AB4JN6FkK9I\nKiV46PzazBaFye9KGmlmm8Pup7gR3Ehrf0+jw7RcMxk4R9LZBC8Qg4CfEHSHlYSticRrx+V6R1IJ\ngQ+yHd0gV16RQneFRDs9S7rLzC7uKYEKpnsqBR3tClgEfFLSLcAz3SlYB+hQHczsO2Z2BfAb4BdR\nG4w0dFQXXwfOAM6XNLM7BesKoXG7HVhjZv+dcOhBID4u9gXgDwnp08NZVCcCuxO6sXKGmV1tZqPN\nbBxByOS/mNnngCeB81PIFZf3/DB/r16klUZ3BUMKPfeYwYACamnkAjOrpWU2SUFjZgt6WoauYGY3\nAzf3tBxZMBn4PLBS0vIw7dsEraR7JV1K4KH1/4bHHgbOBtYCtcAl0YrLt4B7JH2PwKtt3OXO7cCd\nktYSxKe5MGK5eoKkujOzh3tQpoKn0I1GVF0B3UlvqAP0nnq0wsz+BqSaZXR6kvwGfLVbhWp/zaeA\np8LtdQRdhW3z1AHRB/ToQTLoruBI1HNPUujdUy8DE8IFTX0I3p4e7GGZOkpvqAP0nno4jpOGgjEa\nCrzdPg8cLukdSZeGg31fAx4jmIN9r5m91pNypqM31AF6Tz0cx+k47rDQcRzHyZqCaWk4juM4PY8b\nDcdxHCdr3Gg4juM4WeNGw3E6gKRxbX1uFRKS5kqa09NyOIWLGw3HcRwna9xoOAVJ+Ma/JnSv/pqk\nxyX1lfSUpOowT4Wkt8LtL0p6QEHApLckfU3SNxQEtnlB0tA01zpB0gpJK0hYuKcgMM71kl5WEHDp\nX8P0U0M57pP0uqRfhy4tkPQDBUGBXpV0Q5hWKen+8DwvS5qcRpa5oZfhpyStk3RZwrFvSFoVfq5I\nSP+OpL9L+huB0854+gckPSppqaS/SjoiTP9seI4VkvLF3Y6TL5iZf/xTcB9gHEGQnePC/XuBiwlW\nzFaHaRXAW+H2FwlcewwEKglcg88Mj91E4AE11bVeBU4Jt68HVoXbM4Brwu0yYAkwHjg1PP9oghez\n54GPEbgif4OWqe5Dwu/fAB8Lt8cS+EpKJctc4LnwehUETgdLgROAlQTOLAcArxG4Ao+n9yNwargW\nmBOe6wlgQrj9UQK/RoT5qxJl9I9/4p9CdyPSLUjaZ2YDuvka5wATzewH3XmdFNeeCvzdzFZHfe0c\ns97M4j6FlhIYknQ8aUEwnr2SdgOLw/SVwLHJCkgaQvDgjL9x30kQnQ/gTOBYSXEHgYMJIvbtB14y\ns3fCcywPZXsBqANuVxC2Mx668wxgolrcrQ+SNMDM9qWoxx/NrB6ol7SVIJbHx4Dfm1lNeM1FwMcJ\njNbvLfC7hgJ39XF34ScDv0u4bln4/SywQNK9BE4+HacZNxrdiKRiM0vqgt3MHqQb3WykuzZBNLeH\nCOJyFDL1CdtNQF+C1ke827VtzJHE/LGE/Rid+y8I+LqZPdYqMXBf31a2EjNrlPQRAp9V5xOsoD8t\nlPdEC/xDZUO7c3dC9iKCgE3HtT1gZjMlfZQgWtxSSSeYWa93o+5kh49pZEDSvyf0WV+bkP5A2Bf8\nmqQZCen7JN0Y9n+fFPafXytpmaSVCf3GX5T003B7gaSbJT0X9lOfH6YXSfpZ2C/+J0kPJ7zVEZLN\ncAAAA0pJREFUJpP1LUk/lLQM+Kykr4Syrwj7zPtJOhk4B7he0vKwXztp33aB8hZBlwy0uAjvNGa2\nC9gl6WNh0ucSDj8GzFIQswFJH5TUP9W5wrf7wRZ4WZ1NEFAL4HECV/HxfO0e5FnwV2BqqOP+wP8J\n054J0/tKGghMCeu1B1gv6bPhNSVpUrj9ATN70cz+A9hGa0eUzgGOtzTSoCB29QQCr6ECHpR0SthV\n8SUze09SX4IodfeHb2P9gRfN7MrwHADbzex4Sf8GzAG+nORyIwm6GI4gaIHcB5xH0K0xETiYwKfT\nrzKIvcPMjg+vPczMfhFufw+41MxuCbsoHjKz+8JjTxD0778ZvmH+jOANuBC5gcBl+Qzgjzk65yXA\nrxRE53s8If2XBPpZpkDR22iJyZ2MgcAfJJUT/J6+EaZfBsyT9CrBf/IZgjC4WWNmyyQtAF6Ky2Zm\nrwBI+i2wgiBQ1MsJxT4H3CrpGoJxkXvCfNdLmhDK+ESY5jiA+55KSnxMQ8HslvOBXeGhAcB1Zna7\npLkEb3MQPDg+aWYvSGoEyuJdQwpm70w2s43hA/n7ZnaGpC8SDNh+Lfyz/8nMfh2W2WtmAyX9GFhh\nZv8Tpi8CfhN/2CeR+y3gn8zsH+H+PwHfA4aEsj8Wdj0sIDQa4dvvNoIB2jhlZnZk5++g4zi9FW9p\npEcERuK2VolBn/UZwElmVivpKVr6z+uSjCXE+6DT9T8n9lN3JQZATcL2AmCqma0IjdSpSfKn7Nt2\nHMdpi49ppOcx4Evh2ziSqiQdTDBLZmdoMI4ATuym6z8L/Es4tjGc5A/9dAwENod97ol98XvDY2n7\ntg80JM0Lx3kSP1FH3ovLckkSWeb1hCyOk4i3NNJgZo9LOhJ4Phyb2EewFuBRYKakNQTdOi90kwj3\nE8y0WU0Qf3sZwfz/bPku8CJB99OLhIaCoO/6FwoWhp1P6r7tAwozizTiXjrCLsn/6Wk5HKctPqaR\n5yicry9pGMEg52Qz29LTcjmOc2DiLY385yEFC8z6AP/pBsNxnJ7EWxoFiKTfE7irSORbbReZOY7j\n5Bo3Go7jOE7W+Owpx3EcJ2vcaDiO4zhZ40bDcRzHyRo3Go7jOE7WuNFwHMdxsuZ/AQ57Qnkteh1t\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc27c3010b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plot_objective(result=search_result, dimension_names=dim_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们也可以用另一种模型图。这里对角线上显示贝叶斯优化器的每次超参数的采样分布的直方图。直方图下面的图显示在搜索空间中采样的位置，颜色编码显示了采样的顺序。对于大量的采样你希望看到这些样本最终集中在搜索空间的某个区域。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAGECAYAAADDQ9xjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8FHX6wPHPk14IBEhApAVCExBQQKWoqNixndjO3ttZ\nTj1P7/R3eqd3enqW09MT+6nYEBVBBRViBwwg0hHpRSCUkJC++/z+2ElIsrtJNrub3YTn/Xrta2e+\nM/OdZ7Kwz87Md75fUVWMMcYYX2IiHYAxxpjoZUnCGGOMX5YkjDHG+GVJwhhjjF+WJIwxxvhlScIY\nY4xfliSMMcb4ZUkCEJHCJtjH6SJyV7j342ffZ4pI/0js2xjTvIk9TOdJEqraKgT1xKqqKxQxhXLf\nIvIKMFVVJzVtVMaY5s7OJGoRkT+IyA8i8pOI3F+t/AMRmSciS0TkmmrlhSLyLxFZCIwQkbUicr+I\nzBeRRSLSz1nvMhF52pl+RUT+LSLfichqERnvlMeIyDMislxEPhORjyuX+Yl1rYg8LCLzgXNE5Gon\n9oUi8p6IpIjISOB04BER+VFEsp3Xp87xfF0ZozHG1GZJohoROQHoDRwGDAGGishRzuIrVHUoMAy4\nWUTaO+WpwBxVHayq3zhleap6KPAscIef3XUCRgPjgIecst8AWUB/4GJgRAPC3qGqh6rqW8BkVR2u\nqoOBZcCVqvodMAX4g6oOUdVfgAnATc7x3AE804D9GGP2Q3GRDiDKnOC8FjjzrfAkja/wJIaznPKu\nTvkOwAW8V6ueyc77PDxf/L58oKpuYKmIdHTKRgPvOuW/isisBsT8drXpgSLyAJDuxD699soi0goY\nCbwrIpXFiQ3YjzFmP2RJoiYB/qGqz9UoFBkDjAVGqGqRiOQASc7iEh/3Akqddxf+/8al1abFzzoN\nsbfa9CvAmaq6UEQuA8b4WD8G2K2qQ4LYpzFmP2GXm2qaDlzh/NpGRDqLSAegDbDLSRD9gCPCtP9v\ngbOdexMd8f0lX5c0YIuIxAMXVisvcJahqnuANSJyDoB4DA46cmNMi2RJohpVnQFMBL4XkUXAJDxf\nrp8CcSKyDM/9g9lhCuE9YCOwFHgdmA/kB7D9vcAcPMlmebXyt4A/iMgCEcnGk0CudG62LwHOCEHs\nxpgWyJrARhkRaaWqhc6N8bnAKFX9NdJxGWP2T3ZPIvpMFZF0IAH4myUIY0wk2ZlEMyAi7wM9ahX/\nUVW9Wi8ZY0woNfskkZGRoVlZWZEOY783b968PFXNjHQcxpjQavaXm7KyssjNzY10GPs9EVkX6RiM\nMaFnrZuMMcb4ZUnCGGOMX5YkjDHG+GVJwhhjjF+WJIwxxvhlScIYY4xfliSMMcb4FZEkISIvicg2\nEVlcraydMxrbz85720jEZowxZp9InUm8ApxUq+wu4AtV7Q184cwbY4yJoIgkCVX9CthZq/gM4FVn\n+lXgzCYNyhhjjJdouifRUVW3ONO/Ah3rWtkYY0z4RVOSqKKeXgf99jwoIteISK6I5G7fvr0JIzPG\nmP1LUElCRB5uSFkDbRWRTk4dnYBt/lZU1QmqOkxVh2VmWsejxhgTLsGeSRzvo+zkRtY1BbjUmb4U\n+LCR9RhjjAmRRnUVLiLXAzcAPUXkp2qL0vCMr1zf9m8CY4AMEdkI/AXP2NHviMiVwDrg3MbEZowx\nJnQaO57EROAT4B/UbKpaoKq1Wy15UdUL/Cw6rpHxGGOMCYNGJQlVzQfygQtEJBZPS6Q4oJWItFLV\n9SGM0RhjTIQENTKdiPwOuA/YCridYgUGBReWMcaYaBDs8KW3An1VdUcogjHGGBNdgm3dtAHPZSdj\njDEtULBnEquBHBGZBpRWFqrqY0HWa4wxJgoEmyTWO68E52WMMaYFCSpJqOr9oQrEGGNM9Am2ddMs\nfPSxpKrHBlOvMcaY6BDs5aY7qk0nAWcDFUHWaYwxJkoEe7lpXq2ib0VkbjB1GmOMiR7BXm5qV202\nBhgKtAkqImOMMVEj2MtN8/DckxA8l5nWAFcGG5QxxpjoEOzlph6hCsQYY0z0CfZyUzxwPXCUU5QD\nPKeq5UHGZYwxJgoEe7npWSAeeMaZv9gpuyrIeo0xxkSBYJPEcFUdXG1+pogsDLJOY4wxUSLYDv5c\nIpJdOSMiPQFXkHUaY4yJEsGeSfwBmCUiq/G0cOoOXB50VMYYY6JCsK2bvhCR3kBfp2iFqpbWtY0x\nxpjmo1FJQkQuAkRVX3OSwk9O+cUi4lLViaEM0hhjTGQ09p7ETcD7PsonA7c3PhxjjDHRpLFJIl5V\nC2sXqupePE1ijTHGtACNTRLJIpJau1BE0rDBh4wxpsVobJJ4EZgkIt0rC0QkC3jLWWaMMaYFaNSN\na1V9VEQKga9EpJVTXAg8pKrPhiw6Y4wxEdXoJrCq+l/gv84lJlS1IGRRGWOMiQrBPkxnycEYY1qw\nYLvlMMYY04IFfSZhmqesu6Z5la196NQIRGKMiWZBnUmISIqI3CsizzvzvUVkXGhCM8YYE2nBXm56\nGSgFRjjzm4AHgqzTGGNMlAg2SWSr6j+BcgBVLcLTG6wxxpgWINgkUSYiyYACOGNLWC+wxhjTQgR7\n4/ovwKdAVxF5AxgFXBZMhSKyFijAM3hRhaoOCzJGY4wxjRTseBKfich84Ag8l5luUdW8EMR1TIjq\nMcYYE4TGjidxaK2iLc57NxHppqrzgwvLGGNMNGjsmcS/6limwLGNrLdy+xkiosBzqjqh9goicg1w\nDUC3bt2C2FX0secXjDHRpLEd/B0T6kCqGa2qm0SkA/CZiCxX1a9q7X8CMAFg2LBhGsZYjDFmvxbU\nPQkRSQJuAEbjOQP4GvivqpY0tk5V3eS8bxOR94HDgK/q3soYY0w4BNsE9n/AAOAp4Gln+rXGViYi\nqZW9yjqDGp0ALA4yRmOMMY0UbBPYgarav9r8LBFZGkR9HYH3RQQ8sU1U1U+DCdAYY0zjBZsk5ovI\nEao6G0BEDgdyG1uZqq4GBgcZkzHGmBAJNkkMBb4TkfXOfDdghYgsAlRVBwVZvzHGmAgKNkmcFJIo\njDHGRKVgn7heJyJtga7V67KH6YwxpmUItgns3/D01fQLTid/BP8wnamlqR6wa+h+fK1njGmZgr3c\ndC6e7sLLQhGMMcaY6BLscxKLgfRQBGKMMSb6BHsm8Q9ggYgspto4Eqp6epD1GmOMiQLBJolXgYeB\nRYA7+HCMMcZEk2CTRJGq/jskkRhjjIk6wSaJr0XkH8AUal5usiawxhjTAgSbJA5x3o+oVmZNYI0x\npoUI9mG6cI4rYQLk7/kFG7TIGNNYwZ5JICKn4ukiPKmyTFX/Gmy9xhhjIi+o5yRE5L/AecBNgADn\nAN1DEJcxxpgoEOzDdCNV9RJgl6reD4wA+gQfljHGmGgQbJIodt6LRORAoBzoFGSdxhhjokSw9ySm\nikg68AgwH0/LpueDjsoYY0xUCLZ109+cyfdEZCqQpKr5wYdljDEmGjTqcpOIDBeRA6rNXwK8A/xN\nRNqFKjhjjDGR1dgzieeAsQAichTwEJ4WTkOACcD4kETXSIGMv9AUYzVE+vkFG//BGNNYjU0Ssaq6\n05k+D5igqu/huez0Y2hCM8YYE2mNbd0UKyKVCeY4YGa1ZUE/oGeMMSY6NPYL/U3gSxHJw9MM9msA\nEekF2I1rY4xpIRqVJFT1QRH5As8zETNUtXJ86xg89yaMMca0AI2+NKSqs32UrQwuHGOMMdEk2Ceu\njTHGtGCWJIwxxvhlScIYY4xfzb656qJN+SF/WKyh9fl6GK6pHlxrafsxxkQnO5MwxhjjlyUJY4wx\nflmSMMYY41fUJQkROUlEVojIKhG5K9LxGGPM/iyqkoSIxAL/AU4G+gMXiEj/yEZljDH7r6hKEsBh\nwCpVXa2qZcBbwBkRjskYY/Zb0ZYkOgMbqs1vdMqMMcZEgOzrmy/yRGQ8cJKqXuXMXwwcrqq/q7Xe\nNcA1zuxAYLGfKtvg3Stt9bLKaV/vlZqiV9sMIK+O5bWPI5BjyPexfTj0VdW0MO/DGNPUVDVqXsAI\nYHq1+buBu+vZJreOZRPqKquc9vVe+Wqi4/Z7DL6OI5Bj8Pd3aOpjsJe97NU8X9H2xPUPQG8R6QFs\nAs4HfhtEfR/VU/ZRPe/RonY8gR5DtB2PMaaZiKrLTQAicgrwBBALvKSqD9azfq6qDmuS4MLEjsEY\nE62i7UwCVf0Y+DiATSaEK5YmZMdgjIlKUXcmYYwxJnpEWxNYY4wxUcSShDHGGL8sSRhTBxHJEhF/\nz+FEPRG5T0TuiHQcpvmyJGGMMcYvSxKmWXB+0S8TkedFZImIzBCRZBHJEZFhzjoZIrLWmb5MRD4Q\nkc9EZK2I/E5EbhORBSIyW0Ta1bGvoSKyUEQWAjdWK48VkUdE5AcR+UlErnXKxzhxTBKR5SLyhoiI\ns+whEVnqrP+oU5YpIu859fwgIqPqiOU+EXnJqX+1iNxcbdltIrLYed1arfzPIrJSRL4B+lYrzxaR\nT0Vknoh8LSL9nPJznDoWishXAX40pqWL9NN89rJXQ15AFlABDHHm3wEuAnKAYU5ZBrDWmb4MWAWk\nAZl4uiW5zln2OHBrHfv6CTjKmX4EWOxMXwPc40wnArlAD2CMU38XPD+8vgdGA+2BFexrRZjuvE8E\nRjvT3YBldcRyH/Cds78MYAcQDwwFFgGpQCtgCXBItfIUoLXzN7jDqesLoLczfTgw05leBHSuHqO9\n7FX5irrnJIypwxpV/dGZnocncdRllqoWAAUiks++J88XAYN8bSAi6Xi+KCt/Ub+Gp+t6gBOAQU4f\nY+DpE6s3UAbMVdWNTh0/OrHNBkqAF0VkKjDV2W4s0N852QBoLSKtVLXQz3FMU9VSoFREtgEd8SSh\n91V1r7PPycCReJLU+6pa5JRPcd5bASOBd6vtN9F5/xZ4RUTeASb7icHspyxJmOaktNq0C0jGc3ZR\nedk0qY713dXm3TTu374AN6nq9BqFImN8xBanqhUichhwHDAe+B1wrBPvEapa0sD9etXdiNhjgN2q\nOqT2AlW9TkQOB04F5onIUFXd0Yh9mBbI7kmY5m4tnkss4PkiDoqq7gZ2i8hop+jCaounA9eLSDyA\niPQRkVR/dTm/3tuopxeB3wODnUUzgJuqref1xd0AXwNnikiKE8NZTtlXTnmyiKQBpznHtQdYIyLn\nOPsUERnsTGer6hxV/T9gO9C1EfGYFsrOJExz9yjwjtN9/LQQ1Xk58JKIKJ4v9Eov4LmMNN+5Mb0d\nOLOOetKAD0UkCc9ZyG1O+c3Af0TkJzz/B78CrgskQFWdLyKvAHMrY1PVBQAi8jawENiGp9PMShcC\nz4rIPXjua7zlrPeIiPR2YvzCKTMGsG45jDHG1MEuNxljjPHLLjeZ/ZaI/Aeo/YzCk6r6cgRiuRy4\npVbxt6p6o6/1jWkqzf5yU3p6uvbq1Ytde4rYuqPAa3liQhw9OrdvdP2lFetxuXd7lcfFZJAQd2Cj\n661u7969pKZ63//8tWQ9Li33Km+b0JGU2Fb11ltW4WJz/h4Ky8qIQUhPSaY8roByreDA+AISY1xe\n25RrR1bt9K5LRBjQsUONsvX5+eSXeBrolG3cmKeqmfUG1UAZGRmalZUVqupMEObNmxfSz9Y0L83+\nTKJjx47k5uaSt6uQ39z2ImXlNb/4bjhvNJecdlij699dNJ3VeVd7lffpOJlWiaEZYycnJ4cxY8Z4\nlU//dSKzttVstp4Qk8TdBz1HcqzfRjUAuFU55ZlXKd2xi+op8tjRCWyMX87JbX7mtxlLam0VT3LG\nVxz3/CdsLajZZP+sgf3557gTa8b388/c8JHn0YPVt9++rs6AApSVlUVubm4oqzSNJCIh/WxN89Ji\n7klktG3F3248lXZtUgCIjRFOO3oAvz0luC/y9JQT6dTm93gaqECMtKJL2/tCliDqcmyH8QxOH404\nH1Pr+HZc1P2OehMEwPz1m1izY5dXee485ajMoczI783M/Cxc6nmwSmIySGn7FIkJXXj2N6fRvW16\n1TZH9czinrFHe9V1Yu/e3HTEESTFNfvfGsYYP5r95aa+ffvqihUrquYrKlys3rSD9umptG9T/5dp\nQ1W48ymr2EBiXA9iY0JXL/g/k6i0p3wnhRV76JjUlViJbVCd3/6yjive8H54NikujoV/uokdpfns\nKS+gS1I8onnExvVGJKFqPVXl57wdpCYk0LlN6zr3taekhDbJyfM0hMOXDhs2TO1MIjqISEg/W9O8\nNOonoIi0Bbqq6k8hjidocXGx9Oneof4VA603pg1xCW1CXm9DtI5vR+t4v/3RAbDk+5X874HJrFu6\niezB3fntn88gIzWFvL1FNdY7qX9vANontqF9YuXxdPSqT0Tok5nRsPiSaj/obIxpKRqcJEQkBzjd\n2WYesE1EvlXV2+rc0ITdumWbuGvcw5SVeG5y5372E0u+X8n9H93KA999x5Y9nhv6I3p05e4Tx0Qw\nUmNMcxPImUQbVd0jIlcB/1PVvzhPjJoIm/rCF1UJolJxYQkbP1nKF/dewdJft5GWmEhW+7YRitAY\n01wFkiTiRKQTcC7w5zDFYxph17Y9vsu35hMbE8PBBx7QxBEZY1qKQFo3/RVPB2e/qOoPItIT+Dk8\nYe3fPv5gPlef9yzjj3+Ef9w7me1bfSeBSsPGHuy7/Hjf5cYY01ANThKq+q6qDlLV65351ap6dvhC\n2z99OmUBT/x9KuvWbGdPfjGzpi/mjze+hqvC7Xeb4y8czagzajY+Oe6CUV5lxhgTqEBuXPcBngU6\nqupAERkEnK6qD4Qtuv3QB297OvV0Z1fgPqQMgPULKsidvYrDR/fxuU1sXCz/N/Fmluf+wrolG+k1\nJIvswd2bLGZjTMsVyD2J54E/AM8BqOpPIjIRsCQRQvm7i3CNLMV9+r7xaFzDyvn37ukcvmgrp3Q7\niF5tfDdN7Tcsm37Dshu8rxJXOe+tn8MPO1aRmdSac7uPoHdap6CPwRjTcgSSJFJUdW61oQ/BMyqY\nqcblchMb2/gH2YeO7sm0gZu8yjckrWP2oj38e/E3/PPwcfymZ3D3G1SVW3NfYf6uNVVln2z+kWeG\nX8Wgtt2CqtsY03IE8m2WJyLZgAI44/xuCUtUzdCGjTu580/vcPwpj3DWOU/x+sTvaMzT7Kdefiik\nem8XG6PExrpxq/Lggs8pdQWXn+fuWFUjQQCUuSt4efWsoOoNhohcIyK5IpK7ffv2iMVhjNknkCRx\nI55LTf1EZBNwK3B9WKJqZsrLXfzhrrf5IXcNqrA7v4gXX/maSZMD71Yiu0Mn2sR7d/vhcgsVLs/H\ntau0mF/2BDcE8dq9vr+E1/kpbwqqOkFVh6nqsMxM63TUmGgQSOum1ao6FsgE+qnqaFVdG7bImpG5\nuavZ6uNZhY+m/RhwXQkxcfzmwDFe5bsLUvCMLgkJMbEcmFJ3f0r1GdDG9zDG/Vt3CapeY0zLUu89\nCRHx2e1G5b0JVX0sxDGFjEvdfLpxGd9vW8MBKa0ZnzWEA5KD+3L1paiozGd5cbHv8vrMXLKLLdsz\nSG1TBLFKUUU8peXxVcsv7jOU9MTkRtVdaWB6V04+cAifbN6XyNLjU7i613FB1WuMaVkacuM6zXnv\nCwwHpjjzp7FvEPaodOuc95i+aXnV/Ks/z+HNMZfRq3VoL2UcNrwnCQlxlJXVvE8wepTvJqt1KXVV\n8NX6tSgJlBY7vbImuCDBzaCOHbmwz6Gc03NQKMLmvoPPYewBBzN3xyoyE9swrvOhtEusfzAjY8z+\no97LTap6v6reD3QBDlXV21X1dmAo0KBmMCISKyILRGSqM99DROaIyCoReVucPqpFJNGZX+Usz2rs\ngf2Qt75GggDILy/h6WVfNbZKv9q0TuZPfxxHasq+rrYPGdyNKy8/KuC64iSG1PiEmoVlsVAYzz8P\nO41zswdTq4VZo4kIR3Y4iNsPOo1Leh5lCcIY4yWQJrAdgerXT8rw1ce0b7cAy4DKaz0PA4+r6lsi\n8l/gSjwP6l0J7FLVXiJyvrPeeQHEWGXpLt8Nr5bs+rUx1dXr6CP7ctiwHixdtpn09BSyezauu/LY\nmBguGDiI5xfUvOk9/MDO9G3fsK67jTEmVAJp3fQ/YK6I3Cci9wNzgFfq20hEugCnAi848wIcC0xy\nVnkVONOZPsOZx1l+nDTyZ7O/S0q924Sv1UxycgJDD81qdIKodOeII7lh6OG0T04mOS6Os/r257mT\nzwhRlMYY03ANPpNQ1QdF5BPgSDzPSlyuqgsasOkTwJ3su7fRHtitqpUX8DcCnZ3pzsAGZ38VIpLv\nrJ9XvUIRuQa4BiAzM5OcnByfO/5T4gAKK0qr5mMQeha19bt+pBQWFnrFNBwYnt2/an7hnDlNG5Qx\nxhD4yHQuwI0nSfjvcc4hIuOAbao6T0TGBB6eb6o6AZgAnuFL/Q39Weaq4N21P/L9tjV0TE7jwuxh\n9EyLvks29Q1faowxkRJIB3+3AFcD7+FpsP+6iExQ1afq2GwUcLqInAIk4bkn8SSQLiJxztlEF6Cy\nH4pNQFdgo4jEAW2ARj81lhAbx4XZw7gw23pDNcaYxgjknsSVwOGq+hdV/T/gCDxJwy9VvVtVu6hq\nFnA+MFNVLwRmAeOd1S4FPnSmpzjzOMtnamP6tjCN4nK7ycvfS4Wr3pNEY8x+IpDLTYLnclMlF5WP\nAAfuj8BbIvIAsAB40Sl/EXhNRFYBO/EkFtMEpueu4MnJX/PrrgLat07h2nEjGH9kaJ7HMMY0X4Ek\niZeBOSLyvjN/Jvu+3OulqjlAjjO9GjjMxzolwDkBxGRCYOXG7dzz8ie43J6Tth17ivj7xC/olpnO\nYf2sR1hj9meBtG56TES+xHOfARreuslEuamzl1YliOqmfL+k3iSxYvN2np8Z1Q/eG2OCEGjrph/x\ndA8eByAi3VR1fcijMk3K3z2I+u5NrN2+i4ufeZui0vJwhGWMiQINvnEtIjcBW4HPgKnANOfdRMC2\nHQWs27QzJHWdMMx3H1MnDutb53YTv/3REoQxLVwgZxK3AH1VNbiBDKLUzl/z+XV9Hln9O5PSKinS\n4fi1p7CEv/57Gt/P9wwY1KNre+67ZRy9shr/JPmQ7M784ZwxPPPRd+wtKSMpPo6Ljx/KMUN61bnd\nll3e3aOb5i3rrmmRDsFEmUCSxAYgP1yBRIrb7ebZu97mk9e+xlXhJjk1kSv+8hvGXX50pEPz6fEX\nv6hKEABrNuzg7kc+4O2nriImpvEd/11w7CGcMXIA67btoktGG9JS6k+Uw7O7Mmvp6kbv0xgT/QJ5\nTmI1kCMid4vIbZWvcAXWVGa88R1TX/4SV4Xn+nvx3lKe+eNbrF68McKReatwuZn5/Qqv8s1b81my\ncnOj6tSyBbh3Xo572yiSiq6nX6etDUoQAOcccTCHZB3YqP0aY5qHQJLEejz3IxLw9MNU+WrWvp4y\n36tMVfnmI+/ySBMgJsb3R+avvC5a8Qu681Io+xbc26Hsa3TnJWhFw9oiJCfE88r15/D05db5oDEt\nVSBNYO+va7mIPKWqNwUfUtNKSPL9J0hIivdZHkmxsTGcMLofU2curlHevXM7+vc+IOD6tOgtoKR2\nIVr8DpJ2R1XRnvJiUuMSiRXvRBQbE8OY/j0D3rcxpnkItAlsXUbVv0r0OfHCUcz+9KcaZQlJ8Rxz\n9vAIRVS3Wy4/ltKyCmZ+twKXWxnY50Duuenkxg1E5M6rszx3x1oeWjyNlXu20jYhhct7jeay7NFB\nRG+MaW5CmSSapSNOGsyND5/PxH99zK5te+jWtxPX/O0cOnaLvt5iAVKSE7jv1nHcfvVYyspctG+b\n2ui6JOFItMS7NYskHMmO0kJunPM6xS7POFO7yop4bOkMMhPTOLXL4Ebv0xjTvOz3SQJg3BVjOPnS\noyjZW0pq6+RIh9MgaalJ0Pj84JF8OpTmQOmn+8qSToOkk/l0zdyqBFHd5PXzLUkYsx8JZZIIzcDL\nERIbG9NsEkSoiMQhbf+Nli2EipUQ3x+JHwBAqdv3Q3Jl7gqf5caYlingJjEikuJn0ZNBxmKayJ7y\nAraVbK+al4TBSMo5VQkCYGyn/j5vVJ9w4ACvMmNMyxXIoEMj8YxT3QroJiKDgWtV9QYAVX0lLBGa\nkCl2lfD86leYu3M+itItpQvXZ19Jt5QuXut2S23PfYPP4J+LP6GgooQYhNO7DuG3PY6IQOTGmEgJ\n5HLT48CJeAYGQlUXishRYYnKhMUb695hzs55VfPrizbyrxVP8/iQvxPj46zhjK6HcEKnAaws2MoB\nSW3omNy6KcM1xkSBgC43qeqGWkUunyuaqPTdjjleZXllO1hRsMrvNslxCQxu29UShDH7qYD6bnIu\nOamIxOPp8G9ZeMIyxhgTDQI5k7gOuBHoDGwChjjzppkY2f5wr7LMxPb0Tau7t1djzP4rkG458oAL\nwxiLCbOLup9Lkauoxo3rG7Kv8nk/whhjILDWTf8EHgCKgU+BQcDvVfX1MMW2XyrML+J/j37M7BmL\nSGmVxKkXj+a0y44MSd1JsUnc3Ps69pQXUOouJTMxOp8qB1i0Kd9rbIO1D50aoWiM2X8Fck/iBFW9\nU0TOAtYCvwG+AixJhNBfr3qRRbP33Uh+5t5JlJaUM/66Y/1u49IKluyazNrCb0mITaV/+hl0S/W+\ntFSpdXyL6MDXGNMEArnOUJlQTgXeVdUWNwBRUygqmcWW7RewcetYduz+Ky73vj/jqkUbaiSISh+8\nkFNnnTO3/I3vt/+HLcU/sq7wWz7ZeCc/7/ks1KEbY/ZDgZxJTBWR5XguN10vIpl49TNt6rK3eAZb\nd1wGKABl5UsoKf0OuAuAXXkFPrfzVw6wq3QtqwtyvMrn5b1K79bHBxmxMWZ/1+AzCVW9CxgJDFPV\ncmAvYKPNBGB3wdNUJohKpeULcWshAP2H9SQ5NdFru6FH9/NbZ375Jp/le/yUG2NMIAJt1tIPOE9E\nLgHGAyeEPqSWq8Ll54tbPb2tpqYlcfND5xGXEFu1qEOXdlx731l+6+yQ1I8YHyeEHZMHBhesMcYQ\nWOum14CqpmHjAAAgAElEQVRs4Ef2PWmtwP/CEFeLlJw4gsKi92qVCiL7+vwec+ZQhozuww+zlpGa\nlsTwY/sTn+D/Y0qJa8/wjCuZk/dcVVlCTCojMm8IdfjGmP1QIPckhgH9VVXrXdP41Lb1HykpnV3j\njCI97WbWS81LTOkZaRx/zmENrndI+9/SOXUoawu/ITGmFb1aH09KXLuQxW2M2X8FkiQWAwcAW8IU\nS4vldrv5dPI8vpq+iKTk6xh3YQG9+ieQnHg0iQkDgZyg95GZ1JfMpL5B1xNJInINcA1AbOvMCEdj\njIHAkkQGsFRE5gKllYWqerq/DUSkK57LUR3xXJqaoKpPikg74G0gC88zF+eq6i7xDNT8JHAKUARc\npqrzAzqiKPTfhz9mypuzq+Zn58DVd5zM2ZfYfYPqVHUCMAEgsVNvO2M1JgoEkiTua0T9FcDtqjpf\nRNKAeSLyGXAZ8IWqPiQid+FpA/pH4GSgt/M6HHjWeW+2du0oZNq7c73K33nxS8787RHExsX62MoY\nY6JDIE1gv8Tzqz/emf4BqPNXvqpuqTwTUNUCPL3GdsbTdPZVZ7VXgTOd6TOA/6nHbCBdRDo1/HCi\nz7bNu3FVuL3K83cVUVhgj5kYY6Jbg5OEiFwNTAIqm9F0Bj4IYPss4BBgDtBRVSvvbfyK53JUZZ3V\nx6zY6JSFTUlJObO+WMK0jxawo46H1hqre3YHUtOSvMq7ZGXQpm2qjy2MMSZ6BHK56UbgMDxf8qjq\nzyLSoSEbikgr4D3gVlXd47n14KGqKiIBXX+ufoMzMzOTnJycQDavUlZWwcYNO6lwfulPnryCAzql\nk+bjSz0Yl/7hULZt3l31GJ2IcGC3dlVxFxYWNvoYjDEmnAJJEqWqWlb5BS8icdR+fNgHZ4Ci94A3\nVHWyU7xVRDqp6hbnctI2p3wT0LXa5l2cshqq3+Ds27evjhkzJoDD2OeOW17nxwXrapSlpCTw9uRb\nSE5JaFSd/qz7ZRtfz1hMfEIsx5wymA6d0quW5eTk0NhjMMaYcArkiesvReRPQLKIHA+8C3xU1wZO\na6UXgWWq+li1RVOAS53pS4EPq5VfIh5HAPnVLkuFlMvl9koQAEVFZSxdsjHk++ue3YGLrj+W8648\nukaCMMaYaBbImcRdwJXAIuBa4GPghXq2GQVcDCwSkR+dsj8BDwHviMiVwDrgXGfZx3iav67C0wT2\n8gDiC0hsbAxt26Wya+der2UZmdHVjXZxxXYKytfRJiGbxNi2kQ6HNcu3MP29HyjZW8bIEwZy2Bj/\nfUsZY5q3QEamcwPPO6+GbvMNIH4WH+djfaUJh0Qdf97hPP/szBplww/vSfes6HmQ68e8x1iV/xaK\nixhJ4KD0K+jf7uqIxTN75lIe+N1rVS22pk/6gfOuPSZi8RhjwqveJCEii6jj3oOqDgppRE3ovAtG\nkJqSyNQp8ykuKmP0Uf246LLRkQ6ryobCGfyc/0bVvFvLWLLrv2QkD6FD8vCIxPTSI594Nel976Wv\nIhKLMSb8GnImMc55r/yF/5rzfhENuHEd7cadcSjjzjg00mH4tLHwCz/lM8OTJPLyIMP/kKblZRVs\n+GWbV3lFucvH2saYlqDeG9equk5V1wHHq+qdqrrIef0R6yo8rGJjfDfFjfNT3ljbigp5dfF8Vl9y\nEduKCv2uF58QR5ce3pfiYuMC7XHeGNNcBPK/W0RkVLWZkQFubwLUI+1Mat/SEeLISjstZPuYtX41\nR775PK+8/xY9P5nOhU88yMz1v/hd/7LbTiImtubHfsYlo/ysbYxp7gJp3XQl8JKItHHmdwNXhD4k\nUykz+RAO7/A3Fu98lr0Vm0iL78Hg9rfQOqFn8JW73bj37uXBzz8idu9exuUuBOCkOfN5sONHHH3O\nFcTGxEBKCsTsSwqjThzI4+/cyPRJP1C8t5RRxw9k1IkDuebu4EMyxkSfQFo3zQMGVyYJVc2vvlxE\nLlXVV31ubBqtW9rJdEs7GZe7xO/lp0ZRZc/fH+DTfz5KnHvfjejbp8zg9ikz0GvvgHvvhXvu8dq0\nz8Fd6HNwl9DFYoyJWgFfLlLV/NoJwnFLCOIxfoQ0QQDExhJ3/9+47I7r2dyuTY1FOzuksOOTSfCX\nv0Cs9VJrzP4skMtN9fH3PISJUq0SEuh/1ni+3j2H8ybkVpUvvKQrm3p9xUVVnfOa+mTdNc2rbO1D\np4Z924bW50sw+zD7j1DeeG72zWH3R3cMP4LRc1ZRnhTDgvO6Up4YQ+/Pt7K5eBk7StdHOjxjTISF\nMknYmURztHkj8cUuXntrBJ/93wBee3sECUUVtNpWQoW7zOcmJa4KdpYUNXGgxphICOXlpm9DWJdp\nIgnueL756EbyKjxda+X1TuP1N0fQsbAtHZKya6zrVuXheTm8tmIBRRXlDGjXkX+MOJFBGc16XChj\nTB0CGXQoXURuFpHHROTfla/K5ar6u/CEaMKqe3eO73E7nZL6VhW1at2V4w/7B9XH/QCYsGQuzy2Z\nS1FFOQBLdm7l0s/fpajc9xmHMab5C+RM4mNgNp5eYL3H4zTNVlp8Bhf3fIK8knVUaBkdk3p5JQiA\nd1f95FW2q7SYzzeuaoowjTEREEiSSFLV28IWiYm4jKTudS6vcPv+bVDup9wY0/wFcuP6NRG5WkQ6\niUi7ylfYIjNRZ1zWQV5lyXHxHNcl28faxpiWIJAziTLgEeDP7GvuqkAI+ogwzcFNg0eyrmAXH69b\ngVuVDsmteHjkSaQnJkc6tJAJ9TMLzT0OYwJJErcDvVQ1L1zBmOiWFBvH00efwZa9e9hRUkTftpnE\nx9gT2ca0ZIEkicohRc1+rlNqazqlto50GMaYJhBIktgL/Cgis4DSykJVvTnkUZmwWLLmV/4z6RuW\nrN5C907tuPr0ERw5xK4WGmP8CyRJfOC8TDO0dWcBNz4yib0lnmcalq3dyh+e/pAJd53HoF4HRjg6\nY0y0CqSrcOsGvBmb9u3SqgRRyeVWJs1aaEnCGONXg5OEiKzBRyd+qmrXK5qB/L3FvssLfZcbYwwE\ndrlpWLXpJOAcwJ6TaCZGD+7JxBnzfZYbY4w/DX6YTlV3VHttUtUnAGu43UwMP6gbF500lJhq3W0c\nO7Q3Zx11cASjMsZEu0AuNx1abTYGz5lFKHuR3a9t31HApI/ms2Z9Hr17dmD8uENpm54a0n3ccu7R\nnD1mMMvWbiWrUzt6d80Maf2R0NABdqJlH8HUFepjbYq/nWn+AvmS/xf77klUAGvxXHIyQXK53Fx7\n++vk7SwEYHbuaj7/chkvPH4Jaa1CO2xplw7pdOmQHtI6jTEtVyB9N50MvAh8gWfsiE3A+eEIKloU\n7t7Lwi+XsH3jjrDuZ1d+UVWCqLRlaz5TP/PuddUYY5pSoM9J7AbmAyXhCSd6TH5yGi/9aSKlxWXE\nxMZw8hXHcvOzVxMTE8rB/DzKylw+y9dt2Nmw7Ssq+GD2EmavWE+H9FacN3owPTpamwJjTPACSRJd\nVPWksEXSaKEfWnvlvF949vevVM27XW6mPf85fYb34pSrjgv5/pKSfH8M/Xp19Fm+ozQPRclI9NxT\nuPn5KXy/fF3V8vdnL+alm85hQLcDfG5fVFbO5t176NK2DUnxdlvJGONfIN8Q34nIwaq6KGzRNIJb\nS/hhy8UcnPkoSXG+v1QD9fV7c/yUfx+WJJHeOoUe3TJYs35f34n9eh3ASccNrLHerrKdvLjmGVYV\nrgCgZ2pvhslZNRIEQElZBROmz+HJq8/w2tdL3+Ty7Kw57C0to3VSIjeNHcmFRwwJ+TEZY1qGQJLE\naOAy56G6UkAAVdVBYYksALtLclmy/S6Gdno5JPUlpST6LE/0Ux6smBjh2UcuZMasJaxen0efnh05\n/uiDSEyMr7HeK2v/W5UgAFbv/Zk81+uA9xnDqi3e91G+/Xkdj376ddX8npJSHpw6i34HZDI0q3Po\nDsgY02IEkiRODlsUIbCzZDalrjwSYzOCruu4i45k4t/fo6ykvEb5KVeNDbpuf1KSEzjzlEP8Ls8v\n382KgmVe5XtiN5OQ2payvTUTWP+uHbzWnfbTcp91T/tpuSUJY4xPohr6a/pNSUQKgBV+FrcB8uso\nq5z29V6p9vbhkAHUNU5H7eMI5BjyfWwfDn1VNS2YCkTkGuCayvrw/7lGUn2fVSSFK7buqtr8H6ox\njaOqzfoF5NaxbEJdZZXTvt4rX5E+Bl/HEcgx+Ps7NPUxtJRXNB9nNMdmr+b7aulNWz6qp+yjet6j\nRe14Aj2GaDseY0wz0RIuN+Wq6rD614xedgzNRzQfZzTHZpqv0D8Z1vQmRDqAELBjaD6i+TijOTbT\nTDX7MwljjDHh0xLOJIwxxoSJJQljjDF+WZIwxux3RCRLRBZHcP+XicjTkdp/ICxJGGNMCyMiIXu8\nwZKEMSZinF/0y0TkeRFZIiIzRCRZRHJEZJizToaIrHWmLxORD0TkMxFZKyK/E5HbRGSBiMwWEb99\n5IvIUBFZKCILgRurlceKyCMi8oOI/CQi1zrlY5w4JonIchF5Q8Qz/q+IPCQiS531H3XKMkXkPaee\nH0RkVAP/BqeJyBznGD4XkY4iEiMiP4tIprNOjIiscvbhcz8icp+IvCYi3wKvicgAEZkrIj86cfZu\nxEdkScIYE3G9gf+o6gA8Y9acXc/6A4HfAMOBB4EiVT0E+B64pI7tXgZuUtXBtcqvBPJVdbhT59Ui\n0sNZdghwK9Af6AmMEpH2wFnAAPV0cPqAs+6TwONOPWcDL9RzHJW+AY5wjuEt4E5VdQOvAxc664wF\nFqrq9nr20x8Yq6oXANcBT6rqEDzDTW9sYDw1tPQnro0x0W+Nqv7oTM8DsupZf5aqFgAFIpLPvh4F\nFgE+e6UWkXQgXVW/copeY1+npScAg0RkvDPfBk/iKgPmqupGp44fndhm4xl47UURmQpMdbYbC/R3\nTjYAWotIK1WtOeykty7A2yLSCUgA1jjlLwEfAk8AV+BJcn7340xPUdViZ/p74M8i0gWYrKo/1xOH\nT3YmYYyJtNJq0y48P14r2Pf9VHug9+rru6vNu2ncD1/Bc4YxxHn1UNUZ/mJT1QrgMGASMA741Fke\ng+eMoLKezg1IEABPAU+r6sHAtTjHq6obgK0icqyzv08asJ+9lZWq6kTgdKAY+NipJ2CWJIwx0Wgt\nMNSZHl/Heg2iqruB3SIy2im6sNri6cD1IhIPICJ9RCTVX13Or/Y2qvox8Hug8vLVDOCmaus1dDSv\nNsAmZ/rSWstewHPZ6V1VrRznuEH7EZGewGpV/TeeM5JGjf0TdUnCuRm1yLnZkhvpeIwxEfEoni/u\nBXi6QA+Fy4H/OJeNpFr5C8BSYL7TLPY56j4jSQOmishPeO4n3OaU3wwMc24SL8VzT6Ah7gPeFZF5\neHf1PgVoxb5LTYHs51xgsXO8A4H/NTCeGqKuWw6nFcMwVY3WPvuNMaZJOC28HlfVIyMVg924NsaY\nKCQidwHXU/PSWNPHEYVnEmuAXYACz6mq9WxpjGkwEfkPUPsZhSdV9WVf64c5lsuBW2oVf6uqN/pa\nPxpFY5LorKqbRKQD8BmeVgdf1VqnapjLpKSkod26dYtApKHjdruJian/9lCZu9RnudsVAyo1ykQg\nMb7miWKZuwxfn7YiVC4QAbdLvFcSSI6rWV9JRUVVfRtWr87TIIe4rP65JiYlDu3U5QDAc3yCeB1P\nJDT0s6quQitwqdurPFZiiPPzYGy5y43L7WObGCE+Nrbe2NyqlLkrfK6XFBtfZ7ylLhfuat8Lofhs\nK2VkZGhWVlYoqjJBmjdvXoM+16hLEtWJyH1Aoao+6m+dvn376ooV0TgUcsPl5OQwZsyYetd7cOk9\nbCheV6MsM/4Apr3cGZe75ud42bFD+f0ZR9Uoe3zlM+TuWlCjzOUW1uS39yQK4IJuo5g4Yz27S0pq\nrHd8n14885vTapTdM3UGb670dH+z+vbb54VywJvM/u317NdOoaI0lnmTDuaKY4/gpnENeoA1rBr6\nWVU3f9dC/rXSu5ueG7OvYmTG4T63+Wb5Wq578X2v8n+cfxKnDT2o3thKXOWMy3mIPeXFNdY5puMA\nHj6k7qsX7y1ewp0zplfNh/KzHTZsmObmWnuUaCAiDfpco6p1k4ikikha5TSeh1wi1glXtLk06xra\nxrevmk+Pb8vVvW7kz+ccR2L8vl+Xw3t35ZoTj/Da/pLu59M1uXPVfLwksKM4vSpBDG3Xk2v6jOXR\n006idWJi1Xp9MzO4d+wYr/ruO3ksPVLS8Xl6EgIVpbH8/HUWh2VnceXxh4VnJ03g0LaDOemAsYjz\ndxaEYzKPZER7/8c0ul8Wlx41lBjngSkROGv4AE49pF+D9pkUG89fB51LWty+Rwx6tTqA2w4aV++2\nZw8cwKiu3cL2uZpmJtKDbFd/4XnsfaHzWgL8ub5t+vTpo83drFmzGrxuhbtCl+Uv1iX5P2mFu7yq\nfHdhsc76aZUuWf9rndu73W5dvmelLtj1k5ZUlOju0r365dalunT3xhrrFZWV6axVq/WH9RvV7XbX\nWedXq9YokKsh/LfQb1A//WzhUl26YWsD/zJNI5DPqrZtJdv1hx0LdEtxw49p0858/WLRKl27bWej\nYisqL9Vvti3XBTvX1Ps51rZk61Z95MuvQ/rZDh06NKAYTPg09HMN20VeEckGNqpqqYiMwfMgx//U\n81CLv4S1mn0PphgfYiWWfq0HeJW3SU1izMHZ9W4vIvRN29fPV2IsHNXB+/JFcnw8Y7J7eJX7cmR2\nVoPWC0RqfCpjB/m+rNJcZSZmkJkYWJP/A9u25sC2rRu9z+S4BEZl9m3Utv07dKB/hw78odF7Ny1B\nOO8EvofngY9eeMbe/RCYCJwSxn0aY/ZzWXdN8ypb+9CpEYikZQjnPQm3evo4OQt4SlX/AHQK4/6M\nMcaEWDiTRLmIXICnL5LKXhLrbntnjDEmqoQzSVwOjAAeVNU1Tv/sr4Vxf8YYY0IsLPckRCQWT8uk\nqgbZqroGeDgc+zPGGBMeYTmTUE+Xtt1FJCEc9RtjjGka4WzdtBr4VkSmUHMgjMfCuE9jjDEhFM4k\n8YvzisHT/7oxxphmJmxJQlXvBxCRFFUtCtd+jDHRrXrHjc29M879UdhaN4nICGfUpOXO/GAReSZc\n+zPGRCdVnaCqw1R1WGZmSDqTNU0onE1gnwBOBHYAqOpC4Kg6tzDGGBNVwtoLrKpuqFXk8rmiMcaY\nqBTOG9cbRGQkoCISj2d0pmVh3J8xxpgQC+eZxHXAjUBnYBMwxJk3xhjTTITzTEKrP3FtmoG8PMgI\nrCvrFmF/PW5jGiCcZxKzReRdETlZRHwMmuyfiMSKyAIRmVr/2iZkbr450hFExv563MY0QDjPJPoA\nY4ErgKdE5B3gFVVd2YBtK+9fBDTayqoN23nitRzmL9tIh3atuGjccMYfP8Tv+q4KN6+89CVTpyyg\npLiMkaP7cOPNJ9CufatAdhtSO37N5+4bXiY3s4j8Pgn06bmFgd23ERdXSu+0QZzS4VLef38NH37+\nEyUl5Rx9eG9uufwY2qWnBryv0r2vU1zwNOrazLL57Rn55o88Om4s142/iFYJLbtHlQq3myfmfcvX\nMz/jwzff5C8nHMmN515Mh5R9n/2Hk3N5+83vydtewOAh3bn+puPrrrPcxf8encYnb3xPSXEpo04a\nzLX3nUXbzIb/M96at4cnXprJt/NWk5aayNknHcJl40cQE9Ow31nvLl3M07mz2bAnn6GdOnPP6DEM\n7ngALpeb11/4ko8m5VK0t5QjjurLDbedSEYH37HN+fkVkuUxuibvanDsJnSiaUyMsJ1JOCPkfaaq\nFwBX4+kyfK6IfCkiI/xtJyJdgFOBFwLZ397iMm76x3vkLt2AW5VfdxTw6Ksz+eSbpX63een5HN58\n/TsK9hRTXu7iy1nLuPfudwLZbcjdfP5/+OGAEnYdnEjPrK0Mzl5HbFwxipuVBT/y1LJ7eeOj2RQU\nllBe4eLzb5dz1z8/CHg/ZXvfp3jzH6FgA1Lk4pCvVgOQ+O4b3PvpZCgs9Lzc7lAfYmS53VBYyNNf\nfcZL33/JUd/NAaD1+x9ww/sTq477s08W8tQT09m2dQ9ut7Jg/lruvG0i6vY/8PMrD0/l3We+oDC/\niIoyF19Omc9fr3oxgNCU2x54j6/mrsLlcrN7TzEvvvMdr70/p0HbF5SVcufM6azfk48CuVs2cfGH\nk9hRXMTrL3zJGy9+zZ78Yioq3Hwzcxn3/P7NymGDa1i3/Ueyku+jW8ouArsGYFqicD5M115EbhGR\nXOAO4CYgA7gdzwh1/jwB3AkE9O2U88PP7Nrj/WD3+zN/8rm+qjL1o/le5SuWb2Hlii2B7Dpkls1f\ny7Y9JeT39pzgZXfa6rWOK6GQdr1rjgC7ZOUWVq7ZFtC+SgvfIOm/BbQ5aDPpfTaT/OgeAG76YCaP\nn3EBmp4O//oX+PgSadZU4dFH+d1xp7D0lnu5fcoMAG6fMoNJF11bddzTPvT+t7F7114KCkt8Vut2\nu/lk4nde5cvnr+WXJRsbFNrCZRtZu3GHV/n7039s0PY7iou9ygrKSpmycjlTJ8/zWrb6560sXeQd\n28otj5EYY63VjUc4Lzd9j2f8iDNVtfq/xFwR+a+vDURkHLBNVec542L7VP0x/8zMTHJycijbU8Rl\nx3X2WjcxIY6cnByf9Yw7s4vP78BfflnE5i0r/O0+5AoLC8nJyaGosIRzru5HaftYEEjf3Y6YfO8A\new9IpKJnzY/ulxUL2byu4WM6uStORE86GjnSTcz6Cijft6w8Nhbp2ZO41q3h668bfVyBqv65duzY\n0e/nFrQxY1jTuyddtu8k3rXvy7A8NpaKrO4kp7flkLV59D/Ee8xwt7vcb1wnXz8QXyl15dolbNi+\nqt6wCotKueSUrl7lMSIN+lukKdzWsYtXeeqmLZz4my6+zxo2LGX7zl9qlJUUH828rcOrldg9m/1Z\nOJNEX/X1rxJQVX/jSowCTheRU4AkoLWIvK6qF9XafgKecbPp27evjhkzhs3b8xl/20u4a+3y8jMO\nZ8yYUT53NnPGu3z3Tc1bJOnpKUycNJ6EhHD+aWrKyclhzJgxlBaXcfaoB1hzQipFXeI4JHsNfTr/\nWmNddcUw560hlO/dd8+gXXoKk58dT3x8bIP3WVKwgJICT4e8ifMKSH4gv2rZ8xefy9W33hrkUQWu\n+uc6bNgwHTNmTNj29fZnH7Jl8tv8afLHVWWPnXcmN/zuXZLi4njp+RzeeePbGtvExAh3/vlw/MWV\n8+rzzPl8cY2ytplpvDr7AuIb8O+pqLiMM6/9L3uLymqUn3RUf65uwN9i0scf89jGX7zKPzn/Et77\n5ku+/Lzmpde01klMnDqexKSaPy5yf/mVXsn31bs/s38IZ+umDBF5REQ+FpGZla+6NlDVu1W1i6pm\nAecDM2snCH8OzGzDnVccR2K1/4wjBmdx6emH+d3mpltPpGd2h6r51m2S+dNfzmzSBFFdYnICd/99\nPJ2+KyZhl5vFa7uydfe+G4sJMUmc0v4qurU/sKosvXUy9906LqAEAZDY6jriEscCEP9JMZokfH9q\nb0rj47lwxZrQHFAUu2/kcZy5eCXF8fG8ftQRlMTHc+UvG0iK83z2v714FMMP71m1fmJiHLfecTJx\ndfydb3xwPFn99g3j3rpdKn98+tIGJQiAlOQE/nLzqaSlJlaVHdTrAH536dEN2r59SgonZfeumk+I\njeWe0WPol5HJ9bedRK9qsaW1TuKuv/3GK0EADMs+nx/yjqZC7YaECe+ZxBvA28A4PA/WXQpsD+P+\nOPOYQRx7WB8Wr9pCx3ZpZHetu+17ZofWPPfSVSxbupniolIOHtSNhMTIJIhKo04axLSj+zF9ci6r\nyvYwuNPZZHeCUi2ge0pfkmJTOOpfypKVWyguLWfwQZ1JiA88ZpFkWrV/Gdfar5DSy1g87THodyix\nebtJvOgi2LwZDjyw/oqaqY678+mYmMyyzz6le3ZPJG83bS65pOq4k5Li+ccjF7Bm9Ta2b9vDQQM6\nk5aWXOdln8wD2/LMjD+yfP5aSorKGHBYdsD/nkYNy+aDCdexcNlGWrdK4qBenerfyCHAsyefzi+7\ndrA+P5/BHQ+gXXIKAO0yWvHM/65m2eKN7C0s5eAh3XwmiErHD3qdjTuW8svWKcDdAR2DaVnC+Y3Y\nXlVfFJFbVPVL4EsR+aGhG6tqDpAT6E5bpyYxcnCPBq8vIvQf4H0vI5ISkxM4/cKRfpeLCAP7huYL\nPFa6w9wlHJyc7Ck4sBvMmQPbw5rPI6+iAubM4aDK4+6Cz+Pu0bMDPXp28N7eDxHhoKEN//fnS1Ji\nPIcPaXwd2W3bk922vc9lBw30vmfhT5f2/enSvj+WJPZv4UwSlbdCt4jIqcBmoF0Y92cao3t377Lk\nZGjp/f7vr8dtTIDCmSQeEJE2eJq8PoXnwbjfh3F/xpgot2hTvteDYpF6SMw0TDhHpqvsUiMfOCZc\n+zHGGBM+IU8SIvIU+GwuDoCqWqNrY4xpJsJxJpEbhjqNMcZEQMiThKq+2pD1ROQpVb0p1Ps3xhgT\nOmEdvrQevh+DNsYYEzUimSSMMcZEOUsSxhhj/IpkHxTWMYwxxjQBX4MYNVTYzyREJMXPoifDvW9j\njDHBCeegQyNFZCmw3JkfLCLPVC5X1VfCtW9jjDGhEc4ziceBE4EdAKq6EDgqjPszxhgTYmG93KSq\nG2oV2ZiIxhjTjITzxvUGERkJqIjEA7cAy8K4P2OMMSEWzjOJ64Abgc7AJmCIM++XiCSJyFwRWSgi\nS0Tk/jDGFxYulztkdbnVjVuDr8/ldvsc37ihKtzeJ4CqGtJjjSauiqY94fX19w2F6p+RSxv+b6Dc\nZSf8Zp9w9gKbB1wY4GalwLGqWuicfXwjIp+o6uzQRxhaGzbu5N//+Yx589fSunUyZ585jIt+OwKR\nwIx0idsAACAASURBVFv6FlWU8PTPH/LF1vkAHNvxEH7X+0xS45ICqufXPQX89ZNZ5KxcTUpCAucP\nPZhbjx1FXEzDfhss3r2Jhxd/zMJdG+iY1Jqrex/NWQcewiv/+pRP351LSXE5I47rzw3/d0bAxxiN\nZk5ZwGtPzuDXDTvpNaAzV991KoMOzw7b/ubmreZfS6ezLH8LXVPacUPfYzm1y6Cg6y0vdzHhxRw+\n/vQnShJLST/Hxe52u0iNS+KMzqO4rOeJxIr3v4FZK1fz6Bdfs2r7TrIz2nHbcaMZ2zd8x2+ah7Al\nCRH5J/AAUAx8CgwCfq+qr/vbRj0/dQqd2Xjn1fifwE2kvPz/2zvvOKmq64F/z85WtsLu0quwgtio\nFjBKjC1WLET9aewSe4yJiSHGnxqNGjWaxJifNRg0UUTpKBIBQVB6W5q0BRYWdtned2fm/P54b3dn\ndsrW2YL3+/nMZ96775Zz587Meffc+85x8ehjH3E0uwiAwsJy3n1vOdHREUy6dmyT6/vTjul8lb2p\n9vzzrDWUOit4+tTbGl2HqnL3v2fxXfYxAIorK3lr5VocYWH84vyGPaLkV5bys2/eo9hZAcDRiiKe\n2TKXdfO3s/mfW2vzrfginezDBY2Wq6OyYeVuXnr0o9q77d1bD/HE3f/kjQWP0KNv68fKyizL5/5V\n71PpdgJwsCyPKRs+ITU6jjNSTmigdHDeeHsJn8xcByj8Tx553ayZQbGznPf3/xeHhHHbCRd7ldlx\nJIcHps/F6bZmHnuO5fHQ9Ll8fNeNLZLF0PkJpbnpIlUtwopxnQEMAR5tqJCIOERkI5ANLFLVVSGU\nsVVYvXZvrYLwZO78jU2uq7CqhOXZm33SV+Skk19V3Oh6NmRm1SoITz5c51u3PxYeTq9VEJ4srd7t\nk7YrPbPRcnVUPp++ysccU1lRzZez14ekvbkHN9YqiBoUZcb+dS2q1+VyM/8ze4z7VUOKr+lo7uFv\nfNJmbEyvVRC1dakyY8NWn7yG7xehXLiuqfsy4GNVLWyM6UVVXcAIEUkCZorIKaqa7plHRCYDkwFS\nU1ODBqdvC4qLK7hxkm84zPBwR6NkKykpqc3nVBfXl47wm2/titVEiKNRMpVUVvFQmm/sbhEaJVN4\nZQn3uIf5lu8D4Q/5Tu4+bwV/vp7j2qNHjzYd1wEjo7lmiK+pJzKlzEsOz7FqCd0qiv1+vvFHG/ed\n8UdJSQnLli3j6iv6AApRCqW+SiKsTHzaGFRZ7Pf7klRV4pNm+H4RSiUxT0R2YJmb7hWRVMD31jQA\nqlogIkuAS4D0etfeBN4EGDp0qE6YMKHVhG4OhUXlXH/T61RWet8ZXn3VKBoj29KlS73yTV7zCruK\nve/OB8f1ZvIZNzdapopqJ+e9+hYF5d4f+WUnD+W+Rsi0rziHiUtfQ+tZ+07N6EbeW1leaV1T4hot\nVzA8x3XMmDFtOq6ffbSKac9+6pP+5+n3cdKIuhuA+mPVXDbk7efWFe/4pD91ykQm9B/VrDprZFu0\neAbfrNoDEQr3H4No7zG8sOdo7ho+wStt8c49PP3RHJ86//6TK5oli+H4IWTmJlV9DBgHjFHVaqAU\nCLrCKSKp9gwCEYkBLsR+Yrsjk5gQw5TfXE5sbFRt2sgRA7jjtuY9Ozhl+I30jkmuPe8dnczvhjdt\nD0B0RDivXnsZ3brE1Kad1rsnUy4+r1HlB8WnMuXUy4gOi6hNu6jXybx00y0MHt67Nq1rShy//UtT\n9yd0PC6eNJaLrxtbu9EgPMLBbY9c4qUgWpOR3Qbw4LAfERFmzQzDEK7pP5qr+vmfRTaFXzx0EUMG\nd4dqgdkJSEXdz/zkhAHcN+RKnzLnDx3MXePG1G5qcIhw+1mjuGDYkBbLY+jchNrB3zBgoIh4tvOv\nIPl7Ae+JiANLgU33iJXdoTn3nKGMHT2IbdsP07VrLCcMSm12XQNjezLtrMfYWrgfUE5OHEiYn90o\nDXH2Cf356uG7WHfwMAnRUZzcq0eTyl8/8Ax+3PtUthYeondMVwbEWYrrtVk/57stmZSXVjJ81AAi\nItvTT2TrEBYWxsN/vI4b7j2fQ/uPMXh4b5K6tc4MKRB3p53HNf1Hs7PwCAPikunTpWur1JuamsBb\n/7idHTuzqKio5sSTurOj5CDxETGkxfcNWO7RC37ALWeOZFd2LmmpyfRICG3/DZ2DUO5umgYMBjZS\n96S1EkRJqOpmYGSoZAo1MTGRjB41sFXqCpMwTk0a1OJ6IsPDOXtQ/2aXT4iM4exU37vJE08N/GfT\nmenZrxs9+7X+bqZAJEfFMa57aO7Whw3tVXs8qltao8r0iI+jR7xRDoY6QnkLOAYYri15istgMBgM\n7Uoot8CmAz1DWL/BYDAYQkwoZxIpwDYRWY31JDUAquq7amYwGI5bPLc2OxKav1bXkagfxCfj+cva\nSZLQE0ol8WQI6zYYDJ0Ez63NUb3SjPm5kxFK301ficgAIE1V/2tHqGvck2AGg8Fg6BCEMjLd3cAM\n4A07qQ8wK1TtGQwGg6H1CeXC9f3AeKAIQFV3Ad1D2J7BYDAYWplQKolKVa2qObEfqDP2SIPBYOhE\nhFJJfCUiU4AYEbkQ+BiYG8L2DAaDwdDKhFJJPAbkAFuAnwELgMdD2J7BYDAYWplQ7m5yA2/ZL4PB\nYDB0QlpdSYjIFoKsPahqy+MzGgwGg6FNCMVM4nL7/X77fZr9fjNm4dpgMBg6Fa2uJFR1P4CIXKiq\nnh5dfyMi67HWKgwGg8HQCQjlwrWIyHiPk3Ehbs9gMBgMrUwo/7TvBF4XkQwRyQBeB+4IVkBE+onI\nEhHZJiJbReTnIZTPC1WlqqI6aB63Njr6ql+c1S5cLnfQPJVVTr/pTrebardvvOL6VFVUNZinstq3\nDdXKgNfq41al0tVwvpbhbZmsrnbhdvtaK91upaoBmStaWdaqKidud/BxbC6qFVRXOf1+T6qdLlzN\naLfS5SSQx363201VZeDvfUu/84bOTyh3N60DTheRRPu80PO6iNyqqu/VK+YEfqmq60UkHlgnIotU\ndVugdtxawdacKQxNnkJ4WPOCpSx8fwUfvDSPnEP5DD61H5P/cB2njR9aez2vdA5ZhS9S6dxPdEQa\nfZJ+R2LM+Y2uvyCvlL//cS4rF28jPNzBj64YweRf/ZjomMjaPIu/3sFb05ZzKKuAAf2SuefWcxl/\nxhBKq6v4w/pFzM7YilvdXNxvGE+NuYiuUV282vjqk1VMfWoGh/ccpf+w3tz5h+s561Lv+E2rtu7n\nLx99xa7MY/ROSWDyVeO4dPQBtPjP4MrgYH4yrywcy568Edx/xXguHjvUq7yq8trmb3h3+1ryK8s5\ns0c/nj7zQoZ2bX3PnkWV29hw5B6SXD/ntTfTWbsxg9guUVx96Uju+J/xiAhvzl7Jx4s3UlxWydiT\n+vPoTeczsFddwKBlh/bxx3VL2JGfQ9+4RB4ZcQ7XDD6l2TId2JvNof3HuPKhJ4lPjOGq/zmbm+75\nYW3I05agFQtx5v8Jhxwka18XPvi/YaQMuI7bf381uSVlPPefxXydvo/oyHCuGncKD1/zAyLCg7tC\n23DsEE+vW8Sm3MN0j4nj3uFnc+vQsVZ7qnzw/Gxmvb6Q4rxSTj/3JB545Rb6D+sDQHHFt2TmP015\ndXqwJgzfA0Ju/lHVwvoKwsZnlqCqWaq63j4uBrZj+XwK1gKHS2ay7djvmyXf6kVbePUX08g5lA/A\nni0HeeLG18g5lAdASeUaMnIfpNK5H4CK6l3szbmb8updjW7j2V/9h+VfpONyuqmsqGbBx2t4/fm6\nqKwVFdU8/dI8DmUVALD/YC6PPzebvftzeHzNZ0zfs4lKl5Nqt5t5+7fx8IrZXvVvX72b5297ncN7\njgJwYMdhnr7xr2RszazNk5ldwCN/mcWuzGMAHD5WxPSF7+HOfxhcGQD07ZrLC5O+ICZsD1P+uYCN\new55tTN1xzpe3ric/MpyAFYdPchPF02nwhl8BtZcjpV/xerMyazZsA9VKCmtZNrH3/L+x9/yr8/W\n8M7cVRSVVqIKq7cd4KE/f4rTac229hXlcdfiT9iRn2P1v6SQX349n5VZ+5slS1WVkyk/m0pZqTVT\nKy4s5/1/LObTaStb3E+t3oIWPIxDDgLQd1AZv3p2AxsXz+Rfz83h4ddns2zLXtyqlFVW858lG/jb\nrK+D1plXUcatSz5kU+5hALLLS3hq3SLm7bfutz756+dMe+ZTivNKAdi0bDu/vfJFqiqrqXJmsSfn\nVqMgDED7rhEEvf0SkYFYoUxXNaay7NJFVLsKmizE59N8f2yV5dUs/ng1AMdKPqS+6UOpJq/k40bV\nf2j/MbaszfBJXzJvExXl1h9OQVG5jynF5XIz878bmb9/u0/Z5Uf2kVlS19fPp37lW97pYuG0ZbXn\n81duo8rpba66YvRORLzNF+EON1eM2IEqzFzh/Sfxn+82+ciSXV7Cl5l7fNJbi27dj9FnYI5X2pwv\nNjNr2RafvFm5RXy71VICn+5Jp6qeeU6Bj3ZtbpYca7/+jmNHi3zSP5uxpln1eaJlM6iL8GvhCFcu\nvDqLmbNXsTMzx6fMzBXpAU1IAPMObKOkutIn/T+7NwDw2dSlPteOHcpj7RebySubiVvLm9YJw3GL\ntFd0URFZr6qjAlyLA74CnlXVT/1crw1ikpqaMvqd958EIC5yKNJEC9qhvdmUFfvaXbv1SCS5ZyJV\nzgM43b4ToQhHMhGO3g3WX1lRzYG9vj9ygMHDehEWJuTlF5KT6ytDQmI0uQ7/P9a0xBSiHFZfj2Tk\nUJxf6pMnMSWe7v2SAcjOLyG3yDtP767FJHbxbbegLIasgngSukTRJyWxNn1nQQ5VLt91kb5xiVxz\nyaXrVHWMX2Ebide4dk8a/c60ZwDIzU6gsqLONOdwCBouOP3Y5/umJhHfJYqssmKOlft+JgmR0QyI\nT2qybMWF5Rw5lE/X1Gjyc+o+s/AIB4PSejS5Pi9ch0DzfZKL8iM4mhOLMy7S55oAw/p7+8ssKSkh\nLs4yuR6rKOVIWbFPuZjwCAYnJLMv/SDOat+x7DkwleiEEqpd2bVpl138UIvHtoaoXmna69ZXvdJa\nO2BP/YBAbdFGR++Dv/r2v3B5o8Y1lEGHGsLvTEJEIoBPgA/8KQjwDmKSdmJ/jU57jcSo0zmj9/3+\nsgdlwf5lvP/Mv33S//bfKQw5vT95pXPIyPX1JpLW/UPio8c1WL+qcsflr5B1MM8rfeRZg7n7vh8C\nMG/+Qj6Y851P2ZefmsTs3OWszj7olT4kIZkvfnRD7fnyWWt45v6/+ZT/45xfM3qCZYPfvPswd/7x\nQ6/r5560jxd/utCn3IPvX8Y3e/rz3J2XMmFM3brEijWLeWub951zZJiDby5rnWCDnuN64qkxGp32\nGmUl0bz3j+txuers75dfdBqaGM6Mr7xnNl2iIpj/8kTiukSx5mgmv/38A582/nzOZUxoxrpEcWEZ\nN1/4IpfflsYn/1e3RDbxprOZMGFCk+vzRCv+ixY85ZP+v/edRkzqJSxNKuVYUZnXtQtGpXFPvXaX\nLl1aK8u+ojwunP8G7no3gY+NOJ8Jw89i+/z3mfX6Yq9rUTGRvP/dq4THHWDHkUta1CfD8UN7mptW\n1E8QawXwHWC7qv65sRXFRgzm5NTnmiXExTefw8U3jScszNJZUTERTP7DJIac3h+AbrFX0j3+Tmr0\nqRBFr8RHGqUgAESEKS/eQPfedXevg07sycNPTqw9j4uN4vqJY3A4rOGICHdw2w3jGDtyIH8663JO\nTEypzds/Lom/jL/aq40fTBzLtQ/9GIe9kBkRGc7NUyYy+kd1f4anDenNg5N+QFSElccRJqT2nAhd\n7qrtW5UzjLeXjWbNvoHc+MORXDzGe+H64RHn8MM+J9SeJ0RG8ZdzryA52nsRvbWIdKSQ4n6KuNi6\nDQkjT+3Pvbedx/3XnsOZJw+oTU+Mi+aZn11GXJcoAMb26MuvR51XO9tyiHDL0FFcfcLJzZIlPrEL\nj73wk9oxAhg9Lo1bHrigWfV5ItEXQOzduNWStbpa+OjNAZRVnMF9z17PC3dfTkpibG3+Uwb25Nc/\n+WHQOgcldOPZsT8mNtyahQhwxYDh3D7MWri+9YlrGXNhnfOD+G6xPPbPe0noFkeXyOH07fokYRLT\n4r4ZOj8hMzeJSBJwCzAQjxmLqj4UpMw5wHIsp4A1toQpqrogUJmhQ9N0x47vWrzDJDszj6yMHE44\npS/xSbE+16ucR6h07iU6YigRjuQm1+9yudm5JZOISAdpw73X4mvuAI/llXAgM49B/ZPpWk+GzblZ\nuNTN6cm9CQvQ19ysAg5+d5iBJ/UlqXuC3zwFJeXsPphDvx5d6dEtHgB15YBzD4WVfdmVpQzs0ZXU\npMA7xfYU5pJTXsqIlF5Eh0cAICKtZpIAGDl6uK5bu4kwiaCq2sm2nVkkJsQwqH+KV759h3PJLy7n\n5BN6EhXhOzHOryhne342JyR0o2dsfIvlWrJkCcnxA+maHEe/Qa27q0tdObgqd7FrSzjRcb0ZdHLf\n2mvVLhfp+44QGx3JiX39t+s5k6ihuLqS9Lws+sYm0S/O18x2YOdhCrKLGDrmBKJivM1aTnch5VVb\nSYgZb8xNDbTR0fvQUc1NC4Bv8f7DD4qqfk0DC9q+hLXKFsTufbvRvW+3gNcjw3sSGd6z2fU7HGEM\nH9E/aJ6UbnGkdPP/53xacq8G20julURyr+D29qS4GMac5C2HOFLBkUpSFIz1r1u8GJyYzODEpivK\npuCQLoSJpYAiI8IZcUo/v/kG9U5mUJB6ukbHMK7XgCA5moaIcNqYYC22oG5HKuFdUjnpTN9rEQ4H\nI4c0sNHPD/ERUZzdY2DA6/2H9qb/UP9ra+FhiY2eMRuOX0KpJKJV9ZEQ1m8wGAyGEBPKNYlpInK3\niPQSkW41rxC2ZzAYDIZWJpQziSrgReB31D1ooMAJAUsYDAaDoUMRSiXxS2CIqh4LYRsGg8FgCCGh\nNDftBsoazGUwGAyGDksoZxKlwEYRWQLU+gcItgXWYDAYDB2LUCqJWfbLYDAYDJ2UULoKr+8G3GAw\nGAydjJApCRHZh5+Y1qpqdjcZDAZDJyGU5ibPx72jgUmAeU7CYDAYOhEh292kqrker0Oq+irQug5O\nDAaDwRBSQmlu8owVEYY1s2hP1+QGg8FgaCKh/NN+mbo1CSeQgWVyMhgMBkMnIZRK4sfAtXi7Cr8B\neDqEbRoMBoOhFQn1cxIFwHrAN0amwWAwGDo8oVQSfVW1yTEQReRd4HIgW1WbHmfSD6XOKqId4Tik\n6ev0ZcXlRMdGERbWemv8qk5UKwkL8w1u5D+/4tQyIhrIX+2uIEzCcUjwYVVViisrCA8XuoRH16a7\ntBq3uokIi2qUXKHErYrL5aKqvIqYuNaNkFbpKiMyLBrx831waRWo4mjGZ+DWClzuKsIkGkeYb1zq\ntqCkupIu4ZEBA1M1lTKnub/7vhNKJbFSRE5V1S1NLDcVeA34V0sFSM/P4skNC9icf5ikyBhuTzuT\ne4f9oFFl1y3axP898h4ZWw+S3LsrP31iEpdNvrBF8qgq+UUvU1TyNm4tJCpyLClJwcOu7i9eQHre\n65Q5s4iLGMDpyT+nd+x5XnmKqrJYfvQlMsvWES7RDEu6jLNS7/WrLOZv2sFr22YgPXJwRLjpFd6L\nR0+5gd3Fc9hWuBi3uhgcfyYX9XqAuPD227G8/XA2Z93xLHFzvuOUYf158LU7OXH04BbVebA0nS+P\n/oPsir3EOrpyZspPGJNshZGtdpewPucFDpZ8ASh9Yn/IqNTfEuUIHsQJwOXO52jeI5RXLERVOepK\npDryBsb1/DWRYaEJ7Vqfldn72F2cw+Q5f6J7dBz3n3QuN54wutn1bSrYydt7PiWj7HArSmnojITS\nwd85wDoR2Skim0Vki4hsbqiQqi4D8lraeEl1JXd8/QGb860veUFVOa9sXcqHe9c1WDZr31GeuOoF\nMrYeBCD3cD6v3vMmq+Y3XDYYRSVvUVD8Mm4tBKCyag1Hjt1IoMB9OeXrWZ39BGXOLLtP+1l55FEK\nq/bU5lF189mhX5NZthZQnFpOev4M1hx726e+jQcO8+yaDwnvexRHhNVmljOLt3c/zpaChbi0GsXN\n7uJvmH3w2Rb1tTUo7RdP7k9OYvuqXfz2kmcpKy5vdl0lzjxmHPg92RV7rbpd+Sw++gbbC78CYG32\n0xwoWYDiRHGRWfpfVh19vFF1Z+c9QEXF5whKmECv8EKiqj5g+ZGXmy1vUzhcVsg9Kz+kwuW05Kko\n4X83LODLwzubVV9OZT5Pb33DKAgDEFol8WMgDbgIuALLhHRFCNvz4ovDOyio8v1TmZ6xocGyiz/4\nmqqKap/0z95d3CKZikr/7ZPmcufgdhf7zb+veDb1H1pXXOwvnld7fqR8CwVVB3zK7ijwjWk7c/1W\n4vsUeaUJbrpG5vvkPVS+jdzKg37lakucqV2o6hNPUW4xK2atbnY9OwqXUa2+ppMtBQupchWRWbrE\n59rR8m8ocx4JLp/rCOUVvt+L7o4SDhQvospV2myZG8vcg+m1CsKTGRkbm1XfV9lrqXL7fv8N309E\n1cdzRrsjIgOBeYHWJERkMjDZPj0FSA9QVSJQGCSt5tjfew31y4eCFCBY3I36/WhKHwr9lA8FQ1U1\nviUV1BvXoUDzboVDS0Nj1Z6ESrYBqpra3MLtMK4deYwaS1v0oXHjqqod7oW1bTa9kXnXBrn2ZrC0\nmmN/7zWvNupvwD7460dT+hDoc2jrPhwvr47cz44sm/kcOm8fjvcnoOc2kDa3gfeOQn15mtqHjtYf\ng8HQSehw5iYR+Q8wAWu6dRT4X1V9J0j+tao6JtD1zoDpQ+ehI/ezI8vWlhwPn0NH6kOHm0mo6o1N\nLPJmSARpW0wfOg8duZ8dWba25Hj4HDpMHzrcTMJgMBgMHYdQboE1GAwGQyfHKAmDwUZE+onIEhHZ\nJiJbReTndno3EVkkIrvs9652uojIX0Vkt/3A6KjgLbSKjA4R2SAi8+zzQSKyypbhIxGJtNOj7PPd\n9vWBoZatPQk0dp2R+mPc3hglYTDU4QR+qarDgbOA+0VkOPAY8KWqpgFf2udQ98BoGtZzAP9oAxl/\nDmz3OH8BeEVVhwD5wJ12+p1Avp3+ip3veCbQ2HVG6o9xu3JcKwkRCRORZ0XkbyJya3vL01xEJFZE\n1orI5e0tS3MQkYki8pZ9Z3tRe8sTCFXNUtX19nEx1g+1D3AV8J6d7T1gon18FfAvtfgWSBKRXqGS\nT0T6YkV3fNs+F+B8YEYA2WpkngH8yM5/XBJk7DoV9ce4I9BhlYSIvCsi2SKSXi/9Etsf1G4ReSxQ\neZurgL5ANZAZKlkD0Up9APgNMD00UganNfqgqrNU9W7gHuD6UMrbWtjmmZHAKqCHqmbZl44APezj\nPoCn75JMQvvH9Crwa+qcfSUDBapa45PDs/1a2ezrhXb+4556Y9fZqD/G7U6H2wLrwVTqeYMVEQfw\nd+BCrB/EGhGZAziA+u5U78ByAbBSVd8QkRlYpoK2ZCot78PpwDYgmvZhKi3sg6pm28eP2+U6NCIS\nB3wCPKyqRZ434KqqItLmWwLtWWS2qq4TkQlt3X5nof7Ytbc8TaGjjnGHVRKquszPYtsZwG5V3Qsg\nIh8CV6nqc1gOBL0QkUygyj51hU5a/7RSHyYAscBwoFxEFqhqm91ltFIfBHge+KzGJNBREZEIrD+Z\nD1T1Uzv5qIj0UtUs25xUo/QOAf08ive100LBeOBKEbkU64YhAfgLlokr3J4teLZfI1umiIRj+e/K\nDZFsHYIAY9eZ8BljEXlfVW9uT6E6rLkpAE2d3n8KXCwifwOWhVKwJtCkPqjq71T1YeDfwFttqSCC\n0NRxeBC4ALhORO4JpWAtwVZm7wDbVfXPHpfmADVrWrcCsz3Sb7F3OZ0FFHqYpVoVVf2tqvZV1YFY\nYYAXq+pNwBLgugCy1ch8nZ3/uH0oKsjYdRoCjHG7KgjowDOJ1kBVy6jb7dGpUdWp7S1Dc1HVvwJ/\nbW85GsF44KfAFhGp8bM9BWsWNF1E7gT2Az+xry0ALgV2A2XA7W0rLmCtV30oIs8AG7D+KLHfp4nI\nbqz4LDe0g2xtid+xU9UF7SjTcUFnUxJtOb0PFaYPHRRV/RoItAPoR37yK3B/SIXyg6ouBZbax3ux\nzH/181QAk9pUsHakgbHrdHiOcXvT2cxNa4A0+wGiSKy7ozntLFNTMX0wGAydhg6rJMTyBvsNMFRE\nMkXkTntx7gFgIdY+6OmqurU95QyG6YPBYOjsGAd/BoPBYAhIh51JGAwGg6H9MUrCYDAYDAExSsJg\nMBgMATFKwmAIgogMrO+3qjMhIk+KyK/aWw5D58UoCYPBYDAExCgJQ6fAvqPfbrsc3yoiX4hIjIgs\nFZExdp4UEcmwj28TkVliBQnKEJEHROQRsYK5fCsi3YK0NVpENonIJjwelhMrGMyLIrJGrCBDP7PT\nJ9hyzBCRHSLyge0mAhF5XqxAOJtF5CU7LVVEPrHrWSMi44PI8qTtiXepiOwVkYc8rj0iIun262GP\n9N+JyHci8jWWk8ua9MEi8rmIrBOR5SIyzE6fZNexSUQ6ivsaQ0dBVc3LvDr8CxiIFVhmhH0+HbgZ\n66nUMXZaCpBhH9+G5S4jHkjFcpV9j33tFSwvoYHa2gycax+/CKTbx5OBx+3jKGAtMAiYYNffF+vG\n6xvgHCzX3Dup22qeZL//GzjHPu6P5W8okCxPAivt9lKwnPRFAKOBLVjOH+OArVjusWvSu2A5AdwN\n/Mqu60sgzT4+E8s3EHb+Pp4ympd51bw6m1uOkCAiJaoaF+I2rgSGq+rzoWwnQNsTge9UdVtbt93K\n7FPVGr8867AURzCWqBWAplhECoG5dvoW4DR/BUQkCeuPsuaOehpWBDqAi4DTRKTGoV4iVlS6JlSu\npwAABG9JREFUKmC1qmbadWy0ZfsWqADeESsUZU04yguA4VLngjxBROJUtSRAP+araiVQKSLZWPEs\nzgFmqmqp3eanwA+wlNRMtfyWIZYL9xoX2uOAjz3ajbLfVwBTRWQ6llNMg6EWoyRaERFxqKpfl+Sq\nOocQuq4I1jZWtLJ5WHEpOjOVHscuIAZrdlFjNq0fc8Mzv9vj3E3zvvsCPKiqC70SLXfu9WULV1Wn\niJyB5ffpOqyn1M+35T1LLf9KjcGn7mbIHoYVoGhE/Quqeo+InIkVEW2diIxW1eParbih8Zg1iXqI\nyKMeNuenPNJn2bbcrSIy2SO9RERetu3XZ9v276dEZL2IbPGw+94mIq/Zx1NF5K8istK2M19np4eJ\nyOu2XXuRiCzwuGv1J2uGiLwgIuuBSSJyty37Jtvm3UVExgFXAi+KyEbbLu3XNt1JycAysUCdy+xm\no6oFQIGInGMn3eRxeSFwr1hxCxCRE0UkNlBd9t17olqeSH+BFUAK4Ass9+k1+Xz+uBvBcmCiPcax\nwNV22jI7PUZE4oEr7H4VAftEZJLdpojI6fbxYFVdpapPADl4O280fM8xMwkPxIq/nIblVVOAOSJy\nrm16uENV80QkBisS2yf23VYssEpVf2nXAXBMVUeJyH3Ar4C7/DTXC8tkMAxrhjEDuAbLTDEc6I7l\nF+ndBsTOVdVRdtvJqvqWffwMcKeq/s02OcxT1Rn2tS+x7PO77DvI17HucDsjL2G58Z4MzG+lOm8H\n3hUrAt0XHulvY43PerEGOoe6mNL+iAdmi0g01vfpETv9IeDvIrIZ6ze4DCu0a6NR1fUiMhVYXSOb\nqm4AEJGPgE1YwZHWeBS7CfiHiDyOta7xoZ3vRRFJs2X80k4zGADjuwmoW5MQa/fJdUCBfSkOeE5V\n3xGRJ7Hu1sD6o7hYVb8VEScQVWPqEWt3zXhVPWT/AT+rqheIyG1YC6wP2D/uRar6gV2mWFXjReRV\nYJOq/tNO/xT4d82fux+5M4DzVHW/fX4e8AyQZMu+0DYlTMVWEvbdbQ7WgmoNUap6UvM/QYPBcLxi\nZhLeCJZSeMMr0bI5XwCcraplIrKUOvt3hZ+1gBobcjD7saeduSV+8Es9jqcCE1V1k62UJvjJH9A2\nbTAYDPUxaxLeLATusO+2EZE+ItIdaxdLvq0ghgFnhaj9FcC19tpED/z/yQcjHsiybeaetvRi+1pQ\n2/T3DRH5u71O4/lqj+hyiMjtfmT5e3vIYjB4YmYSHqjqFyJyEvCNvbZQgrUX/3PgHhHZjmWm+TZE\nInyCtRNmG1YM6fVY++8by++BVVjmpFXYigHL9vyWWA9iXUdg2/T3ClVt86hygbBNjP9sbzkMhvqY\nNYkOhtj75UUkGWtRcryqHmlvuQwGw/cTM5PoeMwT64GuSOAPRkEYDIb2xMwkOgEiMhPL/YMnv6n/\nUJfBYDC0NkZJGAwGgyEgZneTwWAwGAJilITBYDAYAmKUhMFgMBgCYpSEwWAwGAJilITBYDAYAvL/\nRTV5kc06i7IAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc28784fa58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plot_evaluations(result=search_result, dimension_names=dim_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 在测试集上评估最优的模型\n",
    "\n",
    "你现在可以将最优模型用在测试集上。用Keras重载模型非常方便。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = load_model(path_best_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "之后我们在测试集上评价模型的性能。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 8960/10000 [=========================>....] - ETA: 0s"
     ]
    }
   ],
   "source": [
    "result = model.evaluate(x=data.test.images,\n",
    "                        y=data.test.labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以输出测试集的所有性能指标。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.0363312054525\n",
      "acc 0.9888\n"
     ]
    }
   ],
   "source": [
    "for name, value in zip(model.metrics_names, result):\n",
    "    print(name, value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者我们可以打印分类准确率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "acc: 98.88%\n"
     ]
    }
   ],
   "source": [
    "print(\"{0}: {1:.2%}\".format(model.metrics_names[1], result[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 预测新的数据\n",
    "\n",
    "我们也可以预测新图像的分类。我们从测试集取一些图像，你可以加载自己的图形到numpy数组来代替。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "images = data.test.images[0:9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面是这些图像正确类别数。只在画图的时候用到。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cls_true = data.test.cls[0:9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "获得预测结果的独热编码数组。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_pred = model.predict(x=images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将结果转换成整形。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cls_pred = np.argmax(y_pred,axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe8VNW5//HPg2IQVBCxU86NWECjmGCvV5EoFpQYxViI\nsUQxlngDSYwSTJAYbPwsV1FfoglWEBGJimJXQISASrGAokEvIqJGiWJ7fn/MXjN7Tp19ph++79eL\n19kzs8uj68w6z1p77bXM3RERkdy0KncAIiLVRJWmiEgCqjRFRBJQpSkikoAqTRGRBFRpiogkoEpT\nRCQBVZoiIgmo0hQRSWDdfA7u1KmT19TUFCiU6jBnzpyV7r5pueMoFZVxy6cyTiavSrOmpobZs2fn\nc4qqY2bvlDuGUlIZt3wq42TUPBcRSUCVpohIAqo0RUQSUKUpIpKAKk0RkQTyunsu0lxXXnklAF98\n8QUAr7zyCgATJkyos+/ZZ58NwF577QXAySefXIoQReqlTFNEJAFlmlJSxx9/PADjx4+v93Mzq/Pe\nTTfdBMC0adMAOOCAAwDo2rVrMUKUMnrjjTcA2H777QG49tprATj33HPLFlNtyjRFRBJQpilFF7JL\naDjD3GGHHQA49NBDAXjrrbfSn02ePBmAxYsXAzBu3DgALrroosIHK2U1d+5cAFq1SuVzW2+9dTnD\nqZcyTRGRBJRpStGE55kfeOCBOp/ttNNOQCaL7NSpEwAbbLABAF999VV63z322AOAl19+GYCPPvqo\nSBFLuc2bNw/I/B4MGDCgnOHUS5mmiEgCJc80wzi8W265BYCtttoq/VmbNm0AOPHEEwHYYostAOje\nvXspQ5QC+b//+z8A3D39Xsgwp06dCsCWW25Z77FhHCfAokWLsj474ogjChqnlN+rr74KwHXXXQfA\nKaecUs5wGqVMU0QkgZJnmkOGDAFg6dKlDe4TxuVttNFGAPTs2bMg1+7SpQsAQ4cOBaB3794FOa/U\n78gjjwQyd70BNtxwQwA6duzY6LH33ntvejvevykt0+uvvw7A6tWrgewRF5VGmaaISAKqNEVEEih5\n8/zWW28FMsNH4k3vhQsXApkBrk8//TQAM2fOBDKPzb377rsNnr9169ZAZghLuBkRP09opqt5Xhrd\nunXLed8rrrgCyDxOFxeGHoWf0nKMGjUKSC29AZX93VSmKSKSQMkzzYMPPjjrZ1x4hC74+OOPgUzm\nGf76vPTSSw2e/3vf+x6QeeA/PJ4HsGrVKgC22WabZsUuxTNlyhQAhg0bBsCaNWvSn22++eYAXH75\n5QC0bdu2xNFJMcRvBofvdPjetmvXrhwh5USZpohIAhX9GOXGG28MwEEHHZT1fn1Zam33338/kMlW\nAXbeeWcABg4cWKgQpUDCI5fxDDMIw0/ClHDSMjzzzDN13tt008pfbl6ZpohIAhWdaTbHihUrABg8\neDCQ/Qhf6C9ramC1lM7RRx8NZB6rDAYNGpTeHjFiREljktIIS5zEhQdPKpkyTRGRBFpcpnnDDTcA\nmYyzQ4cO6c/CnTkpvzB+dvr06UCmLzP0aV188cXpfcM0YdIyzJgxA4CxY8em39t1110BOOSQQ8oS\nUxLKNEVEEmgxmebzzz8PZMbyBQ8++GB6O0xLJuUXJpdduXJl1vthWkCNpW25nnjiCSB7ZEsYox2m\nh6xkyjRFRBJQpSkikkCLaZ4//PDDQGbuxT59+gCw1157lS0mqSusCRQejQ0OPPBAAP70pz+VOiQp\nsTBZT9xPf/rTMkTSPMo0RUQSqPpM84svvgDg0UcfBTITdlx66aVAZqo4KZ/46pEjR44E6s7G3qtX\nL0DDi1qy5cuXA/Dcc88B2ZPpHHPMMWWJqTmUaYqIJFD1mWaYtDb0kR122GEA7L333mWLSbJdddVV\n6e1Zs2ZlfRYeo1RfZst3++23A/DBBx8Ame9qtVGmKSKSQFVmmmHCWoA///nPALRv3x6ASy65pCwx\nScOuvvrqBj8Lj72qL7Ple+edd7Jeh6kfq40yTRGRBKoq0wx3Yc8777z0e9988w0A/fr1AzQus9qE\nMs1llENoTYR9v/76awA+/fTTOvuGR/Suueaaes+1zjrrpLf/+te/AlpGo9geeuihrNdHHHFEmSLJ\njzJNEZEEVGmKiCRQFc3zb7/9FsjMhPL222+nP+vevTuQuSEk1SWs25SL4447DoAtt9wSyAxdueee\ne/KKIax2GZ/DUwonDGYP5VXtlGmKiCRQFZnmkiVLgMyKhXFhOIvmX6xc4SYdwKRJk5p9nvvuu6/J\nfcJNolatsvOBo446CoDevXvXOWbfffdtdkzStAceeADI3LQNs7RX6+qiyjRFRBKo6EwzDIbt27dv\n1vtXXnllertahy2sTSZOnJjeHjVqFFB3wo5g4cKFQOP9lKeddhoA3bp1q/PZT37yEwB69OjRvGCl\nYP7zn/8A8Mgjj2S9H6aBiw/7qibKNEVEEqjoTHPMmDFA3cev4n0hZlbSmCQ/ua5rfddddxU5Eim2\n0L8cVoTt378/AOeff37ZYioEZZoiIglUZKYZxnVdf/31ZY5ERJorZJphnfOWQpmmiEgCFZlphjXM\nP/vss6z3w9M/mkZMRMpFmaaISAKqNEVEEqjI5nltYaXCJ554AoCOHTuWMxwRWYsp0xQRSaAiM83f\n//73WT9FRCqFMk0RkQTM3Zt/sNmHwDtN7tiydHP3TcsdRKmojFs+lXEyeVWaIiJrGzXPRUQSUKUp\nIpJAo5WmmW1iZvOif8vN7L3Y6/WKEZCZ9YxdY56ZfWZmv2rimNPN7MNo/0Vm9os8YxhnZkc3sc/v\nYjEuMLNvzKx9PtcthzKVcTcze9rMFkb/7xot3+iYcpTxKWb2qpm9YmYvmNkP8rlmuZSjjKPr3hHK\nLMf9y1HGO5rZDDNbY2YX5HRid8/pHzAc+E097xvQKtfzJPkHtAZWAJ2b2O90YHS0vQWwEuhUa591\nE1x3HHB0gv2PAR4rxv+DUv4rVRkDWwG9ou2NgCXAdpVWxsA+QIdo+0jghXKXUbWUcXTOA4DdgXk5\n7l+OMt4c6A1cDlyQy3mb1Tw3s+5RlnAnsADoYmafxD4faGa3Rtubm9lEM5ttZrPMbM8ElzoEWOTu\ny3I9wN2XA0uBrmY2wsz+ZmYvALeb2bpmdnUUxytmdnoUYysz+18ze83MHgc6JYgR4ATg7oTHVLRi\nlrG7v+/u86LtfwOvAVvnGlupytjdX3D38N88E+ica4zVoNjfY3d/BljVnNhKWMYfuPts4JtcY8tn\ncPsOwCnuPtvMGjvPtcAod59pZjXAFGAnM9sDONXdz2rk2IEkrIzMrDvQDXgrFuf+7v6lmQ0GVrj7\n7mb2PWCmmT0G7An8F9CTVBa0ELgpOt9lpDKMhxu43gZAH+CMJHFWiaKXsZl9H9gJeCnXoEpdxpHT\ngEca+bxaleJ7nFiZyjgn+VSaS6Iauil9gO0tsyzFxma2vru/CLzY0EFm1gY4HLgwx3hONLMDgTXA\n6e7+SXTNB939y2ifvkAPMxsYvW4PbAvsD9zt7t8By8zs6XBSd/9DE9ftDzzj7p/mGGc1KXYZbwTc\nD5zr7p/ncJ2ylLGZ9QFOBlriWr9FLeNmKNf3OGf5VJqrY9vfkeoTCdrEtg3Y3d3rX36wYYcDL7r7\nyhz3v9Pd6+vIjcdpwGB3fyK+g5kdkzC2uIHA3/M4vpIVrYwtdQNiIjDW3SfneFjJy9jMegFjgB+7\n+8fNOUeFK/b3OKlyfY9zVpAhR1HN/rGZbWtmrUjdGAmmAeeEF9EvYS7q9BOa2flmlk8zYCowODRD\nzGx7M1sfeBY4PuoT2ZpUB3aTzGxjYG/goTxiqgqFLGNLpQ63k7pBcG2tzyqmjKNm6ATgZ+6+OI+Y\nqkKRvsd1VFIZN0chx2n+ltR/zHQgfuPmHGCfqMN2IVHfn5ntYWY31XciM9sQ+G9gUq2PegAf5RHj\nGOBNYJ6ZzQduJJVtTwDeJdUHMhZIL2piZpeZWb8GzvcT4BF3/yKPmKpJocr4AFJ/FA+xzNCXH0ef\nVVIZDwc6AmOiGAvZDK1UhfwejweeA3qa2TIz+3n0UcWUsZl1NrNlwHnA8CjOto1dvKoeozSzfwD9\n3T3nO11SXVTGLV+1l3FVVZoiIuWmxyhFRBJQpSkikoAqTRGRBFRpiogkkNcaQZ06dfKampoChVId\n5syZs9LXolm9VcYtn8o4mbwqzZqaGmbPzuUJrJbDzNaqZQFUxi2fyjgZNc9FRBJQpSkikoAqTRGR\nBFRpiogkoEpTRCQBVZoiIgnkNeSoVFavTs0/OmTIEABuuikzE1Xv3r0BGD9+PADdunUrcXQisjZR\npikikkBVZJrvv/8+ALfccgsA66yzTvqzMCj3oYdSk6f/6ldNLqEtFeCf//wnAAMGDABg6dKlzT7X\nY489lt7u0aMHAF26dGl+cFI24Xt81FFHAXDdddcBcPbZZ6f3iX//y0GZpohIAhWdaX744YcADBo0\nqMyRSKFNnToVgDVr1uR9rsmTM+uy3XbbbQDcc889eZ9XSuejj1KrX8QzSoBzzz0XgNNOOy393vrr\nr1+6wOqhTFNEJIGKzDSvvTa1QOGkSal11V566aUmj3nuuecACMt37LLLLgDsv//+xQhRmumbb1LL\nwjz88MMFO2cYQQFw9dVXA5kRF+3atSvYdaR4nn32WQDee++9rPdPOOEEANq0aVPnmHJRpikikkBF\nZpoXXJBaKz7JXbKJEydm/ezatSsA9913X3qfH/3oR4UKUZrpqaeeAmD69OkA/Pa3v837nKtWrUpv\nL1iwAID//Oc/gDLNShbvzx4xYkS9+5x88skAmFlJYsqFMk0RkQRUaYqIJFBRzfN+/foBmZs53377\nbZPHdOrUCcg0w955JzUh89tvvw3Abrvtlt73u+++K1ywkrNXX301vT1w4EAAunfvDsBFF12U9/nj\nQ46kerzyyivp7fCwQ7Duuqmq6bDDDitpTLlQpikikkDZM81nnnkmvf3aa68BmU7fhm4EnXXWWent\nvn37AtC+fXsAnnzySQAuu+yyOsfdeOONQN0BtFJc8bIIN2jGjRsHwAYbbNDs84YbQPHfoUq6YSCN\nCzdt63PIIYeUMJJklGmKiCRQtkwzTNAQ+rgAVq5cWe++YfjQscceC8Af//jH9Gdt27bN2jdMDTdm\nzJg65xw6dCgAX375JZCZ3KN169bN+4+QRk2YMAHIHsge+jLjfc3NFYapxLPLAw88EIAOHTrkfX4p\nrngLIVhvvfUAGDlyZKnDyZkyTRGRBMqWaX799ddAw9klZB6BvPfee4HMnfLGhEwz3JW98MIL05+F\nR+tCxhmmn9pmm20SxS65CRNDh//vUJj+5NBKueuuu4DMnVaAiy++GFDroZKFBxtmzJhR57PQcuzV\nq1dJY0pCmaaISAJlv3ten9DfNXbsWCC3DLO2kEXeeeed6fdmzZpVgOikKZ9++ikAM2fOrPPZ4MGD\n8z7/zTffDGSmDuzZs2f6s4MOOijv80txNTYBTzWMbFGmKSKSQNkzzfqe+nnxxRfzPm94qij+FFDt\nJ43CXfgwZlAKI0zEsGzZMiAzvVehLFmyJOv1TjvtVNDzS3HVl2mG0Q6FaIkUmzJNEZEEVGmKiCRQ\ntuZ5WLu8WCvLhVXt5s6dm36v9uOZl156aVGuvbbbcMMNgcywkfiEHeHRx44dOyY+74oVK4DMUKZg\nn332aVacUlrPP/88kBkqFhceg+7cuXNJY2oOZZoiIgmULdOcMmVKQc8Xhp8sXLgQaPwxrDCESQOg\niyOsFhgemQyPUwIcfvjhQPZDB/WZP39+ejvc+AnT/tWelKNVK/3trwZhxclwQzaukifoqE2/bSIi\nCZR9yFGhhOnHbrjhhgb3qampAeCOO+4AMhOBSHEMHz4cyM4sQgsjPlFLfTbddNP0dsgsG3rk9tRT\nT80nTCmR2n3R8UlVzjzzzFKH02zKNEVEEqj6TDMskREmMG5MeNxuv/32K2pMktKjRw8ge0XQMJqh\n9gD12sI0gHGDBg0C6j6MEPpQpTKFhxxq3zWP3ykvxFSBpaJMU0QkgbJlmo0tnvbII49kvT7jjDMA\neP/99xs8Ty7LHBT6jr0kt+uuu2b9TOL73/9+ve/Hx4H+4Ac/aF5gUjRhKrjad8379+9fjnDypkxT\nRCQBVZoiIgmUrXke5s0Ls6jHhQHQtR+xrO+Ry9C8z2XlSqluoXlXu5mnJnllC4Pag/BwyQUXXFCO\ncPKmTFNEJIGyZZoDBgwAYNSoUen3GlsvqCnhr1cY5nLLLbcAsOWWWzb7nFJZws0+rW1eXaZOnZr1\nukuXLkBmko5qo0xTRCSBsmWaYdXIsNIkwKRJkwAYPXp04vP94Q9/ADJrmUvLE9arDzSovbKFFWcX\nL16c9X6bNm2A6p0wR5mmiEgCZX+MMqxtHt/u27cvkFl1MEwofOSRRwLwy1/+Mn1MuJMaX5FQWqaw\nOmmY6GHYsGHlDEeaEKbsC49ILliwAIBtt922bDEVgjJNEZEEyp5p1ufQQw/N+ikCmYzl17/+NaA1\nzitdGDsdpm0Mox5++MMfli2mQlCmKSKSQEVmmiL1CX3bUl222morAG677bYyR1IYyjRFRBJQpSki\nkoAqTRGRBFRpiogkoEpTRCQBVZoiIglY7QldEx1s9iHwTuHCqQrd3H3TpndrGVTGLZ/KOJm8Kk0R\nkbWNmuciIgmo0hQRSUCVpohIAo1Wmma2iZnNi/4tN7P3Yq/XK1ZQZtbPzF43s8VmNiSH/UfEYnvV\nzA7P8/rPm1mvJvZpY2YTohhnmFnXfK5ZLuUq4+ja65rZK2Y2KYd9S17GsX2PNzPPdf9KU8bv8R1m\n9qGZzctx/9PD/ma2yMx+kef1x5nZ0U3ss2P0/V1jZjktj9nohB3u/hHQKzr5cOBzd7+y1kWN1A2l\n73K5YFPMrDVwPfDfwHJgtpk96O5vNHHoFe4+2sx2Ap4ys808dpfLzNZ1928KEWPkTGC5u3c3s5OA\nvwAnFvD8JVGOMo65EJgPtM1x/1KXMWa2ETAYmF3I85ZSGcv4NuAG4OYEx9zp7heY2RbAfDOb7O7p\nFReLUMYrgXOBY3M9oFnNczPrbmYLzexOYAHQxcw+iX0+0MxujbY3N7OJZjbbzGaZ2Z5NnH5PYJG7\nv+Pua4D7gP65xubu8wEDNo7+0txoZrOAkWa2gZndHsUx18yOjGJsa2bjo79u9wNtcrhUf+COaPs+\n4Me5xlgNilzGmFk34BBgbNLYSljGACOjf2uSxlnpil3G7v4MsKo5sbn7cmAp0DVqZfzNzF4Abo9a\nKFdHcbxiZqdHMbYys/81s9fM7HGgUw7X+cDdZwM5V8T59GnuAFzj7j2B9xrZ71pglLv3Bo4DQiHs\nYWY31bP/1sC/Yq+XRe/lxMz2Br5091BYWwJ7uvtQYBjwqLvvDhwEXGVmbYBfAR+7ew9gBLBr7Hxj\nG2iWpeN096+A1WbWIdc4q0SxyhhgNDAESDzmrVRlbGa7AZu5+9Tan7UgxSzjZjOz7kA34K1YnAe7\n+0mkWnkrojLeDTjHUt1jxwL/BfQETgX2jp3vMjPrV4jY8plPc0lUQzelD7C9Zdaq3tjM1nf3F4EX\n87h+bUPM7OfAZ8DxsffHx5ocfYHDzOx30es2QFdgf2AUgLvPNbMF4WB3P7WAMVabopRx1M/0L3ef\nZ2Z9EsRTsjI2s1bAVVRhl0tClfY9PtHMDiSV2Z/u7p9E13zQ3cNypH2BHmY2MHrdHtiWVBnfHf0u\nLDOzp8NJ3f0PhQown0pzdWz7O1LNpSDe9DFg9ygby8V7QJfY6840/hcwuMLd61v7Nx6nAUe7+5L4\nDrFfhCRCnMst1Znezt0/aeKYalOsMt4bGGBmR0Xn2cjM7nD3QU0cV8oy7kAqY3kuOnYL4GEzO9zd\n5yY9WQUrVhk3153uXt8NmdplPNjdn4jvYGbHFDWySEGGHEU1+8dmtm30Fzoe/DTgnPCigaZu3Eyg\np5l1M7PvkWoKTI6OHRX6qJppKqlO3xBLaKI9C/wsem8XYMcczjUZCF/y44DH8oir4hWyjN19qLt3\ndvca4CTgsVBhVkoZu/sqd+/k7jVRnLOBfi2swsxS4O9xg8zsfDM7q/mRMhUYbGbrRufb3szWJ1XG\nx0d9m1sDB+RxjQYVcpzmb0n9x0wn1Q8ZnAPsE3XYLgTOgIb7Qtz9a+A84HFgITDO3V+PPt6Z1B31\n5roUaGepISsLgOHR+9cDm5jZIuASIP3FaKRP82ZgSzNbTKq/7KI84qoWBSnjJlRSGa+NClbGZjYe\neI5UErQs6loB6AF8lEeMY4A3gXlmNh+4kVSreQLwLql6YywwIxZLvX2aZtbZzJaRqnOGR3E2Opqj\nap49t1Qb6RF31xKVLZTKeO1gZv8A+hd6eFipVE2lKSJSCfQYpYhIAqo0RUQSUKUpIpJAPuM06dSp\nk9fU1BQolOowZ86clWvTrN4q45ZPZZxMXpVmTU0Ns2dX7TwGzWJma9WyACrjlk9lnIya5yIiCajS\nFBFJQJWmiEgCqjRFRBJQpSkikoAqTRGRBFRpiogkkNc4TRGRYvj4448BePfddxvcp1u3bgBcc801\nAOy0004AbLfddgDssssuRYlNmaaISAIVlWmuWLECgOOOOw6AvfdOrYt05plnAqknFwrh008/BeDZ\nZ58F4NBDU9M3tm7duiDnF5FkpkyZAsBDDz0EwNNPPw3Am2++2eAx22+/PQBLly4FYM2a7AVDv/uu\n0CtOpyjTFBFJoOyZZui7ANhxx9SyLSET3HzzzYHCZ5g//OEPAVi5MrUGfXjudtttty3IdSR3//73\nvwH43e9Si0cuWJBaJHLatGnpfdQCaBmWLEmtdXfDDTcAcPPNN6c/++KLLwBIMin666+/3vRORaBM\nU0QkgbJlmiHLC/2XAB99lFpr6ZxzUoveXXfddQW95ogRIwB4++23gcxfOmWYpTdu3DgALr74YqDu\nXdKQgQJssskmpQtMimbZstQ6baNH17cKc+522GEHIHO3vNSUaYqIJFC2TPOf//wnkLlLFjds2LCC\nXWf+/Pnp7SuvvBKAY45JLed8/PHHF+w6kpuQbfz6178GMi2O1EKUGeeem166nOuvvx6Ajh07liJE\naYZQjpDJJPfdd18gMzplvfXWA6B9+/YAbLDBBuljPv/8cwB+/OMfA5ksco899gBg1113Te+7/vrr\nA9CuXbsC/1fkRpmmiEgCqjRFRBIoefM8DGC///7763x22223AbDppvkvzxKa5YccckidzwYMGADA\nhhtumPd1JJnQRRJu+jXknnvuSW8/8sgjQOamUWi6h+aelM/q1auB7O/Zyy+/DMCkSZOy9t1rr70A\nmDt3LpA9lDDcCOzcuTMArVpVbj5XuZGJiFSgkmea//M//wNkhpyEgeYAP/3pTwt2neeffx6A5cuX\np9879dRTATjppJMKdh1p2jvvZNawGjt2bNZnYVKF8CDD448/Xuf48FBCyFJPPPFEALbYYovCBys5\n+eqrrwD42c9+BmSyS4CLLroIgD59+tR7bH0Pq3Tt2rXAERaPMk0RkQRKnmmGoSXh59Zbb53+LJ8+\nqvAY1siRI4HMo1rxoSyhz1RKa968eentMGh9//33B+CZZ54B4MsvvwTgrrvuAuAvf/lL+pjFixcD\nmVZD//79gUxfp4YilU4YGhS+Z2GCjfh9iCFDhgDQtm3bEkdXGso0RUQSKPuEHWFKKIC+ffsC0KFD\nBwDOPvvsJo8Pg+PDz5kzZ2Z9Xsh+Umme+JRdIfMPg9uDNm3aAPCLX/wCgAkTJqQ/CxM9hMkcQgaj\nu+elF+6IX3755UBmIuDnnnsuvU8YvN5SKdMUEUmg5Jnm+eefD8CTTz4JwPvvv5/+LPRvhYziwQcf\nbPJ8Yd/aj+Fts802QKbvRcrn7rvvrvPeP/7xDwCOPvroeo8J0/XVZ8899wSyH8OT0pg+fXrW6/B4\nYxhfuTZQpikikkDJM80f/ehHALz66qtA9p3VRx99FIBRo0YBsNlmmwEwaNCgBs938sknA7Dzzjtn\nvR+WyggZp5TPCSeckN4OrYeXXnoJgNdeew3I/D488MADQPbk1KGPO7wXpvQLZd+zZ8+ixS7Z4n3N\nkBnBcOmll6bfO+qoo4DsSTZaEmWaIiIJqNIUEUnAkqzJUVvv3r29sQ77UnjrrbeATDO8V69eADz2\n2GNAYSb/iDOzOe7eu6AnrWCFKONVq1alt0M5hUcjG7qRF58AIjyocMQRRwDwxhtvAJlVSm+66aa8\n4qtNZdyw2g+n1GedddYB4KyzzgIyc2L+61//AqB79+5AZk2wuLBGVJjco1g3mPIpY2WaIiIJlH1w\ne77+9Kc/AZm/fOEmUqEzTGm++GOO48ePB+DYY48F6mac5513HgB//etf08eEge9hSr/wiOXUqVOB\nzOB30I2/YvvNb34DwFVXXdXgPt9++y2QaSGEn0mEm8AHHnggkD1VYLkp0xQRSaAqM82QrQDccccd\nAGy00UaAVi6sdGG6sDB0JUzQEYYVhZZDyC7jLrnkEgAWLVoEZIYvhWMg8/sgxREenwyryIZp+r7+\n+uv0PmEdqJBxNkeYrDx81+MrT4bJqMtFmaaISAJVmWmGAbVxhx9+OJA9qbFUrpBxNjRRbX3CKoRh\nFdGQaT711FPpfcKdek0XVxzhzvhuu+0GZEYyxD3xxBNAJvscPnw4ALNmzUp8vdDXPWfOnMTHFosy\nTRGRBKo+0wxrH4e7etLyhf60yZMnA9l3VsMa6cOGDSt9YALAwQcfnPU6PCodMs3WrVsDmeVnAM44\n4wwArrnmGiDT112JlGmKiCSgSlNEJIGqap6Hx+XiK0yGVQx1A2jtEdbEHjp0KJC9vna46TBw4EAA\ntttuu9IGJ3WEFRnCKpXhBlGYrQrgzTffBDIrMNQWX0us3JRpiogkUJWZZnyygH79+mXt89lnnwGZ\nuReraT1lSSZMzvLnP/85/V64Ifj73/8egHHjxgGZ4UpSej169AAyQ8XuvffeOvvEh40BrLtuqmoK\nQwnjj9XY76NRAAAG3klEQVSWmzJNEZEEqirTrE/4ixQyijBkITx2pcfqWr5TTjklvT1mzBgAJk6c\nCGT6ymrP7C+lE7L80aNHA5nWYHzA+gcffABATU0NkCnT0EddSZRpiogkUPWZ5i233ALArbfeCsDp\np58OZCZ3kJYvPg3gtGnTgMx63GGCiUoeLL22CCNdpkyZAsDf//739GczZswAMpllmBquEinTFBFJ\noKoyzeuuuw6AP/7xj+n39t9/fwDOPvtsADbeeGMA1ltvvRJHJ5UgjJYIy2WERy0XLlwIaOXKShJW\nE629XemUaYqIJFBVmeZ+++0HwJNPPlnmSKTShUmOd9llFwAWL14MKNOU/CnTFBFJQJWmiEgCVdU8\nF8lVWDPq7bffLnMk0tIo0xQRSUCVpohIAqo0RUQSsLDaW7MONvsQeKdw4VSFbu6+adO7tQwq45ZP\nZZxMXpWmiMjaRs1zEZEEVGmKiCTQaKVpZpuY2bzo33Izey/2uqgzYpjZumb2iplNymHfEbHYXjWz\nw/O89vNm1ivHfY83M891/0pTrjI2swvNbEH079wc9j/dzD6M4lpkZr/I8/rjzOzoJvbpaGaTo9/D\nF82sKp/BLGMZL4u+j/PM7MUc9i9HGfcxs09j/z/+0NR5Gx3c7u4fAb2ikw8HPnf3K2td1Ej1jX7X\n1MUSuhCYD7TNcf8r3H20me0EPGVmm3msw9bM1nX3bwoZoJltBAwGZhfyvKVUjjKO/sAMAnoD3wCP\nmdkUd29qJPqd7n6BmW0BzDezye6+MnbeQpfxJcCL7n6Ume0I/D/gkAKevyTK/D3ez90/SbB/qcsY\n4Cl3b7RyjWtW89zMupvZQjO7E1gAdDGzT2KfDzSzW6Ptzc1sopnNNrNZZrZnDufvRuqXc2zS2Nx9\nPmDAxtFfmhvNbBYw0sw2MLPbozjmmtmR0fXamtn46K/b/UCbHC83Mvq3Jmmcla7IZdwDmOnuX7j7\n18CzwDG5xubuy4GlQNeolfE3M3sBuD1qoVwdxfGKmZ0exdjKzP7XzF4zs8eBTjlcqifwZHTNBcB2\nZrZJrnFWumJ/j/NRwjJOLJ8+zR2Aa9y9J/BeI/tdC4xy997AcUAohD3M7KYGjhkNDAES39o3s72B\nL919VfTWlsCe7j4UGAY86u67AwcBV5lZG+BXwMfu3gMYAewaO99Yq6fpbWa7AZu5+9SkMVaRYpXx\nq8ABlmr+tgMOA7rkGpSZdQe6AW/F4jzY3U8CzgRWRGW8G3COmXUFjgX+i1RFeCqwd+x8l5lZ9rKm\nKS8DA6J99gI6R/9akmJ+jx140szmmNlpSYIqYRkD7GtmL5vZw5ZDF0w+z54vcfdcmqV9gO0ts+zu\nxma2vru/CNTp54j6IP7l7vPMrE+CeIaY2c+Bz4DjY++PjzU5+gKHmdnvotdtgK7A/sAoAHefa2YL\nwsHufmo9MbYCrgJOTBBfNSpKGbv7fDO7GpgGfA7MBb7N4TonmtmBpDL70939k+iaD7r7l9E+fYEe\nZjYwet0e2JZUGd8d/S4sM7OnY/E01I91GXCtmc0jVYG+nGOc1aQoZRzZ093fi5raj5vZInef3sR1\nSl3GLwE17v551PKcSKqCblA+lebq2PZ3pJrEQbx5a8Du7v5VjufdGxhgZkdF59nIzO5w90FNHHeF\nu49uIk4Djnb3JfEdYr8IuepA6q/Zc9GxWwAPm9nh7j436ckqWLHKGHe/GbgZwMxGAYtzOOxOd7+g\niTgNGOzuT8R3MLOcm/+xGD8l1fca/lAuBVraDCDFLOP3op/LzexBYHegqUqzHGUcth+KuvM6NNYP\nW5AhR1HN/rGZbRv9csWDnwacE17U19Stda6h7t7Z3WuAk4DHQoVpZqNCP2QzTQXSd2rNLDTDnwV+\nFr23C7BjEzGucvdO7l4TxTkb6NfCKswshSzjaJ/Nop81wFHAPdHr883srDxCnQoMNrN1o/Ntb2br\nkyrj46N+r62BA3KIsYOZtY5e/hKY5u6rGzummhWyjC11/2CDaLsdqXsU86PXlVTGW8S29wS+aerG\nVSHHaf6W1H/MdGBZ7P1zgH2iDtuFwBlRgI31hTRkZ2B5HjFeCrSz1DCIBcDw6P3rgU3MbBGpO6bp\nyq+hPs21VCHLeFK07yTgLHf/d/R+D+CjPGIcA7wJzDOz+cCNpFpUE4B3gYWkbjDOCAc00t/1A2Ch\nmb0OHExqREdLV6gy3hJ4wcxeBmYBD7j7tOizSirjgZYa9jYPuIbsrr16Vc1jlJZqBz/i7oeWOxYp\nHjP7B9C/CMNKpEJUexlXTaUpIlIJ9BiliEgCqjRFRBJQpSkikoAqTRGRBFRpiogkoEpTRCQBVZoi\nIgn8f6S1hmwDdPmXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc287ae2fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_images(images=images,\n",
    "            cls_true=cls_true,\n",
    "            cls_pred=cls_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 画出错误分类的例子\n",
    "\n",
    "我们画出测试集中被错误分类图像的例子。\n",
    "\n",
    "首先我们获得测试集中所有图像的预测结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_pred = model.predict(x=data.test.images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后将结果从独热编码转换到整数类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cls_pred = np.argmax(y_pred,axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "画出被错误分类的图像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XncVeP+//HXJ1ORFCUh3c5pUIaTTjKEzENSScgUkupk\n6sTX7JRDZg45knBMOXF0EjJVh/STSoNKknksSRQNhuj6/bHXtde+x73Xvee79/PxuB/32ntfa61P\nLvu6P+ta17ouc84hIiKpqZXvAEREiokaTRGRCNRoiohEoEZTRCQCNZoiIhGo0RQRiUCNpohIBGo0\nRUQiUKMpIhLBpuns3LBhQ1dSUpKhUIrDnDlzVjjnGuU7jlxRHdd8quNo0mo0S0pKmD17djqHKDpm\n9nm+Y8gl1XHNpzqORpfnIiIRqNEUEYlAjaaISARqNEVEIlCjKSISQVp3z0Vy4YMPPgCgf//+AJx2\n2mkAnHfeeXmLSTZeyjRFRCJQpikFyWeXAMcddxwAn3zyCQCfffYZoExT8kOZpohIBMo0paDcfffd\nANx1113x97744otSZZo1a5bTmKR61q5dC8DixYsBeOCBB0p9vnz58vj2+PHjAejXr1+pMr7/+uCD\nD85anFEp0xQRiaCgM81Zs2YBMGrUKCDs5/rjH/8YL9OjRw8A9t13XwAaNdpo5lmoUX777TcAFi1a\nBMDnn4ePBpsZAC1btgRg9OjROY5Okrnhhhvi288++ywQZprvv/8+AH65cF+ficuH+/f8d92/njNn\nDgAvvfRSvGzDhg0z/w+IQJmmiEgEBZlpfv311wD07NkTCPu0Nt00Fu7UqVPjZR9++GEA2rdvD8Cd\nd94JwEEHHZSbYCUjRo4cCcCDDz5YaRmfYey88845iUkq57P9Sy65BCjdP1k2k2zdujUQ9kWfcMIJ\n5Y7nrxj9Ph06dACIz76U2K+tTFNEpIgUZKZZq1asLV+9ejUA9evXB2DMmDEArFq1Kl72yiuvBMK/\nSM899xygTLNYLF26FICHHnoICDONxP4u77bbbstdYFKlFStWlPo9YMCAcmX8ONrddtsNgC233DLp\ncd97771Sx/VZayFRpikiEoEaTRGRCAry8rxx48ZAeIntL7n9jaBevXrFyx544IEAjBgxAoD77rsP\ngAMOOACouNNZCocfWrRgwQKg4suxrl27AtCuXbvcBSZVGjRoUKnfmbJu3TogHK7kb/rk++ZPImWa\nIiIRFGSm6Z1++ulAmGmee+65AAwdOjRexn82d+5cIPwL5X9LYdt6662BMJPwNwASTZ8+HQgfbthj\njz1yFJ3k2jPPPAOEVxy6ESQiUuQKOtP0QxT8Xxs/wLVPnz7lytauXRsIB0efccYZuQhR0uSzxu7d\nuwMVD2732afvt/a/pebxj9H6IWe77LJLqd8V8f3i/v8TXzZbj1Qr0xQRicAqGkScqvbt27tcLDI/\nbtw4AObNmweUnhzAx+/vtCc+YpkNZjbHOdc+qycpILmq4y+//BKAkpISoOLJHHbccUcAJkyYAMCf\n/vSnrMSiOs4f/2CLr/OLL74YCAfIJ/L9n/5+RtlM0z9SDeVH0aRTx8o0RUQiKOg+Tc8/zO9/33zz\nzfHP1q9fD0Dv3r1zH5hkTNOmTYEws/jHP/5RrsySJUuAcNxm4vRxUtz81WPZK1///0HiXfSyE4H4\nCVyOPvpoIHy02n+eaco0RUQiKIpMMxWbbbZZvkOQDBgyZAgQTvUH4dK9/mmRZcuWAXDRRRcBpUdT\ntG3bNidxSvrOPPPM+LZf7qLs+Ey/zEVi1lidiUAySZmmiEgEajRFRCIoqsvz7777Dqh4rsXtttsu\n1+FIFmyzzTZAuAohwNNPPw3AlClTAPjxxx8BuPfeewEYO3ZsvOz8+fMBrRVVSPyQwauvvhoov2YQ\nhJfj/lL78ccfBwpzwh1lmiIiERRVpvn8888D4cqFAFtssQUAXbp0yUtMkn1+EPP9998PwMCBA0t9\n7m8MAfz666+5C0wq5DNLX2/+Jo+fROfEE08E4L///W98H59p+mGFhZhheso0RUQiKKpMMzGj8Cqa\nvENqpr322ivfIUgZ3377LQDXXntt/D2fYfrM0mePZQedVzTtm59UvJAp0xQRiaAoMk1/19z3aSU6\n6aSTch2ORPT666+Xe69Tp04p7//AAw8AcOONNwLlR0+kM+mMVI9fNbJz585A6Uda27RpA4Rr2Zft\nn/T7JmaaZfs0C5kyTRGRCIoi01y+fDkAn332WbnP/Lg+KTx+TfNu3brF3/OPxfk6LcsvX5KYnX7z\nzTdAOGrCZyX+kUm/D8AOO+yQkdilaj4j9BmmzyoTP6tsMbSKrhj8GM5CWkCtMso0RUQiUKMpIhJB\nUVyeS3H6/fffAVi9enX8PT/r+gsvvFDlvhU9YlevXj0AbrnlFgCOP/54AJo0aZKhiCVV/lFIXzeJ\nl9WVXWL7Qe9lZzSCwh7MXpYyTRGRCJRpStZssskmQJghAvzwww8p7etncgfYe++9gXBW90MPPTRT\nIUo1+clTfNbob+QAXHPNNUB4Q8jPfzl69GggHPQ+aNCg+D7t2rXLbsAZpExTRCQCZZqSNX71SJ+N\nALz99tulygwfPhyAQw45BAgflUzMQqTw+KFj/nfiCrGLFy8G4IwzzgDCtez9MDPfl3nVVVflJtgM\nU6YpIhKBMk3JusRHJss+PqmMsmZIXKfH908uWrQIgMGDBwPhypI+Oy2GgewVUaYpIhJBUWSajRs3\nBsIVCmfPnh3/rGPHjkA4pdSkSZNyHJ2IVMWvGun7MotpTGZFlGmKiERQFJnmtttuC8CLL74IhHdl\nAX755RcALrjggtwHJiJJ9evXr9TvYqdMU0QkAjWaIiIRFMXluefXsl6/fn2eIxGRjZUyTRGRCNRo\niohEoEZTRCQCS2clPzP7Fvg8acGapZlzrlG+g8gV1XHNpzqOJq1GU0RkY6PLcxGRCNRoiohEUGWj\naWbbmdm84GeZmS1JeL15toIys8Fm9m7wc2EK5fua2bdBXO+ZWZ80zz/azLonKdPAzF4ws/lBnL3T\nOWe+5LGOvzKzd4LzzEyhfD7quIeZLQjOOcvMDkjnnPmSjzo2s2ZmNsXMFgXfj6TPOeepjq9I+G/x\nrpn9ZmbbVHlg51xKP8BQ4NIK3jegVqrHSeE8bYH5QB1gM+A1YNck+/QF7gq2dwBWAA3LlNk0Qgyj\nge5JyvwNGBZsNwZWRjlHIf7kqo6DY34F1I9QPh91XJew378dsDDfdVQsdQzsCLQNtusBHwMtC62O\ny5Q/AZiYrFy1Ls/NrHnwF+QJ4F2gqZmtSvi8l5k9GGw3NrNxZjbbzN4ys/2SHL41MMM595Nzbj0w\nNfjHpMQ5twz4DNjFzG4ws8fMbBrwiJltamZ3BnEsMLO+QYy1zGyEmS02s0lAKrOjOmDrYLsusQr+\nPdU4C12W6zgtuapj59waF3ybgK2I1XmNkc06ds4tdc7NC7Z/BBYDO6UaWw6/x4lOBcYkK5TOY5S7\nAb2dc7PNrKrjDAdudc7NMLMSYAKwh5ntC5zjnBtQpvw7wBAz2xb4BTgWmJZqUGbWHGgGfJIQ58HO\nuZ/NbCCw3DnXwcy2AGaY2URgP2BXoA2xv5CLgJHB8YYB05xzL5Y51d3ABDNbSuwvac+EL1hNka06\nhlgD9KqZOWCEc+6hVIPKYR1jZj2BYcS+gJ1TjbGIZLOOATCzPwB7ALNSDSqXdRx8Xhc4AjgvWWzp\nNJofO+dmJy/GEUArCxeGb2BmdZxzM4FyfVnOuYVmdicwGVgDvE1qGdzpZnYIsYa2r3NuVXDOZ51z\nPwdljgJam1mv4PU2QAvgYGCMc24D8JWZTUmIJ1ybtLTOwFtAJ6Al8LKZ7emcW5NCrMUiK3Uc2M85\nt8TMdgAmmdl7zrk3k5wn13WMc24sMNbMDgWuD45fk2SzjjGzesB/gQtT/G7kvI4D3YDXnXNJ15hO\np9Fcm7C9gVifiFc7YduADs65X1M9sHNuFDAKwMxuBT5KYbcnnHMVLTiTGKcBA51z/0ssYGbVmUr6\nHGBokF2+b2ZfEms851bjWIUqm3W8JPi9zMyeBToAyRrNXNdxYryvmdmjZlbfObcq+R5FI2t1bLGb\nTOOAh51zz6W4W77quBfweCoFMzLkKGjZV5pZCzOrRek+yMnA+f6FmbVNdjwz2z74XQJ0BZ4MXl9s\nZpVeBqTgFWCgvwwxs1ZmVodYv+kpQZ/ITsSyx2S+AA4PjtMEaA58mkZsBS2TdWxmdYPLIcxsK+BI\nYGHwumDqOOjzs2C7PbGbQjWpwSwlw3VswCPAPOfc8DKfFUwdB/s3AA4Ank+lfCbHaV5O7B/zJrE7\no975QMegw3YRQZ+Bme1rZiMrOdb4oOx4YEDQkQyxm0TfpRHj/cCHwDwzWwjcRyzbHkusEVwEPAxM\n9zuY2TAzq6gvayjQycwWAJOI3ZFcmUZsxSBTddwEmGZm84l1cTzjnJscfFZIdXwysNDM5hHr0zsl\njbiKRabquBOxGytHWjik5+jgs0KqY4ATgZeccz+lcvKieozSzF4Aujnnfst3LJIdquOar9jruKga\nTRGRfNNjlCIiEajRFBGJQI2miEgEajRFRCJIazXKhg0bupKSkgyFUhzmzJmzwm1Es3qrjms+1XE0\naTWaJSUlzJ6dyhNYNYeZbVTLAqiOaz7VcTS6PBcRiUCNpohIBGo0RUQiUKMpIhKBGk0RkQjUaIqI\nRJDWkCORVIwePTq+fdZZZ5X67PHHY/O+nnbaaTmNSbJv8uTYbH/r168H4PnnY9NV3nfffUn39eNG\n69evD8CFF4aL0vbpk9YilWlTpikiEoEyTcm6xOxyk002KfXZ2WefDcDq1asBaNOmDQAHHXRQboKT\njHvyyScBOP300wEoO/2kX2co8f169eoBcMUVVwDQsWNHAPbdd18AtthiiyxGHI0yTRGRCJRpSkEY\nOHAgALvvvjsAI0aMiH924IEH5iUmqZ45c+YA0LRpUwC++OKLpPv4DPPKK6/MXmAZokxTRCSCgsw0\nH3nkEQBuuukmAD744IOk+9StWxeAk046CYBBg2KrgO61115ZiFCi8HfIIezDrMzixYtL/QZlmsXm\ntttuA2DIkCEAvPLKK0D43fQS+yn32WefHEWXPmWaIiIRFFSmedFFFwHwz3/+Eyh/161169YA9OjR\nI/6ev9s6ceJEAN566y0gvPv27LPPxssedthh2QhbkmjZsmV8+/fff6+wzIYNG0q97t+/f3x7yy23\nBDSWs9j4qz/fx1lW4vfxiCOOyElMmaBMU0QkAjWaIiIRFNTluX/cruxl+a677grAm2++CYSPViXy\nl26vv/46AIcccggAxxxzTLzMBRdcAMCdd96ZwaglmUaNwlUFOnXqBMAbb7xRYdmyg98hvHmky/Pi\n8tFHHwFw7733AuW/18Van8o0RUQiKKhMs3HjxgCsXLkSCDNK/4jV5ptvXum+06dPB6BLly6l3veT\nBQA8+uijgDLNXGvWrFl82w9a94PZK8s4pfi9/PLLQPiIrH980l/9lR2CVCyUaYqIRFBQmeYll1wC\nwHnnnQfA2rVrAahduzYAvXr1AsIJASAcjvLaa68BsGbNGiDMUqdOnRove88992QtdknNbrvtBkCr\nVq0AZZo12R133FHh+35oYSFNwhGFMk0RkQgKKtP0fR3+bvmnn34KwK+//grApEmTADj33HPj+/Tu\n3RuACRMmlDrWVlttBcCee+4Zf2/UqFHZCFuqwU9E69fbTmXdbf9wg+8rS+wrleKxYMECoPSjkw0a\nNACgVq3Cz+MKP0IRkQJSUJnmzjvvDISPRB599NEADB06FIDzzz8fCB+VBHjnnXcAePfdd0sd65xz\nzslqrJIZ/pHYuXPnAhWP0/T8xC233HILUHr6OCk8b7/9NhBe7S1ZsgQIp4G7/PLL42VPPfVUADp3\n7lzqGH7Zi0KatEWZpohIBAWVaXrNmzcHwn4u/xfrq6++Srqvz1avueaaLEUnmeQnnVV91Tx+BMuY\nMWMAGDBgAFD+qjCxjP9dlr+6ALjssssyGmdUyjRFRCJQoykiEkFBXp57fhiCnxvTDzH5/PPPK91n\n7733Bop34Kwk98wzzwBw1FFHAdC9e/d8hiNJ+Js4vrtt3bp1ADzxxBPxMv4Ry+eeew6AVatWldrn\n2muvjZfdZZddgPBhl1xTpikiEkFBZ5qezxqvu+46IOxQBvj5559LlfWvv/76awCaNGmSixAlTWVn\nbq+qzPLlywFYsWJFVmOSzPKPQ/vfF154YbkyfvrG559/HoBu3boB4QMuAHfffTegTFNEpCgURabp\nnXLKKUCYcUL4qKXPRv3jk9999x2gTLPQjRs3Dggfn6tqcLvny/jJPhL7NBs2bJjpECUH/OQ8vu/y\noYceqrRsvleYVaYpIhJBUWWafi1sn10m8pM5+DurUhxuvvnmau/773//G4CLL744/p4yzeLhH6sE\n6Nu3LxCukV7WySefHN/2q9XmizJNEZEIiirT/PLLL8u955fA0ATDxemxxx4DYPfdd89zJJIrs2bN\nAkpnj5999lmFZX2Zp556KutxpUqZpohIBGo0RUQiKIrLcz+wdc6cOeU++8Mf/gAU1nx7kjq/ZlDL\nli2BcM7MiqQyAF5yy68D5L+HAAcddBAQ3pTzs5ONHDkSgNtvvx2AX375Jb6Pf4zSD3w/88wzgfzf\n9KmIMk0RkQiKItP0a4okDmr3/F8mKW6jR48GYP/9909a1g9u91cXGmaUP2PHjgVg5syZ8fcaN24M\nQL169QD4/vvvgfCBk4q0adMGCIegdenSJfPBZogyTRGRCAo60/zmm2+AcD30ipx44om5CkeyyE/7\nd9pppwHhwPWq+LJNmzbNXmBSpa5duwLh6goAy5YtA8Lvb1n+kWe/9hfA2WefDYRZaiFTpikiEkFB\nZ5qffPIJAFOnTi31vp+cGKBPnz45jUmyw/dLHnDAAUBqmabkn1/jqUWLFvH3/Bo+ZQes+2ne/L2J\nfE+8UV3KNEVEIijoTLMyfvo3gC233DKPkUim9evXr9RvKQ49e/ascLsmUqYpIhJBQWeafpLZshKn\nlFq6dCkQrrEsIpJNyjRFRCJQoykiEkFBX55vv/32QLi+9cSJEwE45phj4mX841ciIrmgTFNEJIKC\nzjTPOuusUr9FRPJNmaaISATmnKv+zmbfAp9nLpyi0Mw51yjfQeSK6rjmUx1Hk1ajKSKysdHluYhI\nBGo0RUQiUKMpIhJBlY2mmW1nZvOCn2VmtiTh9ebZCsrMBpvZu8HPhSmU72tm3wZxvWdmaU2yaWaj\nzax7kjINzOwFM5sfxNk7nXPmSx7r+FFfZymWVx1XUx7r+Cszeyc4z8wUyuejjnuY2YLgnLPM7ICk\nB3bOpfQDDAUureB9A2qlepwUztMWmA/UATYDXgN2TbJPX+CuYHsHYAXQsEyZTSPEMBronqTM34Bh\nwXZjYGWUcxTiT67qODhmJ6ADMC/F8qrj4qvjr4D6Ecrno47rEt4QbwcsTHbcal2em1lzM1tkZk8A\n7wJNzWxVwue9zOzBYLuxmY0zs9lm9paZ7Zfk8K2BGc65n5xz64GpwAmpxuacWwZ8BuxiZjeY2WNm\nNg14xMw2NbM7gzgWmFnfIMZaZjbCzBab2SQgleUNHbB1sF2XWAX/nmqchS7LdYxz7nXg++rEpjrO\njGzXcTpyVcfOuTUuaDGBrYjVeZXSeSJoN6C3c262mVV1nOHArc65GWZWAkwA9jCzfYFznHMDypR/\nBxhiZtsCvwDHAtNSDcrMmgPNgE8S4jzYOfezmQ0EljvnOpjZFsAMM5sI7AfsCrQBdgQWASOD4w0D\npjnnXixzqruBCWa2FKgH9Ez4j19TZKuO06I6zqhs1rEDXjUzB4xwzj2UalA5rGPMrCcwjFgj2zlZ\nbOk0mh8752anUO4IoJWZ+dcNzKyOc24mUK6fwzm30MzuBCYDa4C3Se2v++lmdgixhravc25VcM5n\nnXM/B2WOAlqbWa/g9TZAC+BgYIxzbgPwlZlNSYjn6krO1xl4i9hlZkvgZTPb0zm3JoVYi0VW6jgN\nquPMy2Yd7+ecW2JmOwCTzOw959ybSc6T6zrGOTcWGGtmhwLXB8evVDqN5tqE7Q3E+kS82gnbBnRw\nzv2a6oGdc6OAUQBmdivwUQq7PeGcG5QkTgMGOuf+l1jAzFK+/E9wDjA0yDzeN7MviX2x5lbjWIUq\na3VcTarjzMvm93hJ8HuZmT1LrA87WaOZ6zpOjPc1i92grO+cW1VZuYwMOQpa9pVm1sLMalG6D3Iy\ncL5/YWZtkx3PzLYPfpcAXYEng9cXm1k6l3qvAAP9ZYiZtTKzOsT6TU8J+kR2IpZZJPMFcHhwnCZA\nc+DTNGIraJmu48qojvMnk3VsZnXNrG6wvRVwJLAweF0wdRz061qw3Z7YTaFKG0zI7DjNy4n9Y94k\ndtfMOx/oGHTYLgLOCwLc18xGVnKs8UHZ8cAA59yPwfutge/SiPF+4ENgnpktBO4jlm2PJfYFWQQ8\nDEz3O5jZMDOrqJ9jKNDJzBYAk4jdkVyZRmzFIGN1bGZPA/8PaGOxoSlnBx+pjvMrU3XcBJhmZvOJ\ndXE845ybHHxWSHV8MrDQYkPfhgOnJDt5UT17bmYvAN2cc7/lOxbJDtVxzVfsdVxUjaaISL7pMUoR\nkQjUaIqIRKBGU0QkgrTWCGrYsKErKSnJUCjFYc6cOSvcRjSrt+q45lMdR5NWo1lSUsLs2ak8TFBz\nmNlGtSyA6rjmUx1Ho8tzEZEI1GiKiESgRlNEJAI1miIiEajRFBGJQI2miEgEajRFRCJQoykiEoEa\nTRGRCNJ6Iqgq69evB+C772JzjS5atAiAFStWADBr1iwAXnrppfg+a9fGZrQ/6aSTSh1r8ODBAGyz\nzTYA1KlTJ1thi4hUSZmmiEgEGc00ly5dGt8ePnw4ALfddluFZf3kxwmr28XdcccdpV7ffvvtABx4\n4IEAXHfddfHPDj300DQilnz75JPYCq2TJ08u9f6CBQvi2/7/kT333BOA0047DYC6devmIkRJk7/q\n/PLLLwF45JFHAHj00UfjZb744osK9x05MraSRr9+/eLvVdRm5JIyTRGRCDKaad51113xbZ8tNmzY\nEIB27dqVKuszzTVrwiWkp0+fTlWmTZsGwOWXXx5/73//i63iufXWW1c3bMmhn3+OLV09bNgwAJ58\n8kkAPvoolVWaY6ZMmQLAiBEjAKhfv34GI5R0bdiwAYDHH38cgBtvvBGADz/8sNJ9Ksse//KXv5T7\nvG/fvgDUqpWfnE+ZpohIBBnNNP/617/Gt88880wg7HfaddddK9znp59+im/7fi3fD+ozy7LmzJkT\n337hhRcA6NWrV3XDlhy6+eabAbjhhhsq/Pz4448H4Kijjoq/N2rUKADeeecdAMaMGQOEWeu4ceOy\nE6yk7Pfff49v+yvO//u//ytVZrPNNgNgjz32AKB3796VHs/fE/n009gy8wMGhMukH3nkkUDlbUq2\nKdMUEYlAjaaISARprXvevn17l41p8v0l+8knnwyEl+AVdRb7ge5PP/00AMcee2zG40lkZnOcc+2z\nepICkok6rlevXnzb3/jz/9/5tWmeeeYZIBxWtMkmm8T38UNW/A2GFi1aALB69WoAVq5cGS976623\nAvC3v/0NgMMOOwyAJ554AoAGDRokjVd1HN19990X3z7//PNLfda/f38gvKzu0aNH0uP5G4NHH300\nEF6mQ3h5728cb7XVVpHjTaeOlWmKiESQtcco0+Gzx+effx6Azp07A/Dyyy+XK7tu3ToAjjvuOCDM\nPqrz10cy65tvvgHgl19+ib/nM8ztt98eCG/itG3bttLj+BsI3tixYwF4//33ATj44IPjn/nHc/05\nfTayfPlyILVMU1L322+/ATB16tRyn+2+++4AXHHFFQA0a9Ys5eM2b94cgCFDhgBw9tlnxz9buHAh\nENZxrr/ryjRFRCIoyEyzrH/9618A7LTTTknL+iErr776alZjkuT8sKJff/01/p7PMH0/9d577x35\nuH7iFn/8Dz74oNKyfuhbq1atIp9Hklu8eDEATz31VPy9HXbYAQivFKNkmMVAmaaISARFkWn6fqjD\nDz8cCB+drIgf+O6nomvTpk2Wo5Oy/CB0Pyg9ka/L9u2rf3O6W7duQMUZZqNGjYBwcPwtt9xS7fNI\ncv5R1kS+/9GPjKhplGmKiERQFJnmFltsAYRj7qrKNP1D/LVr185+YFIhP54ysS8zHf7O93/+8x8A\nPv/880rL+ukDR48enZFzS3StW7fO2LGWLVtW7r1jjjkGyN/UgMo0RUQiKIpM07vkkkuA0k+A+AmK\nvR9++AEIJwN44403chSdeP7u6X777QfAjBkz4p99++23AFxzzTWlfpe9Mrjpppvi2w8//DBQfmox\nv8/VV18df2/gwIHp/wMkZaeeeioQTqICmRkL68d9Xn/99eU+u+qqqwDYfPPN0z5PdSjTFBGJQI2m\niEgERXF57juD/cqViQ/vl51wxL/2l+mJj/D5G0qSXY0bNwZg4sSJQDh7P8D3338PhDO3+2FJZSdj\n8ZfxUL6O/QB5PwTNX+JL7nXo0AGA/fffP/7epZdeCkCXLl2qfVzf7eZXqPVdPpD/YYTKNEVEIiio\nTNOvTDhz5kwgnKDDrzVS0dRwla0t4ge3+8cqIZw1vOx6RZIdft2myy67LP6eX11wxYoVQOmMMlUP\nPfQQkF4mI5nhr94S1wf78ccfq308v0Lla6+9BsCOO+4IwPjx4+Nltt1222ofPxOUaYqIRJD3TNNP\n5QZwyimnADB37tyMHT9xILx//M6vqa1pwnIjcdjIxRdfDIRrwHh+MHpif3VZBx10EBAOYJfC0bJl\ny7T292uhX3jhhUDYl+lXnk3nsdtMU6YpIhJB3jPNxIHqlfVV/PnPfwbCSU0T+emnVq1aVeG+vk8E\nwr9aiUspSG75O+l///vfgXBkxIQJEyrdx08NeMIJJwBa57ym8P2XUD7D9FckZVe0LATKNEVEIsh7\nprnLLrufpF0mAAAG30lEQVTEt5977jkgHGPp+ensK5rW/v777wcqf3zOL8IEcMEFF6QXrGScf1zu\n7bffLvV+4hIXTZo0AZRhFjs/YqJPnz5AeIccwoml/WgX/8h0IY6tVqYpIhKBGk0RkQjyfnmeyKfi\n/jG5VHTs2BEI59YrO7DWr2EC8PXXXwPh5Z4UrsT16/38iVKc/GW4H3o2ZcoUoPRNWr/u/T777JPb\n4KpBmaaISAQFlWlWh7/R49cj8evT+McrE4c0rVmzJrfBSaU+/vhjIFwT2/P1qLV9ipe/kevXD/KT\ns6xbtw4IJ99IfDSykAavJ6NMU0QkgqLPND2/vnXi5BBQ+vEuP4t7ixYtcheYVMhP3FH2sclzzz0X\ngN122y3nMUn1JQ4f8uvRJ74H0L9/fyDMPPM98UZ1KdMUEYmgxmSaZ511FhA+muWnhvProEM4OWrX\nrl0B2G677XIZokiN4x979Ov2QDi1o39k1n/vBg0aBORvbZ9MUaYpIhJBjck0GzVqBISrGI4bNw4I\n79hBuH52IT6atbEZPHgwEPZt+pEN9957LwBPP/10vKy/Inj11VdzGaJU4aeffgLguOOOA8LsEsKV\nYP3d8y233DLH0WWXMk0RkQhqTKbp+SUQtBRCYfNPZZ1zzjkA3HPPPUA4VZz/DeGUflI4/BWcn3DF\nXzEA9O3bF4BatWpmTlYz/1UiIlmiRlNEJIIad3kuxcWvFVR2zSApbP7m3IYNG/IcSe4p0xQRiUCN\npohIBGo0RUQiMOdc9Xc2+xb4PHPhFIVmzrlG+Q4iV1THNZ/qOJq0Gk0RkY2NLs9FRCJQoykiEkGV\njaaZbWdm84KfZWa2JOF1VuZ3MrM2CeeYZ2arzazKBcvNrK+ZfRuUf8/M+qQZw2gz656kTA8zWxCc\nc5aZHZDOOfMlH3UcnHewmb0b/FyYQvl81HEDM3vBzOYHcfZO55z5ou9xlWW2NbPngu/yTDNrk/TA\nzrmUfoChwKUVvG9ArVSPE+UH2AxYDuycpFxf4K5gewdgBdCwTJlNI5x3NNA9SZm6hH3C7YCF2fhv\nkMufXNUx0BaYD9QJ6vg1YNcCrOO/AcOC7cbAyijnKMQffY/LlfkHcHWwvTswKdlxq3V5bmbNzWyR\nmT0BvAs0NbNVCZ/3MrMHg+3GZjbOzGab2Vtmtl+EUx0JvOec+yrVHZxzy4DPgF3M7AYze8zMpgGP\nmNmmZnZnEMcCM+sbxFjLzEaY2WIzmwQ0TOE8a1zwXxrYCqhRd9SyXMetgRnOuZ+cc+uBqcAJqcaW\nqzomVqdbB9t1iX2Jf081zkKn7zEAbYBXg3O+C7Q0sypnJ0/nMcrdgN7OudlmVtVxhgO3OudmmFkJ\nMAHYw8z2Bc5xzg2oYt9ewJgoQZlZc6AZ8ElCnAc75342s4HAcudcBzPbAphhZhOB/YBdif0H3BFY\nBIwMjjcMmOace7GCc/UEhhGrnM5R4iwS2arjd4AhZrYt8AtwLDAt1aByWMd3AxPMbClQD+iZ8Iey\nptjYv8fzgR7AdDPbH9g5+PmustjSaTQ/ds7NTqHcEUArC5bUBRqYWR3n3ExgZmU7mVlt4DhgcIrx\nnG5mhxD7EvZ1zq0Kzvmsc+7noMxRQGsz6xW83gZoARwMjHHObQC+MrMp/qDOuasrO6Fzbiww1swO\nBa4Pjl+TZKWOnXMLzexOYDKwBnib1DK4XNdxZ+AtoBPQEnjZzPZ0ztWktaA39u/xMGC4mc0j1oDO\nJ8n/i+k0mmsTtjcQ6xPxaidsG9DBOfdrxOMfB8x0zq1IsfwTzrlBFbyfGKcBA51z/0ssYGYpXxpW\nxDn3mpk9amb1nXOrku9RNLJWx865UcAoADO7Ffgohd1yXcfnAEOD7PJ9M/uSWOM5txrHKlQb9ffY\nOfcDcFawfy1iXQKfVrVPRoYcBS37SjNrEZw4MfjJwPn+hZm1TfGwp1ImpTezi82sqsuAZF4BBvrL\nEDNrZWZ1iPWpnRL0iexELLOoUtAfZMF2e2I3hWpSg1lKpuvYzLYPfpcAXYEng9cFU8fAF8DhwXGa\nAM1J8oUqZhvp97i+mW0WvOwPTHbOra1qn0yO07yc2D/mTSCxw/d8oGPQYbsIOC8Idl8zG1n+MGBm\nWwOHAuPLfNSaKvoaUnA/8CEwz8wWAvcRy7bHEvuCLAIeBqYnxDLMzCrqrzwZWBik9cOBU9KIq1hk\nrI6B8UHZ8cAA59yPwfuFVMdDgU5mtgCYROyu88o0YisGG9v3eE9gkZm9T+wPZNJuhKJ6jNLMXgC6\nOed+y3cskh2q45qv2Ou4qBpNEZF802OUIiIRqNEUEYlAjaaISARqNEVEIlCjKSISgRpNEZEI1GiK\niETw/wHzcqwbcSXHTgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc285c98128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_example_errors(cls_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结\n",
    "\n",
    "这份教程展示了如何通过贝叶斯优化神经网络的超参数。我们用scikit-optimize的（`skopt`）库，它仍然在发展，但它已经是一个非常强大的工具。它能够在少量的迭代中显著地改进手工调优的超参数。这远远优于网格搜索和随机搜索，但对于更困难的问题，它需要花费更多的计算时间，并且最有可能发现较差的超参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 练习\n",
    "\n",
    "下面使一些可能会让你提升TensorFlow技能的一些建议练习。为了学习如何更合适地使用TensorFlow，实践经验是很重要的。\n",
    "\n",
    "在你对这个Notebook进行修改之前，可能需要先备份一下。\n",
    "\n",
    "* 尝试运行100或200次迭代，而不是40次迭代。绘制的landscapes会发生什么变化？\n",
    "* 尝试scikit-optimize中其他的优化方法，例如用`forest_minimize`来代替`gp_minimize`。它们的表现怎么样？\n",
    "* 尝试给优化器用另一个采样函数，例如 Probability of Improvement。\n",
    "* 尝试贝叶斯优化来优化更多的超参数。比如，在卷积层中卷积核的尺寸，滤波器的个数，或者是训练时的批大小。\n",
    "* 加入卷积层数目的超参数，并且应用到`create_model()`函数中。注意，如果你在卷积层之后有池化层，图片会被下采样，所以需要限制卷积层的数目，防止图像变的太小。\n",
    "* 看绘制的图。你是否认为某些超参数可能是无关紧要的？尝试移除这些参数，重做剩余超参数的优化。\n",
    "* 用另一个更复杂的图像数据集文件\n",
    "* 训练多更代（epochs）。验证集和测试集的分类准确率是否有提高？对时间效率有什么影响？\n",
    "* 向朋友解释程序如何工作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## License (MIT)\n",
    "\n",
    "Copyright (c) 2016-2018 by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n",
    "\n",
    "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n",
    "\n",
    "The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n",
    "\n",
    "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}