10b. neural_networks_in_practice_keras.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd \n",
    "import numpy as np\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt \n",
    "import keras\n",
    "np.random.seed(1)   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "=================================================================================================================\n",
    "# Neural Networks in Practice: Keras\n",
    "\n",
    "\n",
    "##### D.Vidotto, Data Mining: JBI030 2019/2020\n",
    "\n",
    "\n",
    "=================================================================================================================\n",
    "\n",
    "\n",
    "In this notebook, we are going to see how to setup and train Feedforward Neural Networks with the [Keras](https://keras.io/) deep learning library. Keras has two possible backends,  [Tensorflow](https://www.tensorflow.org/) and [Theano](http://deeplearning.net/software/theano/).  For this notebook we will use the Tensorflow version, and therefore Tensorflow is also required to run the code in the notebook. As of now (February 2020) Tensorflow is not supported in Python 3.8 (although a new release supported by Python 3.8 is expected in March 2020). Therefore, in order to run the code in the notebook you need to use Python 3.7 or earlier version. \n",
    "\n",
    "scikit-learn also supports methods for Neural Networks; examples of code with scikit-learn can be found in the notebook \"Neural Networkds in Practice: scikit-learn\". \n",
    "\n",
    "Here, we are going to set up Neural Networks for the following topics:\n",
    "\n",
    "1. Regression \n",
    "1. Classification\n",
    "  * Binary Classification\n",
    "  * Multiclass Classification\n",
    "1. Interfacing Keras with scikit-learn for network tuning\n",
    "\n",
    "This notebook is used with the following versions of Python and Keras (and Tensorflow 2.1.0): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.7.4\n"
     ]
    }
   ],
   "source": [
    "# Python version\n",
    "import platform; print(platform.python_version())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.3.1\n"
     ]
    }
   ],
   "source": [
    "# Keras version\n",
    "print(keras.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Regression\n",
    "For the regression example, we will use the [Boston Houses dataset](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_boston.html). The data contains information collected by the U.S Census Service concerning housing in Boston; the sample size is 506, and there are 13 features. The target variable is the median value of houses in different areas of Boston. More information about the dataset can be found [here](https://www.cs.toronto.edu/~delve/data/boston/bostonDetail.html).\n",
    "\n",
    "Let's load the data: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['data', 'target', 'feature_names', 'DESCR', 'filename'])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.datasets import load_boston\n",
    "bunch_data = load_boston()\n",
    "bunch_data.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD',\n",
       "       'TAX', 'PTRATIO', 'B', 'LSTAT'], dtype='<U7')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Feature names \n",
    "bunch_data.feature_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(506, 13)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X, y = bunch_data.data, bunch_data.target\n",
    "X.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Some preprocessing**..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train-test split (separate 10% of the data for the test set)\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_trainvalid, X_test, y_trainvalid, y_test = train_test_split(X, y, test_size=0.1, random_state=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Another split for the validation set (10% of the new dataset)\n",
    "X_train, X_valid, y_train, y_valid = train_test_split(X_trainvalid, y_trainvalid, test_size=0.1, random_state=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5.44520e-01, 0.00000e+00, 2.18900e+01, 0.00000e+00, 6.24000e-01,\n",
       "        6.15100e+00, 9.79000e+01, 1.66870e+00, 4.00000e+00, 4.37000e+02,\n",
       "        2.12000e+01, 3.96900e+02, 1.84600e+01],\n",
       "       [1.05393e+00, 0.00000e+00, 8.14000e+00, 0.00000e+00, 5.38000e-01,\n",
       "        5.93500e+00, 2.93000e+01, 4.49860e+00, 4.00000e+00, 3.07000e+02,\n",
       "        2.10000e+01, 3.86850e+02, 6.58000e+00],\n",
       "       [7.85700e-01, 2.00000e+01, 3.97000e+00, 0.00000e+00, 6.47000e-01,\n",
       "        7.01400e+00, 8.46000e+01, 2.13290e+00, 5.00000e+00, 2.64000e+02,\n",
       "        1.30000e+01, 3.84070e+02, 1.47900e+01],\n",
       "       [3.25430e-01, 0.00000e+00, 2.18900e+01, 0.00000e+00, 6.24000e-01,\n",
       "        6.43100e+00, 9.88000e+01, 1.81250e+00, 4.00000e+00, 4.37000e+02,\n",
       "        2.12000e+01, 3.96900e+02, 1.53900e+01],\n",
       "       [1.20742e+00, 0.00000e+00, 1.95800e+01, 0.00000e+00, 6.05000e-01,\n",
       "        5.87500e+00, 9.46000e+01, 2.42590e+00, 5.00000e+00, 4.03000e+02,\n",
       "        1.47000e+01, 2.92290e+02, 1.44300e+01]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT8AAAEvCAYAAAAzcMYwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAO3klEQVR4nO3dfYxl9V3H8fenbOmjZnkYEFl0aLLRYtOnTJCKMQhVqZDCH8XQVN1Uko0Jamtr2qX+QTRpAtH0wUSbbAp2myCF0FZIqbZkS1P9o1uHggW6RVa6wrrITlNo1Sat2379Y86mI8522XvP3dmd7/uVbO49v3vunN8vmX3vuXfm3E1VIUndPG+tJyBJa8H4SWrJ+ElqyfhJasn4SWrJ+ElqacNaTwDg9NNPr/n5+bWehqR15r777vtGVc2t9thxEb/5+XkWFxfXehqS1pkk/3a4x3zZK6kl4yepJeMnqSXjJ6kl4yepJeMnqSXjJ6kl4yepJeMnqSXjJ6kl4yeppePi2l7N3vy2u2d+jL03XDbzY0hj8cxPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS0eMX5KbkxxI8tCKsT9L8rUkX0nyySQbVzx2XZI9SR5J8muzmrgkTeO5nPl9BLj0WWP3AK+oqlcC/wJcB5DkPOBq4OeG5/xVkpNGm60kjeSI8auqLwDffNbYZ6vq4LD5RWDTcP8K4GNV9d2q+jqwBzh/xPlK0ijGeM/vd4C/G+6fDTyx4rF9w5gkHVemil+SPwYOArccGlpltzrMc7cmWUyyuLS0NM00JOmoTRy/JFuAy4G3VNWhwO0Dzlmx2yZg/2rPr6rtVbVQVQtzc3OTTkOSJjJR/JJcCrwbeGNVfWfFQ3cBVyd5QZJzgc3Al6afpiSN64gfZprkVuAi4PQk+4DrWf7p7guAe5IAfLGqfreqHk5yO/BVll8OX1tV35/V5CVpUkeMX1W9eZXhm37E/u8F3jvNpCRp1rzCQ1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS0eMX5KbkxxI8tCKsVOT3JPk0eH2lGE8Sf4iyZ4kX0ny2llOXpIm9VzO/D4CXPqssW3AzqraDOwctgHeAGwe/mwFPjTONCVpXEeMX1V9Afjms4avAHYM93cAV64Y/2gt+yKwMclZY01WksYy6Xt+Z1bVkwDD7RnD+NnAEyv22zeMSdJxZewfeGSVsVp1x2RrksUki0tLSyNPQ5J+tEnj99Shl7PD7YFhfB9wzor9NgH7V/sCVbW9qhaqamFubm7CaUjSZCaN313AluH+FuDOFeO/PfzU9wLgW4deHkvS8WTDkXZIcitwEXB6kn3A9cANwO1JrgEeB64adv808OvAHuA7wFtnMGdJmtoR41dVbz7MQ5essm8B1047KUmaNa/wkNSS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPU0lTxS/KHSR5O8lCSW5O8MMm5SXYleTTJbUlOHmuykjSWieOX5GzgD4CFqnoFcBJwNXAj8P6q2gw8DVwzxkQlaUzTvuzdALwoyQbgxcCTwMXAHcPjO4ArpzyGJI1u4vhV1b8Dfw48znL0vgXcBzxTVQeH3fYBZ087SUka2zQve08BrgDOBX4SeAnwhlV2rcM8f2uSxSSLS0tLk05DkiYyzcve1wNfr6qlqvof4BPALwAbh5fBAJuA/as9uaq2V9VCVS3Mzc1NMQ1JOnrTxO9x4IIkL04S4BLgq8C9wJuGfbYAd043RUka3zTv+e1i+QcbXwYeHL7WduDdwDuS7AFOA24aYZ6SNKoNR97l8KrqeuD6Zw0/Bpw/zdeVpFnzCg9JLRk/SS0ZP0ktGT9JLRk/SS0ZP0ktGT9JLRk/SS0ZP0ktGT9JLRk/SS0ZP0ktGT9JLRk/SS0ZP0ktGT9JLRk/SS0ZP0ktGT9JLU31f3hIK81vu/uYHGfvDZcdk+NoffPMT1JLnvkdB47VGZOkH/LMT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLU8UvycYkdyT5WpLdSV6X5NQk9yR5dLg9ZazJStJYpj3z+yDw91X1s8CrgN3ANmBnVW0Gdg7bknRcmTh+SX4c+CXgJoCq+l5VPQNcAewYdtsBXDntJCVpbNOc+b0MWAL+Osn9ST6c5CXAmVX1JMBwe8YI85SkUU0Tvw3Aa4EPVdVrgP/mKF7iJtmaZDHJ4tLS0hTTkKSjN0389gH7qmrXsH0HyzF8KslZAMPtgdWeXFXbq2qhqhbm5uammIYkHb2J41dV/wE8keRnhqFLgK8CdwFbhrEtwJ1TzVCSZmDaT3L+feCWJCcDjwFvZTmotye5BngcuGrKY0jS6KaKX1U9ACys8tAl03xdSZo1r/CQ1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9TStP9p+bo2v+3utZ6CpBnxzE9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS1PHL8lJSe5P8qlh+9wku5I8muS2JCdPP01JGtcYZ35vA3av2L4ReH9VbQaeBq4Z4RiSNKqp4pdkE3AZ8OFhO8DFwB3DLjuAK6c5hiTNwrRnfh8A3gX8YNg+DXimqg4O2/uAs6c8hiSNbuL4JbkcOFBV960cXmXXOszztyZZTLK4tLQ06TQkaSLTnPldCLwxyV7gYyy/3P0AsDHJoQ9M2ATsX+3JVbW9qhaqamFubm6KaUjS0Zs4flV1XVVtqqp54Grgc1X1FuBe4E3DbluAO6eepSSNbBa/5/du4B1J9rD8HuBNMziGJE1llM/zq6rPA58f7j8GnD/G15WkWfEKD0ktGT9JLRk/SS0ZP0ktGT9JLRk/SS0ZP0ktGT9JLRk/SS0ZP0ktGT9JLRk/SS0ZP0ktGT9JLY3ykVbSsTS/7e6ZH2PvDZfN/BhaW575SWrJ+ElqyfhJasn4SWrJ+ElqyfhJasn4SWrJ+ElqyfhJasn4SWrJ+ElqyfhJasn4SWrJ+ElqyfhJasn4SWrJ+ElqaeJPck5yDvBR4CeAHwDbq+qDSU4FbgPmgb3Ab1TV09NPVTp2/LTo9W+aM7+DwDur6uXABcC1Sc4DtgE7q2ozsHPYlqTjysTxq6onq+rLw/3/BHYDZwNXADuG3XYAV047SUka2yjv+SWZB14D7ALOrKonYTmQwBljHEOSxjR1/JK8FPg48Paq+vZRPG9rksUki0tLS9NOQ5KOylTxS/J8lsN3S1V9Yhh+KslZw+NnAQdWe25Vba+qhapamJubm2YaknTUJo5fkgA3Abur6n0rHroL2DLc3wLcOfn0JGk2pvlPyy8Efgt4MMkDw9h7gBuA25NcAzwOXDXdFCVpfBPHr6r+EchhHr5k0q8rSceCV3hIasn4SWrJ+ElqaZofeKypY3HtpaTn5lj9fRzzemjP/CS1ZPwktWT8JLVk/CS1ZPwktWT8JLVk/CS1ZPwktWT8JLVk/CS1ZPwktWT8JLVk/CS1ZPwktWT8JLV0wn6en3Si8zMp15ZnfpJaMn6SWjJ+kloyfpJaMn6SWjJ+kloyfpJaMn6SWjJ+kloyfpJaMn6SWjJ+klqaWfySXJrkkSR7kmyb1XEkaRIziV+Sk4C/BN4AnAe8Ocl5sziWJE1iVmd+5wN7quqxqvoe8DHgihkdS5KO2qzidzbwxIrtfcOYJB0XZvVhplllrP7PDslWYOuw+V9JHpnRXMZyOvCNtZ7EDK339cH6X+N6Xx+58ajX+NOHe2BW8dsHnLNiexOwf+UOVbUd2D6j448uyWJVLaz1PGZlva8P1v8a1/v6YNw1zupl7z8Bm5Ocm+Rk4GrgrhkdS5KO2kzO/KrqYJLfAz4DnATcXFUPz+JYkjSJmf0HRlX1aeDTs/r6a+CEeYk+ofW+Plj/a1zv64MR15iqOvJekrTOeHmbpJaM3yqS3JzkQJKHVoydmuSeJI8Ot6es5RynkeScJPcm2Z3k4SRvG8bXxRqTvDDJl5L887C+PxnGz02ya1jfbcMP405YSU5Kcn+STw3b6219e5M8mOSBJIvD2Gjfo8ZvdR8BLn3W2DZgZ1VtBnYO2yeqg8A7q+rlwAXAtcPlh+tljd8FLq6qVwGvBi5NcgFwI/D+YX1PA9es4RzH8DZg94rt9bY+gF+uqlev+PWW8b5Hq8o/q/wB5oGHVmw/Apw13D8LeGSt5zjiWu8EfmU9rhF4MfBl4OdZ/uXYDcP464DPrPX8pljXpuEv/8XAp1i+sGDdrG9Yw17g9GeNjfY96pnfc3dmVT0JMNyescbzGUWSeeA1wC7W0RqHl4QPAAeAe4B/BZ6pqoPDLif6JZcfAN4F/GDYPo31tT5Yvirss0nuG64IgxG/R2f2qy46/iV5KfBx4O1V9e1ktasST0xV9X3g1Uk2Ap8EXr7absd2VuNIcjlwoKruS3LRoeFVdj0h17fChVW1P8kZwD1JvjbmF/fM77l7KslZAMPtgTWez1SSPJ/l8N1SVZ8YhtfVGgGq6hng8yy/t7kxyaF/8P/fJZcnkAuBNybZy/InJl3M8pngelkfAFW1f7g9wPI/YOcz4veo8Xvu7gK2DPe3sPw+2Qkpy6d4NwG7q+p9Kx5aF2tMMjec8ZHkRcDrWf7BwL3Am4bdTtj1VdV1VbWpquZZvnT0c1X1FtbJ+gCSvCTJjx26D/wq8BAjfo/6S86rSHIrcBHLn5LxFHA98LfA7cBPAY8DV1XVN9dqjtNI8ovAPwAP8sP3jN7D8vt+J/wak7wS2MHypZXPA26vqj9N8jKWz5ROBe4HfrOqvrt2M53e8LL3j6rq8vW0vmEtnxw2NwB/U1XvTXIaI32PGj9JLfmyV1JLxk9SS8ZPUkvGT1JLxk9SS8ZPUkvGT1JLxk9SS/8LvvfX3PIicooAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Histogram of the target\n",
    "plt.figure(figsize=(5,5))\n",
    "plt.hist(y_train)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Rescale data with min-max scaler\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "mms = MinMaxScaler()\n",
    "X_train_scaled = mms.fit_transform(X_train)\n",
    "X_valid_scaled = mms.transform(X_valid)\n",
    "X_test_scaled =mms.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Setting the Regression FNN**. We are now going to use Keras to set up a Regression NN. Let's review the components of a Regression network: \n",
    "* the input layer contains as many neurons as the number of features\n",
    "* as we are predicting just one continuous output, we can set up the network with one output layer\n",
    "* the loss function is the mean squared error \n",
    "* no activation function is used for the output layer\n",
    "\n",
    "In Keras, you build the network with the `models.Sequential()` function, which takes as input the layers you want to include in the network. In turn, these layers are declared sequentially, with the `layers.Dense` function. Here, we are going to create a simple network with one hidden layer containing 5 hidden neurons and one output layer. The first hidden layer will be given the ReLU activation function, and it also needs the number of neurons of the input layer:   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "    keras.layers.Dense(5, activation='relu', input_shape=X_train.shape[1:]), \n",
    "    keras.layers.Dense(1)\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The last layer is the output layer; it doesn't specify any activation function, and it is indeed composed of just 1 neuron in this example. Moreover, besides the ReLU function you can choose other activations from the Keras library. [Here](https://keras.io/activations/) you can find a list of activation function available in Keras. \n",
    "\n",
    "An alternative, equivalent way to specify the network just created is the following one: \n",
    "\n",
    "```python\n",
    "model = keras.models.Sequential()\n",
    "model.add(keras.layers.Dense(5, activation='relu', input_shape=X_train.shape[1:]))\n",
    "model.add(keras.layers.Dense(1))\n",
    "```\n",
    "\n",
    "We will only train networks with one layer in this example; if you want to add layers you can do as follows (here we add a second hidden layer with 4 neurons and ReLU activation): \n",
    "\n",
    "```python\n",
    "model = keras.models.Sequential([\n",
    "    keras.layers.Dense(5, activation='relu', input_shape=X_train.shape[1:]), \n",
    "    keras.layers.Dense(4, activation='relu'),\n",
    "    keras.layers.Dense(1)\n",
    "])\n",
    "```\n",
    "\n",
    "or, alternatively:\n",
    "\n",
    "```python\n",
    "model = keras.models.Sequential()\n",
    "model.add(keras.layers.Dense(5, activation='relu', input_shape=X_train.shape[1:]))\n",
    "model.add(keras.layers.Dense(4, activation='relu'))\n",
    "model.add(keras.layers.Dense(1))\n",
    "```\n",
    "\n",
    "...and so on. As you can see, adding a layer is as easy as adding a line of code. \n",
    "\n",
    "\n",
    "Let's go back to our single-layer model, and get a list of its (hidden and output) layers: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<keras.layers.core.Dense at 0x18aab159b08>,\n",
       " <keras.layers.core.Dense at 0x18aaad957c8>]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.layers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also get the layers name or fetch them by index: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'dense_1'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hidden1 = model.layers[0]     # Use model.layers[1] for the output layer \n",
    "hidden1.name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see how weights and biases are initialized: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 3.4500921e-01,  5.8017135e-02,  2.9814649e-01,  2.9226756e-01,\n",
       "         5.4160845e-01],\n",
       "       [ 4.2473042e-01, -1.4750713e-01, -4.4800287e-01, -5.1457298e-01,\n",
       "         2.3564065e-01],\n",
       "       [ 4.1132069e-01,  1.4079702e-01,  4.8360717e-01, -3.1908926e-01,\n",
       "        -5.1186383e-02],\n",
       "       [-2.9594600e-01, -3.9225459e-02, -2.9463777e-01, -3.7478381e-01,\n",
       "         6.9240510e-02],\n",
       "       [-5.1224232e-01,  2.3711658e-01, -5.4733098e-02, -3.0457562e-01,\n",
       "         2.7155495e-01],\n",
       "       [-3.4767610e-01, -4.4668612e-01,  4.7552896e-01, -4.9084458e-01,\n",
       "         1.3802874e-01],\n",
       "       [-1.9538987e-01, -3.5431072e-01, -3.2166567e-01,  4.4577003e-01,\n",
       "        -2.1970055e-01],\n",
       "       [-9.3907744e-02,  4.5959616e-01,  8.7464452e-02,  1.0315555e-01,\n",
       "        -2.1910092e-01],\n",
       "       [-5.2398902e-01,  3.1235868e-01, -2.8578433e-01, -9.7421408e-03,\n",
       "         2.9641128e-01],\n",
       "       [-4.9400330e-04,  6.2059700e-02,  4.3593729e-01,  3.5691756e-01,\n",
       "         2.2118855e-01],\n",
       "       [-1.3550258e-01, -1.3214111e-02, -2.6697955e-01,  2.9133165e-01,\n",
       "         4.8750174e-01],\n",
       "       [ 4.1614354e-01,  5.7224631e-01,  3.9354509e-01, -3.6274493e-02,\n",
       "        -5.1622671e-01],\n",
       "       [ 1.2766302e-02,  5.4085457e-01,  4.5910239e-01,  2.8779089e-02,\n",
       "         4.7506344e-01]], dtype=float32)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weights, biases = hidden1.get_weights()\n",
    "weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 0., 0., 0.], dtype=float32)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "biases # Initialized at 0 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step that needs to be performed is the compilation of the model, which defines the loss function, performance metrics, and optimization method of the model. In this example, we use the MSE both as a loss function and as the evaluation metric, in which case we don't need to specify the `metrics` option (if, for example, you want to use the MAE as metric, you can simply add the option `metrics=[\"mae\"]` to the inputs of the `compile` function). For optimization, we will use Stochastic Gradient Descent (technically the mini-batch version) as optimizer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=\"mean_squared_error\", optimizer=\"sgd\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this [link](https://keras.io/metrics/) you can find the available metrics in Keras; in [this one](https://keras.io/losses/), you can find the available losses; and in [this](https://keras.io/optimizers/), the available optimizers. \n",
    "\n",
    "The model is now ready to be trained. To do this, we use the `fit` function (exactly as in scikit-learn), to which we can also pass other hyperparameters such as the number of epochs and the batch size for mini-batch gradient descent (indicated with $n_B$ in the theoretical notebook of Neural Networks). Furthermore, we can indicate a validation set to assess model performance. (This argument is optional, but highly recommended to help with model tuning). If you didn't create a validation set before, you can alternatively let Keras create it before training; to achieve this, use the `validation_split` argument of `fit` (this has to be a float be between 0 and 1), denoting the ratio of units to randomly split as a validation set.    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "history = model.fit(X_train_scaled, y_train, epochs=1000, batch_size=32,\n",
    "                    validation_data=(X_valid_scaled, y_valid), verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can plot the training and validation MSE observed at each epoch: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe4AAAE9CAYAAADNvYHXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3wUZf7A8c+zm01Cb0pVKTaKCCo2VAxiO3s7Bc9eOH+WU+88FdvZ9fTU0ztEsWJBRc+CNEUk0ntvUkKoIZDeky3P74/Z2d3ZHsgmbPJ9v1682J2dmX12sjvf+T5tlNYaIYQQQiQHW0MXQAghhBDxk8AthBBCJBEJ3EIIIUQSkcAthBBCJBEJ3EIIIUQSkcAthBBCJJGURO1YKXUs8FXAol7AU8An3uU9gGzgWq11YbR9HXLIIbpHjx4JKacwlJeX06JFi4YuRqMmx7h+yHFOPDnGibd06dI8rfWh4V5T9TGOWyllB3YBpwL3AAVa65eVUo8C7bTWj0TbftCgQXrJkiUJL2dTlpmZSUZGRkMXo1GTY1w/5DgnnhzjxFNKLdVaDwr3Wn1VlQ8DtmittwGXA+O8y8cBV9RTGYQQQoikV1+BezjwhfdxJ611DoD3/471VAYhhBAi6SW8qlwplQrsBvpprXOVUkVa67YBrxdqrduF2W4kMBKgU6dOJ3355ZcJLWdTV1ZWRsuWLRu6GI2aHOP6Icc58eQYJ97QoUMjVpUnrHNagD8Ay7TWud7nuUqpLlrrHKVUF2BvuI201mOBsWC0cUt7SmJJm1XiyTGuH3KcEy+eY+x0Otm5cydVVVX1U6gklZ6ezmGHHYbD4Yh7m/oI3CPwV5MDTARuBl72/v9DPZRBCCFEPdq5cyetWrWiR48eKKUaujgHJa01+fn57Ny5k549e8a9XULbuJVSzYHzgG8DFr8MnKeU2uR97eVElkEIIUT9q6qqokOHDhK0o1BK0aFDh1rXSiQ049ZaVwAdgpblY/QyF0II0YhJ0I5tf46RzJwmhBBCJBEJ3EIIIUQSSY7AXVPe0CUQQgjRiEUb3padnc1xxx1Xj6WJLjkCd2XUqcyFEEKIJqM+hoMdOI+roUsghBBiPz3z41rW7S6p03327dqaf1zaL+LrjzzyCN27d+fuu+8G4Omnn0YpxaxZsygsLMTpdPL8889z+eWX1+p9q6qq+L//+z+WLFlCSkoKr7/+OkOHDmXt2rXceuut1NTU4PF4+N///kfXrl259tpr2blzJ263myeffJLrrrvugD43JE3gdjd0CYQQQiSR4cOH88ADD/gC94QJE5g2bRoPPvggrVu3Ji8vj9NOO43LLrusVj27R48eDcDq1avZsGED559/Phs3buSdd97h/vvv509/+hM1NTW43W6mTJlC165dmTx5MgDFxcV18tmSJHBLxi2EEMkqWmacKCeccAJ79+5l9+7d7Nu3j3bt2tGlSxcefPBBZs2ahc1mY9euXeTm5tK5c+e49ztnzhzuu+8+AHr37k337t3ZuHEjp59+Oi+88AI7d+7kqquu4uijj6Z///489NBDPPLII1xyySWcddZZdfLZkqONW0vGLYQQonauueYavvnmG7766iuGDx/O559/zr59+1i6dCkrVqygU6dOtZ78JNL9Pa6//nomTpxIs2bNuOCCC/j111855phjWLp0Kf3792fUqFE8++yzdfGxJOMWQgjROA0fPpw777yTvLw8fvvtNyZMmEDHjh1xOBzMnDmTbdu21XqfQ4YM4fPPP+ecc85h48aNbN++nWOPPZasrCx69erFX/7yF7Kysli1ahW9e/emffv23HDDDbRs2ZKPP/64Tj5XkgRut/HPZm/okgghhEgS/fr1o7S0lG7dutGlSxf+9Kc/cemllzJo0CAGDhxI7969a73Pu+++m7vuuov+/fuTkpLCxx9/TFpaGl999RWfffYZDoeDzp0789RTT7F48WL+/ve/Y7PZcDgcjBkzpk4+V3IEbjCybgncQgghamH16tW+x4cccgjz588Pu15ZWVnEffTo0YM1a9YAxt28wmXOo0aNYtSoUZZlF1xwARdccMF+lDq65GjjBkjwfcOFEEKIZJA8Gbf2NHQJhBBCNGKrV6/mxhtvtCxLS0tj4cKFDVSi8JIncCMZtxBCiMTp378/K1asaOhixCRV5UIIIUQSSaLALVXlQgghRPIEbqkqF0IIIZIocEvGLYQQohai3aozmSVR4JaMWwghhEiiXuVCCCGS0tRHYc/q2OvVRuf+8IeX41pVa83DDz/M1KlTUUrxxBNPcN1115GTk8N1111HSUkJLpeLMWPGMHjwYG6//XaWLFmCUorbbruNBx98sG7LfoCSJ3BLVbkQQoj98O2337JixQpWrlxJXl4eJ598MkOGDGH8+PFccMEFPP7447jdbioqKlixYgW7du3yzZRWVFTUwKUPlUSBW6rKhRAiKcWZGSfKnDlzGDFiBHa7nU6dOnH22WezePFiTj75ZG677TacTidXXHEFAwcOpFevXmRlZXHfffdx8cUXc/755zdo2cNJojZuybiFEELUXqRbcQ4ZMoRZs2bRrVs3brzxRj755BPatWvHypUrycjIYPTo0dxxxx31XNrYkidwy3AwIYQQ+2HIkCF89dVXuN1u9u3bx6xZszjllFPYtm0bHTt25M477+T2229n2bJl5OXl4fF4uPrqq3nuuedYtmxZQxc/hFSVCyGEaNSuvPJK5s+fz4ABA1BK8corr9C5c2fGjRvHq6++isPhoGXLlnzyySfs2rWLW2+9FY/HqOV96aWXGrj0oZIocEtVuRBCiPiZt+pUSvHqq6/y6quvWl6/+eabufnmm0O2Oxiz7EBSVS6EEEIkkeQJ3FJVLoQQQiRT4JaqciGESCaRenMLv/05RskTuKWqXAghkkZ6ejr5+fkSvKPQWpOfn096enqttpPOaUIIIercYYcdxs6dO9m3b19DF+Wglp6ezmGHHVarbZIocMtVmxBCJAuHw0HPnj0buhiNUkKrypVSbZVS3yilNiil1iulTldKtVdKTVdKbfL+3y6RZRBCCCEak0S3cb8JTNNa9wYGAOuBR4EZWuujgRne57FJVbkQQgiRuMCtlGoNDAE+ANBa12iti4DLgXHe1cYBV8S1Q6kqF0IIIVCJ6vGnlBoIjAXWYWTbS4H7gV1a67YB6xVqrUOqy5VSI4GRACd1sZ309nsfUdHiiISUVRgzDLVs2bKhi9GoyTGuH3KcE0+OceINHTp0qdZ6ULjXEhm4BwELgDO01guVUm8CJcB98QTuQIO62vWSFWugY5+ElFVAZmYmGRkZDV2MRk2Ocf2Q45x4cowTTykVMXAnso17J7BTa73Q+/wb4EQgVynVxVuwLsDeuPYmVeVCCCFE4gK31noPsEMpdax30TCMavOJgDmr+83AD/HtUDqnCSGEEIkex30f8LlSKhXIAm7FuFiYoJS6HdgO/DG+XUnGLYQQQiQ0cGutVwDh6uiH1X5nknELIYQQyTNXubRxCyGEEEkUuKWqXAghhEiiwC1V5UIIIUQyBe6GLoAQQgjR8JIocEvGLYQQQiRP4JaUWwghhEiiwC29yoUQQohkCtxSVS6EEEIkT+CWqnIhhBAiiQK3ZNxCCCFEMgVuybiFEEKI5AncUlUuhBBCJFHglqpyIYQQIpkCt2TcQgghRBIFbsm4hRBCiKQJ3B6PZNxCCCFE0gRuLRm3EEIIkTyB2+2RwC2EEEJI4BZCCCGSSPIEbrcEbiGEECKJAre7oYsghBBCNLjkCdzSOU0IIYRInsDtkYxbCCGESJ7A7ZI2biGEECJ5ArdHepULIYQQyRO43R6pKhdCCCGSJnDLlKdCCCFEEgVuGQ4mhBBCJFHgljZuIYQQIokCt1uqyoUQQojkCdwyjlsIIYRIosAtNxkRQgghICWRO1dKZQOlgBtwaa0HKaXaA18BPYBs4FqtdWGsfXlkOJgQQghRLxn3UK31QK31IO/zR4EZWuujgRne5zFJG7cQQgjRMFXllwPjvI/HAVfEs5H0KhdCCCFAaZ24TFYptRUoBDTwrtZ6rFKqSGvdNmCdQq11uzDbjgRGApzUxXbS1Ht6s/aM0Qkra1NXVlZGy5YtG7oYjZoc4/ohxznx5Bgn3tChQ5cG1FRbJLSNGzhDa71bKdURmK6U2hDvhlrrscBYgEFd7fpQ504yMjISVEyRmZkpxzfB5BjXDznOiSfHuGEltKpca73b+/9e4DvgFCBXKdUFwPv/3kSWQQghhGhMEha4lVItlFKtzMfA+cAaYCJws3e1m4Ef4t5p7to6LqUQQgiRXBKZcXcC5iilVgKLgMla62nAy8B5SqlNwHne5/EZMzgR5RRCCCGSRsLauLXWWcCAMMvzgWGJel8hhBCiMUuamdOEEEIIIYFbCCGESCoSuIUQQogkIoFbCCGESCJJF7gXZxc0dBGEEEKIBpN0gfuP78xv6CIIIYQQDSbpArcQQgjRlEngFkIIIZKIBG4hhBAiiUjgFkIIIZJIUgTuEuNeJUIIIUSTlxSBu5rUhi6CEEIIcVBIisANqqELIIQQQhwUkiNwS9wWQgghgGQJ3BK5hRBCCCBpArcQQgghIAkD98a0G6E8v6GLIYQQQjSIJAnc/qryVOWGnYsbsCxCCCFEw0mOwB3cxK2So9hCCCFEXUvOCKiks5oQQoimKUkCd1CglsAthBCiiUqOwC1V5UIIIQSQLIE7JONOkmILIYQQdSwpImBIxbgEbiGEEE1UkkRAybiFEEIISJbALW3cQgghBJAsgTuYBG4hhBBNVJJEQBn+JYQQQkDSBO4g2tPQJRBCCCEaRHIE7uAJVyRwCyGEaKKSInCHVJRL4BZCCNFEJTxwK6XsSqnlSqlJ3uc9lVILlVKblFJfKaVS49iL9akEbiGEEE1UfWTc9wPrA57/E3hDa300UAjcHnMPwSm3BG4hhBBNVEIDt1LqMOBi4H3vcwWcA3zjXWUccEXM/QQvkMAthBCiiUp0xv1v4GHAjLQdgCKttcv7fCfQLeZeQjqn6ToroBBCCJFMUhK1Y6XUJcBerfVSpVSGuTjMqmGjsFJqJDASoHPnjpbXVq1aScEuR90VVlBWVkZmZmZDF6NRk2NcP+Q4J54c44aVsMANnAFcppS6CEgHWmNk4G2VUinerPswYHe4jbXWY4GxAL169dRQ5Xvt+OP6wbEZCSx605OZmUlGRkZDF6NRk2NcP+Q4J54c44YVtapcKXVDwOMzgl67N9q2WutRWuvDtNY9gOHAr1rrPwEzgWu8q90M/BCrkME15dLGLYQQoqmK1cb914DH/wl67bb9fM9HgL8qpTZjtHl/UOs9SOAWQgjRRMWqKlcRHod7HpHWOhPI9D7OAk6Jd9uwbySBWwghRBMVK+PWER6He15/JHALIYRoomJl3L2VUqswkt4jvY/xPu+V0JIFUMGN3LuWQb8r6+vthRBCiINGrMDdp15KUVvz3oIhD0F6m4YuiRBCCFGvogZurfW2wOdKqQ7AEGC71nppIgtmed9wC51VEriFEEI0ObGGg01SSh3nfdwFWIPRm/xTpdQD9VC+KGT2NCGEEE1PrM5pPbXWa7yPbwWma60vBU5l/4eD1V7Y+dYkcAshhGh6YgVuZ8DjYcAUAK11Kf75xxMubFW59CwXQgjRBMXqnLZDKXUfxs1ATgSmASilmgENO1m4xxV7HSGEEKKRiZVx3w70A24BrtNaF3mXnwZ8lMByWYTNuCVwCyGEaIJi9SrfC9wVZvlMjDnH64c3cm/wHE5v2w7jicddb28vhBBCHCyiBm6l1MRor2utL6vb4kQohzdy68DcWzJuIYQQTVCsNu7TgR3AF8BCajE/eSJ4JHALIYRo4mIF7s7AecAI4HpgMvCF1nptogsWyAzXEriFEEI0dVE7p2mt3VrraVrrmzE6pG0GMr09zeuNO6UZNdrOaNcV/oXSxi2EEKIJipVxo5RKAy7GyLp7AG8B3ya2WEFsdo6p/pTWlPuXScYthBCiCYo15ek4YB7GGO5ntNYna62f01rvqpfS+UsCxFdVvnRbIZePnkuVUzJyIYQQjU+sjPtGoBw4BvhLwO01FaC11q0TWDYf8209gdcZEQL3k9+vYV1OCZv3lnFcN7kJiRBCiMYl1jjuWBO01IvwndMkoxZCCNH0HBSBOV4yjlsIIURTlxSB26yij1lV7nYxuGYecstPIYQQjVVSBG5TzM5pc9/gifIXucC2uP4KdZCYsT6XLxdtb+hiCCGESLCYw8EOBmbntJhV5UXGPOYdVGk9lOrgcvu4JQAMP+WIBi6JEEKIREqKjLu2ndOkolwIIURjlRSB2x+6Y3VOk5AthBCicUuKwK3C3dokSq9y3bD3QhFCCCESJikCd1jhArc2Mu5nUz6i5Y76u124EEIIUV+SInCHzZ+jtHGnKjc9pt2csPIIIYQQDSU5AnfcVeXSxi2EEKJxS5LAHSZyxzFz2rLthQkojRBCCNFwkiNwh1vocYGz0vhnCkq47/18WSKLJYQQQtS75Ajc4SL3jGfghc7w8hHgqq73MgkhhBANITkCd7ThXe4a2Pe794k15ZYWbyGEEI1NwgK3UipdKbVIKbVSKbVWKfWMd3lPpdRCpdQmpdRXSqnU2DuL8brHGXax1jB+4Xamr8utdfmFEEKIg1EiM+5q4Byt9QBgIHChUuo04J/AG1rro4FC4PZYO4o5nYrb21FNh+bYj323mjs/WVKbcgshhBAHrYQFbm0o8z51eP9p4BzgG+/yccAVB/xmkTJuqSwXQgjRyCS0jVspZVdKrQD2AtOBLUCR1tocy7UT6HbAb+Q2A7cEaiGEEI1bQm/rqbV2AwOVUm2B74A+4VYLt61SaiQwEqBTp06ke5f/peZe3kr9LwBuWyp2Tw2rViyjYIei9549dA7YR3V1je9xZmbmgX6cpLC/n7OsrKzJHKOGIse4fshxTjw5xg2rXu7HrbUuUkplAqcBbZVSKd6s+zBgd4RtxgJjAQYNGqTzvMsnegYz0jOJ42zZ2FObQ1UNx/frA70zoOALCOiH5khNhWpjqFhGRkZiPtzBYtpkYP8/Z2ZmZuM/Rg1MjnH9kOOceHKMG1Yie5Uf6s20UUo1A84F1gMzgWu8q90M/FDbfVfjMB44mhn/e6SqXAghRNOQyIy7CzBOKWXHuECYoLWepJRaB3yplHoeWA58UNsdV2tv4E7xVqC7Iw8HE0IIIRqThAVurfUq4IQwy7OAUw5k3yEZt6sqUikO5G2EEEKIg05SzJwWrBrvnC1mxv3DPVBRICm2EEKIRi9JA3dQxg2w9Tco2mZZT+K4EEKIxiY5A3dwGzfA17fAjoWW9U7Wq+uvUEIIIUQ9SJrAbbf5Jz4Nm3GH8Y7nmUQWSQghhKh3yR24AzNuIYQQoglImsCdEjbjlsAthBCiaUmawG1XAYFbe3uV22PfEVQIIYRoTJImcNvCZdwedwOVRgghhGgYSRO4w1aVxxG4L7HNJzv9eijbl6iiCSGEEPUmaQJ3usPue+wP3K4Ia/vdljLVeJC/ORHFEkIIIepV0gTuNs0cvsdVZht3HIE7De885rZ6uRGaEEIIkVBJE7jbtfAHbqd3ivXNucUxt0vHe0/ugM5tQgghRLJKmsDdtrm/B7kTo9p8856imNs1U8b9uHFVH1gBXNXgcVPtcvOfGZuodh2cHeO0zPMqhBCNWtIE7nbN/Rm32xu47bj55Kg3o27ny7gj3kEsTs93hAk3MW5eNq9N38iHc7IPbH8J4pG4LYQQjVrSBO6eh7T0PXZ5i52Cm6zWJ4OK/DHSzTbuvI0HnnVvmERFjZFpV9bEbl9vCJJxCyFE45Y0gfuWwT18j82MOwVvdbX2RNyuuVlVPu1R4/afB0hxcLeVS9gWQojGLWkCt92muLBfZwBcvsAdOWCHtfmXui7WQccjGbcQQjRqSRO4wR+U3Nootl0dnB3EGpLEbSGEaNySKnCbMWk3HQBY5Oldyz0c3NXcdUECtxBCNG5JNSuJGZR26o6cXf06O3RHbqrNDpQCt7ezmt0Rfd1YZTmgrRNHH7QlE0IIUReSKuM+on1z3+NtujOeWhdfwcvd4b8n124zj78t/WCfx0WGgwkhROOWVIH7kT8ci912gJHTWQ6FW2u3jU6etnQZDiaEEI1bUgXutBQ7Gcccuv87qE26nLcZlnxkPE6i24dK2BZCiMYtqdq4Adx1lVG6ndHbud8fBlVFcOJNyZVx13KEnBBCiOSSVBk3gDtaI+4tU2JsHZBxl+6BdRNhysNQFeZmJVXeedCdlWGjYW2uH5ZuK2Delrz4NwgwadVurn13ftzrS+c0IYRo3JIu4446wUinfiGLsjyd6WXbYzwJrCovzIYJNxqPjzwHjr3QuqGyG5m2s9KSme9PC/vVY4zAm/3yxbXe9t7xy2u1vnROE0KIxq1xZdxhqr7X6R4BzwLCbvle/2Nneei+zPt3OyvCZty3rrgOfn0hemHrkCfOiCyd04QQonFLusDtidaGa0+1PK3UqTzuvC38ulUl/sfOytDXfYG7MmzntA6VW2HWKzFKW3fibduXjFsImL8ln4VZ+Q1dDCESIukCd3AAs2SYNmvN/9OumymmJWFVl/of11SEvm4PzLgbvnNa1JqGANLGLQSMeG8B141d0NDFECIhki9whwtgKc2M/4OGe+Xo9tb1PE7/48DA7QwTuG3eavegzmkHMgFLvME3HFe820rcFkKIRi3pO6ct31EEd82BXUtD1s3RHawL3IGBO7Cq3Ajc63NKOPLQlqSm2GJWle+P8hoXrdP3b6pVt1uqyoUQQiRhxh0cuFftLGZBSTsYcB0AN3T4wvfanuCM213jf2ypKi8np7iSP7w5m39MXGsss9V9VXlZlWu/t423jVuqyoUQonFLWOBWSh2ulJqplFqvlFqrlLrfu7y9Umq6UmqT9/92tdmvO0zntD3FVb7HpbY2XFP9FJ+7hlFKc+uKETPuSoorjdeWbSs0ltnsvteCM25V2/uAe5VXhwbuKqeb/LLqmNu6ovbK85OMWwghGrdEZtwu4G9a6z7AacA9Sqm+wKPADK310cAM7/O4ueMIYEt0bx533R76QmDmXGWtKg9JaKMMB0vZz8CdX14TsuyG9xdy0vO/RNzGbFOPu3NalMy8vNpFaZUz4utCCCEOfgkL3FrrHK31Mu/jUmA90A24HBjnXW0ccEVt9hsufgV2GIu771jMzmnhA7dSCju1qzpv19xo1567OXT2tCVmhh+B+XlccbZxR6tRP/6Zn+n/9M9x7UcIIcTBqV46pymlegAnAAuBTlrrHDCCu1KqY4RtRgIjATp16kRmZiYAZWWhQXbduvW0KdoEQElJ6Jhsj1bYVFBE270MAJe9BSU521m+aC42PJSXl5GZmcmgyipaAlkb15Kf3xLzRqBZWVk4AgK3Wa5oUrRRRT5nzVZOSs0Ju06s/cxfsIAtzWNfZ0Vbz8zabxn9E7f0S7O8VlZWFtdnEftPjnH9KCsrw7zkleOdGPJdblgJD9xKqZbA/4AHtNYlKs7xVFrrscBYgEGDBumMjAwA0hbPhApr8O7btw8ZA7sB8O+1c6G4yPK6Gxu2CFlyyuEn0j5nFdcvH45OGcYnLR4gI2MIrGsB5dDrsM706nciLDHW79WrF/ZNS3zbm+WKJnXBrzztHENb3Z6MjHetL06bHHU/tp+n4NGak04+hSMPjTAmPWA/p556Kt07tIi6TuYOFx/fc4HlpczMzLg+i9h/cozrhxFQjNkQ5XgnhnyXG1ZCe5UrpRwYQftzrfW33sW5Sqku3te7AHsjbR9OvFXGgZbpoyO/2PtiqDZuMvKnlBn+5WZHtqDOaTaP05Jxx8Pt0dyS8jNXlH5Zu7uTUPs2bumcJoQQjVsie5Ur4ANgvdb69YCXJgI3ex/fDPxQm/2G63zljBHM76h5iCodYfx0u56+hy5tMwLrqgmQv8m7c+twsP+bfRqX2uO/W9ee4ir2lPh7vVO6J+x6kTqVKW+VX/xt3BK5hRCiMUtkxn0GcCNwjlJqhfffRcDLwHlKqU3Aed7ncQs3nrnGFb2XdynNKcM7u9qtU+HSN32vjV3pD6opysPnpbdAZkCRnJUhWfJptnVxl3fwyzOsC8J1hAMWZBXQ49HJ5AYGefwZd+D49YoaF394czbLtod2bJOMWwghGrdE9iqfo7VWWuvjtdYDvf+maK3ztdbDtNZHe/8vqN1+Q5fVuGJXXafjHYrV9QQ46Rbf8reXWQPlIboACrb4389ZETKOO2R8eBQhgdQdOhyrl9rNrzOmArBoq/VwmIE7cMrTzXvLWJ9Twt+/XhnmHSVyCyFEY5Z0M6d9evup9OnS2rKsJtysLEGuq3kKrhgDjmaW5cVE6Mjl23llyMxpV9tnx1fYcNyhY7l/TXuIx3ffA4TOhW5WlQeOX09LMSaH2bKvnDvGLabK6S9ffWTcHo+Oa9IYIYQQdS/pAvexnVvx9KV9LctiVZUDrNU9YOD1Ics1NkrTu0TcLlzGfSCcNdEDngoaie7LuAPauANnUftl/V4WBmTp9dHEPXrmZk56/hdyisPcDlUIIURCJV3gBkixW4NbPIE7mmkZk0KWfeo6l7nufti2z4NVXx7Q/gNVVVfC5L/B+OFhXw/OuG3eBYFt+8GTxwVuUh9zlf+yPhewTjUrhBCifiRn4LZZi10dUFUeT9hyuT383TmSX9wnAJCa3owtp71kWWez7kaRWY2+/LMDKm+gqupqWPw+bJwa1/pmUA4cDhbthiNxTmkuhBAiSSVl4LbbrGnpu79l8f7sLONJHHXFNW4PX7szuMP5d8AIinuOupYnnLf61nFjo5rUWpVr7uY89pWGVoXbAuY218742obzyqrJK6v2Re7AzmnB87UHZun1kXGb7xDvZDpCCCHqTlIG7uCqcoDnJ68H4uucFVy17vJo3B5NVVCgXuYJmrglY1TU/f7p/YVc++58Y6z2D/dAjTH0y4H/rmAeV0DntOWfE1xHYH6yQc//wqDnf/E991gCd+Qy1Ecbt/keEraFEKL+JWfgtkUOGfFknNVBgdvt0bi1xqP9+3BAD5MAACAASURBVP3RfTqfu4dROjjg5mXHWKcJBeDpNoA/sG7NKzeC9vLPYNtcwBq4tSsg4/7hbk4PGhMe0sbt/azWjNv6GbWGPmob2enX0yxncbiPXKfMYywJd9Mze9O+sPMHJAOttUxQJBqFpAzc9oA27lZp1unW42njDZdxezzafyOSASMopiUaG3MODehEpuzhd1hRgDPwjXev8O7YCNIpAVOkWjJuoAMlWKmwzwKDtSfMyWeIbRUArbb9FL6MCSDnwKbnxg8WcdXb8xq6GPul56gp/PGd+Gc9FOJglZyB25vqpabYfBmpKZ5YEpJxuz2MnZWFMrdW/sPyv1X5/hVtEQL3vt+DsmDv4wpjW0vGHTQBSzMVY3iYsmbceWXV/LrBOr17Q8XPwFqAmb/v5dp35luq9IU42MS6ja4QySApA3dqilHsq0/sRnCteTxVYcEZd43bw8KtBbi193Ck+G95acluVYTDVbDFMl+62zysvsAdkHE7rUOo2uG/L3hn8sNMwOLdzhsQb/5wER/M2WpZJzBY1mcbd+DFyl/GL2dRdgFlNa4IWwkhhKgLSRm4O7dJZ9J9Z/Lc5ceF9DCPJ3AFz7RWUmkEm0me0/k27XIY9pTvNWvgtmbcbrt3FrbKIjz5W5ma+ggX2xZQVm7cUtAXuFVAMAuaOa2j8t+CdEH6fbQqDN/mbWa3W/aVhXwel0f7agt0PXQZMw9J4EQwOug1IYQQiZGUgRvguG5tSLHbfBOUgFFdG61z2uiZmwGodlpnQiuuNKqva3Awttkd0Kyd77XM3/dRbd5ZLKiqfMG53xgPfn6cZplP08e2gzNtq2mJd0axCmNGs8A2bu2yVpXfkWIdz92ieLPluVlVbg4Bc9hC/2SuOKZ8rUvmEQ53q9F4bz8qRKKE6wMiRGOStIHbFJhx3/rRYjbmhmakpve8Y71DMu6q0Bt/BNqH0XP8kv/M8i37p3M4pS39twRN3zwZgBEpM7GbndyqjY5ngVXlOsxc5YG0x1oW89OZGbc9zFA4p8efZ9f2lFXldPPLutxabWM2R/y01n+LUl856/kiItEmr8qhx6OT2VkY/q5uTUVheQ3HPhHfpEENTeJ2w8sprsTZyM4FB5OkD9y2OMYkndunI2D8oKeszqHKaf1CmRk3GJ2/HvxqheX1ue7jACirdvOPIyeQUf0aY9yX4dIx3tsZOo473E1GLIJe993W0xu4g2eNA2uwrO0564XJ67njkyVkFdV+PvbPFmz31WKY7+uqw4x75Y6iBpkPffXOYp6euBatNd8u2wnA+pzSGFs1bku3FYZ06oxXYXkN5dX11/dBwkXDqqhxcfpLv/LYt6sbuiiNVtIH7mapEXp6BzA7jhVXOrn782WMm5fte81uU0GBu4bvlu+ybP+k61ZG1DxOtu5CVk1bsrVxU5KY1cI1FbSgkq4qz78sZuAOzrgV59mWoKqKgfBj2ANvQFLbdCM732iPL3fuX8B99affI5flAF0+ei5n/nNmne0vFqfbw/iF27l6zDw+npdNZUCTSlMfsn4gf9UTnpvOOa9l1lVRYtrfjPulqet5O3Nz7BVFVGZiZN7TIF4yzj5+SR+437tpEEceaswp3rVNuuW1Q1qm0cxhD+lFvr3AyIT7d2tDSlDgDqcGB/M9/QDIL/MH3miBu1Q3gz2rWJt+O++m/tv/QsA47vnuvnzjHmLZTrlrLOt0UoW8l/o6Q1YbE8EEd8YDcHo8+905zTcLWpTNdhVV8uYvm+L6UbnqeLL0cMfY49F8Mj+bypq6u2sbwLh52Tz23WpfU0pt2uvfm5XFyh1FsVdsonJL6u82sB7LdWz8f8N3f8vilWm/x15RxKW2lW+3fryYnqOmJKYwjUzSB+6eh7Tg9WsHAtC6mYMFo4b5XpvzyFCWPXke1S7rCd58PuaGE0mxKUpiBO5A+eX+E1C0auFd+hBfVXkgm7Pc97iMdJ513kiJbu5bNmDdq/D8ob7nLbzjvNMKNzFp1e6w070GZrm1vV71d+aLHLnv/mwpb/yy0dejPdq50DwmC7Py2Z6fmHbh6etzeeqHtSHZfiTl1S4GvzSD+Vvyo65XEXQhUJvj+sKU9Vw+em5c5RGJFfizlL6S9c+8eK9t9pz5+75EFKdRSvrADdClrZFpn9e3E53bpNOuudELPN1hp1mq3TLGGvxVOakptpCq8lgKyv3Z8LQ1e+CP43zPX3SO8D3eqQ8J2dajFc3Ktvmel5NOCS140nlLyLoKDxfbFnCKeykA2uPi3vHL2VVobfO9wLaYy+dcgZ3o2WekiVHimXe83BvQzHWj9dw3O6RcN3YBQ15NTDV3lbcKe19Z9Cxub0kVBeU1bNhTwu7iKl75aUPU9Zs5rM0u8Xaukeq9yBpiQp7AdzRrTRrD36jHo5P557To3+GDge+YN3A5GrNGEbg7tkpn4WPDeODcYwD48b4zeXP4QN/rwVXlZsadZreTYreFBPZoAtf9dcNe9h5xIdiMaVfX6h6+13aHCdzzPX1pUZbte16ujXHgbkLb6ZtRw+jUt3jA+QGALzAHZ/mvOt6lbcVWWisju9VhOsztLqpkWkAP8ED+YBzevtJqNu+N3FPf2EfkedQDLd1WyCt1cOIxmwuC75IW7JQXZ3Dic9N9N2Wxx+jImB7UX8Lp0XHNx76/nbaagvIGmJDHmnEbT2rT7DHqIOxUZV6sjsnc0sAlic1XUyWRO2EaReAG6NQ63XdCP6xdcy4f2M33WvDwLzPjTk+1hW0zro0NOaXorsZ9vV3aP296Kc1C1l2vj6B5pT+AlmHUFDjDBO5htmWW5ykR+sqabdvNMbJPrUMz76vensfdny/z7sdFX5Udsk7gPb7X7S5h1Ler8Xg0L0/1B9pI577Ai4loF0FXj5nH25lb8Hg0e4qreP3n36PUBETej9lBL96OcOZJe8m2wqgZYJrd+nNwuT2+C5toY4PrI3An61SyZXXYm7zHo5MZ9e2qmOsF/qn8kwXFf/y+WLSdHo9OZsrqnNoWMSG01pRWGccx1X7wn7JdknEn3MH/LagDwRk3GNWiaSn2mFlYLDd9uIjPD/8H+3pczkrdi1WeXoA/kG7zGEPRPI7mbNSHWbad4j4VAFeYwP2f1P9anrdTZYxzvAzAebYltKOEI1QurZRRdd7S+7/LGdprfU+Jf5rVUSlfMCXtMRaPudPoxen9eQUeols+WsQXi7aTW1pluegxq46DY1hglbLL7YmZ3VQ63dz/5XLe+nUza3YXh11nZ2HkYWDmTWbizaICg+5dny1l897wQ7uCT+7WzxUtcNdtJ7lgS7cV0uuxKSzaWpDQ94lmf6uay6oOLHBvzSunx6OTmeHtofzFoh3cMW4Jl/xndsRtAv/e7v3IuE3vzsqq9TbR/LBiF0UVMUaVBFmzq5ieo6b4LiLSHYk9ZeeWVB1wp8/GNpfDwahJBO5wGZHZDn6gGTfAN5ttrBv8GlWkca3rGfpUfUgPZWTWz7pu5MuTvqDouomW+3uXnXQPK/VRALhICbvfYGfbV3EoRi/zNxxjmJX2oO81c7Y2lzN6u+9Jto0AnJw7gT0F/qAZ+Fvztwtaf4RmEA8+BTrd/mEcbo/2VetFUlHj9nUEixQPznolcvu4+SeLN4sKDKw/r8vl3NdnWV7XWvP+7Czf0DhTYO1BtN7y1c66P1Fd+fZc/v71SgDmbzGGEwbfXCbQ0hi1CVVO9wGdkPc3eyoNk3F/OGcr3wcNuYxkxQ7jpiATV+72LftlfS5rdgXfVc8vsKxmEN+f+QXq4NTgsy2/nPu/XMH9X66IvbKXx6NZkGV0qDTnE4hn+OuBOPXFGYx4b8EB7cN1gP0KGkN/hERrEoG7JkxG5DQnNAnTS7u2XB6Pr73VSQqVpPOKazj53S9ijqc/JW2ORXUZwBbdleLm3QGYvtZ/4nIF/Bl2p/WK+l49vRcEGfaVluXRMu5AKuC0tnnnXn9VYsBvxfzhzdmUZ6nqrHF5WLqtMKTNe8LiHb4ObJ8u2Mb7s7f6Xvtu+U7emL7Rsn5FjeuA7ultBtR4s6hdRVVRX9+WX8Hzk9czNijDCsyyw9XaUFMOFQUJqSpfvr2Ir5caJ2vzpjqRMpkfVuzi6jHz+GFl5GCY8WomfZ6aFvU9XW4P09bsCXvi3J+M9dkf14W9Beizk9bxwFfxBTCzKLWZhcsyHMy72f6UP57JneJlzglQmwmF/vb1Sp6fvB6AIm8HWrMD5bLthWzNK4+47YFYcYDDGs1jvb+tOzJtcmxNInD3OKRFyLJ9pUZmWhcZt8utySs1Aqa5t/W6OytO+zfVpGK32bDbFRobX5/2HaOct/Nk0SX+7QPaxj/v+Leo73WMbWfY5a28Gbc7RuC2BbSV79ybZ5nxLGtfGT0enezrZf/w/1Yxe5N/8hin28PVY0JPxOMXbfc9nrpmD2/84g/UD361kjdnbOLe8f42+/Jq935PklFc6WStt3o9UhZcUuXk1Bd/8T1/8vs1Ufe5tzR8LUWN2+O7sAjbdv/OmfBKz4RXlTu87ZrBfTVMP6wwstFo1dKBzSUF5TVh23DfztzCXZ8t5Zf1oZn9/kxf+eHcrbFX8nK5PewoqKCoosZy4WA+DHvhFLS9KfC87/Zl3LUvf11m3Obbq1rMsxA4EVRRhfGbTPcG7qvensfQf2XWWfmg9pnu7qJKdhWFXoj4mtSi1NM8+f0aejw6OexrdTn7YmPVJAL3R7ecHPG1cDOR1ZbT7eHh/xmdZgK/coUV/qtk833cGr5wD6MM/9jtahy+x8XuNKIZagufqbTwBu78knIozwN3+JO4LaCENZVlvgK7PDBnc17YbUw7CsJnC7GqxgEmrfIHiYoal+/kWpse/QA3fbCQ//xqzG4V6cp8xfaiWk34ESkLcoVp3zc99t1qKPDOfR8mqHw0dyvLttfNvZ/NwB0peJrvHzyVbyQbc402/rdmbGL0zM2+47jDOzFRQXnosauLLGjOpjxKI9wX4LHvVnPWKzMZ+Ox03zS64P89havVMAPNsu2FHPX4VN84/XBV5ftT/toE2VjM99/fJN68mE5kVXmkC8NIBr/8K2e8/CsPfLnc8t0MbGqL5NMF27zrhK5Ul3Oc/2fGpkY5v0KTCNwdWqax/MnzLMsGdTfuAGYPM/d3bQWeFAIfmyfA5ql2X2Yf7mpym+7oe1wZJgju0f67lQ2zLw9bBrOqPKN6Jrx6JBW//TvkR9GKCvrZ/OPIPVX+Km+XB9o2Tw27b9Nj34UfJlPbquK/fLGcvd4MsMblYeWOorhPrCt3+tvlIwX92lZxhssawPq3Gjsry3c8zalRTeE+/zPeauLA6XX3l3lsIh1n87sV73wEZjDbsKeUV3/6nenrclm3u4RZmyJPgFEXU9ne8MFCbvxgUdjXJizx1yRNWe0feWEe83Cf3Vw211srNNd74WkZDuZ9sl/lr8OM26yVUQdY/d4q3RF7pf20v00+36/Yze97/B0+zd9lPEc83MVCXU6b/Nr0jY1yRsMmEbgB2rXwB6UnL+nLJ7efAhjVPQcqUgDJ907W0jzV7rs5SLgvZSGtfY8368P4ypXhe35fzb1kVL9Ojm4ftQy+W4l6zZ45lZ6jppAXMEnJPSnfW9bx1JSj0Qy2raFVzZ79Pk/ZnWVcbZtFvF2YdhdX+Y7N4uwCLh89l7dn1n6O6EjBvrZX7Ob92IMFnlR2FVWyaGsBm3JLQ26YEXzCCyzXPyaurfWwouALLrNGI9L3rNrlJgUXR+yeHNdE3cGrVDpdXPTW7Ki1FM6gqub9HZ4W2H5aXOkM204b2Cs8WsZt/h3M/irmBUy4mdMifVemr8tly76ysJnfoq0FZP4euUNgbZi1IQd6LVCX1ffBDqSTpStc8hLHVyRcLVHwd02EajKBG6Cttyf57Wf2pHmq0a5sZimXDegasn6HFtEzUFOkKqZCX+BO8f3gft0QfuL9n9yDGOc6j5U7i3nENZLjq97jHdelTPOcQhVpXFD9ctQypClrMOmoirjMNo+PPx7rW9YG60nSU12O1jA+9UUe3v2XsNm+KRUnKsJY8r/rj3kt9R0GqdrP87zJ29FtQ27wEC3jV/9iynvwtHFb1dU7rUPHIrV7huvJHE1lhElCgi+yrhu7gPPemOUbU2sKvr978KQjny/cRiSTVu0OyZQjzTuwNa/M0llw2fZCdhRUUOn0cLd9ItduewbWT4z4XqbgGBX89Q1XRRwc+NxhAt3kVTm1uknHRW/OZui/MqO3X3vfJvgYg9FXYkdBhe/v5/B2NNWWwK1xe7Sl+j3QnZ8sYdhrv0VsV73lo8VxfJLYzIz7QCv4XG6dsNtlHkhfDUuzkjnlaZjInRc022G4v2tdZtymA+2pvq+0mhveX8iExTvqqEQHJr5xSI3E3EfOCTnhPHVJXzRGMH9rhDGRSnZeORn/yoy7zSfSXOfm9KjNUu0opUixKUtVb6A/O/9q3ScteNk1IuB5S9/jF50jUMC37jNZnH5P2P2dYNvMCan/hTz4L+MBcCjrj0Q5y9EeY5kdT9S26o3pN/OFayijXHeGvHYIRhbVVpUzkM2s0T3iHuJW5m3zTEvxn9GcO5eRnf4nhtc8wfUp3mFhWnPpf+dYtv09t5Sf1u5h5Y4iHr6wd8A+axm4I3zu4DHpCg9X22dTUnGqZb1qp/X9gt8/UgKxs7CCe8cvJ+PYQ/n4VqMGaG9pFduC5niv8p5Q1+wq4fzXf2Oedz5+s8d2786t6Ki87ell0TPENbuKQ9r0y2Lcjx5Cs323RxM0Qyz3eDsg3p1xVMz9gb+JYm3QWP7An6gZUMP9FgsraiztlynevgCBa7o9mkmrdvt66EeS6J7M/ow79KJo6bZC/jphBZPuOzNmVbjTHf13ur/+OmEFK7bHrlL+fU8p4+ZnM6yN9XgF/obcZlV50CFdtLWAa9+dzzs3nBR2O1MiArfLo30XdtFMXpVD/25tOKKDvw9SRY2LlTuKmLM5j4Vb87n25MPrvHy11aQy7hZpKbQO+mHcdmZPbj+zp2XZ4e2NP9rxh7WJa7+R2oYCq8rhwHuw/9z/dWa5+zPOfQHvui9lH+3Y7vHfkOSXM7/AHeZPambKwdXpOMt5unCU76l/nK8mO/16Hk35AoBmGO3RI8wg6v1MD5xrjEuv8XauO9W2nu/TnuLhlK/i/kxm9pqWYhyjrXnlvDTmQwAutPnbQ3MLI1zwfLqUtzO38P5so6OY1pp/TLT2Ij/hiLZRyxB8cxFTjdtjCViX2ebxL8e7bPjmOct6zhprFhFclR5p1jXzBLx2t39Mcsarmfzxnflh1wOjmaHK6bZkEJVOt//vrqNfbF7ynzn8/Rvr7GN7gqrIw5U3eChaXfb83RhU22K+//qcEsb8ZmTK4bLy4NEA5s/LE9QrfW+EJgBnlM6H++Pe8cs4/43f+HbZzpAMz8xmV+8q5qYPF/Hz2j18uWg7f3p/AU/9sIZt+RUhNUqRyhytZszkcntYkh3/hD3fLttFVhzDyx77bjXjF25nW4n1eAV+5yPNnLbOe4E2O6AvxaRVOSHfrURUlcdzsaO15p7xy7j4Lf/kPuXVLvo+9RPPTFprlM2tD4oJZppU4I6X3ab49W9nW64MA42+/sS49mNmTi281fKRAnyLOHuKpvW/jJuco6jGX4V/Sc0Lvsd9Bg3lhZPm8pTzZst23ZTR27aLst4dy+4sp7/L3+HMPCHMvNf4fHel/AhAe0JnGrv3nKMYfKQxH7s5ZeudKcYt+QbYIs+n3KdLawaozb65182qsy8Wbee+L5bzr59+p9w7FeypNv90q+/NiD5/9POT1+PxaEqrXSHZoSNgmsiHU77kSpvxw9ywp4QTn5tumXjlLNsqjldG+V1ubcn02ipjvYo8a/ZWVuHfvsrpDqmqjxS4y6q9N0spreaT+dnGvsNcRAS3A+4uqqQkIKuvcrrxeH/K2hNa2xDrRLO3xDrOvcbtIb+sms17S5mxPhetdUigdtdhVhR84xxzzzd9uMg3kiHcb8fsBW8y22iDq8oj9WQPPNYHknHvLTVmG5u0KoeNuWX8dcLKkLkOAv+GszbuY+SnSxm/aDtzN+dbLtxicXk0VTWxA8dbMzZxzTvzD3hMdrD23ubDvZXW42V+l40yhr87mDmULfBYvPrT77wXMO8DwCvTNvBlwBDTuhBPxzuzXIG/X7PWNHBEze6iKooqasgtiT4/RCJJ4I6g16EtaZXuYMpfzuL+Yf4Zz64Y2JWLj+8S1z4Cq8ojGX/HqVSEuRqcdN+ZIcuO72atAdjw3IW+IF6pU+nWthnd2jVjn7ZmmHPS7mes4zUG2LKoTPPf/OTmknct6/0wZyXT0h6l+6ThvmVvOf7DxLQnfM97KKOjlV0p3+Q1NUHV4ilBdyrrr7IwT8fNirfwQ9pTjHa8BVjv0zx15XZGbxrKPx3vAdDHFjA+fGnsmyvsKakK22wROL/z3SkTeSN1DKt2FnHD+wspKK+xzML1aerLTEx7EjBOQJZhLt6fiy0ol1i91d9vIae4KiTjLql0MW+LdahdQXmNJWA+9cNarnt3PieoTaTjPyZuj6ba6aYNZb4hf8Ne/40Bz/zsW6eyxo1HeSfm2JoXEtCqYpy0coqtJ6Bqp4cL/j2bc1+fxe3jljBx5e6Q6sut+eUR20RjjbkO9nbQjTPMC53ADnDh2kKDZ7ozLzyDx3GXhDRdGCsEziRXm2GJLreHV6Zt8AXnU16YwU0fLrSsUxT0PQyX8QUPRdUYF1EXvTk7Ypt8jSu+jHtdjvGd3lOLCV/i0bWNcVGdU2b9G1cE9OuIdHcw8+8S/L3ZUVhhCfI/rc3l0Thu9LIku4AXJq+jxuWhuNLJpFW7I647YuwC8iPcTXDDnhKcbg93f7405LVwIzWKK52c9tIMTn1xRswygtGvKfg3eaASFriVUh8qpfYqpdYELGuvlJqulNrk/b9dtH0cDPp2bc1Np3f3Pf/3cKMd/O8XHBuybs8wE72Av6o8XAbfPC0lbEfgXoe24K6zj+Slq/rz7d2DeeO6AbRrkcrIIf6Z1dIddpo3b8Hrzmt4oasxt3mfzq3I10YvdaejlW/d8+3Gl3Jvr2t8y4ID0EzuoLfajm2Pf1a2y+zz6aD8GXdm2t+4zT4VpYyA2IyqkP0E3mL0bNtKfkx7guF2o5r9UIzquwvtRqefwJNQByJnHs299yXvQDGnqPUBr2jf+2XtK/f1EG9JBS+mvEc/lc0RzZ0craxZ8mX/nUteWeTJakbYZ1BS6fJ1MGxLKc87PgKsk9gAzP19N51bGye0of/K5Ntl1hnMfs8t5fr3FrK5yP9ZT3xuOiM/tZ4oNm7N5ru0f/CaY4xv2Udzt1LlcrMyfSQ/pT1ifOKg70uVy0Pr5sb7T1+7m0v/OwetjekytdbkBIyc6KuyOQRrlWxwm/fKnUWWTkT3f7mCTxdkW9a5YvRc7v8i/JwCtZ1aNaTa3fu0o/eYQviLj2lrrHe8M79LAckfWmvWBWW05vsFBpvaZE9fL93J25lb+GDOVl8QWpxtHbMfnOWHy/hSgnqqVdS4WbWzmHU5Jfz31/CBe8Oe0pC/RfAXQmvty7Rr6ri92OzBX+oMzrj9x/J170yJWuMbNeD2aF/gDD4WNhV/08uOggr6PTWNTbmlfDh3K+/N3sqPK3fz169WcO/45WzzXsztKKjwTbIFRifYL8Jk8TsLK7jw37N5acoGZoa5H3hhmLnlS6uccc+Z4PZobvt4CVeGmUEwnLW7ixn47M8htWDBEplxfwxcGLTsUWCG1vpoYIb3+UGvQ8s0RpxyOO/fNMi37J6hR9G7cyvLeq3Tw3fIMqcp7NImPeS1SBPApNptPPqH3ow45QhOPKIdV55g3KDksYv68IfjOvvWW/7U+VzxwJs8cZsRkPt1a0O5985kjs79QvbbpZUdBowIWV4bT6R8hk0p7KU7WZ9+G1farRMcBN7JrJ/3TmQvO97njWEteLF/+NuLgtETPpLm3nb2d1PfYELac7TCuIL9R8onbEm/EYWH9+dk+W7i8NmwGq5PmckXqc/z4saLmZ72MHM7PO/b3wC1GdB0xDjhdqDYEpBfcnzAC1PWk51fwacnbGRF+p99r9mDAneqcnLW0f6ajMAZr+46+0jf47xKzYtT1nPbx+F7Kps3jDnF5u+d//zk9fyy2ujJepgKP0FOjctD8/RUX9mKKpyMnZXF8LELmL0pj8e/87f5T0l7jF/SHrJsvyOoqjpwshxf2cNc5ATeKjYwY9qfW3l2DgjS5p4Cb6gRmMV/5niBGal/C2njNk+mVQHBKjuvgkVBbb1mRh940XjJf6wdHwMFV6Nf9vNZ3Gv/jlS7Ij/CxV/wEMNwGXe1OzRrNe8xX+l0Rzw3fLYgIADlb4Fn2sJa/1DPn9bu8f29CsqqmbMpr846tJkXZSXV1mPiG5rn9lg6Vw79VyY7Cio48rEpvOYN6KVVTsuwNrtSEfsY7CiosNwUaNKqHMpr3ExYssM3Jj63tMrXPm/+Tc96ZSanvWTNiMtr3CHZvnmcFm61NiOazEm0AgXW4GitKaqoCbk4NJkzFgb3po/k47nZFFU4o96XABIYuLXWs4Dg3hGXA+O8j8cBVyTq/evaS1cdz7l9O1mWfX3X6fz29wzf87TgbrbAWyNOwOb9lnZsbcyK1iOgx6Ldphh32ym8cs3xlu1Soty+b8wNJ5H98sW+570ObelrP2rTzME9I66k+LSH4Y8f+dbZ0+MK1hx+PalDHoQr3/Etn3L2JH7QZ0V8r3BsSuPw1NAtzx+wKx3+6nkzA77TPomHHf6OalfOvZwOK/3V80YHuPGYp+pDowTuP9p/40LbIo5RRhA737aEn0f24daUnwCjHf6333NxT3+aVWm308llBE/zPuUA3crX+R6PTn2Lwba1LEq/h5H2H1ma/n884u2MF6i/yuKs9U+Hzvw/1QAAIABJREFUfP5Aqbjo17U14ZwY0DFub4WHqWtyIv4oT+pk9+7PerI4WvkvBG48rTvhtEoztk1XxonIvCnH3M15FARlDWZbvelA2ne11ny2YBszA8Y7V+xP4A64qDUzs+BqyqVpf+Zu+/ecaV/LkbbQiwszOAXWjL80dX3Ieq4wVeXRWKqmtaaFq4iHHF+zfk8p938ZfkKkB75awWPfreaYJ6aybndJ2GNSGvT5KitKGTx/JEd6/95xZaE53tqxNf/zLQpsfpqzOY8bPljIi1NCj0M0C7Lymb4udOiqGaBLaqxlKyg3Pku4eTHu+sxas7S7qMrSUXfzvrKI866f9cpMy02BzIsum1K+i7niSqfvO1xU4fQNSQz+Xo/J3MKxT0xjjnfCnpIqp69pLXiIpynwbm7mhWTgZyypcjF87AIuems2H8/dymcLtlFQXkNheQ13frKEed5JgQJHzYDxXf2/z5aG9IUw70sQa0RTfQ8H66S1zgHQWucopTpGWlEpNRIYCdCpUycyMzPrp4T74aUzmzEvx8XgLpU4qlPYV+nhgh4OurW00bpwI5mZZtWR5oqjHJzSWfOY9wJ/+dIldG1pI/BAtE7lgD5vc2B5+hmwbCMdjnuMmtT2lLY22ukzlxi9I1ud+Cq6aBvNcPGM/W5mV/flX453w+5PYyOr140cmTXOt+yqBVfTqtJf/Zzm9Fe/9rHt4IuOn3J6ydSYZb0rZRJn2NawxtOTVq3bQISmoBtSZnADM8i1d6G1u5LXUt+BT/wXIBc7lvCs/QPIBRQUb5oZfkdeNjwc5T1BPuYwAvb1KZmWdRQenm83JaRMKbhoG9BhLxUn5IW/BaQ7xzhhXmefSc+cSjpUeniPS0LWO/eIFG5ruwWKwOG98EnBxeq0O1jo6eNb75w2+/i5uSK3QnNKZzuL9niDSqVx0dPCWzOxdncJp9nWMWv2Njbr7nRvbSMn4IS+NO3P3OO8nwWevlGPUyz/+moGo1dYs4lL35pFj9b+E9UTKZ+Sr9swxn1ZxP30LltIgWrLdt2JkioXfx7zE1n7/CfT5lTRQZXysGNC2O1Ps60jffsWMjOLKC6v4mT1O25sLNt3jG+dG+zT2aPbM/b7NE7slMKby+KrHh817lcuPdJBql1hc1czxLs81m1Wzdn1/vvjAvIqPThwcbV9Ft+5z6Sa1JBe3OUrJtK9cD7/SCnjJueocLsMsWbdOo4Dtuzay3XPTOGing5Lw5U57/yi33fw8Q+5rCtwc2EPh292wXKn5rn5/kA0P+1eJrgzGO6d/uHjC1uwr8JDTrmHI1rb2JVr/K2LqtwETifzzZLt9FJ7eWFh6DEN7ny3Pai9d+7mfC5+K7TGI/AcOHPmTJRSbMnydhbbsYMdxcZ3/93fsmjlHSz03rQlzNge/cLxhg8WcteANN5ZWc3FPY0N95VYyzTxp5m0TlMs+N0fuHu1hnX5MHO5/14M036dzYY9xvF7+kcjMXji+zUojHTEvPix42HEW9O46qhU2qQpsks8TF1TxYbte7n3hDSapSiaOxR79xjH96kf1kb9DAftOG6t9VhgLMCgQYN0RkZGwxYohhFB/0cydKjx/6rqVXy5eAdnDT7NN/zsCXsWR3dqxdnHHBplD7WVEXP5l/3LGfNjCvrIjqhfnwtZU906hSO7nw6zjoDs2ZCV6Qva1YP+zPd7DuHSM06g+Vf+9nNL0D7qXNj8S/Buffrbsulvy4bK2L3rO7nDz0L2rP0Dy/Nu1Zv8T1p2hjJ/ta6zRWe6lu/hWcc4yzatgiaoudf+PQMqQtumWlDNF6n+aveXLz+G404bxouLpnKaXs7l9rls6341hYecwoXD+rG202Ra/O89cAEO+MR9PtWk8ppjDIdSxLz+z3HneQPp8Oa1gD/j/uf5nWg2q8Z/J7iUZgwdOpS2K2aRW1HK1Wf0ZdH/jE48PQ5tBaXQjGoeazuDT4r786W3jD2qxnPHOX05ob0L75B+OqhS7rV/xxrH8b72yVSceFAhY/C7tEnnrLKp/OYeQC7WGfyCgzZApQvWF3hoSQXvOV7ndLtxQvvQ7e9M2a1tM8tUsy9XPgNpRlkBftpmPfkerkJrKRQetLfS8MvU56Ecfuu6mS+nLSI7/VnfZ79lcA8+npft66PQY/lJPHhud5bv3Riyz3B+zHLyY5aT28/syZNDO0LkW4GHNWWrkyfaTmdCuvH+R6i9vOIaHrLenJ013J6Kb4RAPI7t3RfWwcZCN3lOzSfrajikpXGMW1NOFanU4KB7SxetloxmovM2zjl5MKf0bE9FjYsHv1rBngozaGm6qALuT/mWN1zGb3nIkLMZ8OzPIRnp3kprNb7TQ9igfSB+KmgP3t/k8ScPpkPLNOaUrYNNW5my1VpbUep9urLAjvFDi25VcSpQzepiB+AkePJE3ekYaOZgyjR/s9Zz157K1WPmM3uXf+WjjjsBZoeeI4LrSipcMH+3m7bt2zB1zR4Ob280Z9bY0vjbb5Uc26kVPz04xPh827aG7C9Yffcqz1VKdQHw/l838wkmoReu7M93dw/2BW2AO87qVcdBOz49D2nBRT1TUcdeZCw480E4x9uTvOcQ6H668XjIQ3DTDzDCX/2dlvEQ193xMM37WOeCp8WhcMFLcMqf4crwmTyA0xbQ7q/DVF2mhPYLsOh+RtjFtvKAjiZdBsD9q+AwY5ITR6feYbcxfes2evT/zfGNseC4ayyvD7Rtpo/NP4PScR3TwVnF0rOX80nqP7naPoe/7nyQ54a0QM15nRb/+5Nl+14qh+ZUcbV9NkPsq3m06k067AmYSER5OIRirs4bY9kOh/Fj793F6FuhNfRTW+mhcjik0vixH2fbysiqD5iT9oBl0ysGduW4Q6wn21Kac81JRt+JCw7Zx6r0O3nb8WbI8Xjs7I684niPD1L/BfjvZW+6qH/nkG0AzrEt9wVtgCPVbq4+8TDmPnqOZY6EzvjbFy/u34UBhxvNC/26tmbaA0YzzhHewF2GvwPo16lGcF70+DDfsts/tJ5EFz9+Lk9c3MfXGxrgHvv3fDvDek9206VhZlBMp5p5afeyed733PSOv920c+t0XrqqPyueOo9xt53ClSd0C7tPgNMq/DVA4TphTkh9hjvtxt2yurazdnJNxcmrV/bhUts8Dlf+6uvTbWtx/c/oexE4P7/Zbrsq/U4+TX0JgAtzxnC1fTbn2ZZy12dLOfG56Zz5z5mWTnXNCL0Ie+iblRGrkevaoRT5msIAvljkf3zS879w1dtz+WRB5FkIIb625IFqM+P3XMpg25qQ7N90/5crQmbN69KmWch6z09eF7IsGnPOAnN4mXnx+ntuKVrruPuH1HfgngiYg4xvBn6o5/c/aNhtihOOOMg61XfqC08Xw7lPwwk3Gsv6humGcOyFcN8yuPRNaBlQyX/uM9DmcDjuavi/eXD63XDRK9DiELjwnwE78J5kbvwex1O5cMGL3ve6HDoEzLp190J4cK0RnFt3g5ONWdsWXDqDb8/8Ec54AEZ8GfnztD3C+L99T2jXHdK8nQlbH2ZZ7ffO1qrr9Z4jrPtpbT2ZH6qCJspwVsLbp9Jq3j+ty98aCDOeDSnW161eZ2z7z/0LSnbDjgWWdcZ3+ADWWeeWp7IACrfxj4uO4bIBXTn3mDZMTnuczLS/0TzHGI7U35Yd8n6/3tyZVm8PhExr+apxcH6/TmQ+lMHb3aaTTo1v9EGgjO5GoD7Ols3rLT7hh3vO5CpvkOrGPt5KHcOrfbeyYMAUNjw8kN/v78nMq+24sdagfH52Ma9d1ZtuaVXc0/wX2jezc26fTixIv8+3zuhrjuHWwT0AOPdw6F2+BPDPQdAyoFZkkG0j7Sihmdu/rJfKwRGQcR3azJix8Oe/nu1b9nfHBB5KmUBflc3Tl/Zl4WPDfBcffbq04rYzevLX847hhSuPI91h40iVQ1dVwKiU8ezN81ePL3hsGCMGHUbbZg7OPuZQ/r+9O4+zqf4fOP56z2qYYcyMsc7YZSc7IdHXTyHKV0QlXy2i0J76Vvr27VfKV1LSQn1VIiQtSvwUkXVkXwYZMRoMM8Y2lpn5/P74nLvMuDPIepv38/G4j7nn3HPPPfc8ztz35/P+LGfwDbYj4gd3N2Ht8x15o1cDXuth+66kmGj3+7IJoAwHKcJJHu9Yg7ubV6BZQCKtnEJO5dji9AxcQGJoP2qVCmVZkYfomdCHt0Le5suQF9z7mRLyMmFOn4Yba8W655doG7CWUOz65gFbuL1qNvXEFuyCArxmAhSoHhvunu45wmuCJtekTXlHSJyPmPBQhByeDprCziJ9COUUD5XeQMdano6cpTjEf4LHU5QTzAl9irnOyAlfft116LyHGvoyqLLNvrULWJtrfbkSRXipWx3CQ30nomPyTIFdrkQRVp/DbHPefkvNf5Kbn7cdcHd2PNtdKy9ZqlxEpmDzsTEikgy8ALwKTBORAcAuoOel+nx1gSLKwLP7ICif24xGV7UPb62H2YcvLQZC9b/BnlVQryekJ0GUM7StVldYO9UWGIpE2kAWXdVdw+SuLyEnC4KLQtsnaBHh6iTotDbWuAm2+mhPv+Nz+GE4tH3CLoc5BaUiXuPhH/iZa2Jrw4nD8OltkLKGTo1rkLMxlIDskxDfClo8CCs+gKxMqHWLZz7wYrFwbD/M6A+nz32cZvipVFqf8urxemw//OE1tCq6GjUOJniWi5UCVwbhzfpEtRrC2Dtegr0F32fcpcqaUXA4GdbZQs6sEnfRPeMTojhCk4pRhGybDYnfurdfVP8HXsq6m7lO+1xEjqeGeFv2HMhYyaj052gQFEt8YDpBGxPoyXS7QeKnAFQGXoiug/fcPSWXjYTjO8HkUHf9dH6t9jeo1gG8M4OvlKd7iThaDfgvpaZ1hbXHmXl/IsWWLwIfU+GPr7uFiH2eAvDsv4dz6OuHPBv8uxTU60l499zZi66By+gauAyWjIGKU2kdF8rS9YcJCw7k+a61IeskBIWyIimN1HWemeYeCZrh2cnsx2xv7vgW0Hsy1WIjcnUadY0EaVMjhqKTx7jzi6EhwSwLepifs+uxN2IKDzWLBK8YEpy8lNeD59iPuLM8AeOPQKrtKxEjh5k1+Do7FM5r6HhggNC5flk6RpYm+MORrIy8CWcmYl7bc5e7ivY/VcP4MhE2/6uTnV/iuyfZFdOar9cGckNMBjh9pT6vNp/Pi/cnJCiA46ey3Pd8/99b6/HiNxs5mZXDuD6N6By0kqPBMewIq80tb9usUVkOkkExvn24A9+/N5z+x+1ETu9VXkS7lIl8X/QQc2nNnLLvUzN9AQCrc6q5h52Gc5w+betSIiyYKSt2Ma3dIcp93587IiaxNDWYppVK8lnIy4zfEcPorNvPvCjy6NeyIpOW2pr6xH5N6JC2Hf6A6GLBkAHNK0exPCmNdjVjuatlJSqULMq65AwaVYxk5Jwt7DxwnMyTJwl5ryUJTa/nX6f6cEezeMqUKOLznujNK0fRo1EFGsRFMnfjXtbtyWDepn1Uiw0/ozNaz8YVKB4WzMTFSfT70DNT5JLh7Sn9Sv7f6ZIFbmNMfs29HfJZr642wWdJU58v72DvCtpga8YDvRoOi+a5E1pQKOAUICJy9+wHoNencPIwBATCkrfgcIpN9Rcva1P7Lp1egZBi0KA3LBsH5ZvYNDpAsWho9TB8MYDGDa+FTlvgyF6bhQB4cgcc3Wdr7we22QJGeCmY+YA7ILpFlIMj+U8GcYbMdNt3oMkAW0j4/Rf4Zqjn9aHrYPEb8PNrdnnJWNg6xwZ0X6KqQprXpCaJ39l0/wYbdLrf3p8D36fQ+OBGQlI3wOd35np73NZJvB8xl8zmnQjauxr2/yP3/j++hQCg31l+PWKP+Ohgs/lbOO3UOrbPs4+8MnYTO9nzM9Go+BEI9z2RSIvtY2D7GJBAMNkEb5xOKcmTil4/3T58ObYfZj/CHWlJ9IgIIKBFEhw7CK9Xgab38UKrASSGhsNaqBmwm5p43WRi5QT7d8u3kLTIZnV2LbPXWE1PAC+bnQLHdrqXe9QMg03QNnA9R0PXwtFKuY/phCejE7DthzMOuWHSBBpGFj1jPUBwpi3gNT3ku2Nox3jY1KcjYbsW2EzWiveI5z223zOVoKnPuLdrlj6bZnWqQ0g4BBXhxrgapElJ+jSLo9eafhw8kkmsGQbTHiAcqP9YItHFQsg+dpCl7gzKJvrH7wdnAsR24fbcxeckE8kRd9AG3P0PADYUuZekrOFUbvcUg9pVRSZ1BeDpOukMXiiIRBG8azFDgqDL0HFsSTlM4paNPNC0BCt+mELxm59n2qL1zFyfxqyQ54mrOpJJzizCJQJOQJrtSFquRChkwMu31uWdBb8xzJlo64asxdzAOoi9jzYD6pCaXYzMLfNgdiIxBxIZO+J1AEzqVuJkHzFx1+SqeXdtUM49n/k1ZSJsx9Hl8zlRtzfTtxlGzd3qHjHxao/6BAYIC7emuoN676ZxxEYU/NsrF3rXlMuhSZMmJiEh4ewbqj9twYIFXO0dAC+anBxY8Apc2xdKVsr9WtqO3IWKs1k1Cb4ZYp+3G24LBScy4IdnbYp9XZ5526OqQuk6nlr7E7/B605hZtByiK0J2VnwkpNaHbLGFha+fRQScnfAy1dcC6jTHeZ4TZPQ9wvbQXD5eBiyGvZvhql97A/zKacWcE1n+O1Hm1m42gSGQLbXsLbWj8Li0Z7lhn1tgSd954V9ztO74cBWmOAUHELC7b5X5N9Pw6cRGbB9vj2e2Y8WvG21v/kuwJyPml2g92T4/ilY/m7+29W7HereBlN6Q8uHYKmduIna3XM3zUTGwyGv8eLVO9qsz83/gQntfe76xH1LKPJBqzM/b73vkQBndd9PUL4RfHgT7PL0XRgY9zXv7nZGKDyfBqs/9fwPgu1T8+UDrKrxCI23voEpWYmWx0bT6dgsRgR/7N4sq9kDpMR3I65aXU8W7vAfMNozioPo6vZcBIfZAhpAt3Fw7Z3uuxZmDj/I3M37+G3/URoXTaVN4/oEhHkND537nC1st30CbniWpF9mMHD2AYYGzeTmh9+C2Jokpx9n2/6jtKtRCsnJhkO/IzHVVhljPJOHeNHArYBCFrgvppwc2JMAcc18v26MzQYsHQcLR0Lj/tB1jK3NH0mBctfC0ndssGzzmOd9e1ZBaAmIcdr8Zw2CNZOhSjvYscCzXetHoXxju59Tx2D+i3aCnVpdbE16s01T8mSSzRLsW2+zDMbY4LTHadPu9KqtlScthC8GFPydY2vDoKXwUWf4Pf+JSwrkHRjumW2bMca3Kvg9JeIhw3lP9/GQ8BHE1oLkBOgy2gatFN+zubmVb+z5zvmJquKulRWo/XPw68dwyEeHqTaPw6JRZ9/H+ShZqeCCSZUbYEfBwyAJDLH9VxImQoWmkJzPbUu9m2e8VWwNvy8ms0gZwk7kmUypfBP7v+CtUhubTTpXeb9Dq4dh13JI9qSRD946legvnV75xWLtdX/aq+04b6EjMp6trV6nxne9fH9m7W5wuxPQ32kF+wseigW4zwMA9y+AyIp2TP0n3T37O5oKn/eF3U6bRrlr7Tlf4bnVMo3vgc6jbbYw+zSs+i8c3A7L30VePKyBWxVMA/elt+z7qbS4sZun7f58HNkLv7xpOwCeyIC9a2HxGPsDkbdpweXgbzD3n9DkH7Z/QV6pW2FcU/t8hJOeNcamgDd+adv9e0+2afpVTirzuqHQYrBtsljyNsx91rO/vLVgsLWmoFD48WVInA2Vr4cSFaD7O+4aC0/ssE0VruWbR8F3j8OdM+0P2u4VNkPS+hFbKFk8Bto/e0anQd5tA3tz3/nsDD0m5i6YBBfNv39Cv2/ASdOe4ZqboddkGwC/c2aiK1Pf9+eXrGQ/t3h5GFMPcvK5jeo/U2FSF88PvcsDi2yBpHZ3ePU8bikZXR0Oeg2LrNwWknz3pneLqgodnoPp93jWNexrC41eVjZ5k6bbR/sutPiSXz8UgKrtbaYHoM90+OwKdH2qeJ0t1Jyt4FMQV4EhMMRm0j7tkavAUaC+M2xBadaD7lUauNVZaeC+9K7KczxrkO1c1ejugrdLS4KM3fbH38UY2LXUPub/CwavtIEiJ9umHjfNgi5v2G1PZ0LGHk8GAeCbYbZA4Co0jKph+xE8n27H3XsHZmNsN+iCpKyDlRM4vvn/KJq5x9biM9NtcC7XyNaQbv/E7ifhI2g5GOJbwpbZEBQC0/KcgxcO2YBSvJztiPa719S+rmPevQImOoWiYevh9yVQqbXtbOmaE6HxPXYEBsDbzeBAoieIVmwNgUG2ttrhObtuUlfbIfOTW6F+b7jNK00/wqtjZWRFGyS8gzPA8D22nV3Es33vzyCuuadZJhexWZjat9isjzEwrpltMugyxha0JuceErmo9VTaNKwOPzxjR46smmQLbXV72CGgOVnw35tzn8sXnVkEh22AMXXt8zs+t/M8jG9pC2VtHoWtc2HLN7Bvk23GKV7OFpRcBaSC9P7MXtMnfPT2rt0Nuo6FkV6zD3r1/cil/xz4dRKsdWZTjKpiCzV/pkmj6b32O6z80HYSPUcauNVZXZVB5S/mL3uOs0/bjEDkedQGwQYIY8B1s41Du21Qq3bjBR3Owh/ncX3r6+CrwbBxJjy6xQ5J3PCFbXPNc3MPtz2rIDDU1pCP7ss9auJ0pn28VtkuuwJ3Tja81cg2Q9z3k2ffp0/AV4Pg5FE7xbArK7JxFkzvB/cvhHnPQdsnoXKeKYdzcux+MtMhtLjNOri8GGXnO2g+0I7CcGVvXAE6JAKe8QoOm76CbfOgm9OW/ctY+7k3j7Lp8D9+hQ4jzjwnO3+xgXfgLzbovF7VFnSWvAXAgnZf5b6WE7+37eZtn7TZkONptukjpJhNdze+x47MOLIX2j0NL8VAbB0YdG433wBsSjwozA6LXPG+zd6krLUdOn8eBdFVbMHBGFtw2LfRZjlOHbVBv+ubthDyxxo4ecQWsMBToAA7zLVEBZslSv8dPmgP4aVh4GK7n83f2NfXT/c0Xaz+xL7vrUa+mx9GeDobcuqYzWjNe942Ea35zGYzytSDvettB9WyDSC6KlK5jQZuVbC/bFC5iug5vjzc5/nUcRuYXD/QF8OOhVA0GsrU9aw7fcL+0J8tI+C9/Z8dsXH4DxvQS+e5gdCpY7DnV4ipbodynu3zz+V4fWU5lo2HmOosSA7KfS0bYwtJ13Q+t++2a7k91vyaeS6nrXPtKI2m93pGkZwrY2zTVVikLQwg8O0wG9jv+xECgqFs/TPfk5MFgc5ERsfTbHZo3TSo0dE9bFVE8g3cV+2Up0opdUFCil7coA1Q5foz151vEL6QYZbFy53Ztg+2Zpu35n6hn+8rsLdw2mCTF5y5bd0e57ZfgPjm577tpVajo338GSI2aINngqfu4+2EU8Wi83+PK2iDp/DSIJ/Ocz5c7pnTlFJKqb+uwOD8g/ZFooFbKaWU8iMauJVSSik/ooFbKaWU8iMauJVSSik/ooFbKaWU8iMauJVSSik/ooFbKaWU8iMauJVSSik/ooFbKaWU8iMauJVSSik/4hc3GRGRVOAcb/yq/qQY4MCVPoi/OD3Hl4ee50tPz/GlV9EYU8rXC34RuNWlJyIJ+d2JRl0ceo4vDz3Pl56e4ytLU+VKKaWUH9HArZRSSvkRDdzK5f0rfQCFgJ7jy0PP86Wn5/gK0jZupZRSyo9ojVsppZTyIxq4CwERiRORn0Rks4hsFJGhzvooEZknItucvyWd9SIiY0Vku4isE5FGV/Yb+A8RCRSR1SLyrbNcWUSWO+f4cxEJcdaHOsvbndcrXcnj9iciEikiM0Rki3NNt9Rr+eISkUec34oNIjJFRIrotXz10MBdOGQBjxljagEtgMEiUht4GphvjKkOzHeWAW4CqjuP+4Hxl/+Q/dZQYLPX8kjgDeccpwMDnPUDgHRjTDXgDWc7dW7eBOYYY2oCDbDnW6/li0REygNDgCbGmLpAINAbvZavGhq4CwFjTIox5lfn+RHsD115oBswydlsEtDded4N+NhYy4BIESl7mQ/b74hIBaAzMMFZFqA9MMPZJO85dp37GUAHZ3tVABEpDrQFJgIYY04ZYw6h1/LFFgSEiUgQUBRIQa/lq4YG7kLGSWNdCywHShtjUsAGdyDW2aw8sNvrbcnOOlWwMcCTQI6zHA0cMsZkOcve59F9jp3XM5ztVcGqAKnAR06TxAQRKYZeyxeNMWYPMArYhQ3YGcAq9Fq+amjgLkREJBz4AhhmjDlc0KY+1unwgwKISBdgvzFmlfdqH5uac3hN5S8IaASMN8ZcCxzDkxb3Rc/zeXL6B3QDKgPlgGLYJoe89Fq+QjRwFxIiEowN2pONMTOd1ftcaUPn735nfTIQ5/X2CsAfl+tY/dR1wC0ishOYik0rjsGmZoOcbbzPo/scO6+XANIu5wH7qWQg2Riz3FmegQ3kei1fPDcCScaYVGPMaWAm0Aq9lq8aGrgLAae9aSKw2Rgz2uulr4F+zvN+wFde6+92euS2ADJcaUjlmzFmuDGmgjGmErYjz4/GmL7AT8Dfnc3ynmPXuf+7s73WUs7CGLMX2C0i1zirOgCb0Gv5YtoFtBCRos5vh+sc67V8ldAJWAoBEWkNLALW42l/fQbbzj0NiMf+s/Y0xqQ5/6xvA52A40B/Y0zCZT9wPyUi7YDHjTFdRKQKtgYeBawG7jTGnBSRIsAn2P4GaUBvY8yOK3XM/kREGmI7AIYAO4D+2EqIXssXiYi8CPTCjkhZDdyLbcvWa/kqoIFbKaWU8iOaKldKKaX8iAZupZRSyo9o4FZKKaX8iAZupZRSyo9o4FZKKaX8iAZupQoBEckWkTVej4JmGzvffVcSkQ0Xa39KqYIFnX0TpdRfQKYxpuGVPgil1IXTGrdShZiI7BTeSr0sAAABe0lEQVSRkSKywnlUc9ZXFJH5zj2s54tIvLO+tIh8KSJrnUcrZ1eBIvKBcw/nuSISdsW+lFJ/cRq4lSocwvKkynt5vXbYGNMMO8PYGGfd29jbYdYHJgNjnfVjgYXGmAbYOcI3OuurA+OMMXWAQ0CPS/x9lCq0dOY0pQoBETlqjAn3sX4n0N4Ys8O5Ec1eY0y0iBwAyhpjTjvrU4wxMSKSClQwxpz02kclYJ4xprqz/BQQbIz596X/ZkoVPlrjVkqZfJ7nt40vJ72eZ6P9Z5S6ZDRwK6V6ef1d6jxfgr3LGUBfYLHzfD7wIICIBIpI8ct1kEopS0vFShUOYSKyxmt5jjHGNSQsVESWYwvydzjrhgAfisgTQCr2DlwAQ4H3RWQAtmb9IKC3yVTqMtI2bqUKMaeNu4kx5sCVPhal1LnRVLlSSinlR7TGrZRSSvkRrXErpZRSfkQDt1JKKeVHNHArpZRSfkQDt1JKKeVHNHArpZRSfkQDt1JKKeVH/h/amZQfykAWIgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.DataFrame(history.history)[1:].plot(figsize=(8, 5))\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('MSE')\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model seems to have converged, without overfitting. Let's assess now how to explore the output of the Keras model. Here we can obtain a summary of the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_1 (Dense)              (None, 5)                 70        \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 1)                 6         \n",
      "=================================================================\n",
      "Total params: 76\n",
      "Trainable params: 76\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now check the estimates of the weights and biases of the hidden layer: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(13, 5)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hidden0 = model.layers[0]\n",
    "weights, biases = hidden1.get_weights()\n",
    "weights.shape   # Shape: n.features x n.nodes "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0.39586022,  -1.3965946 ,   0.02745466,  -3.0985963 ,\n",
       "         -4.774773  ],\n",
       "       [ -1.1492051 ,   0.06476866,  -0.5094994 ,  -0.32451457,\n",
       "          0.53492266],\n",
       "       [  0.8083154 ,   0.45100728,   1.2954806 ,   0.09208159,\n",
       "          0.47569904],\n",
       "       [ -0.7439928 ,   0.31776437,  -0.6883689 ,   0.15028778,\n",
       "          0.5030628 ],\n",
       "       [ -2.1004004 ,  -4.0892043 ,  -0.11636277,  -0.70726097,\n",
       "         -2.4464087 ],\n",
       "       [ -2.8046834 ,  -2.6861882 ,   8.862835  ,   4.2580667 ,\n",
       "          0.05489183],\n",
       "       [ -1.0692184 ,   5.0049076 ,  -0.7177499 ,  -1.69726   ,\n",
       "         -0.9026026 ],\n",
       "       [  0.51035804, -10.745302  ,  -0.06063677,  -1.3544239 ,\n",
       "         -1.915006  ],\n",
       "       [ -2.022579  ,   2.8751047 ,  -2.0021107 ,   0.74168605,\n",
       "          1.3019769 ],\n",
       "       [ -0.24542502,   0.49112397,  -2.4183533 ,  -0.9556505 ,\n",
       "         -1.2456515 ],\n",
       "       [ -2.6645422 ,  -0.9323434 ,  -2.1411583 ,  -1.9317623 ,\n",
       "         -0.93130386],\n",
       "       [  1.817475  ,   0.9988836 ,  -1.8321708 ,   1.5617864 ,\n",
       "         -0.17923424],\n",
       "       [ -1.7215374 ,  -7.914845  ,  -4.541823  ,  -3.7974484 ,\n",
       "          0.41978535]], dtype=float32)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 2.7319481 , -1.101117  , -0.24770424,  4.2602057 ,  4.710392  ],\n",
       "      dtype=float32)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "biases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have the `pydot` library installed, you can visualize your model: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOgAAAD/CAYAAAD7eppbAAAABmJLR0QA/wD/AP+gvaeTAAASSklEQVR4nO3dP2wT5/8H8PeRP0xtUCXSSKWVSltgomLo0KVRWxZUXdSF0pKmYYDqssFPHW0x0NERY5DD6tiCBdnqmAwZGqlSJSMGMKVUl4KaO1TpzEYCPN+B33M9n8+Jz3G4z5O8X5Kl5Hx+ns+dn/f9eRInllJKgYhE2pd1AUTUGQNKJBgDSiQYA0ok2GB8wdraGi5duoQXL15kUQ/RnjQwMICrV69ibGysZXnbGXRpaQmVSuW1FUZEQKVSwdLSUtvytjOoduPGjR0tiIj+Y1lW4nLegxIJxoASCcaAEgnGgBIJxoASCcaAEgnGgBIJxoASCcaAEgnGgBIJxoASCcaAEgnGgBIJxoASCbZjAfV9H5VKBRMTEzvVhTHy+Tzy+XzWZZCBOn4edLsuX76Ma9eu7VTzO67ZbOLu3bu4c+cOarUaqtVq1iX1rNls4sCBA0jzF1Y7fT4xi7/SGq9fUm07bccCOjc3Z3RAC4UCAODnn3/edltXrlzZdhvbsby8nPo1SqkwGAAQBAFGRkb6XVpX4vUrpeD7Pt5++20A2da203YsoKbToepHQLPUbDYxPz/f02ujgz6rAHSqf3R0NPx6t4YT6OM9aLPZRKVSgWVZmJiYwP379xPX830fs7Oz4Xr677DE71lrtVq4zurqaksb+vXz8/Pwfb/tkqdTH1mIb1c32+n7Pmq1WrjO/Pw8LMvCzMxMy361LCt8dFpWKBRQq9VangN6vy+WUn8aOuT69fl8vmWM6Mfs7Gz4muhz0e3qNHb19jabTczMzPRvzkHFlEollbB4S7ZtK8dxVBAESimlyuWyAtDSlud5yrZtVS6XlVJKLS4uKgCqXq8r27bD9VdWVpRSSrmuqwAox3HCNgqFgnJdVymlVBAEKpfLdd1HL+LbkFZ0u+Lfd9pO/Xx0nSAIlOM4CoBqNBrhtsbr021FlyVtQy6XU7lcLvX2S6l/s+Vxul/P89pqXVlZaRtj0W31PC+stduxW6/XE9vbDABVKpXal8cX9BLQarXasuOVevWGxHegDm28MD1QknZ40puld5pS/73J3faR1nYDmtRGt9sZX6derysAqlAobLutXmuXVH+325XL5VoCE39doVBQAMIDv65Vh1Gp7seuPkGltaMB1UeopE47HX3jj6T1k5bpvsrlcuLO2KqPtCQFtN9t9VK7pPrTbpfrumEYo6/TB45isRgui16pKdXb2E1jRwO6nTdjq3biyxqNRsvOih6Nu+kjLQZ0dwS0WCwq27ZVo9FIfJ0+8AdBEF6Op+lrVwU0eim8VTud2tbX+fGQbtVHWhIDutnlWpq2eqldUv1bbZfuR1+e6jNi0uv0WbRcLqtqtRreO8f7SjN209jRgBaLRQW0T8TEi9br5XK58PLU87wwYN2+WdFLW71ju+0jLUkB1Uf/arW67bZ6rV1S/Ztt18rKSngP2W17+oBv23bbc72M3TR2NKB6Zsy27fAopWe5okfM6Kxd9OG6bstzegdEJ5r0xJDeSboffV+hbdZHWtH+e735j9bjeV6q7dRHdL1OLpdrGzzxmVE9Kxnd7/qWIDqgupnFTdp+KfUnzQBrug19wtCvd1235RI3OtkYfV30XlTrduz2akcDqtSroOid7ThOy7R0dEe4rhv+aMRxnLbLjuiGdlqm3yig/R50sz7SSHozetkvndrpZjv1INMDrFgsth0oXNcNn9dnpvh+11cZuVwuXLZVQLeqO8v6u61N9xV/vZ7VTRoX+j41STdjN+ns240dDyj113aPyFkzsf6kyaHXpVNA+XEzov9348YNnD59OusyWjCgAvm+n/i1KUyqP5/Pt/xK3xdffJF1SS321C/Ld/t7nK+uOF5fW3H6Uxr6617ayJJJ9b/33nsAgGKxiAsXLmRcTbs9FdB+DpSdHHSSB3Q3TKr/woULIoOp8RKXSDAGlEgwBpRIMAaUSDAGlEgwBpRIMAaUSDAGlEgwBpRIMAaUSDAGlEgwBpRIMAaUSLCOn2b55ptvXmcdRJTAUrHPBq2treHSpUt48eJFVjVRj+7duwcAOHbsWMaVUFoDAwO4evUqxsbGWpa3BZTMNTk5CQAolUoZV0L9wntQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwfgftg31+PFjfPXVVzhw4EC47P79+wCAI0eOhMuCIMDS0hLeeuut114jbd9g1gVQb/7991/cvn078bl//vmn5fvHjx8zoIbiGdRgH330ER48eLDpOh9++CH++OOP11QR9RvvQQ127tw5DA0NdXx+aGgI586de30FUd/xDGqwhw8f4oMPPth0nT///BOHDx9+TRVRv/EMarDDhw/jxIkTsCyr7TnLsnDixAmG03AMqOGmp6cxMDDQtnxgYADT09MZVET9xEtcw62treGdd97By5cvW5bv27cPjx8/xtjYWEaVUT/wDGq4sbExjI+Pt5xFBwYGMD4+znDuAgzoLjA5OdnVMjIPL3F3gSAIMDo6io2NDQCvfrzi+37LbxmRmXgG3QUOHDiAU6dOYXBwEIODgzh16hTDuUswoLvE1NQUnj9/jufPn2NqairrcqhPjP1d3JWVFTx69CjrMsRYX18Pv3727Blu3ryZYTWyHDp0CJ9++mnWZfTE2HvQpB/OE3Vi6DA3+xK3VCpBKcUHHx0fpVIp62G6LUYHlGi3Y0CJBGNAiQRjQIkEY0CJBGNAiQRjQIkEY0CJBGNAiQRjQIkEY0CJBGNAiQRjQIkEY0CJBNvTAfV9H5VKBRMTE1mXQpTI2L+o0A+XL1/GtWvXsi6jZ81mE3fv3sWdO3dQq9VQrVZTt7HZB98LhQKOHDmCzz77DCMjI9splXq0p8+gc3NzWZewLYVCAb/88gt+/PFH1Gq1ntpQSsHzvPD7IAjCDzufPHkS8/PzmJqagu/7/SqbUtjTATXdlStXcOXKlW23Mzo6Gn4dPVN+/PHHuH79OgDg/PnzaDab2+6L0tlTAW02m6hUKrAsCxMTE+F/pI7zfR+zs7PhektLS+Hy6D1rrVYL11ldXW1pQ79+fn4evu+3XUp26qPf8vk88vl8z68fHR3FxYsXUavVsLy83PLcbtpPYilDAVClUinVa2zbVo7jqCAIlFJKlctlBUBFd4Pnecq2bVUul5VSSi0uLioAql6vK9u2w/VXVlaUUkq5rqsAKMdxwjYKhYJyXVcppVQQBCqXy3XdRy/i2xCVy+VULpfbVhtBELRtoyn7qVQqddwuExhbedqAVqtVBUA1Go1wmR540TdQhzbelx7kSQM5vgyA8jwv/N7zvFR9pLVZuPrVhqn7iQHNSNqAOo6T+EbFB0306B9/JK2ftEz3VS6Xw7N11FZ9pJVFQE3ZTwxoRtIGtNMbm3RUTzNQk5Y1Go2WwVUoFLqqpVc7HVB9pRE9c5myn0wP6J6aJEqj0wRSN44cOYJqtYp6vQ7HcfDTTz9hdna2r328Tr///jsA4PPPP297jvtph2V9hOgVUp5Bi8Vi4gQDYkdpvV4ulwsvuzzPC4/u8fWTlgFouWSr1+up+kgrqaZ+taEnamzbblluyn4y/QxqbOVpA6pnEW3bDmcO9awg8N/sop6oiD9c1215Tg+Y6ESTnvDQg0r347puy6DarI+0ov0n3cd1M4vbqQ09I2vbdstkjkn7iQHNSNqAKvVqAOiJCcdxWqbxowPQdd1wyt9xnHBAxAfKZsv0kR4J91ab9ZF2HyQ9orYKaKc2dN36xyRJTNhPpgfU6H+eVCqVcPbs2axLIcEWFhYwOTkJQ4f53vpNIiLTMKBEgu3pj5tJ1O3/PTX1ko3SYUCFYfAoipe4RIIxoESCMaBEgjGgRIIxoESCMaBEgjGgRIIxoESCMaBEgjGgRIIxoESCMaBEgjGgRIIZ/WmWmzdvYmhoKOsySLCbN29mXcK2GPsnT/bv34/19fWsyyADDA8P49mzZ1mX0RNjA0rtJicnAQClUinjSqhfeA9KJBgDSiQYA0okGANKJBgDSiQYA0okGANKJBgDSiQYA0okGANKJBgDSiQYA0okGANKJBgDSiQYA0okGANKJBgDSiQYA0okGANKJBgDSiQYA0okGANKJBgDSiQYA0okGANKJBgDSiQYA0okGANKJBgDSiQYA0okGANKJBgDSiQYA0ok2GDWBVBv1tfXsbCwgPX19XDZgwcPAADFYjFcNjw8jO+//x6Dg3yrTWQppVTWRVB6y8vLGB8fBwAMDQ0BAPRbaVkWAGBjYwMA8Ntvv+GTTz7JoEraLgbUUOvr6zh48CCePn266Xpvvvkmnjx5guHh4ddUGfUT70ENNTw8jDNnzoRnzyRDQ0M4c+YMw2kwBtRgk5OT4WVsko2NDZw9e/Y1VkT9xktcg718+RJjY2N48uRJ4vMHDx7E2toa9u3jcdhUfOcMtm/fPkxNTSVewg4PD2NqaorhNBzfPcOdPXu25Uct2vr6Oi9vdwFe4u4Chw8fxl9//dWy7P3338fDhw8zqoj6hWfQXeCHH35omc0dGhrC1NRUhhVRv/AMugs0Gg0cO3asZdm9e/dw9OjRjCqifuEZdBc4evQojh8/DsuyYFkWjh8/znDuEgzoLjE9PR0GdHp6OutyqE94ibtLPHr0CO+++y4A4O+//8ahQ4cyroj6wdiA7t+/P/HHC0Rxw8PDePbsWdZl9MTYgFqWha+//po/64t4+vQpLMvCG2+8kXUpYiwsLODWrVswdJib/XnQ06dP4/Tp01mXQYJtbGzg1q1bWZfRM04SEQnGgBIJxoASCcaAEgnGgBIJxoASCcaAEgnGgBIJxoASCcaAEgnGgBIJxoASCcaAEgnGgBIJtqcD6vs+KpUKJiYmsi6FKNGeDujly5fx3XffoVarZV1KT1ZXVzEzMwPLsjAzM4OlpaXUbei/Y5T0mJ2dRa1WQ7PZ3IHqqRt7OqBzc3NZl9CzZrOJ27dvY25uDkEQYHx8HF9++WXqg41SCp7nhd8HQQClFJRSOHnyJObn5zE1NQXf9/u9CdSFPR1Qky0vL8O2bQDAyMgIvv32WwDo6XJ9dHQ0/HpkZCT8+uOPP8b169cBAOfPn+eZNAN7KqDNZhOVSgWWZWFiYgL3799PXM/3fczOzobr6UvH+D1rrVYL11ldXW1pQ79+fn4evu+H//V6qz66pcMZ5zhOy/f5fB75fD5V21Gjo6O4ePEiarUalpeXW54zYT8ZTxkKgCqVSqleY9u2chxHBUGglFKqXC4rACq6GzzPU7Ztq3K5rJRSanFxUQFQ9Xpd2bYdrr+ysqKUUsp1XQVAOY4TtlEoFJTrukoppYIgULlcrus+ehUEgQKgqtVqy/JcLqdyudyWr4/vh6S2o9toyn4qlUodt8sExlaeNqDValUBUI1GI1ymB170DdShjfelB3nSQI4vA6A8zwu/9zwvVR+9WFxcVLZthweftDYLaNLzpuwnBjQjaQPqOE7iGxUfNNGjf/yRtH7SMt1XuVxODMxWffTCtu3wbNWLtAE1ZT8xoBlJG9BOb2zSUT3NQE1a1mg0WgZXoVDoqpZelctlVSwWt9VGN5e40TOXKfuJAc3ITgc0eim8VTud2q7X6+FZIjr4tuojjXq9vq1L43hNSfS93+LiYtv60vcTA5qRtAEtFosKaJ9giA8avV4ulwsvuzzPCwdOt/dW0Uu2er2eqo9uJb1GD/a0OoVHT9TYtt2y3JT9xIBmJG1A9SyibdvhzKE+MwD/zS7qiYr4w3Xdluf0gIlONOkJDz2odD+u67YMqs366JYOTlI70ZncbmZxo9sQD4wOZ3Qyx6T9xIBmJG1AlXo1APSllOM4LdP40QHoum445e84Tjgg4gNls2X6SJ90b7VZH93S25H0iF4SbhXQTm3oujebeDJhP5keUKP/eVKpVOI/T6JNLSwsYHJy0th/nrSnfpOIyDQMKJFgRv/7wd0o/ruonZh6yUbpMKDCMHgUxUtcIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsEYUCLBGFAiwRhQIsGM/osKRN0ydJib+3GzX3/9FY8ePcq6DDLAoUOHsi6hZ8aeQYn2At6DEgnGgBIJxoASCTYI4P+yLoKIkv0PnxArdIiSjs4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras.utils.plot_model(model, to_file=\"./img/neural_networks/keras_model.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The network hyperparameters can be retrieved via: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'batch_size': 32,\n",
       " 'epochs': 1000,\n",
       " 'steps': None,\n",
       " 'samples': 409,\n",
       " 'verbose': 0,\n",
       " 'do_validation': True,\n",
       " 'metrics': ['loss', 'val_loss']}"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "history.params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now assess model performance on the test set: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "51/51 [==============================] - 0s 137us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "6.476306522593779"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mse_test = model.evaluate(X_test_scaled, y_test)\n",
    "mse_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now see how to perform predictions; we will use the first 5 units of the test set: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[26.609444],\n",
       "       [23.78115 ],\n",
       "       [19.127851],\n",
       "       [20.119244],\n",
       "       [19.274319]], dtype=float32)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred = model.predict(X_test_scaled[:5])\n",
    "y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Other Hyperparameters. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Learning Rate**. To set the learning rate, simply initialize a `optimizer.SGD` object and set a learning rate: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "sgd_optimizer = keras.optimizers.SGD(lr=1e-4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we re-compile the model just estimated and re-train it, the model will continue the training from the current state of the weights and biases (this is call \"warm start\"). If you want to re-train the model from scratch, you should create a new one. Let's continue the training of the neural network, with the new learning rate (the model is compiled with the optimizer just created): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=\"mean_squared_error\", optimizer=sgd_optimizer)\n",
    "history_2 = model.fit(X_train_scaled, y_train, epochs=1000, batch_size=32,\n",
    "                    validation_data=(X_valid_scaled, y_valid), verbose=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAE9CAYAAAAbGFuyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZhc1X3u+++v5qquntQttUaQBEICSSCBzOABC7BNPHJ87MTgCRNiQuwTbJ8b4oPzPB6Sm+RgfK/n4RLPsc0QbB/jQPCAkTGJjZFAA6MEEhpbUs/d1dVd47p/rGqpaVrdEqi6tFvv53n66aq9d+1ae9Wu/a699qoqc84hIiIiwROqdQFERETkpVGIi4iIBJRCXEREJKAU4iIiIgGlEBcREQkohbiIiEhARWpdgGPV2trqFi5cWOtiTGuDg4PU1dXVuhjTnuq5+lTHU0P1XH0bNmzodM7NHDs9cCG+cOFC1q9fX+tiTGvr1q1j7dq1tS7GtKd6rj7V8dRQPVefme0cb7q600VERAJKIS4iIhJQCnEREZGACtw1cRERCZZCocCePXsYHh6udVFOeIlEgvnz5xONRo9qeYW4iIhU1Z49e6ivr2fhwoWYWa2Lc8JyztHV1cWePXtYtGjRUT1G3ekiIlJVw8PDtLS0KMAnYWa0tLQcU4+FQlxERKpOAX50jrWeFOIiIiIBpRAXEREZI51OH3He888/z4oVK6awNEcWuBDvGczXuggiIiInhMCF+N7eIfLFcq2LISIiAfLxj3+cr33ta4fuf/rTn+Yzn/kMl112Geeeey4rV67kZz/72TGvd3h4mGuuuYaVK1eyevVqHnjgAQCeeOIJzj//fFatWsXZZ5/Ntm3bGBwc5M1vfjPnnHMOK1as4I477njZ2xW4j5g5YOuBAVbMa6x1UURE5Bh95udP8OS+/uO6zrPmNvCpty6fcJkrr7ySj370o3zoQx8C4M477+S+++7jYx/7GA0NDXR2dnLhhRfytre97ZgGl331q18FYMuWLTz99NO84Q1vYOvWrXzjG9/gIx/5CO95z3vI5/OUSiXuvfde5s6dyz333ANAX1/fS9ziwwJ3Jg6wZe/L33ARETl5rF69moMHD7Jv3z42bdpEc3Mzc+bM4ROf+ARnn302r3vd69i7dy8HDhw4pvU+9NBDvO997wNg2bJlnHrqqWzdupWLLrqIf/qnf+Lmm29m586dJJNJVq5cya9//Ws+/vGP87vf/Y7Gxpd/Mhq4M/GwGZv39HHV+bUuiYiIHKvJzpir6Z3vfCd33XUX+/fv58orr+SHP/whHR0dbNiwgWg0ysKFC4/5W+Wcc+NOf/e7380FF1zAPffcw+WXX843v/lNLr30UjZs2MC9997LTTfdxBve8AY++clPvqxtClyIJ2NhNu7urXUxREQkYK688ko++MEP0tnZyW9/+1vuvPNOZs2aRTQa5YEHHmDnznF/7XNCF198MT/84Q+59NJL2bp1K7t27WLp0qVs376dxYsXc8MNN7B9+3Y2b97MsmXLmDFjBu9973tJp9N897vffdnbFLgQT8cjPNXez56eLPObU7UujoiIBMTy5csZGBhg3rx5zJkzh/e85z289a1vZc2aNaxatYply5Yd8zo/9KEPcf3117Ny5UoikQjf/e53icfj3HHHHfzgBz8gGo0ye/ZsPvnJT/LII49w4403EgqFiEajfP3rX3/Z21TVEDezjwAfBAz4F+fcF8bMN+CLwJuALPAB59yjE62zMRmlH7jv8f38xWsWV6fgIiIyLW3ZsuXQ7dbWVn7/+9+Pu1wmkzniOhYuXMjjjz8O+B8sGe+M+qabbuKmm256wbTLL7+cyy+//CWU+siqNrDNzFbgA/x84BzgLWa2ZMxibwSWVP6uAyZtlsQiIc6a08BPHt17xGsRIiIiJ4Nqjk4/E/iDcy7rnCsCvwXePmaZK4DvO+8PQJOZzZlsxR945UKebO9n3TMdx7/UIiIi+LP2VatWveDvggsuqHWxXqCa3emPA/9oZi3AEL7LfP2YZeYBu0fd31OZ1j7Riv/b6nl88f5t/O//eJpXL2klGg7kJ+VEROQEtnLlSjZu3FjrYkyoaiHunHvKzG4GfgVkgE1Accxi432i/kV95GZ2Hb67nba2Nv7roQd5x6IyX3psgL/9zq+54vTYcS79yS2TybBu3bpaF2PaUz1Xn+p4akxWz42NjQwMDExdgQJueHj4qPfbqg5sc859C/gWgJn9E/5Me7Q9wIJR9+cD+8ZZz63ArQBr1qxxa9euZS2w0z3Gzzbt488uOY+LTmupwhacnNatW8fatWtrXYxpT/VcfarjqTFZPT/11FPU19dPXYECLpFIsHr16qNatqr90GY2q/L/FOC/A7eNWeRu4P3mXQj0Oecm7Eof7R/fvpKFLXV89I7H6NYPo4iIyEmm2heTf2xmTwI/Bz7snOsxs+vN7PrK/HuB7cCzwL8AHzqWlafjEb787tX0DBa4/l83KMhFRGRcE/20aJBVuzv9NeNM+8ao2w748Mt5juVzG/ncn53D3/zbJt765Yf4xnvPY+V8/TiKiIhMf9NiWPfbzpnLXddfBMA7vvFf/PSxsZfeRURE/Hed33jjjaxYsYKVK1ce+jnQ9vZ2Lr74YlatWsWKFSv43e9+R6lU4gMf+MChZT//+c/XuPQvFrivXT2Ss+c3cff/eBUf/tGjfOyOTfxxRw+feNMy6hPRWhdNRERG/Mf/gv1bJl/uWMxeCW/830e16E9+8hM2btzIpk2b6Ozs5BWveAUXX3wxP/rRj7j88sv5u7/7O0qlEtlslo0bN7J3795D387W23vi/W7HtDgTH9GSjvP9P7+A6y5ezB2P7OI1n32AL9+/jb6hQq2LJiIiJ4CHHnqIq666inA4TFtbG6997Wt55JFHeMUrXsF3vvMdPv3pT7Nlyxbq6+tZvHgx27dv56//+q+57777aGhoqHXxX2TanImPiEVCfOJNZ/LWs+fyxfu38v/8aiu3Pridq1+5kD9/9SJm1Okz5SIiNXOUZ8zVcqSv67744ot58MEHueeee3jf+97HjTfeyPvf/342bdrEL37xC7761a9y55138u1vf3uKSzyxaXUmPtrK+Y188+pXcM8Nr+Y1Z7Ty1XXP8pqbf8PX1j1Lx0Cu1sUTEZEauPjii7njjjsolUp0dHTw4IMPcv7557Nz505mzZrFBz/4Qa699loeffRROjs7KZfLvOMd7+Af/uEfePTRCX+fqyam3Zn4WMvnNvK195zHtgMD/P2/P8ln73uGz/3iGc5fNIO3r57HFavmkYiGa11MERGZAm9/+9v5/e9/zznnnIOZ8dnPfpbZs2fzve99j1tuuYVoNEo6neb73/8+e/fu5ZprrqFcLgPwz//8zzUu/YtN+xAfsaStnn+99gKePTjA3Rv3cc+Wdj7+4y185udPcsmyWbxl5RzWLp1FMqZAFxGZbkZ+WtTMuOWWW7jlllteMP/qq6/m6quvftHjTsSz79FOmhAfcfqsev7nG5bysdefwR+2d/Pzzfv4xeP7uWdzO6lYmAsXt/CKhTN4x3nzmFWfqHVxRUREjuikC/ERZsZFp7Vw0Wkt/P3blvPwjm7u3dLOH3d085unD3LLL55mYWsdq+Y3sfrUZs6YleaUlhRt9QlCofF+t0VERGRqnbQhPlokHOJVp7fyqtNbAdjROcj/eWwvT7b38+C2Tn7y2N5Dy8YiIeY3J5nfnOKixS0sak1Rn4iyoDnF/OakAl5ERKaMQnwci1rr+NjrzwD8xxH29g6xo3OQnV1Zdndn2d2T5Zn9A9x8X8cLHhcOGc2pKItb0zSmokTDhmEsn9fAW1bOZW5Tgoh++1xETkLOOcx0kjOZI30E7kgU4pMwM+Y3p5jfnOI1S144r2+owO7uLJlckR2dg+zpydI5kGdH5yC7u7MMF0o44J4t7Xz2vmeIhUMsnV1PLBJiUWsdp81MU3aO/qECqViEtoY485tTLGxNMVwoUyyXmVWfIJsvks2XGMwd/h8Nh5hRF6M+EaHsIFcs0ZqOM6MuRlQNBRE5gSQSCbq6umhpaVGQT8A5R1dXF4nE0Y/HUoi/DI3JKI3z/I+tXLj4yL9n/uzBAR7a1sne3iGe3j/Avt4h7t7UR77oP7ZgBsfY+JpQfTxCU12UXKFMcypGOuEbCCNOm5mmPhEhGYsQNiMSMgZyRdoa4tTFI6zfXySzeR+rT2mmNR0jFg5hZjjncA56snmaUzFCIaN/uED/UIH6RJRkNIzDEY/4Ef7ZfJFUTLuYyMlu/vz57Nmzh46OjskXPsklEgnmz59/1MvrCDsFTp9Vz+mz6l8wzTlHNl8CIBkN0zWYZ7hQYk/PENsODtCQiGIG3YN56mIRUvGw/x8Lk4pFKJTLdGfy9A0VCIUgHArRN1SgZzBP92CenmyeaDhE/1CBnmyeZ/YPEDJjqFDi3i37Jy/0xscO3YyGjbp4hIHhIqWyOzStMRmlM/Pin3+Nho1EJMxArkhrOk5zKkp9IkKx7FjQnKIhGSUVC7Nsdj0HB3I0JCKY2aEeh0KpTGs6TjQcYmC4SDRstDUkKJTKLJiRIhEJV7bZGMwVaU7F6B8u0pXJ0TWYJ5srMqcpSWMyyupTmujLFugazNM/VGBf3xDbOwZpTEZZMa+Rc+Y36WOFIlUWjUZZtGhRrYsxLSnEa8TMB+OImfX+THnBjBQXnXbks/rjoVgqM1wsk80VKTsolMokY2F2dA4SDhlPbHqMs85ezZPt/XQM5BjKFxkulGlIRjB8eHdn8/QM5jmlJUVrXZz2vmFC5nsVMrkSw4US9YkInZkcXRnfqEjGwjy1v5/+oQL9w8VDPRFjhQzKx7FnYiKRkLF8bgOpWISebB7nIBELs7cnS/9wkZnpOKEQFEuOcMg4tSVFQyJKoeQYKvhGzdzGJPOakxgQj4Y5ZUaKsnP0DOYxM/LFMqfPSjOYL5KO+wZL/1CBPV0lWvf2kckVyeaLZHL+UklmuMishjgNySj18Qg92QKLWutY3Fo37sBJ5xzPd2V5qr2fbQcypGJhTptVR3MqxpzGJLPq45ScI18sk4qF1Z0pMo0oxE9CkXCIdDhEOv7Cl7817RsS/dtDnHdqM+ed2ly1MuSKJbZ3DLJgRopsrghAKh4hGQ1j+C77kvNd8/limd09WRKRMAf6hymVHcWyo+zcod6GplSUlnSclroY4ZDRPZjnQP8wm/f0URf3wVqfiNKYjLJ0dj0Dw0U27u5hw84e1j/fw1ChxPzmJEOFEs7BZcvaaEpF2VdpnERCIbL5ItsOZjjYnyMSDpGKhQkZ/GF7F+39wy/pksjNjzx01MumYmEak/6yBQZ1sQh9QwV6s3n6h4tHvR4z33gJVS6lxKNh5jQmmFkfJ2xGYypKXcz3nOSKJZqSMc6e38ishjgtdXEKpTLZfInWdIyhQond3UNEw0ZDMsoz+wcOjdmIRkLEwkY0HCISDlEXCzOrPkEmV2QwVySdiDCUL7G3d4hYOEQ6ESEeCZGMhlnYWseMuhiRkFF2fvxJe98Q2XyJaNiYmU7gcDQl/biQsY2bXLFErlimPVPmV08eYH//MDPTcfKlMpGQ0dYQxznozOTZ2zvEvKYEZ85poH+oyHCxxNymJPWJCK4MJefIDBcZKpSIhP3zPHcwQ65YpiUdoykZIx2PkIiGGC6U6c7mKZTK5Apl9vZmaamL05SKkk5EOHVGHclYmHJl/w3iQFfnHEOFEoWSozebpy4eOebBWHL8WNAqf82aNW79+vW1Lsa0tm7dOtauXVvrYgRKqewIGQwXyuzsHiQSCtGUipIrlomGjN09WeriEXqzBaJhIx2P8uuH/siSM5eTjkeoi0eoi/tLJalYmIMDOTozOYolR108wtYDAzyzf4BMrugHTDrI5Io0paI0JKIsaUuztK2epbPrKZUdu7qz9GTz7OrK0p0tUCiVSUT8mIVypRE00hgazBXZ3jFIrliiWPY9CEOFEqWyIxYJ0ZMtHLqMMtVi4RCFcnnCBlJjMkokZORLZWam4wzmixwcyB3XcSbHy0gnyEjZZtX7wajxaJjWuhjFsiMZDTOYL5Kp9MoM5oqV1zrG7MYEbQ0J5jUlyeQK9Gb9eJRo2MjkivQPFWmpi9GYihIOGe29QzRXGkO9ldex7Bw92QKJaIimZIwZ6Ri92QKZXJHuwRzRcIjOTI7nDg4SrTTCHDCjLkY2V6RrME9uTC9aMgJzmuporovRnIrRkIiAQaHkDjXSE9Ew5cq4mpEGeDIWJhULU5+IMLshQa5YJlcsVy4j+vdBXTxCUzJKUypKYyrGrPo49YkIhZIjFQszuzFBfaWHayhfomswRyQUIhYJ4ZyjM5OnuS5KU9L/+FUscrjhVC47dnQNsmVPH43JKA3JKMVSmZ5snsFcCTPfeH5sV69vtNXFSMUjpONhls9tpFAqM1QoMas+zqktdYcGFRdLZfKV95wZfrsKZVJxXwcH+nKUnSNTqU/wPZBhM2ZW1jWYKxKPhhjKl/xxIR7Z4JxbM3af0pm4yHEQrpwJJmNhls1+8c8Vzmp48WjT9tYwa5fPHnd9TakYZ7QdHkdxrL0iLen45AsdpVyxxK6uLJ0ZP94iEjaS0TD7eodoTEaZ15ykf6hI2TkWttTRWh+jUHTkS2UKo/66B32vQToeIZ2I0D/kD1Lzm5MUS46+oQL5UplsrsSu7uyhMQ6pmO8paEr5g3DIjIHhAqGQ0Zct8MyBAeKRENFwiL29QzRVyhSLhOjeu4M3vnoNM+piDOaKJKIh8kXHgYFhcoUyc5t8IO7szrLtwACNyRiJqF/PUL6EmREyDg3cLJbLlMqOha11pOMRDvbn6B3y9WLm66U5FfW9DyFjTlOSgeECXZk8/cMFnj2YwfBjWMqVj6/2DPp57X3DlJ1vXKUTEdJxH2x18Qh1sTB7e4fpGBhm0+5e7tm8j1TMN/y6B/3ZcDIaJhYJ8dhwkb6hAsWyO9TbVio7EtEQqViEcMhfyhvMFWnvG8I5aEpFScUitKRjlMp+nMmV58+gVGnslR10DAzTlIoxoxLU0bCRjIXJFcr85+atJJsa6Mzk2NOTZaDSMxSr9KwA5Etlf8kNw8xfxhvKlxjMl8jkii9oKMbCIfKlsu/tqSwzkZHXaKKfnR4ZQDyvKYlzjng0TFcmd1S9WLFwiHg0dGi7xhMJGeGQ37bhQvkF00Y3esIhO+ZGcWqCcTsKcRGZUDwSZklbPUvajuFBL+EXfxcc+0MmtW7dnnEbQGfxwoZWSzrOuacc++Wj0Q2tqTTSgzryqZGx4xxKZUd21BiMIylULi+83HESi4s7Wbv23Jf8+FyxRF+2QDwaJhEN+R6Ykjt01pwvlukbKtA3lOf5zizFctk30gYLtPcO+V6I4SLzmvxloVIZ8sUSZQet9XH6snna+4b95Y5BfzmsezDPhYtbWL2giZXzG+kfKjBcLBM2o7kuSjruP747mCsearSNlLMzk2d7Z4a6mL8EtL9/mG0HM5TLDgfEI34b+ocL5ItlZtTFSSciDAz73pA5jclDl6BGBjE7Bw7Hpt19FCvjlHzjK8xju3p46gh1pxAXEQmY0aE7XgCHQ0Z9Ijrpek6U75SIR8LManjh2WYsYqNuh5hZH2dmffxFn/SZSiPlnNWQ4Ky5L+5xOx5eeVrri6Zd/cqFfPGq8Zc/MV5BEREROWYKcRERkYBSiIuIiASUQlxERCSgFOIiIiIBpRAXEREJKIW4iIhIQCnERUREAkohLiIiElAKcRERkYBSiIuIiASUQlxERCSgFOIiIiIBpRAXEREJKIW4iIhIQCnERUREAkohLiIiElAKcRERkYBSiIuIiASUQlxERCSgFOIiIiIBpRAXEREJKIW4iIhIQCnERUREAkohLiIiElAKcRERkYBSiIuIiASUQlxERCSgFOIiIiIBVdUQN7OPmdkTZva4md1mZokx8z9gZh1mtrHy9xfVLI+IiMh0UrUQN7N5wA3AGufcCiAMXDnOonc451ZV/r5ZrfKIiIhMN9XuTo8ASTOLAClgX5WfT0RE5KRRtRB3zu0FPgfsAtqBPufcL8dZ9B1mttnM7jKzBdUqj4iIyHRjzrnqrNisGfgx8C6gF/g34C7n3A9GLdMCZJxzOTO7Hvgz59yl46zrOuA6gLa2tvNuv/32qpRZvEwmQzqdrnUxpj3Vc/WpjqeG6rn6Lrnkkg3OuTVjp1czxP8U+BPn3LWV++8HLnTOfegIy4eBbudc40TrXbNmjVu/fv1xL68ctm7dOtauXVvrYkx7qufqUx1PDdVz9ZnZuCFezWviu4ALzSxlZgZcBjw1plBzRt1929j5IiIicmSRaq3YOfewmd0FPAoUgceAW83s74H1zrm7gRvM7G2V+d3AB6pVHhERkemmaiEO4Jz7FPCpMZM/OWr+TcBN1SyDiIjIdKVvbBMREQkohbiIiEhAKcRFREQCSiEuIiISUApxERGRgFKIi4iIBJRCXEREJKAU4iIiIgGlEBcREQkohbiIiEhAKcRFREQCSiEuIiISUApxERGRgFKIi4iIBJRCXEREJKAU4iIiIgGlEBcREQkohbiIiEhAKcRFREQCSiEuIiISUApxERGRgFKIi4iIBJRCXEREJKAU4iIiIgGlEBcREQkohbiIiEhAKcRFREQCSiEuIiISUApxERGRgFKIi4iIBJRCXEREJKAU4iIiIgGlEBcREQkohbiIiEhAKcRFREQCSiEuIiISUApxERGRgFKIi4iIBJRCXEREJKAU4iIiIgGlEBcREQkohbiIiEhAKcRFREQCasIQN7P3jrr9qjHz/ke1CiUiIiKTm+xM/H+Ouv3lMfP+/DiXRURERI7BZCFuR7g93n0RERGZQpOFuDvC7fHui4iIyBSKTDJ/mZltxp91n1a5TeX+4qqWTERERCY0WYifOSWlEBERkWM2YXe6c27n6D8gA5wLtFbuT8jMPmZmT5jZ42Z2m5klxsyPm9kdZvasmT1sZgtfxraIiIicVCb7iNm/m9mKyu05wOP4Uen/amYfneSx84AbgDXOuRVAGLhyzGLXAj3OudOBzwM3v6StEBEROQlNNrBtkXPu8crta4BfOefeClzA0X3ELAIkzSwCpIB9Y+ZfAXyvcvsu4DIz06h3ERGRozDZNfHCqNuXAf8C4JwbMLPyRA90zu01s88Bu4Ah4JfOuV+OWWwesLuyfNHM+oAWoHP0QmZ2HXAdQFtbG+vWrZuk2PJyZDIZ1fEUUD1Xn+p4aqiea2eyEN9tZn8N7MFfC78PwMySQHSiB5pZM/5MexHQC/ybmb3XOfeD0YuN89AXfXTNOXcrcCvAmjVr3Nq1aycptrwc69atQ3Vcfarn6lMdTw3Vc+1M1p1+LbAc+ADwLudcb2X6hcB3Jnns64AdzrkO51wB+AnwyjHL7AEWAFS63BuB7qMuvYiIyElswjNx59xB4Ppxpj8APDDJuncBF5pZCt+dfhmwfswydwNXA78H3gn8xjmnL5ERERE5ChOGuJndPdF859zbJpj3sJndBTwKFIHHgFvN7O+B9c65u4Fv4Ue6P4s/Ax87el1ERESOYLJr4hfhB57dBjzMMX5funPuU8Cnxkz+5Kj5w8CfHss6RURExJssxGcDrweuAt4N3APc5px7otoFExERkYlN9o1tJefcfc65q/GD2Z4F1lVGrIuIiEgNTXYmjpnFgTfjz8YXAl/CjzQXERGRGppsYNv3gBXAfwCfGfXtbSIiIlJjk52Jvw8YBM4Abhj1jagGOOdcQxXLJiIiIhOY7HPik30ZjIiIiNSIQlpERCSgFOIiIiIBpRAXEREJKIW4iIhIQCnERUREAkohLiIiElAKcRERkYBSiIuIiASUQlxERCSgFOIiIiIBpRAXEREJKIW4iIhIQCnERUREAkohLiIiElAKcRERkYBSiIuIiASUQlxERCSgFOIiIiIBpRAXEREJKIW4iIhIQCnERUREAkohLiIiElAKcRERkYBSiIuIiASUQlxERCSgFOIiIiIBpRAXEREJKIW4iIhIQCnERUREAkohLiIiElAKcRERkYBSiIuIiASUQlxERCSgFOIiIiIBpRAXEREJKIW4iIhIQCnERUREAkohLiIiElAKcRERkYBSiIuIiASUQlxERCSgFOIiIiIBpRAXEREJqKqFuJktNbONo/76zeyjY5ZZa2Z9o5b5ZLXKIyIiMt1EqrVi59wzwCoAMwsDe4GfjrPo75xzb6lWOURERKarqepOvwx4zjm3c4qeT0REZNoz51z1n8Ts28CjzrmvjJm+FvgxsAfYB/yNc+6JcR5/HXAdQFtb23m333571ct8MstkMqTT6VoXY9pTPVef6nhqqJ6r75JLLtngnFszdnrVQ9zMYviAXu6cOzBmXgNQds5lzOxNwBedc0smWt+aNWvc+vXrq1dgYd26daxdu7bWxZj2VM/VpzqeGqrn6jOzcUN8KrrT34g/Cz8wdoZzrt85l6ncvheImlnrFJRJREQk8KYixK8CbhtvhpnNNjOr3D6/Up6uKSiTiIhI4FVtdDqAmaWA1wN/OWra9QDOuW8A7wT+ysyKwBBwpZuKi/QiIiLTQFVD3DmXBVrGTPvGqNtfAb4y9nEiIiIyOX1jm4iISEApxEVERAJKIS4iIhJQCnEREZGAUoiLiIgElEJcREQkoBTiIiIiAaUQFxERCSiFuIiISEApxEVERAJKIS4iIhJQCnEREZGAUoiLiIgElEJcREQkoBTiIiIiAaUQFxERCSiFuIiISEApxEVERAJKIS4iIhJQCnEREZGAUoiLiIgElEJcREQkoBTiIiIiAaUQFxERCSiFuIiISEApxEVERAJKIS4iIhJQCnEREZGAUoiLiIgElEJcREQkoBTiIiIiARXAEHe1LoCIiMgJIXghfuCJWpdARETkhBC8EC8VINtd61KIiIjUXPBCHKBzW61LICIiUnMBDfFnal0CERGRmgteiFsIOrfWuhQiIiI1F7wQj8ShQyEuIiISwBBP6ExcRESEoIZ4704oDNe6JCIiIjUVwBCPgytD93O1LomIiEhNBS/Eo0n/f8/62pZDRESkxgvIdDcAABEXSURBVIIX4pEENC6Abb+sdUlERERqKnghDnDG5fDcA7ouLiIiJ7WAhvifQGEQnn+o1iURERGpmWCG+MLXQDQFT/+81iURERGpmWCGeDQBy/87bLoDBjtrXRoREZGaCGaIA7zqBigOwR9vrXVJREREaqJqIW5mS81s46i/fjP76JhlzMy+ZGbPmtlmMzv3qJ9g5lJY+mb4w9ehS58ZFxGRk0/VQtw594xzbpVzbhVwHpAFfjpmsTcCSyp/1wFfP6Ynufwf/Re//PpTUC4fh1KLiIgEx1R1p18GPOec2zlm+hXA9533B6DJzOYc9VpnLIKLPgxP/Rx+9uHjWFwREZETX2SKnudK4LZxps8Ddo+6v6cyrf2o17z2Jijm4D+/AI3z4dUfhVjdyyqsiIhIEJhzrrpPYBYD9gHLnXMHxsy7B/hn59xDlfv3A3/rnNswZrnr8N3ttLW1nXf77be/8DnKBc586gvM6niIXGwGW8/4K7paz6/eRk1zmUyGdDpd62JMe6rn6lMdTw3Vc/VdcsklG5xza8ZOn4oQvwL4sHPuDePM+/+Adc652yr3nwHWOueOeCa+Zs0at379Eb43fduv4d8+APkBWPYWOP+DsHjty9+Ik8y6detYu3ZtrYsx7ameq091PDVUz9VnZuOG+FRcE7+K8bvSAe4G3l8ZpX4h0DdRgE9qyevgL38LK94JT/87fP8K+PIauO0qePoeGOqF9s0vefUiIiInkqpeEzezFPB64C9HTbsewDn3DeBe4E3As/jR69e87CdtOQ3e+S146xfh3hth04+gaxs8c+/hZRoXQCELc1ZBKAwWgkSj/+KYxnkQScK+xyDZBIMdMHuln9+/D065COaeC0/+H8hnID0bBvZB80LIZ/1gO4Adv/O356zyPQMLLoCuZ6FU9PN7d0I4BrGUf1zfHmhdAk2n+udKNEJ6FvTsgGw3zDrTX+sv5iCXgVyf/9a6eL3/nx+EwhBkO/00V4amU8A5eO43gIOmhVDf5h+f2Q+zz4HQOO045/xzDnZCy+lg5v8mkunw9RFJQMM4YxOz3f6v9fSjex2dg3IRisN+e45GuTz+9kz2PJNtm4jICarq3enH24Td6eMpFWGgHbbe50OpfaMP0xmL/fThfigVINcP9bP92Xr2BPkWOAv5MB6RnAFDPcCo1ywU8eE+3Pfi5eONEI5Atmv89cfq/WNd2a8n1++n5zMvXK55oa+j5Az/nfV9e324h8JQLkGyGdo3+cYK+EbQ7BX+Z2NLBSjlYW9lmMPss6Fupq/j3IBvyDTMhZ6dcMqFvvHUvcOXqZTzj5l3HvS3Q12rb/zUz/W/K5854F+vxa/1z/HcA75u5pwDdbP8a9y/1zfEWk73yzbO9+sol+Dgk758S9/on3f/Zlj0Wv/cw30wd7Xfxvo50PEMDB70jZTmhb7s5ZJvZEST/jH5Qd+gWvgqX0cAA/uhbxfMW+PnJRr8YxrmsXXzI5yx5HS/zr7dvh46t0E46huKpbyf/vhP/L457zywMKSaIVa5/lgq+LI2zvPl6dvty7bnEb8tuX6YcRo0zIPdD/uGYmqG3+/zg74xO9gBrWf457Owr6sDj/ttiqV9mTd81/dwtZ0F+zb6umiY559/13/B/FfAzGXw60/71/fUV/r7z/4a2pb717lzGyx5Awz3+m0s5nxZQ2G/b0USlX245Buwndv8/p5ohESTf5yFfMNrYJ8vc7nkX5/Mft9Qbt8E8Qbo3QUHn+Cp3V2c2f5TuOpH/jl6nvf7dzHnG7nJJv9a9e/z+0vrEuh42q93wQX+eZzzddMwz5ehv9JoH+z0+9fMZYffKyP72sEn/foH2qF5kX+fWcjvR6kW2HKnf8+d/Wd+/ZkDvo7qZvrGa6IRMP+FVv37fB01zIdI7HDDs1z2/3MD/pg28l7q3+dPDmYsfuH7OJfx+16qxT9HKPLiBuzohvDuP/qTimjS7wPO+ddzwfmV8h02aXd6Ydh/0+Zkunf41z5W5/etcPTwfj5yezzZbn8cAr9fx9N+3xmZNpnhfr9vzDn76JZ/ubLdvv4TDUe3fDGPReO1uSZ+vB1ziL8UI585b3/MHzyyXb7CZ50J3dv9gaLpFGhbATt+6/9HEv6gt/U+/6Yy8wfR3p2VAOiE01/nD7LhKITjgPNhE01C51bfQ1Au+LDJ7Pc7lpn/7fS6mf52YciXo3+fP7BjPhDzg36d9W2+7MWcD85Ui1//3NWw6w8+/MpFP35g6Rv9G8uVD7/B07Poev4JWpav9QeuwpA/yLmyL+++jf4gO3ulP+CbwcABqGuBwS5/kCsVfHmjST8/HPMHqf59vuzZbl+OVIuvj8Guww2AplP9eIZsJ2y+w0+LJHzd9e/zB+dw1B8c4w3+dn7Ql/0Q82UNx33wZ7sOz7eQD7We531dg98OVzqKHaOyXvAH9MEOf2BrmOufY2zjR2S0UMS/947KqH1ttEQjFPN+v84c8O+x4b7xVzHrLL+/N8zzvZHd2/30ZLM/tkQSvueyXPTHn2jKH7tiad9YGN34n322f77MAQhF/fEvN1BpzHRQGOgkOvM0f7wYKXck7o9DFvINo3Sbf7/MOce/9wY7D29nftAfF3J9kGr1jdIDT/rGYLwetv7CB3Oi0b9fZ6/0j2vf7Nff87w/FkQS/hg05xw/7/TLfGMwmvTlidb5nsL+ff7Y3rnVNzJ3/qff3tMu9c+XG/D1nOuDxlP88ah3lz+ODuz32z2w3/egxuv9sbJ1iT9Odm712z9jsT+Wt5zuy7b5Tn8sTM984Wux6GLfwOzf68tZLvnGdTjuG2cWgoF27BN7FOJydGo2SGW8ru181r8xx3aTj22Zjxx8XPnwGWU46tfnnH+D1M30B9JQ+PDz5fr9Y+INcOCJSuPEfA9OOFI52875N1Vdq2/ANS7wZ7Ojy1zM+xAfuazRMNeXabjP3y4O+wNQz07/ph9o548bH+f8c1f58iaa/LL1bb5xs28jDHX7x0aS/rljaX9m1vN8Zfti/mCYOeDPdBrm+LPTgf2VnoYn/Los7Nfdcro/IOx7DPau95d6Zi71z/HcA/6soOkU3+hsmOsbmJ3P+IPhzKW+7lzZnyk55w9soYg/Ex3s8I3EWWf5g1LXs/456mb65ywXD/d4Rev8wWrkMkm83jdUO57yB8nkDL9NqRb/mHIBdv4XnHWFD5py0a8/VudvDxyAp37mewMaFxzuWZi5lOd+exunNZR9YzWW9utv3+Tn5wb861LX6oOuc6uvp8HKAbu+za8v1+/v182sfHzVfJnilV6VWNpfrku1+h6ozEH/2s04ze9Pezf40HBl34gdOat+7n5/lh6K+H0jN+B7uiJJ/zpnDvrtKwzBjgd9z8DIekt5Xz8Hn/LlLOZg/hofRAP7fVk6n/F137fXB8fMM/1Jxcg+2bnVvxapGT6Udv3eh9KCC/zjisO+l6Nnh39MOOZfq5HAiaV9Xde10tkzQKv1+jIlmw730oUrjYGu53yDoTjsyxpJ+P3YlaBlib/dMMefiScafHAO9fj34FCv317MlzvVAt3P+XrrevZww8hCvqetVPDTBjv8a5Tr971YdTN9ufvbfU/FSC9i/16/r/fuqrz3837Zupl+me7n/H431H34cudgh5833Od7DBsXHG7gNFb2pSM1xMaK1fvl4g3++JHrOzw9P+DLOGMxdt0DCnE5OhppOjVUz9WnOp4aL6meRy4HvNwxKQP7fQiDbyiFxwz1Kgz5RuPRGrnMF4kf3fKjL2uEY77BONTrGzOlog/iSKUnwDl/QhCre+HJwtgcHjn5cO7QCcyRRqdP1Ze9iIiIHHasg1CPpH726JW+eP6xBDgc/UDaQ09Zec7R17eTTf5/OPLC6/Jm/rLAiJEGx3gNmaNs4AT3V8xEREROcgpxERGRgFKIi4iIBJRCXEREJKAU4iIiIgGlEBcREQkohbiIiEhAKcRFREQCSiEuIiISUApxERGRgArcd6ebWQews9blmOZagRPk91inNdVz9amOp4bqufpOdc7NHDsxcCEu1Wdm68f7on05vlTP1ac6nhqq59pRd7qIiEhAKcRFREQCSiEu47m11gU4Saieq091PDVUzzWia+IiIiIBpTNxERGRgFKIn4TMbIGZPWBmT5nZE2b2kcr0GWb2KzPbVvnfXJluZvYlM3vWzDab2bm13YLgMLOwmT1mZv9eub/IzB6u1PEdZharTI9X7j9bmb+wluUOCjNrMrO7zOzpyv58kfbj48/MPlY5VjxuZreZWUL78olBIX5yKgL/l3PuTOBC4MNmdhbwv4D7nXNLgPsr9wHeCCyp/F0HfH3qixxYHwGeGnX/ZuDzlTruAa6tTL8W6HHOnQ58vrKcTO6LwH3OuWXAOfi61n58HJnZPOAGYI1zbgUQBq5E+/IJQSF+EnLOtTvnHq3cHsAf+OYBVwDfqyz2PeC/VW5fAXzfeX8AmsxszhQXO3DMbD7wZuCblfsGXArcVVlkbB2P1P1dwGWV5eUIzKwBuBj4FoBzLu+c60X7cTVEgKSZRYAU0I725ROCQvwkV+nqWg08DLQ559rBBz0wq7LYPGD3qIftqUyTiX0B+FugXLnfAvQ654qV+6Pr8VAdV+b3VZaXI1sMdADfqVyy+KaZ1aH9+Lhyzu0FPgfswod3H7AB7csnBIX4SczM0sCPgY865/onWnScafpYwwTM7C3AQefchtGTx1nUHcU8GV8EOBf4unNuNTDI4a7z8aiOX4LKmIIrgEXAXKAOf2liLO3LNaAQP0mZWRQf4D90zv2kMvnASPdi5f/ByvQ9wIJRD58P7JuqsgbUq4C3mdnzwO34rscv4LtwI5VlRtfjoTquzG8EuqeywAG0B9jjnHu4cv8ufKhrPz6+XgfscM51OOcKwE+AV6J9+YSgED8JVa5PfQt4yjn3/46adTdwdeX21cDPRk1/f2V074VA30h3pYzPOXeTc26+c24hfhDQb5xz7wEeAN5ZWWxsHY/U/Tsry+vsZQLOuf3AbjNbWpl0GfAk2o+Pt13AhWaWqhw7RupZ+/IJQF/2chIys1cDvwO2cPh67Sfw18XvBE7Bv3H/1DnXXXnjfgX4EyALXOOcWz/lBQ8oM1sL/I1z7i1mthh/Zj4DeAx4r3MuZ2YJ4F/x4xO6gSudc9trVeagMLNV+IGDMWA7cA3+5ET78XFkZp8B3oX/ZMtjwF/gr31rX64xhbiIiEhAqTtdREQkoBTiIiIiAaUQFxERCSiFuIiISEApxEVERAJKIS5ykjGzkpltHPU30becHeu6F5rZ48drfSIyscjki4jINDPknFtV60KIyMunM3ERAcDMnjezm83sj5W/0yvTTzWz+yu/wX2/mZ1Smd5mZj81s02Vv1dWVhU2s3+p/P70L80sWbONEpnmFOIiJ5/kmO70d42a1++cOx//zWZfqEz7Cv4nPM8Gfgh8qTL9S8BvnXPn4L+z/InK9CXAV51zy4Fe4B1V3h6Rk5a+sU3kJGNmGedcepzpzwOXOue2V34gZ79zrsXMOoE5zrlCZXq7c67VzDqA+c653Kh1LAR+5ZxbUrn/cSDqnPu/q79lIicfnYmLyGjuCLePtMx4cqNul9DYG5GqUYiLyGjvGvX/95Xb/4X/JTaA9wAPVW7fD/wVgJmFzaxhqgopIp5ayCInn6SZbRx1/z7n3MjHzOJm9jC+gX9VZdoNwLfN7EagA/9LYQAfAW41s2vxZ9x/BeinPUWmkK6Jiwhw6Jr4GudcZ63LIiJHR93pIiIiAaUzcRERkYDSmbiIiEhAKcRFREQCSiEuIiISUApxERGRgFKIi4iIBJRCXEREJKD+fx8ntQsGJTV6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Because of the warm start, the loss starts already at low values \n",
    "pd.DataFrame(history_2.history)[1:].plot(figsize=(8, 5))\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('MSE')\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "51/51 [==============================] - 0s 98us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "6.150179432887657"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mse_test_2 = model.evaluate(X_test_scaled, y_test)\n",
    "mse_test_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This model performs worse than the first one, a sign that the learning rate (or the number of epochs) did not improve performance. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Momentum**. Setting the momentum is also very easy; you can just add the `momentum` hyperparameter to the `optimizers.SGD` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "sgd_momentum_optimizer = keras.optimizers.SGD(lr=1e-4, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Where `momentum` is the momentum hyperparameter. You can give such object as input of the `optimizer` argument in phase of model compilation, as done above. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Adam Optimizer**. The ADAM optimizer can be selected via: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "adam_optimizer = keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time, `beta_1` and `beta_2` refer to the first and second momdent hyperparameters of ADAM. Once again, you can pass this optimizer as input of the `compile` method. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**$l_2$, $l_1$ regularization**. The $l_2$ regularization parameter can be set for each layer, when creating the model. The larger this parameter, the simpler the model will be (i.e., the closer the final weights will be to 0). Here is an example of a model built with two layers, both with $l_2$-regularized weights (the output layer is also regularized): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_regularized = keras.models.Sequential()\n",
    "model_regularized.add(keras.layers.Dense(5, activation='relu', input_shape=X_train.shape[1:], \n",
    "                     kernel_regularizer=keras.regularizers.l2(0.01)))\n",
    "model_regularized.add(keras.layers.Dense(4, activation='relu', kernel_regularizer=keras.regularizers.l2(0.01)))\n",
    "model_regularized.add(keras.layers.Dense(1, kernel_regularizer=keras.regularizers.l2(0.01)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model_regularized.compile(loss=\"mean_squared_error\", optimizer=adam_optimizer)\n",
    "history_3 = model_regularized.fit(X_train_scaled, y_train, epochs=1000, batch_size=32,\n",
    "                    validation_data=(X_valid_scaled, y_valid), verbose=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAE9CAYAAAD9MZD2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3wc5X33/c9vD9LqfLRl2TLYxMZgbHxABnIyKiYhQICceAoh4FAaP2lyh5DepUB6P7mTNq80lD6lpaW0bkgwLSX4JkkhgZCAYwVIgYDB2ICJbRxsy2fZlqyztLvX/cfMymtbtixbq/XOft+v175m5pprZ397abW/ua6ZnTHnHCIiIpLbQtkOQERERE6eErqIiEgAKKGLiIgEgBK6iIhIACihi4iIBIASuoiISABEsh3AyaitrXVTpkzJdhiB1tXVRUlJSbbDCDy1c+apjTNPbZx5q1atanXOjRtqXU4n9ClTpvDqq69mO4xAa25upqmpKdthBJ7aOfPUxpmnNs48M9t8tHUachcREQkAJXQREZEAyFhCN7MZZrY67XHAzG41s2oze8bMNvjTKr++mdm9ZrbRzNaY2fxMxSYiIhI0GTuG7pz7HTAXwMzCwDbgJ8AdwArn3HfN7A5/+XbgMmC6/7gAuN+fiohIQAwMDNDS0kJvb2+2QzmlxWIxGhoaiEajx/2csTopbhHwrnNus5ldDTT55cuAZryEfjXwkPPuFvOSmVWaWb1zbscYxSgiIhnW0tJCWVkZU6ZMwcyyHc4pyTnH3r17aWlpYerUqcf9vLE6hn4t8Ig/X5dK0v50vF8+Cdia9pwWv0xERAKit7eXmpoaJfNjMDNqampGPIqR8R66mRUAVwF3Dld1iLIj7u1qZkuAJQB1dXU0NzefbIhyDJ2dnWrjMaB2zjy1ceYdTxtXVFTQ2dk5NgHluN7e3hF9ZsdiyP0y4DXn3C5/eVdqKN3M6oHdfnkLMDnteQ3A9sM35pxbCiwFaGxsdPrNY2bpd6VjQ+2ceWrjzDueNl63bh1lZWVjE1COi8VizJs377jrj8WQ+3UcHG4HeAJY7M8vBh5PK7/RP9v9QqBdx89FRCTbSktLj7ruvffeY9asWWMYzdFltIduZsXAR4D/N634u8ByM7sZ2AJc45c/BVwObAS6gZuG2357z8CoxisiIpKrMtpDd851O+dqnHPtaWV7nXOLnHPT/ek+v9w5577snHufc262c27Ya7rubNfPHkREZGRuv/12/vmf/3lw+Zvf/Cbf+ta3WLRoEfPnz2f27Nk8/vjjx9jC0Hp7e7npppuYPXs28+bNY+XKlQC89dZbnH/++cydO5dzzz2XDRs20NXVxRVXXMGcOXOYNWsWjz766Em/r5y+lnt/IklPf4KignC2QxERkRH61k/f4u3tB0Z1mzMnlvO/rzznmHWuvfZabr31Vr70pS8BsHz5cp5++mm+9rWvUV5eTmtrKxdeeCFXXXXViM7Gv++++wBYu3Yt77zzDh/96EdZv349//Iv/8JXv/pVrr/+evr7+0kkEjz11FNMnDiRJ598EoD29vZjbfq45PylXzfu1tmSIiJy/ObNm8fu3bvZvn07b7zxBlVVVdTX1/P1r3+dc889l0suuYRt27axa9eu4TeW5oUXXuCGG24A4KyzzuL0009n/fr1vP/97+c73/kOd911F5s3b6aoqIjZs2fz7LPPcvvtt/P8889TUVFx0u8rp3voAOt3dTC74eQbQkRExtZwPelM+sxnPsNjjz3Gzp07ufbaa3n44YfZs2cPq1atIhqNMmXKlBH/Dty7LtqRPvvZz3LBBRfw5JNPcumll/K9732Piy++mFWrVvHUU09x55138tGPfpRvfOMbJ/WecjqhG7B+d0e2wxARkRxz7bXX8oUvfIHW1lZ+/etfs3z5csaPH080GmXlypVs3nzUu5Qe1cKFC3n44Ye5+OKLWb9+PVu2bGHGjBls2rSJM844g1tuuYVNmzaxZs0azjrrLKqrq/nc5z5HaWkpDz744Em/p5xO6IWRMOt3KqGLiMjInHPOOXR0dDBp0iTq6+u5/vrrufLKK2lsbGTu3LmcddZZI97ml770Jb74xS8ye/ZsIpEIDz74IIWFhTz66KP8x3/8B9FolAkTJvCNb3yDV155hdtuu41QKEQ0GuX+++8/6feU2wk9GmL9Lh1DFxGRkVu7du3gfG1tLS+++OKQ9Y51ZbspU6bw5ptvAt6FYIbqad95553ceeehF0u99NJLufTSS08g6qPL6ZPiYtEw29p66OqLZzsUERGRrMrpHnosEqIf2LC7k7mTK7MdjoiIBNTatWsHz2BPKSws5OWXX85SREfK6YReGA3TD6zf2aGELiIiGTN79mxWr16d7TCOKbcTeiREfyTE+l06MU5ERPJbTh9DB5g2vpT1uriMiIjkuZxP6GfWlbFBPXQREclzOZ/Qp9eVsqO9lwO9uvOaiIgM71i3Q81lOZ/QZ9SVAaiXLiIieS3nE/qZfkLXBWZERGQknHPcdtttzJo1i9mzZw/ewnTHjh0sXLiQuXPnMmvWLJ5//nkSiQSf//znB+vec889WY7+SDl9ljvApMoiiqJhnekuIpJrfn4H7Fw7fL2RmDAbLvvucVX98Y9/zOrVq3njjTdobW1lwYIFLFy4kP/8z//k0ksv5S/+4i9IJBJ0d3ezevVqtm3bNnhVuLa2ttGNexTkfA89FDKm15WyQT10EREZgRdeeIHrrruOcDhMXV0dF110Ea+88goLFizgBz/4Ad/85jdZu3YtZWVlnHHGGWzatImvfOUrPP3005SXl2c7/CPkfA8dvGH3X6/fk+0wRERkJI6zJ50pR7vd6cKFC3nuued48sknueGGG7jtttu48cYbeeONN/jFL37Bfffdx/Lly/n+978/xhEfW8730AGm1BSzp6OP3oFEtkMREZEcsXDhQh599FESiQR79uzhueee4/zzz2fz5s2MHz+eL3zhC9x888289tprtLa2kkwm+fSnP81f/dVf8dprr2U7/CMEoodeX1EEwI72XqbWlmQ5GhERyQWf/OQnefHFF5kzZw5mxt/8zd8wYcIEli1bxt133000GqW0tJSHHnqIbdu2cdNNN5FMJgH467/+6yxHf6RgJPTKGAA72nqU0EVE5JhSt0M1M+6++27uvvvuQ9YvXryYxYsXH/G8U7FXni4QQ+4T/R769vbeLEciIiKSHYFI6BMqDvbQRURE8lEgEnosGqampEA9dBERyVuBSOjgHUff0a4euojIqe5oPxeTg06kjQKT0CeUF7GjTT10EZFTWSwWY+/evUrqx+CcY+/evcRisRE9LxBnuQPUlRfy+pb92Q5DRESOoaGhgZaWFvbs0cXAjiUWi9HQ0DCi5wQmodeUFrKvu594IkkkHJiBBxGRQIlGo0ydOjXbYQRSRjOfmVWa2WNm9o6ZrTOz95tZtZk9Y2Yb/GmVX9fM7F4z22hma8xs/rAvkIwPztaWFuAc7O/WfdFFRCT/ZLor+w/A0865s4A5wDrgDmCFc246sMJfBrgMmO4/lgD3D7v1ve8OztaWFgLQ2tk3asGLiIjkiowldDMrBxYCDwA45/qdc23A1cAyv9oy4BP+/NXAQ87zElBpZvXHfJF4H/gnVtSUFACwt7N/dN+IiIhIDshkD/0MYA/wAzN73cy+Z2YlQJ1zbgeAPx3v158EbE17fotfdnQuAV3eiRU1fg99b5d66CIikn8yeVJcBJgPfMU597KZ/QMHh9eHYkOUHfG7BjNbgjckz3n1IV5/9jHaK2fSNeBVfWn121S0bTjp4MXT2dlJc3NztsMIPLVz5qmNM09tnF2ZTOgtQItz7mV/+TG8hL7LzOqdczv8IfXdafUnpz2/Adh++Eadc0uBpQCNE8Nu3unlMK8J5xzR5p9TOWEyTU1nZeo95Z3m5maampqyHUbgqZ0zT22ceWrj7MrYkLtzbiew1cxm+EWLgLeBJ4DUbWwWA4/7808AN/pnu18ItKeG5o/KDPZt8meNmpJCWjs05C4iIvkn079D/wrwsJkVAJuAm/B2Ipab2c3AFuAav+5TwOXARqDbr3ts4YJDznSvKS3QWe4iIpKXMprQnXOrgcYhVi0aoq4DvjyiF4jEYO/GwcXqkgL9Dl1ERPJSbl9SLVLoJfRkAoDK4gLae5TQRUQk/+R4Qo9BvBfavV+7VRZFaevW79BFRCT/5H5CB2j1fqZWWRylvWeAZFJ38RERkfyS4wndu5gMresBqCiKknTQ0Rs/xpNERESCJ7cTeigCRdWDCb2q2Lv8a1uPht1FRCS/5HZCB6g985Ahd4A2nekuIiJ5JvcT+rgzB3voqYS+XyfGiYhInsn9hF57pneDlu59VBR5Q+766ZqIiOSbYCR0gNYNVGnIXURE8lQAEvp0b9q6nooiJXQREclPuZ/QK0/3runeup5IOERZYURnuYuISN7J/YQeCkPNtMEz3SuKo+qhi4hI3sn9hA7esHvab9F1+VcREck3wUjoNdNh/3uQiFNZHKVNZ7mLiEieCUZCrzodXAIOtFBRFKVdQ+4iIpJngpHQK0/3pvs3U1kc1YVlREQk7wQjoVf5Cb1tM1X+PdF1xzUREcknwUjo5Q1gYdi/WXdcExGRvBSMhB6OQMUkaNtMuX9xmQO9Oo4uIiL5IxgJHbzj6PvfozymhC4iIvknQAn9NGjfRnksAmjIXURE8ktwEnppHXTtpjwWBuCAfosuIiJ5JFgJPRmngg5APXQREckvwUnoZXUAlMf3ATqGLiIi+SU4Cb3US+gl/a2AeugiIpJfApfQI917KC4I6xi6iIjklcAldDp3UhaLqIcuIiJ5JTgJvbAUCkqhYxflsaiOoYuISF7JaEI3s/fMbK2ZrTazV/2yajN7xsw2+NMqv9zM7F4z22hma8xs/ohfsHQ8dO5SD11ERPLOWPTQ/8A5N9c51+gv3wGscM5NB1b4ywCXAdP9xxLg/hG/UmkddO6mvEg9dBERyS/ZGHK/Gljmzy8DPpFW/pDzvARUmln9iLZcXAPdeymLRdVDFxGRvJLphO6AX5rZKjNb4pfVOed2APjT8X75JGBr2nNb/LLjV1wNPfsoj0V0lruIiOSVSIa3/0Hn3HYzGw88Y2bvHKOuDVF2xE3N/R2DJQB1dXU0NzcPrpva2sXkrlb279pOe3eclStXYjbUZuV4dXZ2HtLGkhlq58xTG2ee2ji7MprQnXPb/eluM/sJcD6wy8zqnXM7/CH13X71FmBy2tMbgO1DbHMpsBSgsbHRNTU1HVwZXQNbfsS8aXU89d423v+hhcSi4Qy8s/zR3NzMIW0sGaF2zjy1ceapjbMrY0PuZlZiZmWpeeCjwJvAE8Biv9pi4HF//gngRv9s9wuB9tTQ/HErrgGgNtQF6PKvIiKSPzLZQ68DfuIPeUeA/3TOPW1mrwDLzexmYAtwjV//KeByYCPQDdw04lcsrgag2g7eoGV82Um9BxERkZyQsYTunNsEzBmifC+waIhyB3z5pF7U76FXug6gWCfGiYhI3gjOleIAirweejkHAN2gRURE8kewEro/5F4S9xK6jqGLiEi+CFZCj1WChShKtAPqoYuISP4IVkIPhaCoilj/fgAdQxcRkbwRrIQOUFxDpK+NcMjUQxcRkbwRvIReVI1176UsFtExdBERyRvBS+jFNdCzX7dQFRGRvBLAhF4F3Xspj0V1DF1ERPJGABN6DXTvo6wwrB66iIjkjeAl9KJqSPQxrjChY+giIpI3gpfQ/YvLjI/2qIcuIiJ5I3gJPVYBQG24Rz10ERHJG4FN6DXhbjr74iSTLssBiYiIZF4AE3olAJWhbpyDrn4Nu4uISPAFMKF7PfRy6wZ0PXcREckPwUvoRV4PvYwuQAldRETyQ/ASemE5AKXJTgA6dGKciIjkgeAl9FAYCisoSnYA6qGLiEh+CF5CB4hVEEt4PXT9dE1ERPJBYBN64cABQD10ERHJD8FM6EWVRAa8IffOPiV0EREJvmAm9FgFob52wiHTSXEiIpIXAprQK7Hedt0TXURE8kZAE3oFKKGLiEgeCWZCL6qE/k4qCzTkLiIi+SGYCd2//GtdYR8H1EMXEZE8EOiEPj7apyF3ERHJCwFN6N713MdFujXkLiIieSHjCd3Mwmb2upn9zF+eamYvm9kGM3vUzAr88kJ/eaO/fsoJv6jfQ68O96iHLiIieWEseuhfBdalLd8F3OOcmw7sB272y28G9jvnpgH3+PVOjH/HtapQN519cZxzJ7wpERGRXJDRhG5mDcAVwPf8ZQMuBh7zqywDPuHPX+0v469f5NcfOb+HXhHqJpF0dPcnTmgzIiIiuSKS4e3/PfDnQJm/XAO0OedS4+AtwCR/fhKwFcA5Fzezdr9+a/oGzWwJsASgrq6O5ubmI140lOhjIdC/5z3gHH658jmqYsE8XSDTOjs7h2xjGV1q58xTG2ee2ji7MpbQzezjwG7n3Coza0oVD1HVHce6gwXOLQWWAjQ2NrqmpqbDq4Bz8JsoU2qLYTvMnr+AaePLjqwnw2pubmbINpZRpXbOPLVx5qmNsyuTPfQPAleZ2eVADCjH67FXmlnE76U3ANv9+i3AZKDFzCJABbDvhF7ZDGIVFCdTt1DViXEiIhJsGRuHds7d6ZxrcM5NAa4FfuWcux5YCXzGr7YYeNyff8Jfxl//K3cyZ7MVVVKU8O64pjPdRUQk6LJxYPl24E/NbCPeMfIH/PIHgBq//E+BO07qVWIVFMZTCV2/RRcRkWDL9ElxADjnmoFmf34TcP4QdXqBa0btRWOVRLu8EXv10EVEJOiCe+p3rIJI3wFAPXQREQm+QCd06z+AmXroIiISfMFO6L3tlBaGldBFRCTwAp3QSfRTW+g4oCF3EREJuGMmdDP7XNr8Bw9b9z8yFdSo8C//OqGwXz10EREJvOF66H+aNv+Ph637o1GOZXSl7ole0EunErqIiATccAndjjI/1PKpxU/o4yK9dPRpyF1ERIJtuITujjI/1PKpxU/oNRHdE11ERIJvuAvLnGVma/B64+/z5/GXz8hoZCfLT+hV4V4ldBERCbzhEvrZYxJFJhSWA1AZ6qajdwDnHCd6e3UREZFT3TETunNuc/qymdUAC4EtzrlVmQzspPk99ArrZiDh6IsniUXDWQ5KREQkM4b72drPzGyWP18PvIl3dvu/m9mtYxDfiYsWQShKmesG0G/RRUQk0IY7KW6qc+5Nf/4m4Bnn3JXABZzqP1vz74leQhegy7+KiEiwDZfQ07u1i4CnAJxzHUAyU0GNmlg5RYlOQAldRESCbbiT4raa2VeAFmA+8DSAmRUB0QzHdvJiFcQGE7qG3EVEJLiG66HfDJwDfB74Q+dcm19+IfCDDMY1OmIVFMQ7APXQRUQk2IY7y3038MUhylcCKzMV1KiJVRBt2w6ohy4iIsF2zIRuZk8ca71z7qrRDWeUFZYT7j8AqIcuIiLBNtwx9PcDW4FHgJc51a/ffrhYBdanhC4iIsE3XEKfAHwEuA74LPAk8Ihz7q1MBzYqYpXYQDeVhUroIiISbMc8Kc45l3DOPe2cW4x3ItxGoNk/8/3Ud8g90XUMXUREgmu4HjpmVghcgddLnwLcC/w4s2GNkph3PfcJBbpBi4iIBNtwJ8UtA2YBPwe+lXbVuNzg99DHF/SxTfdEFxGRABuuh34D0AWcCdySdrcyA5xzrjyDsZ08P6HXRnp5Rz10EREJsOF+hz7chWdObX5Cr4700nFACV1ERIIrtxP2cPx7oleFenRSnIiIBFqwE7rfQ6+0Lg5oyF1ERAIsYwndzGJm9lsze8PM3jKzb/nlU83sZTPbYGaPmlmBX17oL2/010856SAKSsFClFs3/fEkvQOJk96kiIjIqSiTPfQ+4GLn3BxgLvAxM7sQuAu4xzk3HdiPdwMY/Ol+59w04B6/3skJhaCwnDLrAeBAj4bdRUQkmDKW0J2n01+M+g8HXAw85pcvAz7hz1/tL+OvX2Rpp9WfsFg5pa4LgDYldBERCaiMHkM3s7CZrQZ2A88A7wJtzrnUAe0WYJI/PwnvuvH469uBmpMOIlZBsX9P9P1d/Se9ORERkVPRsFeKOxnOuQQw18wqgZ8AZw9VzZ8O1Rt3hxeY2RJgCUBdXR3Nzc3HjGFOr2MgvgOAF155nZ4tGX3LgdPZ2TlsG8vJUztnnto489TG2TUm2c0512ZmzXjXg680s4jfC28AtvvVWoDJQIuZRYAKYN8Q21oKLAVobGx0TU1Nx37xnVPob90EwOQzZtC0YPIovKP80dzczLBtLCdN7Zx5auPMUxtnVybPch/n98wxsyLgEmAdsBL4jF9tMfC4P/+Ev4y//lfOuSN66CNWWE7Evyd6W4+G3EVEJJgy2UOvB5aZWRhvx2G5c+5nZvY28EMz+zbwOvCAX/8B4N/NbCNez/zaUYnCvyd6JGS0deukOBERCaaMJXTn3Bpg3hDlm4DzhyjvBa4Z9UD8hF5dFNJZ7iIiEljBvlIcHLwnelGcdvXQRUQkoPIgoXvXc58YG9AxdBERCaw8SOh+Dz3aq2PoIiISWHmT0McVKKGLiEhw5U1Cr4n00a6T4kREJKCCn9D9e6JXh3vo7IszkEhmOSAREZHRF/yEnroneqgbQL10EREJpOAndL+HXo5/x7VunekuIiLBE/yEHo5AQdnBW6jqxDgREQmg4Cd0gKIqipMdgBK6iIgEU54k9EqKBtoB2KchdxERCaD8SOjF1RT4CX1vpxK6iIgET34k9KJqwr37KYqG2dvZl+1oRERERl2eJPQq6N5HTWkBe7vUQxcRkeDJj4ReXA29bYwridKqHrqIiARQfiT0oipwSSYXD9CqY+giIhJAeZLQqwGYVNinHrqIiARSfiT0Yi+hTyzsZl9XP8mky3JAIiIioys/EnpRFQB1kW4SSafruYuISODkSUL3eug1Ye/yrxp2FxGRoMmPhO4PuVdZJwB7OpTQRUQkWPIjoccqAKPSvB76ro7e7MYjIiIyyvIjoYfCEKugLHkAgB3tSugiIhIs+ZHQAYqqiPa1UR6LsFMJXUREAiZ/EnpxNfTsp76iSAldREQCJ38SelE19OyjriLGzgNK6CIiEix5lNC9G7TUl8d0DF1ERAInfxJ6SS1072NCRYzWzj4GEslsRyQiIjJqMpbQzWyyma00s3Vm9paZfdUvrzazZ8xsgz+t8svNzO41s41mtsbM5o9qQCXjoL+DhlKHc7BLw+4iIhIgmeyhx4H/6Zw7G7gQ+LKZzQTuAFY456YDK/xlgMuA6f5jCXD/qEZTOh6AM4q6Adiyt3tUNy8iIpJNGUvozrkdzrnX/PkOYB0wCbgaWOZXWwZ8wp+/GnjIeV4CKs2sftQCKvES+uRC7+Iy7ymhi4hIgIzJMXQzmwLMA14G6pxzO8BL+sB4v9okYGva01r8stHh99Br3X4KIiHe29s1apsWERHJtkimX8DMSoEfAbc65w6Y2VGrDlF2xH1OzWwJ3pA8dXV1NDc3H1ccBX17+QCwYfVvqI0t5NV3NtNcvOu4npvPOjs7j7uN5cSpnTNPbZx5auPsymhCN7MoXjJ/2Dn3Y794l5nVO+d2+EPqu/3yFmBy2tMbgO2Hb9M5txRYCtDY2OiampqOL5jEALwIMyZWMXNgPFv3ddPUtPBE3lZeaW5u5rjbWE6Y2jnz1MaZpzbOrkye5W7AA8A659zfpa16Aljszy8GHk8rv9E/2/1CoD01ND8qwlHvt+hdu5laW8x7e7tIJo8YABAREclJmeyhfxC4AVhrZqv9sq8D3wWWm9nNwBbgGn/dU8DlwEagG7hp1CMqGQ+du3jf1FL64km27u/m9JqSUX8ZERGRsZaxhO6ce4Ghj4sDLBqivgO+nKl4AO/EuM49nF1fDsC6HQeU0EVEJBDy50pxAGX10LGdGRPKCBm8vaMj2xGJiIiMivxK6BUNcGA7sTCcMa6Ut7cfyHZEIiIioyLPEvokSMahczcz68tZt0MJXUREgiG/Enp5gzc9sI05kyvZ1taja7qLiEgg5FdCr/ATevtWFkypAuCV9/ZlMSAREZHRkWcJ3b+SbPs2ZtaXU1wQ5tX39mc3JhERkVGQXwk9VgkFpXBgG5FwiHmnVaqHLiIigZBfCd0MyidB2xYAFkypZt2OA3T0DmQ5MBERkZOTXwkdoPoM2LcJ8BJ60uk4uoiI5L78S+i102Hvu5BM0DilipKCMM+8vXv454mIiJzC8jOhJ/qgbQuFkTAXzRjHinW7dKMWERHJafmX0Gume9O9GwH4yMw6dnf0sWZbexaDEhEROTn5l9Br/YTeuh6AP5gxnnDI+MVbO7MYlIiIyMnJv4ReXANF1bB7HQCVxQV8aFot//X6NhIadhcRkRyVfwndDCbNh22vDRb9P42T2dHey/Mb9mQxMBERkROXfwkdoGEB7H4b+rzbp14yczzVJQUsf3VrlgMTERE5MXma0BsBB9tWAVAYCfOpeZN45u1d7NbNWkREJAflZ0Kf1AgYbHlpsOj6C09nIOF45LfqpYuISO7Jz4ReVAmTzoMNvxwsmlpbwkVnjuPhlzczkEhmMTgREZGRy8+EDnDmpd6JcZ0HT4Rb/IHT2d3Rx0/f2J7FwEREREYufxP69I8CDtb/fLCo6czxnDWhjH/61Ubi6qWLiEgOyd+EXj8HaqbBGz8cLAqFjFsvmc6m1i6eXLsji8GJiIiMTP4mdDOY+1nY/JvBu68BfHTmBKaPL+X+5ndxTheaERGR3JC/CR3g3GvBQrD6kcGiUMj44kXv452dHaxYp7uwiYhIbsjvhF4xCaZdAq8+AH2dg8VXzZ3I5Ooi/vaXv9PlYEVEJCfkd0IHuOh26N4Lv106WBQNh7jjY2fzzs4O/o+uHiciIjlACb2hEaZfCv99L/TsHyy+fPYEzju9ir/95Xo6++JZDFBERGR4SugAi74Bve2w8juDRWbG/7ribFo7+/iX5nezGJyIiMjwMpbQzez7ZrbbzN5MK6s2s2fMbIM/rfLLzczuNbONZrbGzOZnKq4hTZgFC/4YXvke7Fw7WDzvtCqunjuRf3t+E9vaeoRriAEAABZySURBVMY0JBERkZHIZA/9QeBjh5XdAaxwzk0HVvjLAJcB0/3HEuD+DMY1tD/4OhRVwVO3QdrP1f78Y2cBcPfT74x5SCIiIscrYwndOfccsO+w4quBZf78MuATaeUPOc9LQKWZ1WcqtiEVVcEl34QtL8Ka5YPFkyqL+OMPT+W/Vm9n9da2MQ1JRETkeI31MfQ659wOAH863i+fBKSfTt7il42tuZ/zbtryi69DV+tg8Z80TaO2tIBv/+xtXWxGREROSZFsB+CzIcqGzJxmtgRvWJ66ujqam5tHNZDiiZ+ncfvX2PuDz/HWObd7V5QDPn46PPjWfv720RUsmHCqNFvmdXZ2jnoby5HUzpmnNs48tXF2jXVm2mVm9c65Hf6QeupSbC3A5LR6DcCQtzxzzi0FlgI0Nja6pqam0Y+yYi/jnv0mTTWtcO41AHw46Xjp3ud5YkucWz7zYQoj4dF/3VNQc3MzGWljOYTaOfPUxpmnNs6usR5yfwJY7M8vBh5PK7/RP9v9QqA9NTSfFR+4BRoWwFN/Bge8MMIh4y+uOJut+3r4u1+uz1poIiIiQ8nkz9YeAV4EZphZi5ndDHwX+IiZbQA+4i8DPAVsAjYC/wZ8KVNxHZdQGD5xP8R74ae3DJ71/uHp47j+gtP41+c28czbu7IaooiISLpMnuV+nXOu3jkXdc41OOcecM7tdc4tcs5N96f7/LrOOfdl59z7nHOznXOvZiqu41Y7HT7yl7Dhl/Df/zhY/P99fCazJ1Xwp8tXs2VvdxYDFBEROUhXijuW85fA2VfCs9+ELS8DEIuG+efr52PATQ/+lv1d/VkNUUREBJTQj80MrvonqJwMj900eK33ydXFLL2xka37e/j8g6/oWu8iIpJ1SujDKaqEz/wAOnbCz+8YLL7wjBru++x83tzWznVLX2JPR18WgxQRkXynhH48Js2HhX8Ga34Ib/5osPgjM+tYesN5bNzdyafu/w1rW9qzGKSIiOQzJfTj9eE/g8kXwE/+BN57YbB40dl1PLLkQgbijk/d/xvuW7mRvngii4GKiEg+UkI/XpECuO6HUDUFHvkstKwaXDV3ciVP3/phLjm7jrt/8Tsu+btf88Qb20kmdZlYEREZG0roI1FcDZ/7ERRXwbIr4d2Vg6sqiwu4/3Pn8dAfnU9pYZRbHnmdq+57gUd+u4UDvQNZDFpERPKBEvpIVU6GP/qF11N/+Bp4/eFDVi88cxw/+8qH+P+vmUPvQJI7f7yWBd9+lq/+8HVe2NBKQr12ERHJgPy5y8hoKpsANz0Jj94Aj38Jfv8cXHaXd0Y83mViP31eA5+aP4k3Wtp5bNVWnli9ncdXb2diRYyPzarnkpnjuWBqDeHQUPelERERGRkl9BNVVAU3Pg7P3Q2/vgs2PguLvgFzrvOOtwNmxtzJlcydXMn/umImz7y9ix+/1sJ/vLyZ7//m99SWFnLF7AlcOWci80+rIqTkLiIiJ0gJ/WSEwtB0B5z5MXjqNu+677/6NjTeBPNu8IbnfbFomCvnTOTKORPp6ovz6/V7+Nma7fzwla0se3EzkyqL+Pi59Vw5ZyLnTCzHTMldRESOnxL6aJg41zuu/u6v4JV/g1//jddrr58DM66AGZdB3SwIeacslBRGuHx2PZfPrqejd4Bn1+3ip2/s4IEXfs+/PreJM2pL+PiciXx6/iROrynJ8psTEZFcoIQ+WkIhmH6J99j/Hrz1X/DOk9D819D8HSgo8y5Qc8ZFMKnR2wmIVVAWi/LJeQ18cl4D+7v6efqtnfz0je384682cO+KDZw/tZprzmvg8tn1lBTqzyUiIkNThsiEqinwoVu9R+du7/j6tlWw5SVY8ZcH69VMh0nnQUMjNCygqu4crjv/NK47/zR2tvfyo9daeGxVC7c9tob//cRbXD67nmvOa+D8qdUakhcRkUMooWda6XiY+1nvAdC9D7a/Btte95L8uyu8S8oCRIq8nntDIxMaFvDl8xbwpaaLWLV5P8tf3cqTa3bw2KoWTq8p5lPzGvj4nHreN640e+9NREROGUroY624GqZd4j0AnIO2LbDtVWh5FVpegZf/dfAe7FY+icZJ59HYsIC/nDefp/ZO4NHX93DPs+u559n1zKwv5yMz61h09nhmTazQmfIiInlKCT3bzKDqdO8x69NeWbwPdr7pJffUY90TxIBPhSJ8qm4WXRfM59X4GfyfnRO491ft/MOKDYwvK+Ty2fV8aFotHz6zlsJIOKtvTURExo4S+qkoUggN53kPvuiVde7xe/Fegi9Zt5yL+ju5CLi3sprdZefwZl8dv3mljAdemsjt0bM5d0odH5xWy4em1zKjrkzH3UVEAkwJPVeUjvN+/jbjMm85mYA9v4OWVwi1vMKE7a8zoetVLgn3QBiSGK1bqnnv97W8/ctxNIfGE6mZyvjTpjN12kymT5tBLBbL7nsSEZFRo4Seq0JhqJvpPc5b7JU5Bx07YecaQtteY3zbFipaf885+zZR1PPfhPYlYR+wGhLO2B2upatoElSeRkHtFCrq30dp3fugZBzEyr2piIjkBCX0IDGD8nrvcealABT6DxIDcGAbbdvfZevv3+HAjo0k9m2muHMb9Z2/ob7lp4TeOPLGMeeHyxl4YzyR4kosVg6F5RCr8BJ+YWpa7k0LSiBaDJEYRIu8QweRIojGvGk44p0f0L3Pu+59tGhMm0dEJMiU0PNFOApVU6ismkLlOYsGixNJx/a2Hp7buY9dW9+lY+e7dOzfTe+BVmJ9+xgfb6OytYMK66E22kJFqIdS100s2Uk02TeyGCwMLnFwvqDE+1mfhb34QhEIF/iPiLfsnLeDkFq2kFc/FPbmQ+FDly3sXeRnsCx93eH1U1MboixtO4c/L5QeS+phgPlTvPnUJFzg1Rlcn1bP32Zh7x5vdAXAJf2pA9zB+d52b2eqqCrtdRhiu0eZjtY5FM7BQA8UFI/O9kRkVCih57lwyJhcXczk6mKY2QBcNLiurbufx37xPNGGM1mzp5N393SydV8Puzt62d89QJQ4pXRTZj2U0U1luJ8JxUlqC5NUFyapjCaoiCYoi8QpDg1QbANEY6VESyop7NhCIf0UJrqIWoJQcgBLxiHRD4k49HdDMu4loXifN8LgEt65Ay4ByeShyy55ZFlqmgPeD/DSWL3acIl/qJ0P8/dRzGvX/g6IVXo7OKkED0Ps0Nhhr5m+7njrDbfu+F7rgt4+WB07iW0cpV5qOVzgfw4TB3f8Ev1e+4RCx/96zvk7df7UuYM7d9EiCEU5fke5XbM71m2cj7HuqM/zyucdOAAby0f0nJG9zjDPA+/vEIl5nQTgiLZN30lObS99/qjr0mM7rF4o7I1IptalylM7+xY6+HdN/W1THYSRxDIMJXQ5qsriAqZVhWlaMPmIdd39cXa097L7QB+7O3rZ09HHno4+dnf0sa6rn/1d/ezb38/erj56B5JDbH36IUvRsFFaGKGkMEJxQZiiaJhY1JsWFYUpiISIhELEoiGvrCBMNBzyH0Y0HKKoIEwkZETCIUoKwjgH0UiIaMgRDUHUkkRDECFJNJQkgiMSckQMIpYkYo6IJQmbty6M/893+I5EcsBf9telf+HCoV8ALul/qae+oA+r55KQjPPOurc4a9oZx+jp++XJuNdTP/wL6ogpRyk/junhIwSHT8FL5j37D9YtKPZjPc4vwKN+cR3lS+y4t3H4cw7Wa9+1k6K6uhOP46hftBz8O6e+pJNx75EanUkmhtlO2nx6EsAOHQEa6Erb1nE66sjMMUZsjjmac5R1ZiTCA1BYNqLnjPh1jvU85yDR5+1g9nce+tk9mZ21o+2ApRaTcejuOvL/Nz2JD/5dU0nc/14ZcSxHp4QuJ6S4IML7xpUe15XqegcSdPTGOdA7QHvPAG3d/SSS0NUXp6N3gAO9cTr74nT6057+BD0DCXr6E+zr6qd3IEFfPEk84egd8NcNJI69Ez8KzCAaCpF0jpAZ0bBREAlhZhgQDYeIhI1oOIwBSeeI+DsZffEEsUiYpHMkkkUURkOUFET87wAbHAFPze/fV011TzXGwe+JkHnrQmaEQ8ZAwhEy73XjySSxaJjCSAjDCIUgEgoRDhnOObr6E5QWRkgkvUaKhI24//xwyIvbqwsHegeoLSlIe+Mc8cVx+NdIT2+CooIwXX1xuvoTVEWilBRGGEgkSSQdRf4OlXOOgkiIkBnxpMOASDhER+8ApYURIiHD4R36SSQd8aTX1iGDpINIyGvzSNjY29lPaWGEpP+HjycdJYWRIb/208OPhkKYwTvd7zDjtLO8wvRcnLaQ+kylf7QOlh1Z75C6aYWp95xIOpIOEs7hnPMHJbz35xwURcP0DCQGd0STztE3kCAaDhGLhnG4g6+fFpvzFw7Zf0mve1jch7yv9OfC4N/pyOcdLD+8HVLzkXCIwkiIgog3fevtdUybPoP+RJKBhPcZLQiHBt+j+e87Fg3T3jPgf04csaj3v+Ic9A4kKYiECIe8/4940hEOeZ9b0t+7OxiLcwfjNTP64gkKI2ES/v9Jf9zbZkrqf8nrKBh7u/opCIcojIYH//eSzjGQSB6yL5D6n03Np6R+EjyYfoeol/q32tvV77UFRsx//yn9cS/e3tT3mw3uJqdt/zSORgldMi7m97bHlRWO2jad8xLAQMIxkEzSH0/S058gkXT0xhP0x72eY388yUDCEU96OwTxpCOeSDLgTwfLkn69RNKvc7AsZN4/VH/ce53Ul0g84f3DD/hJ0/ASU188SUHE6I8nMTPC/hdMV3/C21knecgXkAO6445wzwD4y6kvq2SSwTicc0TDIQYSycEEmXqfCefFnkh6X2iRkPf6kbCfMBOOSNi85OK/31QSDZmRGOKL+3gUF3g7Ffu7B0bjz5p5b67JdgTBt0ZtnC1K6JKTzIxI2IiEoYjcvyJec3MzTU0fzHYYw0rtSIH3Nwj7lxpO9cwBfzQhOTgCMJBIknTeYZXUjlBhNER3vzd0bEA47O34hENG30CScNjrxSb8nZb+RJLKogL64glCISORcEQjIbr74gxuZDDIQ2dTvayXXnqJCy+8MK23dfBJ6U8/2KOyIcrSK6bPHtob6+n3elipkRAzCPs7TsmktwMWDRnd/QmKC8LE/Z2r1CgPMLizlj5gMnTv8NAeYnrdQ+I+WrnfFkP1Lg85f5MjX6ejNz64Y9gXT7Lqld/yoQ++3xvNCofo6k8wEE9SVBD2diQTDjPo93dKnXMU+j3S1HI0HMI5b8Qr6RwRf5QsnkwdurPBUaxU3OmjXakRNee8Uxd6BxIUhMPEk97nMPUBCYe8neN4wlEWi+Ac9CcSgzvbqVG4Q0bPSc0fOaKRWntovdSag6MaqW06f0Qi5P+fgKMgHB4cXUiNZjjc4GhdMgmn38VRKaGLyHFL7UgdzjuX4dDllFh06B2ukZYDFBUcuq50BLcUfrc45J38KaOmuODQ9m8pCTGp8uDPUSvV3GMqNHyVsWNmHzOz35nZRjO7I9vxiIiI5IpTJqGbWRi4D7gMmAlcZ2YzsxuViIhIbjhlEjpwPrDRObfJOdcP/BC4OssxiYiI5IRTKaFPAramLbf4ZSIiIjKMU+mkuKF+TnrED2jMbAmwBKCuro7m5uYMh5XfOjs71cZjQO2ceWrjzFMbZ9eplNBbgPRLkjUA2w+v5JxbCiwFaGxsdE1NTWMSXL7yfk7VlO0wAk/tnHlq48xTG2fXqTTk/gow3cymmlkBcC3wRJZjEhERyQmnTA/dORc3s/8B/AIIA993zr2V5bBERERywimT0AGcc08BT2U7DhERkVxzKg25i4iIyAkyN5I7MZxizGwPsDnbcQRcLdCa7SDygNo589TGmac2zrzTnXPjhlqR0wldMs/MXnXONWY7jqBTO2ee2jjz1MbZpSF3ERGRAFBCFxERCQAldBnO0mwHkCfUzpmnNs48tXEW6Ri6iIhIAKiHLiIiEgBK6HnMzCab2UozW2dmb5nZV/3yajN7xsw2+NMqv9zM7F4z22hma8xsfnbfQW4xs7CZvW5mP/OXp5rZy347P+pf8hgzK/SXN/rrp2Qz7lxhZpVm9piZveN/pt+vz/LoM7Ov+d8Xb5rZI2YW02f51KCEnt/iwP90zp0NXAh82cxmAncAK5xz04EV/jLAZcB0/7EEuH/sQ85pXwXWpS3fBdzjt/N+4Ga//GZgv3NuGnCPX0+G9w/A0865s4A5eG2tz/IoMrNJwC1Ao3NuFt5luq9Fn+VTghJ6HnPO7XDOvebPd+B9AU4CrgaW+dWWAZ/w568GHnKel4BKM6sf47Bzkpk1AFcA3/OXDbgYeMyvcng7p9r/MWCRX1+OwszKgYXAAwDOuX7nXBv6LGdCBCgyswhQDOxAn+VTghK6AOAPhc0DXgbqnHM7wEv6wHi/2iRga9rTWvwyGd7fA38OJP3lGqDNORf3l9PbcrCd/fXtfn05ujOAPcAP/MMa3zOzEvRZHlXOuW3A3wJb8BJ5O7AKfZZPCUrogpmVAj8CbnXOHThW1SHK9DOJYZjZx4HdzrlV6cVDVHXHsU6GFgHmA/c75+YBXRwcXh+K2vgE+OcgXA1MBSYCJXiHLw6nz3IWKKHnOTOL4iXzh51zP/aLd6WGH/3pbr+8BZic9vQGYPtYxZrDPghcZWbvAT/EG578e7xh3tQdD9PbcrCd/fUVwL6xDDgHtQAtzrmX/eXH8BK8Psuj6xLg9865Pc65AeDHwAfQZ/mUoISex/xjWQ8A65xzf5e26glgsT+/GHg8rfxG/wzhC4H21HCmHJ1z7k7nXINzbgreCUS/cs5dD6wEPuNXO7ydU+3/Gb++ejXH4JzbCWw1sxl+0SLgbfRZHm1bgAvNrNj//ki1sz7LpwBdWCaPmdmHgOeBtRw8tvt1vOPoy4HT8P6Br3HO7fP/gf8J+BjQDdzknHt1zAPPYWbWBPyZc+7jZnYGXo+9Gngd+Jxzrs/MYsC/453TsA+41jm3KVsx5wozm4t30mEBsAm4Ca/Tos/yKDKzbwF/iPcrmdeBP8Y7Vq7PcpYpoYuIiASAhtxFREQCQAldREQkAJTQRUREAkAJXUREJACU0EVERAJACV0kj5lZwsxWpz2OdXW1kW57ipm9OVrbE5FjiwxfRUQCrMc5NzfbQYjIyVMPXUSOYGbvmdldZvZb/zHNLz/dzFb49xBfYWan+eV1ZvYTM3vDf3zA31TYzP7Nv3/2L82sKGtvSiTglNBF8lvRYUPuf5i27oBz7ny8K6r9vV/2T3i3HT0XeBi41y+/F/i1c24O3jXU3/LLpwP3OefOAdqAT2f4/YjkLV0pTiSPmVmnc650iPL3gIudc5v8G/jsdM7VmFkrUO+cG/DLdzjnas1sD9DgnOtL28YU4Bnn3HR/+XYg6pz7dubfmUj+UQ9dRI7GHWX+aHWG0pc2n0Dn7YhkjBK6iBzNH6ZNX/Tn/xvvjnEA1wMv+PMrgD8BMLOwmZWPVZAi4tHeskh+KzKz1WnLTzvnUj9dKzSzl/F2/K/zy24Bvm9mtwF78O5oBvBVYKmZ3YzXE/8TQLcjFRlDOoYuIkfwj6E3Oudasx2LiBwfDbmLiIgEgHroIiIiAaAeuoiISAAooYuIiASAErqIiEgAKKGLiIgEgBK6iIhIACihi4iIBMD/BZVuFQAXLSOzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.DataFrame(history_3.history)[1:].plot(figsize=(8, 5))\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('MSE')\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "51/51 [==============================] - 0s 137us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "20.95019011404"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mse_test_3 = model_regularized.evaluate(X_test_scaled, y_test)\n",
    "mse_test_3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the performance of the model on the test set drops (which means the $l_2$ regularizer should be tuned better). [Other regularizers](https://keras.io/regularizers/) available on Keras are:\n",
    "* keras.regularizers.l1 for $l_1$ regularization (the larger its hyperparameter, the sparser the network)\n",
    "* keras.regularizers.l1_l2 for a combination of $l_1$ and $l_2$ regularization. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Early Stopping**. To set early stopping, you should set a Keras [callback](https://keras.io/callbacks/); this is simply a function that you declare before training, and that Keras will compute during the training stage. The callback for early stopping requires a `patience` hyperparameters, which says for how many epochs after the best validation metric has been observed the algorithm should continue, and a `restore_best_weights` (set to True) to return the set of weights and biases obtained at the epoch which led to the best validation metric. \n",
    "\n",
    "Here is an example of how to set early stopping with Keras, using 10 epochs for the `patience`: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Early Stopping (wait 10 iterations with no progress on the validation set)\n",
    "early_stopping_cb = keras.callbacks.EarlyStopping(patience=10, restore_best_weights=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And it can be used as follows (we reinitialize a 1-layer model to check how early stopping works): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize a new model\n",
    "model_early_stopping = keras.models.Sequential()\n",
    "model_early_stopping.add(keras.layers.Dense(5, activation='relu', input_shape=X_train.shape[1:]))\n",
    "model_early_stopping.add(keras.layers.Dense(1))\n",
    "model_early_stopping.compile(loss=\"mean_squared_error\", optimizer=adam_optimizer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The early stopping object must be given as input to the *fit* method: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "history_4 = model_early_stopping.fit(X_train_scaled, y_train, epochs=2500, validation_data=(X_valid_scaled, y_valid),\n",
    "                                   callbacks=[early_stopping_cb], verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the model has stopped much than the 2500 epochs given as input: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAE9CAYAAAD9MZD2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZwdVZ338c/vLt339t6dpdNZIAmENYEEAgRwQsuqqOCCDogQkTGPwijqDCLO87g8+qgMMzLyiPggsikqyODAAIIY0kCUNRCSQAKJgSSdvbP0vtzlPH9UdfdNpzvdndzbnXvv9/161auqTp2qe+7pm/zqnDpVZc45REREJLsFRrsAIiIicvAU0EVERHKAArqIiEgOUEAXERHJAQroIiIiOUABXUREJAeERrsAB2Ps2LFu6tSpo12MnNLa2kpxcfFoFyOnqE4zQ/WafqrT9Et3nS5durTBOTeuv21ZHdCnTp3Kq6++OtrFyCl1dXXU1taOdjFyiuo0M1Sv6ac6Tb9016mZrR9om7rcRUREcoACuoiISA5QQBcREckBWX0NXUREskssFqO+vp6Ojo7RLsqIKC8vZ9WqVcPeLxKJMHnyZMLh8JD3UUAXEZERU19fT2lpKVOnTsXMRrs4Gdfc3Expaemw9nHOsXPnTurr65k2bdqQ91OXu4iIjJiOjg7GjBmTF8H8QJkZY8aMGXYvhgK6iIiMKAXzwR1IHSmgi4iI5AAFdBERkf0oKSkZcNt7773HzJkzR7A0A8vqgN7YHhvtIoiIiBwSsjqgb2vKj9seREQkfW644QZ+9rOf9ax/5zvf4bvf/S7nnHMOJ510ErNmzeKRRx4Z9nE7Ojq46qqrmDVrFnPmzGHx4sUAvPnmm5x66qnMnj2bE044gTVr1tDa2sqHPvQhTjzxRGbOnMkDDzxw0N8rq29bc260SyAiIgfqu//9Jm9tbkrrMY+bWMa3P3L8fvNceumlfOUrX+Gaa64B4MEHH+TJJ5/kq1/9KmVlZTQ0NDBv3jwuuuiiYQ1Ou+222wBYsWIFq1ev5vzzz2fp0qX8/Oc/57rrruPyyy+nq6uLRCLBE088wcSJE3n88ccBaGxsPMBv3CurW+hJRXQRERmmOXPmsH37djZv3swbb7xBZWUlNTU1fPOb3+SEE07g3HPPZdOmTWzbtm1Yx12yZAlXXHEFAMcccwyHH344a9eu5fTTT+cHP/gBN910E+vXrycajTJr1iz+/Oc/c8MNN/D8889TXl5+0N8ru1voo10AERE5YIO1pDPpkksu4aGHHmLr1q1ceuml3H///ezYsYOlS5cSDoeZOnXqsO8DdwM0Mj/96U9z2mmn8fjjj3PBBRdw5513cvbZZ7N06VKeeOIJbrzxRs4//3y+9a1vHdR3yu6AroguIiIH4NJLL+Xzn/88DQ0NPPvsszz44IOMHz+ecDjM4sWLWb9+wLeUDmj+/Pncf//9nH322bzzzjts2LCBGTNmsG7dOqZPn86Xv/xl1q1bx/LlyznmmGOoqqriM5/5DCUlJdxzzz0H/Z0yGtDNrAK4E5iJ16D+HPA28AAwFXgP+JRzbrd5Fyp+AlwItAGfdc69tr/jJ53DOaeHFIiIyLAcf/zxNDc3M2nSJGpqarj88sv5yEc+wty5c5k9ezbHHHPMsI95zTXX8IUvfIFZs2YRCoW45557KCws5IEHHuDXv/414XCYCRMm8K1vfYtXXnmF66+/nkAgQDgc5vbbbz/o75TpFvpPgCedc5eYWQFQBHwTWOSc+5GZfQP4BnAD8EFghj+dBtzuz/crlnAUhBTQRURkeFasWNGzPHbsWF544YV+87W0tAx4jKlTp7Jy5UrAe6FK35Z2c3MzN954IzfeeONe6RdccAEXXHDBAZa8fxkbFGdmZcB84JcAzrku59we4GLgXj/bvcBH/eWLgfuc50WgwsxqBvuczngi7WUXERHJNplsoU8HdgB3m9mJwFLgOqDaObcFwDm3xczG+/knARtT9q/307bs70M6YklKI+kuuoiISK8VK1b0jGDvVlhYyEsvvTRKJdpXJgN6CDgJ+JJz7iUz+wle9/pA+us332fYm5ktBBYCFEw4kmeX/IWx0ay+++6Q0tLSQl1d3WgXI6eoTjND9Zp+I1Gn5eXlNDc3Z/QzMmHq1Kk8//zz+6QP9l0SicQBf9+Ojo5h/T0yGdDrgXrnXPfpy0N4AX2bmdX4rfMaYHtK/ikp+08GNvc9qHPuDuAOgMKaGW7O3FM5YtzAz9mV4amrq6O2tna0i5FTVKeZoXpNv5Go01WrVg37/eDZ7EDeh94tEokwZ86cIefPWNPWObcV2GhmR/tJ5wBvAY8CC/y0BUD38/UeBa40zzygsbtrfn86YrqGLiIikulR7l8C7vdHuK8DrsI7iXjQzK4GNgCf9PM+gXfL2lq829auGsoHdMaT6S6ziIhI1sloQHfOLQPm9rPpnH7yOuDa4X5GZ0wBXUREhq6kpGS/t6Jlq6wfTabb1kRERHIgoHeohS4iIgfAOcf111/PzJkzmTVrVs8rTLds2cL8+fOZPXs2M2fO5PnnnyeRSPDZz362J+8tt9wyyqXfV1Y/yx3UQhcRyVp//AZsXTF4vuGYMAs++KMhZX344YdZtmwZb7zxBg0NDZxyyinMnz+f3/zmN1xwwQX8y7/8C4lEgra2NpYtW8amTZt6ngq3Z8+e9JY7DbK+ha5BcSIiciCWLFnCZZddRjAYpLq6mrPOOotXXnmFU045hbvvvpvvfOc7rFixgtLSUqZPn866dev40pe+xJNPPklZWdloF38f2d9C121rIiLZaYgt6UwZ6HWn8+fP57nnnuPxxx/niiuu4Prrr+fKK6/kjTfe4KmnnuK2227jwQcf5K677hrhEu+fWugiIpKX5s+fzwMPPEAikWDHjh0899xznHrqqaxfv57x48fz+c9/nquvvprXXnuNhoYGkskkn/jEJ/je977Ha6/t92WgoyL7W+gK6CIicgA+9rGP8cILL3DiiSdiZvzrv/4rEyZM4N577+Xmm28mHA5TUlLCfffdx6ZNm7jqqqtIJr2Y88Mf/nCUS7+vrA/oelKciIgMR/c96GbGzTffzM0337zX9gULFrBgwYJ99jsUW+WpsrrL3UwtdBEREcjygB4w06A4ERERsjygG2qhi4iIQJYH9ICZrqGLiGSZgW4Xk14HUkdZHdB1DV1EJLtEIhF27typoL4fzjl27txJJBIZ1n5ZPco9YKaALiKSRSZPnkx9fT07duwY7aKMiI6OjmEHZvBOfCZPnjysfbI6oJvptjURkWwSDoeZNm3aaBdjxNTV1TFnzpwR+ays7nJXC11ERMST1QHdu4auFrqIiEhWB3RvlLta6CIiIlkd0CviO9VCFxERIcsDelGymU610EVERLI7oJtLalCciIgI2R7QcbptTUREhBwI6J3xpJ44JCIieS+7A7rzutu7Eup2FxGR/JbVAR0cAZK6dU1ERPJelgd0KKRLt66JiEjey/qAHqFLt66JiEjey4GAHlMLXURE8l72B3Tr0jV0ERHJe1kf0AuJ6eEyIiKS97I+oEc0KE5ERCQHArppUJyIiEhGA7qZvWdmK8xsmZm96qdVmdnTZrbGn1f66WZmt5rZWjNbbmYnDeUz1EIXEREZmRb6+51zs51zc/31bwCLnHMzgEX+OsAHgRn+tBC4fSgH9+5DVwtdRETy22h0uV8M3Osv3wt8NCX9Pud5Eagws5rBDlZITC9oERGRvJfpgO6AP5nZUjNb6KdVO+e2APjz8X76JGBjyr71ftp+RUwtdBERkVCGj3+mc26zmY0Hnjaz1fvJa/2k7fMaNf/EYCHAyTUBInTx5uo11HW+l5YC57uWlhbq6upGuxg5RXWaGarX9FOdpt9I1mlGA7pzbrM/325mfwBOBbaZWY1zbovfpb7dz14PTEnZfTKwuZ9j3gHcATB3YtBF6GLcYVOprZ2Rya+SN+rq6qitrR3tYuQU1WlmqF7TT3WafiNZpxnrcjezYjMr7V4GzgdWAo8CC/xsC4BH/OVHgSv90e7zgMburvn9iZgeLCMiIpLJFno18Acz6/6c3zjnnjSzV4AHzexqYAPwST//E8CFwFqgDbhq0E8wozgQp0G3rYmISJ7LWEB3zq0DTuwnfSdwTj/pDrh2eJ8SoDgQY5Na6CIikuey+0lxZhQFdNuaiIhIlgf0AEUBXUMXERHJ+oAetZie5S4iInkvywO6ee9D16A4ERHJc1ke0ANEUQtdREQk6wN6od62JiIikgsBPUaHWugiIpLnsjygG4V0qoUuIiJ5L8sDeoACp7etiYiIZHdAxwgroIuIiGT89amZZQEKXKduWxMRkbyX3S10CxBKqoUuIiKS/QHddRGLx/He7SIiIpKfsjygGwCF6HnuIiKS37I8oHvFj9Clp8WJiEhey/KAntpC18A4ERHJX1ke0P0WunXpaXEiIpLXsjugk9Llrha6iIjksewO6KnX0DUoTkRE8liWB3TvGnpE19BFRCTPZXlA94pfqGvoIiKS53IioOsauoiI5LssD+gpXe5qoYuISB7L8oCe0uWuFrqIiOSx3AjoaqGLiEiey/KA3t3lrtvWREQkv2V5QO8dFNcRU5e7iIjkr+wO6BjOAkRMLXQREclvWR7QwUJRoqYHy4iISH7L+oBOOEJxQIPiREQkv2V/QA9FKQrEdduaiIjktewP6OEIUetSC11ERPJaxgO6mQXN7HUze8xfn2ZmL5nZGjN7wMwK/PRCf32tv33qkD4gFKHIYhoUJyIieW0kWujXAatS1m8CbnHOzQB2A1f76VcDu51zRwK3+PkGF4oQsZhuWxMRkbyW0YBuZpOBDwF3+usGnA085Ge5F/iov3yxv46//Rw///6Fo3qwjIiI5L1Qho//H8DXgVJ/fQywxzkX99frgUn+8iRgI4BzLm5mjX7+htQDmtlCYCFAdXU1O5vKCCXa2dawk7q6ukx+l7zQ0tKiekwz1WlmqF7TT3WafiNZpxkL6Gb2YWC7c26pmdV2J/eT1Q1hW2+Cc3cAdwDMnTvXjRlfQ3tTA5HiMmprz0xDyfNbXV0dtbW1o12MnKI6zQzVa/qpTtNvJOs0ky30M4GLzOxCIAKU4bXYK8ws5LfSJwOb/fz1wBSg3sxCQDmwa9BPCUcp0KNfRUQkz2XsGrpz7kbn3GTn3FTgUuAZ59zlwGLgEj/bAuARf/lRfx1/+zPOuX1a6PsIRShAg+JERCS/jcZ96DcAXzOztXjXyH/pp/8SGOOnfw34xpCOFo5S4Dpp61JAFxGR/JXpQXEAOOfqgDp/eR1waj95OoBPDvvgoQjhZCftelKciIjksex/UlwoQth10RaLM5QeehERkVyU/QE9HAEglOwillBAFxGR/JT9AT0UBaCQLtp1HV1ERPJU9gd0v4UeIUZbLD5IZhERkdyU/QE95Ad069JIdxERyVu5E9DV5S4iInks+wN6uPsaekwtdBERyVvZH9BTWuhtXbqGLiIi+Sn7A7rfQo+YutxFRCR/ZX9ADxUC/jV0Pc9dRETyVA4EdF1DFxERyf6AHu69bU1d7iIikq+yP6D7LXRvUJwCuoiI5KccCOjeNfTiQFxPihMRkbyV/QHdH+VeGoqpy11ERPJW9gf0YAFgXgtdAV1ERPJUaLQLcNDMIBylmLha6CIikreyP6ADhAopcTFaOnUNXURE8lP2d7kDhKIUBWK0KqCLiEieyo2AHo4Qtbha6CIikrf2G9DN7DMpy2f22faPmSrUsIWiRK2LVr2cRURE8tRgLfSvpSz/3z7bPpfmshy4cISIddHaqUFxIiKSnwYL6DbAcn/roycUIYIGxYmISP4aLKC7AZb7Wx89oQgFdNEVTxJLJEe7NCIiIiNusNvWjjGz5Xit8SP8Zfz16Rkt2XCEoxS4LgBaO+NUFBWMcoFERERG1mAB/dgRKcXBCkUIJzsBaFFAFxGRPLTfgO6cW5+6bmZjgPnABufc0kwWbFhCEUJ+QNfAOBERyUeD3bb2mJnN9JdrgJV4o9t/ZWZfGYHyDU24N6BrYJyIiOSjwQbFTXPOrfSXrwKeds59BDiNQ+m2tVCEQE8LXQFdRETyz2ABPZayfA7wBIBzrhk4dIaTh6ME4h2AU0AXEZG8NNiguI1m9iWgHjgJeBLAzKJAOMNlG7pQBMNRgB7/KiIi+WmwFvrVwPHAZ4G/d87t8dPnAXfvb0czi5jZy2b2hpm9aWbf9dOnmdlLZrbGzB4wswI/vdBfX+tvnzrkbxGKABChSy10ERHJS/sN6M657c65LzjnLnbO/SklfbFz7t8GOXYncLZz7kRgNvABM5sH3ATc4pybAezGO2nAn+92zh0J3OLnG5qwF9AL9bQ4ERHJU/vtcjezR/e33Tl30X62OaDFXw37kwPOBj7tp98LfAe4HbjYXwZ4CPipmZl/nP0LRQEoCcZoVkAXEZE8NNg19NOBjcBvgZcY5vPbzSwILAWOBG4D/gbscc51R916YJK/PMn/LJxzcTNrBMYADYN+kN9CH1OYpLlDAV1ERPLPYAF9AnAecBleq/px4LfOuTeHcnDnXAKYbWYVwB/o/8lz3S3w/k4W9mmdm9lCYCFAdXU1dXV1jGlYyyyg2LWxdv0m6up2DqV40o+Wlhbq6upGuxg5RXWaGarX9FOdpt9I1ulgT4pL4I1sf9LMCvECe52Z/W/nXN/Xqe7vOHvMrA5vMF2FmYX8VvpkYLOfrR6YAtSbWQgoB3b1c6w7gDsA5s6d62pra2FtAlZCTVmYzWVV1NaeOtSiSR91dXXU1taOdjFyiuo0M1Sv6ac6Tb+RrNPBRrl3jz7/OPBr4FrgVuDhIew3zm+Zd9/mdi6wClgMXOJnWwA84i8/6q/jb39mSNfPAQpKAKgKx2lqjw2SWUREJPcMNijuXmAm8EfguylPjRuKGuBe/zp6AHjQOfeYmb0F/M7Mvg+8DvzSz/9LvEfKrsVrmV865E8q9AN6qIumVgV0ERHJP4NdQ78CaAWOAr5s1nOZ2/AGspcNtKNzbjkwp5/0dcA+feLOuQ7gk0Mrdh8FxQCUhzppategOBERyT+DXUMftEv+kFBQCkB5oJOmDrXQRUQk/2RHwB6M30IvDXbSFU/SEdMrVEVEJL/kRkAPFUIgRIl1AOhedBERyTu5EdDNoKCYYryArm53ERHJN7kR0AEKSom6dgDduiYiInknhwJ6MRHX3UJXl7uIiOSX3AnohSUUJFoBaFQLXURE8kzuBPSCYsIJr8u9sa1rlAsjIiIysnIooJcSjnst9N1taqGLiEh+yaGAXozFWimNhNjVqha6iIjkl9wJ6IUl0NlCVXEBu9XlLiIieSZ3AnpBCXS1UllUoBa6iIjkndwK6PF2xhYF1UIXEZG8kzsB3X+FanUkwW69QlVERPJM7gT0Ai+gTyjsVJe7iIjkndwJ6JFyAMYVdNIeS9DepTeuiYhI/si5gD421Amg6+giIpJXciiglwFQFfSeFqdudxERySc5FNArAKj0A/qOls7RLI2IiMiIyqGA7nW5VwTaANjRpIAuIiL5I3cCeqHX5V6KF9C3NXWMZmlERERGVO4E9FABhKKEupqoKi5gW7MCuoiI5I/cCejgdbt3NDK+tJBt6nIXEZE8kpsBvSzCdnW5i4hIHsnJgF6tFrqIiOSZHAvoZdDRRHVZhB0tnSSSbrRLJCIiMiJyLKD7LfTyCImko0H3oouISJ7IwYC+h8kVUQDqd7ePcoFERERGRm4F9GgltO9hSmUEgPrdbaNcIBERkZGRWwG9aAy4BJOicUAtdBERyR+5F9CBaGw3Y0sK1EIXEZG8kZMBnbZdTK4sYuMutdBFRCQ/ZCygm9kUM1tsZqvM7E0zu85PrzKzp81sjT+v9NPNzG41s7VmttzMThr2hxZVefO2nUyujKqFLiIieSOTLfQ48E/OuWOBecC1ZnYc8A1gkXNuBrDIXwf4IDDDnxYCtw/7E3ta6Ds5rKqI+t3txBLJg/waIiIih76MBXTn3Bbn3Gv+cjOwCpgEXAzc62e7F/iov3wxcJ/zvAhUmFnNsD40JaAfOb6EeNKxfqda6SIikvtG5Bq6mU0F5gAvAdXOuS3gBX1gvJ9tErAxZbd6P23oCkogWNAT0AHWbm85mKKLiIhkhVCmP8DMSoD/BL7inGsyswGz9pO2z7NbzWwhXpc81dXV1NXV7bX99GAJu/62ks32OgBPv7ScSMPqAy5/vmlpadmnTuXgqE4zQ/WafqrT9BvJOs1oQDezMF4wv98597CfvM3MapxzW/wu9e1+ej0wJWX3ycDmvsd0zt0B3AEwd+5cV1tbu3eGVROpKS+k5tz3M/GVRSRLxlBbOzudXyun1dXVsU+dykFRnWaG6jX9VKfpN5J1mslR7gb8EljlnPtxyqZHgQX+8gLgkZT0K/3R7vOAxu6u+WEpqoK2BgCOrC7l7a3NB/gNREREskcmr6GfCVwBnG1my/zpQuBHwHlmtgY4z18HeAJYB6wFfgFcc0CfWjweWrxG/6xJZbyzrZmOWOKgvoiIiMihLmNd7s65JfR/XRzgnH7yO+Dag/7g0gnQvBWcY9akCuJJx+qtzcyeUnHQhxYRETlU5daT4gDKJkK8HTr2cMLkcgCW1+8Z5UKJiIhkVu4F9NIJ3rxpCzXlEcaWFLJsowK6iIjkthwM6BO9efMWzIyTD6/g1fd2j26ZREREMiwHA7rfQm/2BsifOm0MG3a1saVRL2oREZHclYMB3X9arB/QT5vmvbDlpXW7RqtEIiIiGZd7AT0cgWgVNHkB/diaMsqjYZ5bs2OUCyYiIpI5uRfQAconQaP3WPhgwDjnmPEsWrWduN68JiIiOSo3A3rVdNi1rmf1/OMn0Nge4+V31e0uIiK5KXcD+u71kIgDMP+osRSGAvzprW2jXDAREZHMyNGAfgQkYz3d7kUFIWqPHsdjy7cQU7e7iIjkoBwN6NO9eUq3+6fmTqGhpZNFq7YPsJOIiEj2ypuAftZR45hQFuGBVzaMUqFEREQyJzcDeukEKCiFHat7kkLBAJ+cO5ln39nB5j16yIyIiOSW3AzoZjBhJmxduVfyp+ZOwQH3vbB+dMolIiKSIbkZ0AEmnADbVkKydxDclKoiLjpxIvf89V22NXWMYuFERETSK4cD+izoaoHd7+6V/E/nHU0i6fjJojWjVDAREZH0y+2ADrD59b2SDxtTxKdPPYwHXtnIuh0to1AwERGR9MvdgF59PBSUwPq/7rPpH8+eQWEowA+eWDUKBRMREUm/3A3owTAcfga8+9w+m8aVFnLdOTP486rtPPXm1lEonIiISHrlbkAHmPp3sHNNz5vXUn3ufdM4ZkIp337kTVo646NQOBERkfTJ7YB+5Lne/O3H99kUDgb4wcdnsa25gx//6Z0RLpiIiEh65XZAH38sjD0a3vyvfjefdFgll592GPf89V2Wrteb2EREJHvldkA3g5kfh/eWwJ6N/Wb5+geOYVJllC//dhmNbbERLqCIiEh65HZAB5h9uRfYX72r381lkTC3XjqHbU0dfOPh5TjnRriAIiIiBy/3A3rFFDj6Qlh6N7Tv7jfLnMMq+ecLjuaPK7dy71/fG9nyiYiIpEHuB3SAs26A9j3w7L8OmGXh303n3GPH8/3HV/Hyu7qeLiIi2SU/AnrNCXDyAnj5DtjR/4j2QMD490/NZkpVEV/89VI27mob4UKKiIgcuPwI6ABn/y8IF8Mj10C8s98s5dEwdy6YSyyR5HP3vEJThwbJiYhIdsifgF48Fi76CdS/Ak9cP2C2I8aV8PMrTubdhla+8KuldMQSI1hIERGRA5M/AR3g+I/B+74Gr9074Kh3gDOOGMtNnziBv/5tJ1/67evEEskB84qIiBwK8iugA5z9P+HI8+CJr8OGFwfM9omTJ/Pdi47n6be28fWHlpNM6nY2ERE5dOVfQA8E4RN3erezPXAF7PzbgFkXnDGV6y84mj+8volvPbpS96iLiMghK2MB3czuMrPtZrYyJa3KzJ42szX+vNJPNzO71czWmtlyMzspU+UCIFoBlz0ALgH3XTzgU+QArqk9gi+cdQS/fnEDX39oOXF1v4uIyCEoky30e4AP9En7BrDIOTcDWOSvA3wQmOFPC4HbM1guz7ij4DMPQ0cT3HcRNNb3m83MuOEDR/OVc2fw+6X1fOHXS2nv0kA5ERE5tGQsoDvnngP6PqHlYuBef/le4KMp6fc5z4tAhZnVZKpsPSbOhs88BK0NcNcHoGFtv9nMjK+cexTf++hMFq3ezhW/fEnPfRcRkUPKSF9Dr3bObQHw5+P99ElAar93vZ+WeVNOhQX/DbF2uOsC2PLGgFmvmHc4P73sJJbXN3LxbUt4Z1vziBRRRERkMJbJgV5mNhV4zDk301/f45yrSNm+2zlXaWaPAz90zi3x0xcBX3fOLe3nmAvxuuWprq4++Xe/+11ayhpt28SJb3ybULyVlTO/yZ7KWQPmfWd3gp++3klnwnHVzELm1YTSUoZDQUtLCyUlJaNdjJyiOs0M1Wv6qU7TL911+v73v3+pc25uf9tGOhJtM7Ma59wWv0t9u59eD0xJyTcZ2NzfAZxzdwB3AMydO9fV1tamr3RnngW/+jizV3wHLvw3mHtVv9lqgYvO7uDa37zGz9/YTUu0mv/5oeOIhIPpK8soqaurI611KqrTDFG9pp/qNP1Gsk5Husv9UWCBv7wAeCQl/Up/tPs8oLG7a35ElU+Gf3gapr8fHvsKPP7PkOj/WvmE8gi/WziP/zF/Or9+cQOfuP2vrN2uLngRERkdmbxt7bfAC8DRZlZvZlcDPwLOM7M1wHn+OsATwDpgLfAL4JpMlWtQkXL49ANwxpfhlV/Arz4GTf2fW4SDAW688FjuvHIum/a0c+GtS/jFc+tI6CE0IiIywjLW5e6cu2yATef0k9cB12aqLMMWCML534PqmV5L/ednwkdvh6Mu6Df7ucdV86evzuebD6/k/zyxynsQzUeOY970MSNccBERyVf596S44Tjx72Hhs1A6EX7zKfjjDdDV2m/W8aURfnHlyfz003NobI9x6R0vcrbPUxAAABUjSURBVM39eg2riIiMDAX0wYw7Cv7hz3DaF+Gln8PPToe/PdNvVjPjwydMZNE/ncXXzjuKxat3cM6Pn+Xfnnqbls74CBdcRETyiQL6UIQj8MEfwVV/hGDYu67+hy94D6TpRyQc5MvnzOCZfz6LC2dO4KeL13LGDxfx7396m50t/b+LXURE5GAooA/H4WfAF/4Cf/fPsOL38JPZsOQWiHX0m72mPMp/XDqHR//xTM44YqwX2H/0DNf//g1Wbmoc4cKLiEguU0AfrnAEzvlf8MUXYOr74M/fgZ/OheW/h2T/L245YXIFP7/iZJ7+6ll8/KTJPLZ8Cx/+v0v46G1/4TcvbaCpQ4+RFRGRg6OAfqDGHQWf/h1c+ShEK+Hhf4CfzYM3fgeJ/q+XHzm+hB9+fBYvfvMcvvXh42jpjPPNP6xg7vf/zML7XuWRZZv0jHgRETkgufPM0tEy/SxvJPybD8PzP4Y//A9Y/AM4/Vo48VLvvvY+yqNhPve+aVx15lSW1zfyh9c38ceVW/jTW9sImNein3/UOObPGMvsKRWEgjrvEhGR/VNAT4dAAGZdAsd/HN55Ep7/d/jj173u+JmfgLmfg0n7vuLdzDhxSgUnTqngWx8+jtc37uG5d3bw3Jod/PSZNdy6aA2lhSHOOHIMp04bw9zDKzluYhlhBXgREelDAT2dAgE45kJv2vQaLL0bVjwEr/8Kxh0LMz/uBf2xR/azq3Hy4ZWcfHglXz3vKBrbYvzlbw08v2YHz69p4Kk3twEQDQc5fmIZx9aUcfSEUo6tKeXoCWWUFOpPKSKSzxQFMmXSSd50/ve9EfEr/hMW/x9vmjALjr4QjjzPyxPY96Uu5UVhLpxVw4WzvNfCb23s4NX1u3j1vd28tbmJ/1q2ieaO3mv1U6qiHDOhjGMnlDKjupTp44qZNraYogL9iUVE8oH+t8+0SDmc8g/e1LgJ3voveOsReO5mePYmb0DdtPkwZR5MOQ1qTvDude9jQnmED58wkQ+fMBEA5xybGztYvaWJ1VubWeXPF63aRuqj5CeURZg2tphp44qZPraYw8cUc1hVEVOqogr2IiI5RP+jj6TySd5gudOvhbZd3hPn1v4Z3vuLF+QBQlGYdDJMPhnGHw/Vx8PYoyBUsNehzIxJFVEmVUQ559jqnvSOWIJ3G1p7pnU7Wnm3oYUnVmxhT58R9GNLCphcWdQT4A+rKmLXzgRH7GqjpjyiwXgiIllEAX20FFV5A+lmXeKtN22GjS/Bxpdhw4vw4u2Q6PK2BUIwZgaMPwaqjoCqaVA1HSqnQekEMOs5bCQc5Nga7xp7X7tbu1i/q42Nu9rYsKuN+t3efNnGPTy+YkvPW+JuemUxoYAxsSLKlKookyuKmFgRZWJFhEkVUSZWRKmpiFAYyv73v4uI5AoF9ENF2UQ4/mPeBN572HeuhW1vwva3YNtbsHkZvPUouETvfuEiqJzqBfiKw7zjlE3yp4lQWgNB789cWVxAZXEBs6dU7PPx8USSLY0d/PfiF6iaMoONu9vYsKudjbvaeObt7exo3veRteNKC5lYEWVSRYSJ5VE/6Hu9BpMqo1QWhbGUkw0REckcBfRDVTAM44/1plSJGDRuhF3rYNe73rT7XS/4/20xxPq8Dc4CUDwOisdDyXgoqfbnqcvVhIrHMaWykuPGBKk99bB9itMZT7C1sYNNe9rZvKeDzXva2bynnU172lm9tZlnVm+nI7b3k/Ii4UBPgO8N+JGegD+hXK18EZF0UUDPNsGw1xqvmr7vNuego9Hrvm/a5E2Nm6BlK7TsgJZtsONtb57s54l0wQLmhcrgnSl7BXuKx1NYMp7DS6o5vGI8TB4PBZP26up3zrG7LdYT5FMD/qY9HazeOvRW/qTKaE/Xvlr5IiJDo4CeS8wgWuFN1ccNnM856NgDLdu94N6yvWd5z9rlTIiadzKw+XVo3QGun2fUhyJQNKZnsqIxVBWPpapoDDOLqqBiLEz0txdPgWglnS6wVyt/024/8DcOr5U/qbK3pa9WvoiIRwE9H5l5t8tFK2Hc0XttWh2uY0JtbW9CMgFtO/sE/23Q1uCN1G9t8Lbvfs9b7xz4LXKFkXIOLxrL4d0nAsVjoHwM1IyBorG4oiqaApVsixezsbOI9c1BNjd2sLlxeK18L+CrlS8i+UUBXfYvEOy95s7MwfPHu6B9lxfku4N936m1ARrrYcsb3omBP5rfgHJ/OgogEO7tBSiugvFjSUSqaA6UsYsytseL2RwrZkN7EevaOnl7SwHPrI4P2srvDvjdrfzqsgiRsFr5IpLdFNAlvUIF3q10pROGlt856GrxA31q4E85GehO37qSYFsDFe27qQD6GUWAi5aSrKqis6CClmAFjVbKzmQpW+PF1O8u4r3NEZ5ri7CLMna5UhopxhGgLBKiuixCdVmE8aWFjC+LUF1W2LNeXRZhXGmhAr+IHLIU0GV0mUFhqTdVTh3aPom4NwZgn16ABqxtF8G2nRS1NlDUtpPxbe8yo60BYm29+xf2LjoCtIfLaQ2U0dRZzJ7tRezcHGVHrJDdrphlrogmimny58nCcgqKK4iWjaGkfAzjyot7An73ScC40kJd1xeREaeALtknGILisd7UZwzAgLravEsBPScAu/wTgJ0U+dO4jkbvLoGOTbiORmjfg6Xe8w/ggBZ/2gytrpBGimlyxTRRxGpXxMsU0xUshsIyrLCY1hjsXvcSBUWlFBaVUVRSRnFJOSXlFZSXVVJeUUG4sMR7uY+IyAFSQJf8UFDkTeWTh5TdwL8c0OoH+dRpT89yUfsegi27KWnZRXWblx7o2k4o1kJhZwuhTv+EYMPgn9lOhPZAlFigiHioiGS4CBcuxgpLCUZKCEVLKYyWEikupSBShIWLIBz17jgIF0HYn6euh6K9eXTCIJLTFNBFBmIGhSXeVD6p/yxAxJ/6Fe9iyeIned8ps4m3N9Pc3EhLcyOtTXvoaGuio7WJWHszifZmEp0tWFcLFmsl2NlGuK2NKNsoYj3F1kEBHRTRQdgSA33afrlQBEIR/0QgJdh3T6FCLy1U2LseLPTnBb3z1OWhpnUv9/NmQRFJDwV0kUwKFRAPl0HFYYQqoLIGKoe4q3OO1q4Eu1u72NHaxTutXexs7WJPcytNzc20tjTT2tpCR3sL7W0tdLa3kuxqI0KXN5k3j/rr0XgXUeuiLBSnJBCjOBCjKNBF1BqJsp1CYhS4LsJ0EUp2EUp2Ekh2Eeh72eFgWMA7SQgWeAMo9wr84f1s2zfv1I1bIPBq74lHMNxP3oL+j9n3M3WyITlAAV3kEGVmlBSGKCkMMaWqaEj7JJKOlo44je0xmjpiNLZ7U5M/39YeY01HjOaOOC0dcZo74jR1xGjp9JabO2J7vX4XIECSAmIUEKOQOAXEKAklqChwlIUdpQWOslCC0nCSklCCkmCS4lCComCSokCcSCBBNJAgYgkKLU6hxSkw7zgh4oRcDEt0ebc8Jjq9xxt3tUFij3dLY882f7u/PDUZg/VprXHvRUg9U6B32YL+cj/znm3d6cE++/mTBbwJ85ctZZn9bLOUbd3LDJJvf8fvJ5+/Pql+Lby85gCObynrg+VLLcdg33Mkj99dr8M5fvf2Q4MCukgOCQaM8qIw5UXhA9rfOUdbV8IP8DGaUgJ/S2eMls4ErZ1xWjvjtPjzps44mzvjtHYmaG3rTW/tGlrL3gyKC0IUFwYpKggRDQcpKggSLQhSVBQkGg4SLQhRVJCSHg6yft07nHz80RQHExQHExQFE0SDCaKWIBKIEw0kCBMj6GJYIuafGHSmnCT4Jw/dJxHxTu/FR8nuKe6vx/0pmbIc956g2LOe6J33HCeecqyYNyYD5+3nXMq6n7bXtmQ/21w/x+izX/fyAZoBsPaAd89jfU8Sepf/LungryEO7OStv20DU0AXkR5mRnFhiOJC7778g5FMOtpiCVo6UoJ894lAl38C0JPmLbfFErR3xWnrStDcEWdHcydtXQnauvz0WMKLW76733xzCN8JCoIBCoIhCkIFFIQChIMBCkIBQgEjHAwQDhohfx4OBggF+qQFAoSCRjicku7v250nFDDCoUBv3p70AMHUY3Sn9exvBM0IBmzv5UCAQICevMGAETCG9tTDwU4YBjh5+MuSJZx55hmDnFikbmMYx096d4kMeuLS9xgMufy9ZRrO8fs5UUrjydXmDRuZMnnSIPU4jOPz2oB/dgV0EcmIQKD3kkG6OOfojCdp60qw+LklnHjyKSkBP+Evx2mPecuxeJKuRJKu1Hk8SSzhrccSjrg/jyWSdMSStHTE6fLT40lHVzxJPJkk7ueJJRzxpDcfDaGAEQh4JxDdgT4UMALmp6WcFAT99H2WzXpOFgL+iULAjF07C7hn63s96wF/n5711OXA3mmpJxpmYAS9vAHzGqVmmOHlpXcf849n3fv5x+k+nGH+8VKP3SdP9z4923vTe4/jbex7nJ5GMN0N6N4d9jrOXp+Xsm8K63N8gLd2vcXxE4/fq1zdUn9Bvce3fvP2unvALQroIpI1zIxIOEgkHGRMNMCR40tHrSzOOeJJ5wX6vQK+txxPJumKe/PufPFEkliy92ShO18i6R0r6c8TqZPz5vFE93JywLx7pTtHwt8nmXKcZPc8Ce2JBPGkwzmHc9DU6ejY5T2EqTuvc3j7OG8f5xxJBwnnepf9Y0BKI5zevEnnetbdXuuj9McbacsGblWnkwK6iMgBMDO/ix6i5MYI+bq6Ompr54/Y53UHeAc9Jw8Ot1eg707rXU7Zl5STgj77dp9A9C73Hqf3hKN71e177EHKsNcJTM+xXMqyt/bKy68w95RT9to/tWXf3aZ3+CdMfb5/X7NuGnjbIRXQzewDwE+AIHCnc+5Ho1wkERHJkNQu8eB+Opmz2ebSAEdPGJmepEPm0VFmFgRuAz4IHAdcZmb7eam3iIiIdDtkAjpwKrDWObfOOdcF/A64eJTLJCIikhUOpYA+CdiYsl7vp4mIiMggDqVr6P1dQNlnaICZLQQWAlRXV1NXV5fhYuWXlpYW1WmaqU4zQ/WafqrT9BvJOj2UAno9MCVlfTKwuW8m59wdwB0Ac+fOdbW1tSNSuHzhjXKtHe1i5BTVaWaoXtNPdZp+I1mnh1KX+yvADDObZmYFwKXAo6NcJhERkaxwyLTQnXNxM/tH4Cm829bucs4N/lxHEREROXQCOoBz7gngidEuh4iISLY5lLrcRURE5AApoIuIiOQAc1n8dHwz2wGsH+1y5JixQMNoFyLHqE4zQ/WafqrT9Et3nR7unBvX34asDuiSfmb2qnNu7miXI5eoTjND9Zp+qtP0G8k6VZe7iIhIDlBAFxERyQEK6NLXHaNdgBykOs0M1Wv6qU7Tb8TqVNfQRUREcoBa6CIiIjlAAT2PmNkUM1tsZqvM7E0zu85PrzKzp81sjT+v9NPNzG41s7VmttzMThrdb3DoMrOgmb1uZo/569PM7CW/Th/w30+AmRX662v97VNHs9yHMjOrMLOHzGy1/5s9Xb/Vg2NmX/X/7a80s9+aWUS/1eEzs7vMbLuZrUxJG/Zv08wW+PnXmNmCgy2XAnp+iQP/5Jw7FpgHXGtmxwHfABY552YAi/x1gA8CM/xpIXD7yBc5a1wHrEpZvwm4xa/T3cDVfvrVwG7n3JHALX4+6d9PgCedc8cAJ+LVr36rB8jMJgFfBuY652bivTPjUvRbPRD3AB/okzas36aZVQHfBk4DTgW+3X0ScMCcc5rydAIeAc4D3gZq/LQa4G1/+f8Bl6Xk78mnaa96nOz/Az4beAwwvAdJhPztpwNP+ctPAaf7yyE/n432dzjUJqAMeLdv3ei3elB1OgnYCFT5v73HgAv0Wz3g+pwKrExZH9ZvE7gM+H8p6XvlO5BJLfQ85XefzQFeAqqdc1sA/Pl4P1v3fwDd6v002dt/AF8Hkv76GGCPcy7ur6fWW0+d+tsb/fyyt+nADuBu/1LGnWZWjH6rB8w5twn4N2ADsAXvt7cU/VbTZbi/zbT/ZhXQ85CZlQD/CXzFOde0v6z9pOm2iBRm9mFgu3NuaWpyP1ndELZJrxBwEnC7c24O0EpvF2Z/VK+D8LtzLwamAROBYrzu4L70W02vgeox7fWrgJ5nzCyMF8zvd8497CdvM7Maf3sNsN1PrwempOw+Gdg8UmXNEmcCF5nZe8Dv8Lrd/wOoMLPu1xOn1ltPnfrby4FdI1ngLFEP1DvnXvLXH8IL8PqtHrhzgXedczucczHgYeAM9FtNl+H+NtP+m1VAzyNmZsAvgVXOuR+nbHoU6B5huQDv2np3+pX+KM15QGN3l5J4nHM3OucmO+em4g0wesY5dzmwGLjEz9a3Trvr+hI/v1o9fTjntgIbzexoP+kc4C30Wz0YG4B5Zlbk/1/QXaf6rabHcH+bTwHnm1ml33tyvp924EZ7YIGmkZuA9+F16SwHlvnThXjXxRYBa/x5lZ/fgNuAvwEr8EbHjvr3OFQnoBZ4zF+eDrwMrAV+DxT66RF/fa2/ffpol/tQnYDZwKv+7/W/gEr9Vg+6Tr8LrAZWAr8CCvVbPaB6/C3eOIQYXkv76gP5bQKf8+t3LXDVwZZLT4oTERHJAepyFxERyQEK6CIiIjlAAV1ERCQHKKCLiIjkAAV0ERGRHKCALpLHzCxhZstSpv09jW24x56a+jYqEcms0OBZRCSHtTvnZo92IUTk4KmFLiL7MLP3zOwmM3vZn4700w83s0X+e50Xmdlhfnq1mf3BzN7wpzP8QwXN7Bf+O7j/ZGbRUftSIjlOAV0kv0X7dLn/fcq2JufcqcBP8Z5Pj798n3PuBOB+4FY//VbgWefciXjPXH/TT58B3OacOx7YA3wiw99HJG/pSXEieczMWpxzJf2kvwec7Zxb57/QZ6tzboyZNeC98znmp29xzo01sx3AZOdcZ8oxpgJPO+dm+Os3AGHn3Pcz/81E8o9a6CIyEDfA8kB5+tOZspxA43ZEMkYBXUQG8vcp8xf85b/ivVUO4HJgib+8CPgigJkFzaxspAopIh6dLYvkt6iZLUtZf9I5133rWqGZvYR34n+Zn/Zl4C4zux7YAVzlp18H3GFmV+O1xL+I9zYqERkhuoYuIvvwr6HPdc41jHZZRGRo1OUuIiKSA9RCFxERyQFqoYuIiOQABXQREZEcoIAuIiKSAxTQRUREcoACuoiISA5QQBcREckB/x9i18HqsBxxFAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.DataFrame(history_4.history)[1:].plot(figsize=(8, 5))\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('MSE')\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Saving your model**. You can save your model in the hard disk as follows:\n",
    "\n",
    "```{python}\n",
    "model_name.save(\"my_keras_model.h5\")\n",
    "```\n",
    "\n",
    "And you can load it in a new session with: \n",
    "\n",
    "```{python}\n",
    "model_name = keras.models.load_model(\"my_keras_model.h5\")\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Classification\n",
    "### 2.1 Binary Classification\n",
    "For the binary classification case, we use the \"circles\" dataset already encountered with kernel SVM's. Let's load it and view it again: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVIAAAE9CAYAAABZd+pgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2df5QeVZnnv0+/6TZ2gBVe/BGB7hDGM0OCG8EQYUQPKy6L0TPISBhmepgoaIZmceQc95yBzdkZ1zEzx9GjB0dktnUY0O5lXJlxh1WEEZTJmVGEDib8SCQmIQ0ZUEN0IhLQJP3sH/W+SXV13ap7q+6turfq+ZxTp/utt96qW1X3fu9zn+f+IGaGIAiCUJyBuhMgCIIQOiKkgiAIJREhFQRBKIkIqSAIQklESAVBEEoiQioIglCSBXUnwAUnnngiL1mypO5kCILQMDZt2vQcM78yub+RQrpkyRJMT0/XnQxBEBoGEc2k7ZemvSAIQklESAVBEEoiQioIglASEVJBEISS1CakRHQKEX2biLYR0eNE9KGUY4iIPkNEO4joESI6q460CoIgZFFn1P4QgA8z88NEdCyATUT0TWbeGjvmHQBe19veBODm3l9BEARvqM0iZeZnmfnh3v/PA9gG4KTEYRcD+CJHPADgFUS0uOKkCoIgZOKFj5SIlgA4E8D3El+dBODp2Oc9mC+2guCMqSlgyRJgYCD6OzVVd4oEH6m9Qz4RHQPg7wFcx8w/T36d8pPUmaiJaB2AdQAwMjJiNY1CO5maAtatAw4ciD7PzESfAWBsrL50Cf5Rq0VKRIOIRHSKmf8h5ZA9AE6JfT4ZwDNp52LmCWZeycwrX/nKeSO4BMGY9euPimifAwei/YIQp86oPQH4GwDbmPlTisPuBPAHvej9OQD2M/OzlSVSaDVPPWW2X2gvdTbt3wzgCgCPEtHm3r7/DmAEAJj5rwHcBWA1gB0ADgB4Xw3pFFrKyEjUnE/bLwhxahNSZv4XpPtA48cwgP9aTYoEYS4bNsz1kQLA8HC0XxDieBG1FwQfGRsDJiaA0VGAKPo7MSGBJmE+IqSCkMHYGLB7NzA7G/0tK6LXXAMsWBAJ84IF0WchfGrv/iQIbeGaa4Cbbz76+fDho58/97l60iTYQSxSQaiIiQmz/UI4iJAK1pBRQNkcPmy2XwgHEVLBCv1RQDMzAPPRUUBtE9OsyqTTSf+Nar8QDiKkghVcjgIKxdLNq0z6w0uTqPYLAcHMjdve+MY3slAtRMyRfMzdiNKPn5xkHh2Nvh8djT6rjhsennvO4WH18XUyOpr+DLrdo/e6aBHzwEC0v9NhHh+vO9WCCQCmOUVzxCIVrKAa7ZO2P81y+/3fB048cb61WeV4d13LV3Wcaujovn1H7/WFF4CFC4HJSeDQIXvR+lCs9saSpq6hb2KRVo+J5aiy3NJ+Y2rpuk5/1nFZ95XcRkerT7tQHigs0tpFz8UmQloPus11lTimiYxKnIoIUVb6dK+Tddz4uL6Q2qwIbD4jIRsRUsEb8iy3uMjoWFs6Ap53nqz0xMmykE0t0rgV2+nM3W9CVVa7IEIqFETXyjQ9Z5ZVmrSkstKg26zNs9r6QpbcOp2511cd1/9eR0SHhyPrNZnuos1ysUirQ4S05RQRRJe+tyyhcSEieVZbnvDlfZ/lI41H7ZOWqG5lkoX4SKtDhLTFFC1oLi2dLNExEX3dZm3evai+V1mg8e/7XZhMnnOe9ZrVLE97PrZaDi5aIE1ChLTFFBVEG743VcFUiU5akzdL9HXvLU/kVN9niZ3qPDpCVNQidWl9imWbjwhpiykqiGUtUh3x0m3y2hCWPJEzSU9ZKz0t3TYrjiKIrzUfEdIWo1NAVM3FMhZKkYJZRPRdNkezBK+olZ5MN6AftXcZoZfofz4ipC2maLO2rO+tSMH00SrSjdpXQR0WaacjPtM+IqQtx0ZndFOKnNd3P13d6TMZgWUakNKxvov4g5uECKmgxFWTLq1g9q/V7Uabqm+orwV0cjJKd/9++r0Mqk6DqSAODTEPDpoJcJb1XXeFUhcipIFRpZiY9H80Je4HzOryE0IhDEU8bI35LzKKq+mBKRHSgKi6wOpaMETFp33TKdy+F8JQxEN3hFX/naoq7az7bWtgykshBXALgJ8AeEzx/fkA9gPY3Nv+ROe8oQtpHQU2WZjizde0gmeKTuH2vRCGIh4mFmm3mx1oNJ3pyrdKxTa+CulbAZyVI6RfMz1v6ELqQ4E1GQuvg1ik1WHiI1VVmP17Mh1Q4ZubwzZeCmmULiwRIZ1LXQVWJ9BQVNDzIsIhFMIqxaOsj1w3al+m0vY5KOiKkIV0H4AtAL4BYLnOOUMX0jpqe92O52UEPV7w+lF7oNwUclVThXhU+f5DsbJ9IVQhPQ7AMb3/VwP4YcZ51gGYBjA9MjJi/QFWTdW1va5fraiPNI22Ng/zsCVuOnlI3oEZQQppyrG7AZyYd1zoFmkdmER6bSHWUDq2JouxNQeBcBSVkHq9+B0RvYaIqPf/KkTLR++rN1XNRLV4XZLRUXvXVC0Wp9rfFkwWElRhsmjg2BiwezcwOxv9HRvTv44QUauQEtHtAL4L4NeJaA8RXUVEVxPR1b1DLgXwGBFtAfAZAJf3agXBMhs2AMPD2ccMD0fH2cKGYDSRtHdh+uylkqqYNDM19E2a9sVINvHGx902+cQ/p8akuW1jOkJBD/jqI3WxiZCWpyq/mfjnyqGqjEwnyBb0UAkpRd81i5UrV/L09HTdyQiWqSlg3bq5PrbhYWBiQvxnvrFkCTAzM3//6GjkCli/PmrOj4xEn+X9lYOINjHzyuR+r4NNbWBqKioMAwPR36mpulNkFqgQ6iXLFypBpOoQIa2RvuU3MxM1vmZmos9ZYlqF8EqgIhxcBeyy8pmPlX/tpLX3Q99C8ZG6XJ+oynQJ9WE7TyTnW02es+0BQkiwyT9MO15XJXBtLyyhYStglzdMeHTU7dy1IaASUgk21UhWoGD37vn7BwaibJuEKPKD2WRqSgIVbUOVH/tEQ2PS82CSpgYnJdjkIaYdr6vswC6BivaR5wMfGdHPa20LToqQ1sjYWFRrj45Gtf3oaHYtbmPEiyCoyBLJfj7TGQHXp03BSRHSmjGx/EyFVxBMUIlkt3s0n6XlwW43/XxtGuorQhoY0uQWXJEmkpOTwHPPzc1nyTx4443uW0q+d7laUHcCBEHwh77VafobwF1wMjnSrt/fOn7tuhGLtGJ8r1kFoQguW0ohjLQTIa0QnZFMIrSCMJcQRtqJkFZIXs1aZMho45CaREgQwry1IqQVklezhtCEcYrUJEIKIXT7EyG1hI4hlVezVtGE8drga31NIqQRRLe/tHGjoW9Vj7XXHZued5zrsfTej6G3seqbIDgEIS5+Fwq6hlRezeq6CeO9wReCMwzw3KwXaiFNXUPfqrZIbRpSLpfe8N7g895k5jDSaImiebHq31UJZBo9d4Qyf2cQ6fS9NAXxEMtTtL6o+ndVI0LqkFAyQSjp9BrvzXo7FK0vqv5d1aiEVHykFggiqohw0uk1ofhxS1K0B0nVv/OFWoWUiG4hop8Q0WOK74mIPkNEO4joESI6q+o06hLKZCKhpNNbQujUaIGi9UXVv/OFui3SWwFclPH9OwC8rretA3BzBWkSBDUtMeuL1hcbNgBDQ3P3DQ3p/S7k+qlWIWXmjQB+mnHIxQC+2HNPPADgFUS0uJrU2UN6yxTA9kMrc77kbwG3Zr0HGaZMfcGc/TnvegDQ6RztmhdEeUlznFa5AVgC4DHFd18DcF7s830AVuad06fF7yTAo0EyUj8+bn9pzKLnq/oFBp5h8oJGeZ0yfL99+Bq1zxHSr6cI6RsVx64DMA1gemRkxPoDLIpONNL3Hj9O6N90P+KdjIDbDOEWDQlPTjJ3OnbT4iqtnpDVqUFHJH2//VCF9H8B+N3Y5ycALM47p08WaV5vGd9rYCfkrfubtWU9GFWNVKTLUl4aXXV38rh7lU6FnyWEOiLp8e0zc7hC+k4A3wBAAM4B8KDOOX0S0rzM43sN7ATVTetsqlomq0Yq8pDz0tgyi9TGfBI6Iunp7R/BSyEFcDuAZwEcBLAHwFUArgZwde97AnATgJ0AHtXxj7JnQpqXAVWZq595Gtncz7ppnS2tVGWVwCJmf1YaW+gjNRE4leWq6+by8PaP4KWQutp8ElLm7CaRKnMly/HgIHO32xBhLWORJk2YPjo+FJOaqdtVn8/14PHx8aO+2U4n+lwzNprcKpEcH58fa/TViBAh9ZS0zKVjsPlUSxuTddOjo2oRK2qRmqYtS+i7Xb1zqO6h283383poktl+vK46aLhGhNRjkplL1zDzxW9UiCxrLSvQU8RHapKmvCBYngmmc46sdHnqJLSt73n1la95W4Q0IHTF1JdIphPiJa3fzM1r5xX5TRydB59VwrO6SyW3Tie9EvE4bG2rm16R+sqXLoIipB6gmxl0ewd5WWvXnePLmE55PpWs85Tp0lXlMgkeYFpf+eTtECGtGdPMENejbpd5aMiPjJSJDzm+jBDltTWz7qNsAC0+9GdwcO53g4MevuzimNZXPtUtIqQ1UzYz1G3oaeFDji/TNHZpzeZt8d4FyVpzaMjTF14M0/rKJ2+HCGnN+JQZnGH7JovUHnXVWLYsUh8qI8eY1leqDhBikbZQSFtQPuz02o5/72LNCpumfZ7/RddK1Rmh0agaN/01qPYlHytQn7dDhLRmVF0nPehrbY8y4wjj/UiLDuuMnz9NLG36cNN8mQMDc0dNpHWSTBPIeCZoRY07H9WryeqOWwcipDUzOcl8zDHp5SiuH8GjY/HlNYNdTRZiU6R0S3j8eejMJOVDwK4GTD0jdRnoIqQ1otszpvHlJa8XdnxzMX2dzYkNstJuen1fO01WxOSkmYjWaaCLkNZIa0YrqcgaMmlimZataXQnNtC5Tt5LNJlYodNpvFiqyDMyul2/DHQR0hox6RnTsJhC8Y7qcV+pLcvMZGKDvBpNt2JIBrpMh77awlMrN8vI6D8Sn5IuQlojNi1SnzKVFkW6Bbmepk5nYgOdMfVp4eS8l5o1jNRVc0Ql4HkTqFRAlpHhY94WIa0RWz7SIOMQeeZ42jIjVXZlsNk7QFeUTbs4la09dcy+mgitk4IIac3o9pvLIrRMx8z5AuPihkwetkntlPfCdF+QaX/bsrVnXmXW7dbWzHFpHLhovYmQphBaM9lJX20XHdSBo83XtI7q/f26FpxJmtNK5uBg9mQFOs9Ap8TrqsL4uH6Ay0btaepeqdhKdVEOXQm0CGmCEJvJ1i1S2x3UVf4L1fT+RW4oL82uukjoprXIiK0sd4arqeltPhsPcdV6EyFNEGIz2br423wIeQKWNv9mkRvKS7OrLhK2mgOmzzyry5SJCWfaBS3w7iOuRtqKkCYIdUiz1WaQzYdgImBlxr7npblui7Toc8oKNJWZcT/tfPHn7dOMIBYRi7QiIQ3RIrVOlRaprQetuk4/YNIXpfh3eT5SHWw1B7KeU5Z/1mSYqQkh+rg0EB9pRUJa5EHruL9CCl6Vzm3xG1YFlVRbman1dIJJyUkMbLwcW+fIsjDzlovNs2iLpDG4jKuHRO0rEFJmswedpzmmMQRvKJrbVILWbyrGlxO2bfqH3jw19VfGK4SsVkRDrUuf8FJIAVwE4AkAOwBcn/L9ewHsBbC5t71f57wu+pHmtYJV3xM1NB+bRLFdF+7QHN5Fouj955a1frH4q5zjnZAC6ADYCWApgCEAWwAsSxzzXgCfNT23CyHNK6t5EwsFT9FRPGm/tV2zhCYgpv5klWWafJ55mbCRNXq1qIR0APWxCsAOZt7FzL8C8HcALq4xPZmMjGTvV30PAE89ZT89lTI1BaxbB8zMRMVyZgYgSj827UGMjQG7dwOzs9HfsTG76duwARgenrtveDja7yNlMsTMTPQ37XlmZcKZmegdTk0Vv7agpE4hPQnA07HPe3r7kryHiB4hojuI6BTVyYhoHRFNE9H03r17bac1t6xu2GCmLV4yNQUsWQIMDER/+4Vu/XrgwIG5xzLPv+G6xGtsDJiYAEZHozSNjkafbQu2LcpmCJUgpmXSOAcORO9SsE+amVrFBmANgC/EPl8B4K8Sx3QBvKz3/9UAvqVzbldj7fNaqGkj/4KZAT/Ll2lzQuT+tRoYJdZG9axNm/iqc5u6XgRt4KGP9FwA98Q+3wDghozjOwD265y7zklL4vm4yHzBtZHlZzTxQRapbbx+MI5Ie04mvtM8QSzSX1XIxUchXQBgF4BTcTTYtDxxzOLY/5cAeEDn3D7M/hRa/CMzmqYbec87bnw8u3C3HZNoft7zyjtXGysvC3gnpFGasBrAdkTR+/W9fR8F8Fu9//8CwOM9kf02gN/QOa8PQhpaj5zMMd26Hdrz+jhmuQi8fTAVk9WkMRXBPCtXKi9jVEJK0XfNYuXKlTw9PV1rGpYsORpgjTM6GgVavaMfmU8GlYAogKETvBkYiIpoEbx9MDWhykCdDnDbbWaBNNV7IYoi/4I2RLSJmVcm99cZtW80ofXIORL57nTmf6cb7S0ajSby+MHUhKqL1OyseW+EvL57gaPqbFIpaWZq6JuNpr0vQ7OdYdqpW6fpnRZI0tnKjqP1+kEXxKaTvcFDR6u+NfjoI3W1lRXSBue7iKwbLFqAiwx7tDEZQVNfls0JZUZHo+fctMqGqw/qipAaEFzE3ZQiE1/kFcQi0+jZKMxNfllplraO9d3UyiWFqoO6IqQGBBdxN8V0KrasiTLyzpnc2j5hSRl0BTJrztaGIRapx0LaZCOHmc1vUOf4rMIrE5bYQfdebS4W77n/WXykHgtp41tGpjeoY/XV9dAa/7Ji6FrfWXOdmlQwgTzbKrVehDSFrBfgeUVcHpMb1B1uWNdDa/zL6qFrkZZZ6rrI9VqECGkCk1GPbSij84jf+KJF6oLpqZXSSHQzra3JcT3zP/tQFkVIE+hUtoG0bOxTpCtTi62USikzVNd0uQaPLFJfyqIIaQKdytajfFQtRWZwb2KUPFTSVKdIn12PZurypSyqhLS1Q0R1Rs2pRukFP+N9HkVusCHDDRtB2kTXX/oS8LnPRd/3x1QSAQsWRH+TYyunpqIx/cxH9xEBa9dWMmF2cthn2rQDQLS/tmGhcdLUNfTNlo9Utxb0wXdjFVOLtBX+joaQ5baJv8caTUCVQe1DFoQ07eeTJ4BZYhvsBM46TE5GSyurcm1y7fWGDj9sJHmVZF8oaww0Zbl4s5LuussyswhpYdIG+egsSR60H3VyknloSH1jyX5iPkQBBD3y1KgvlDVapHmdDupsKImQWsAkmB1c7CVeY3Q62SIar1lUtUrQNUmD0bVIa6wg8zTcRExtZ0MR0hLkTTQevI5kWaBp1Xwja5KWoOsj7R9bg8smT8PrNGhESAtSpEtlcC1bHV+F6RZUTdIy4pZBv/XhmW9bJ35RR8NIJaSy1EgOWV0v4hBFr250NJrs3dcl1VNJrk9fFt2lSQTBEmkr5bjIhrLUSEF0ulR2u1E3PeZo2aHG6seARnYZHRURdY2ttTW8WKPDDmldZyvNhmlmauibzaa97nwd3qLj59Jp2g8NMQ8MZB8jflH32JokQnpbFAJFfKQAjgNwWsr+/5j1u7o31z7SYPKbSaFL9hsdGJjbV7Txfb4CQadmtznaJDBcx8eMhRTAZQCeAbAZ0dryZ8e+e1j1O5MNwEUAngCwA8D1Kd+/DMCXe99/D8ASnfO6itoH19/cpLDkLWvRuAhboOgM8dGJvHg2s5MNqjB6igjpZgCLe/+vAvADAL/d+/x91e90NwAdADsBLAUwBGALgGWJY64B8Ne9/y8H8GWdc7vqRxocZQqLSXeFoGqXwCkyoUzae2+gRVrFLamENCt60GHmZ3t+1AcB/CcA64nojwCwuTd2HqsA7GDmXcz8KwB/B+DixDEXA7it9/8dAC4gsh1ibjBl1jNfv35uCDSN4WFgcrLhETbP2LAheu5F6L/3qSngF7+Y//3wcHT+QKlzkqEsIX2eiE7rf+iJ6vmIxG25hWufBODp2Oc9vX2pxzDzIQD7AXQtXNsIneCmlwHQtEKnKiy60+0ANYVFBQBzw9MqhobU773fT2jfvrnfd7vBv88ydkNp0szUyILFCgCvw/zm9iCAK1S/090ArAHwhdjnKwD8VeKYxwGcHPu8E0BXcb51AKYBTI+MjFgz5XX8Ll4HpHQcvCbT7QTc9GscWQHA/kQywNxO9w0e0uulj/TIAcBjAP4YAAF4OYC/AvDdvN9pnPdcAPfEPt8A4IbEMfcAOLf3/wIAzwHRIIKsrYruT/F8F7y7SXe6HW9qB4GZ82f3aPTkEOnUFbXX6ZD/JgCnAPgOgIcQRfLfbGj4pvEQgNcR0alENIQomHRn4pg7Aazt/X8pgG/1bqYydPwuQU8APTWlbsYz19jDWUgl7oLJGiDx1FN6fu4+J5xgJXl1MzYWuexnZ6t13S/QOOYggBcRWaMLATzJzLNlL8zMh4joWkRWZwfALcz8OBF9FJHq3wngbwB8iYh2APgpIrGtlJGRdJ2J+110jvGSvr9MxeholBsFP0iOgzx8WH3syEggNXlDSDNT4xuibkkfReQbfQ2AfwRwR97v6tyq7JA/OZm+yGYQreCsrjRB3EDL0O361H93Jl2lGtK0dw1KNO2vYuY/YeaDzPwjZr64J6atIGsM79QUcOWVwAsvzP9dRUvblCPLYslrxnvZTaHhZL2vZAYF0rs4qXoPet988pw0dQ19c9khX2f+YyAKjnpP0UWpxsc97qbQYEzeV1qQqdtlvuCC1gYRbQSiIPORlsd0blLvKdq3S7drVLBjaz1Ft3+PSnC7XTvLNAeIra5RIqQWMB2dFwR5YlfUz+Z159qA0amcdOZGyKoAA8C0jrbVRVGE1AIm+dOkae+14WZy043qXBswpjV+YIGmInW0rTlaVEIqEzsbYOqP14m/9Hu0zMxEr3ZmJvpca+xGp69iMmiRHHoadOfawFENDe4qRlcHFmhK6x574EC0X4Xz4aNp6hr6VqWPdHhYvUSzTkvWO8NNxxHcv+kiLgGxSKtBNS1iA9wtRaxL8ZF6JKTM6mZ4Ud3wzsequpFOx8z3kBU5Dqzgeo2pX8hrP5IeRcuaRO09ElIVRX0wqi5UnU416Z5H3vhtUzEtaqoL85FuaMxcr2EtQpqBjZqq8RZp0b6H0sS3Q9pyMFkVXsNRlVnvlhoJeTMRUlu1W9HzeKczpjPj59HAJS1qQWfNrJY/W6+n0QtxMxFSm0JWpDb00v8fH6ed1dTXKbDe1RSBoiui8WfbAH+oCXUuNVK76LnYTITUB4PJ2/yu0x8xL8Fe1hQBYmKNjo+nP/fBwbkrwzbsHVRRlkVIFYjBlIFuZ/w8YfS2pggIVdN+4cJ0/7WOK6BhFZqvi9+1Ap1ljVo70ZFub+W83tB1zbbbJG68MVqLKc7QELBoUaQXcQ4cmL8mUxp57y0wssqy8zKcpq6hbzaj9q1umZrO0iLWplvSMqrpuPo6fVgV4HocAqRpX4zWN/2TOTNtFutW1jKWKer+MJnpqWUZOW9u6yK3LkJaEB+CUV5gMuN6gwunE8qYTFm/jYtzt8s8NNSaSk+nMVWkDKuEtPU+0jxqXSvbF+Izq+ggE5OYUWQWjj5ZSzjEOeYY4KqrWrOYoc66f1bLcJq6hr5VuWZTKzCdls3EIpWIvptmT8szbp7rWHykFQsps5R1o4CGSQ5teWE/gqkjXidDtty5n+cbLZrFREiF4uRZpAMDxWqZlhf2I5hUKLrHtty576qOFiEVijM5mS2kRXNnlYXd92aFbvp0Kx+ppJy8cq+EFMAJAL4J4Ie9v8crjjsMYHNvu1P3/CKkJVDlPtVImTLLpeoW9rIloklzo+pWPuI2cYJvQvqXAK7v/X89gI8rjvtFkfOLkBYkrytN1gqUNmdsic++f8wx80XDVBCyXBMuxMWl9auq0NIsTd+t8ADxTUifALC49/9iAE8ojhMhrZI8C3F8PH1cd5kJhnUmKy7bRM0Lltls7rq0BCcn5/cFBaLJSEQkK8E3If33xOefKY47BGAawAMA3q17/qJC2voKPK/ZqDvpc5lmv25XKxM/at45bfpkXfoms0YxBYpOmcsbwl1lma1cSAHcC+CxlO1iAyF9be/vUgC7AZyWcb11PdGdHhkZMX5A4lLifBEoMq7b9AHqXsN0vessK9emReoygNawSLxOmTP1Nrkus75ZpFpN+8RvbgVwqc75i1ikEuTk/FyrWmDKZrNZ1yIdGjIrMZOT1awf5XKmcBP/aADoPCrVMVlZ0eXj8E1IP5EINv1lyjHHA3hZ7/8TexH+ZTrnLyKkDavsi6M7fY7uZvoATa5VZCZ4121BW2aSamLmBo2X1ylzRRpBLsusb0LaBXBfTxzvA3BCb/9KAF/o/f+bAB4FsKX39yrd84tFahkdK3FgwN4DTE62kVVifPTJlHX8MWf7QxviyC9jkerUry7wSkhdb0WE1Mfy6A06A5ddLg2cVeJ8qgF1rV2dzGbSX7SIsHoQWS3qI83Liq3xkbreXEXtPch79aA7cFn1gFx0qO+XGF98MiY1cRlTLH5M0drfI6vB1HjP8402Lmpf5+aiH2nZvBe0CJe5eZs+w7QH6ItFapIOHfHXeW5F792XZ1aAuusAEdKSlMl7db98KxStCYo+OJvN5CowsYx1rc24f7h//v6ghaxWQp417osVX5A6jRIR0pKUyXsBGwDlKfLgdJ1n8YBU3csMm7zkvPsr00uiwRapD606EdKSlAmiBmkA2Mq1WR0BTSPW8e5OJtPOxS07VxOVmFrGWc+3SKjaxBr3xYo3wJcki5CWJO1FDg1FXfvyXm5wBoDNXKtjXZlGrHWHSk5Ouhubrupva6PyKboyaD8So3NtH8w7A7LKUJW3IkJqgaIDTXypTbWxrfw6YVeTiHWW0OhadmVqMdsvNJmx8lZq1dkaNpFJ1is3fRVlhFeE1AEmTfagDIC6x4vnCZWuQGaVvjL3ouvniU8HmNW3p4w/NGsLeDKTJFkeIpN6smwdKELqgOCa7Lq4vGgGNwEAABfPSURBVDHdc2fVPJOTegLpyiIt2vQ28ftkpTsp0FnHNwSVAJreetmsLULqANdN9tqsWJd9P1Vzmub1xE4+gLQJn5MlIstHOj4+1zfTH+Kq86Czhq3qCGH8vkx/m0YLhJQ5PTuohLE/ejhJ2caWCKkjXIld7X5VF6OR0qJz8Vn2834fn4kqeZ7++ZPpTIvaj4+nC2x8y4rulxFSHVNK9RvT9DSoaa9iclItjmn1jlikngqpK4J3G5g0WU1zvG7UvmzaVOJVtGkPFJ+KMC96kqwYTKcZDBjVY1PFKsRHqrk1QUiD7Hsax1Rskjk56wHoRu1tpM1E5HUmdzEVUd0XHlQ0cy5lk25qdEjUvkVC2iqLNM0sKGKR6poXpmlLojJrkkGgtKh9keBS4OTFDcu6sKp0g4mQBkBy1KNOZ39v0fWRqkQjz0daZukQVRBKZRGaBsJMn0uW9R3MC0+naE820/qjKoNchNRzVLpT9xDyUqhG/+g2Y210gcpKW9qkIFVYhcn7SpvLVRWEC4yiYyt8dWGJkFZIkdpRt493cGKahi0zxLb/I6+J7fIlBOzjzKLoaF9fPRoipBVR1F+jG/8IqnmvwmY/VZvOMd0gUiNeQjXkCWXt3fwMESGtiKI1bNneQsFhYoHlNfFtWXIm/stGvAT36AhlVa/XBiKkFVHU52My5NpX/5ETqjBZklG+uGNaXkJpbMbl6rZWRUgroozPJ5nhGraMeTFch3WrCCv7ZlYFgo/+UxHSirBZi/pYI1eOjbBuVr/PvOmDyr4EeYmF8TGiL0JaIbbddq02ZmyYJUVGIiWn9Sv6Enw0qwLBx0fnlZACWAPgcQCzAFZmHHcRgCcA7ABwve756xZSwSI2LDodx7Or0uqjWRUIPhrzKiEdQD08BuC3AWxUHUBEHQA3AXgHgGUAfpeIllWTPMEbxsaAiQlgdBQgiv5OTET7del0zK5JBGzYYPYbFSMjZvuFI9h49VVBkcjWdHGi+wH8N2aeTvnuXAAfYeb/0vt8AwAw81/knXflypU8PT3vlEJbITL/ja1yMTUFrFsHHDhwdN/wsL+KIGRCRJuYeWVyf10WqQ4nAXg69nlPb58gmDE66ub4qSlgyRJgYCD6OzU1/5iQzCqhMM6ElIjuJaLHUraLdU+Rsk9pJhDROiKaJqLpvXv3Fkt0RSTL3zXX5JdHIYGOiPXZsCGyAuMMDwMXXDDfWh0e1mvW9y3NmZnIep2ZiT6rxHT3bmB2NvorIpqJyav1hjTHaVUbgPuhCDYBOBfAPbHPNwC4Qee8PgebdDre1+1Q954iUYisfqRFIvI+hpQbgI8BpjjwKWp/5OLZQroAwC4ApwIYArAFwHKd8/ospLpDQaU8ZuCDiEk03gjd+sqHV5uFSkhr8ZES0SVEtAeR1fl1Irqnt/+1RHQXADDzIQDXArgHwDYA/4eZH68jvTZ56im7x7US1cOp8qFJNP4IeU1xEy+ID6+2EGnqGvomFmnD8cFs8b0NWhE6j8HkdfnwarOATxZpm0mLeyTRjXcAgTrmy6IKHtnq+6mDROMxNQWsXTu3ZxcQfV6//uhnEyvTh1dbiDR1DX3z2SJlTp8g3ebsOEXPFxStHztbL3lB07irWGdOUhvloQrgY7DJ1ea7kMZxMYxb5iG2jK25Uz3EdqeFtKa46fJbPudXEVIPKZuJyq4qLGhg8pICU4Uyyc3Ke2nn6As2cHTCrRCnihQh9ZCyjnWTWfWlV05BmhQpSVAmuarfdjpqIW7C5OUqIZVgU42U7eqR5phXDSvP6pXTyoCVLiYvKbC+O2WSqwoK3XabOt62fv38wJSK0HqRiZDWSNmuiGmB46uvNot6mvTxayUmLymwvqVlkluk04JufRJElD5Jmpka+hZK096VS80kgBBYa7R6xEdqDVVeC2nJcYiP1E/qDvLKSEcNCkbtn++O8ge7k14LRJX5z7Zw11F2REiFVLIs0rpF3jsMHkhgxmklTE7OjdJ3u+VEtI7nK0IqpJLVqV+EIIZhyW2Ty0SnfrEtfHU9XxFSQUlaQTAdjdJ4gTUsubZdJr4+b12BtC18dbmkREgFI7IyaiubrYYl16Zw+Py8dSvctGPKCJ9YpCKkQZCVUdvUbD2C4U3bFD+fn7dKIAG9DvhF70F8pCKkQZCVUVsZ6S9Qcm01x31+3v3hnsmt08kfeVdW+CRqL0IaBKqM6rOF5JSaHJVVPm/TW8wSyqzx+D75eU0QIW0hrsq9zWaVr0EUn6iqGVvkOm1zAYmQtgzXhc+GAPocRPGNIs/b9DdFhK9JU+TpIELaMmxYA66txSZaLL5gKmKTk+nvQscXm5VPmtbiECFtGWUDFFVYE0XTWEXhDF0ATCqpvOi6VGxHESFtGa7mOrVZqGw3JW3RhCapSSWVFV0P7b5dI0LaYNKsp7JioCpYgN102wxu2KIJLgeTe8iKrouIzkWEtKHkOfuLNk+z+gfaTr9JGqvoU+lzv01d0vJF/76Sz7kJFUdVeCWkANYAeBzALICVGcftBvAogM2qG0jbmiKkOiLjohBkBR5sWqRFEItUn/jwzKwFEZvgyqgK34T0dAC/DuB+DSE90fT8TRBS3cztYnIM3wIP8Qql22UeHHRb6JsmLDoVQ+jBtarwSkiPXFyEVImuVWTbevIt8JAmakNDkaCW7cOaJRxNEZYy3ZqE+YQqpE8CeBjAJgDrcs61DsA0gOmRkRHrD7BqdC1N26OMspr0dYiJK9dFkyxOFXmti2637hSGR+VCCuBeAI+lbBfHjskT0tf2/r4KwBYAb9W5dpssUmZ3o4zqbtIzuwn8hOYDLfp+VWvGxy37plUerlEJ6QLj1fI0Yea3WzjHM72/PyGirwJYBWBj2fOGwIYN0Wqe8eVrVasrjo1lr96oQ9ZSuXWu6jgyEq1smra/KCGtmtxf5bX/bvqrvALZ73xqCti3L/vcixYdxIEDe7Bt20t2EtsgFi5ciJNPPhmDg4N6P0hT16o2ZFikABYBODb2/3cAXKRz3iZYpMzV+ul87UvoohkekkVaNK15U9gBzDfeuIu/+929PDs7W8GdhMPs7Czv3buXd+3aNe87KCzSWta1J6JLiGgPgHMBfJ2I7untfy0R3dU77NUA/oWItgB4EMDXmfnuOtJbF2NjwO7dwOxs9Les1ZmFysIbHXV73ThTU8CSJcDAQPR3amru+ukA0OlE1tn69dH3RdiwIbKy4/i6lnpR61nHuv61X3sJL395F0RknrAGQ0Todrt46SUDSz1NXUPfmmKRVomu5VfX1Hy2LVMfovIu+wnrWKR3372Vn3vO+m01hq1bt87bBx+j9q62tgipTTGYnMxfKtdltDtPMEJqjutgUnEVeeZpvxscnNtt7KGH5gtF1Tz77LP8O7/zO7x06VI+/fTT+R3veAc/8cQT/OSTT/Ly5cudXPOll17iyy67jE877TRetWoVP/nkk6nHiZC2QEhtd3tysRKkidBn+WjzvrNNFdZqFb0y8n6XJhRlzmfK7Owsn3POOXzzzTcf2ff973+fN27c6FRIb7rpJv7DP/xDZma+/fbb+bLLLks9ToS0BUJq00LTPZdJVyRToddpilZhkVbVx9SH8fwmQuriudx33338lre8JfW7uJA++eSTfN555/GZZ57JZ555Jv/rv/4rMzM/88wz/Ja3vIVXrFjBy5cv540bN/KhQ4d47dq1vHz5cj7jjDP4U5/61LxzX3jhhfyd73yHmZkPHjzI3W43NeAmQtoCIbVZEHXPZSLeRazXvBUnk5sLgSs6tV98CKvOqCsfXBUmQuoivTfeeCNfd911qd/FhfSFF17gF198kZmZt2/fzv3y/clPfpI/9rGPMTPzoUOH+Oc//zlPT0/z29/+9iPn+dnPfjbv3MuXL+enn376yOelS5fy3r175x1nIqS1RO2F8qii7EX6V+qeyyTabRptjkfndYPIL3+53nEmpPVZBdTp7vfznJmJpGXfvmhjPtrnM613gc6zTOvFUBd19r09ePAgPvCBD+D1r3891qxZg61btwIAzj77bPzt3/4tPvKRj+DRRx/Fsccei6VLl2LXrl344Ac/iLvvvhvHHXfcvPNFejiX0j0X0tQ19K0NFmkdPtL+sTp+MlcTS2fNYlSWyUm1da5Kt45LoogPuYpeFHVbpPfee69W0/5P//RP+cMf/jAfPnyYDx48yJ3YXI7/9m//xhMTE3zGGWfwbbfdxszMzz//PN9xxx38rne9i9/3vvfNO7c07UVI52A7am8zkFBW6E2a+mWbw/Hp5lTirUp3ViCsjLtFd8amMs+4bh/p7Owsr1q1iicmJo7se/DBB/n++++fI6TXXXcdf/KTn2Rm5ltuuYUj+4959+7dfPDgQWZm/vSnP80f+tCHeO/evbx//35mjgJXK1asmHfdz372s3OCTWvWrElNnwhpS4TUd8qKc7JLlk2hil9DR7BVlLFIs9AZaVbWSqw7as8cWZRr1qzhpUuX8rJly3j16tW8ffv2OUK6fft2fv3rX89vetOb+Prrr+dFixYxM/Ott97Ky5cv5ze84Q183nnn8a5du3jz5s185pln8ooVK3jFihV81113zbvmiy++yJdeeimfdtppfPbZZ/POnTtT0yZCKkLaGFwJla3z5wlx0YlBdKYzLBtwNBXStiHBJqEWXARH8oIZZYd2lj1/cghrEuZi6UoLRvXpD5G1GXAUSpKmrqFvYpFWj6v+l1mWmY3mpc3z2w7I5E3KXKWPtI1I016EtHJc9YvMEgtX87DmiZHqunXMnVpV1L6NiJCKkFaOy5E6aWJhu/uXrhhlXVcVGPN1Nn8R0mxESEVIK6fqkTp1jQxSXbfbjQJLyf2Dg3a6krkY+y9Cmo0Em4TKqXqOz7pG2qjOv28f8Ktfzd9/3HHl53Otcl5aoRgipIIVkkM8R0ejz64KfV0Ra9Pz//SnbtLRFH70ox/h8ssvx2mnnYZly5Zh9erV2L59O3bv3o0zzjjDyTU3btyIs846CwsWLMAdd9xh5ZwipII1qrSc6prlXnXdbjf9+KLC7tM4+yNYThQz45JLLsH555+PnTt3YuvWrfjzP/9z/PjHP7aSXBUjIyO49dZb8Xu/93v2TprW3g99Ex9pO6hrlvsqgl9VTOVn5CN1kKi6ptHrs3btWv7KV76i/F6CTSKktePDUh5VY+ueqwqkGQmpg0TVNY1eH5tC6mw5ZqG9FF1COHRsLIsNeLpcdI2JOnjwIK699lps3rwZnU4H27dvBxBNo3fllVfi4MGDePe73403vOENc6bRe+c734kLL7zQefoA8ZEKDli//qiI9ukPaxTy8XLop4NELV++HJs2bco97tOf/jRe/epXY8uWLZiensavet0j3vrWt2Ljxo046aSTcMUVV+CLX/wijj/+eGzZsgXnn38+brrpJrz//e8vnD4TREgF63hpUQWEl8tFO0jU2972Nvzyl7/E5z//+SP7HnroIfzzP//znOP279+PxYsXY2BgAF/60pdw+PBhAMDMzAxe9apX4QMf+ACuuuoqPPzww3juuecwOzuL97znPfizP/szPPzww4XTZ0Rae9/1BuATAH4A4BEAXwXwCsVxFwF4AsAOANfrnl98pPXiwzIaoVOFj9m4Q76DRNUxjd6DDz7IJ510Eg8PD/MJJ5zAy5YtS02biY+Uou+qhYguBPAtZj5ERB/vCfofJ47pANgO4D8D2APgIQC/y8xb886/cuVKnp6etp9wQYukjxSIjBeX/UoFc7Zt24bTTz+97mR4S9rzIaJNzLwyeWwtTXtm/idmPtT7+ACAk1MOWwVgBzPvYuZfAfg7ABdXlUahOFV3zheEuvEhan8lgC+n7D8JwNOxz3sAvKmSFAmlsRXBFoQQcCakRHQvgNekfLWemf+xd8x6AIcApA2RSFvWT+mHIKJ1ANYB0cgFQRCEqnAmpMz89qzviWgtgHcBuIDTHbV7AJwS+3wygGcyrjcBYAKIfKTGCRaEFsLM5ZcibiCmsaNafKREdBGAPwbwW8x8QHHYQwBeR0SnEtEQgMsB3FlVGgWh6SxcuBD79u0zFo2mw8zYt28fFi5cqP2bunyknwXwMgDf7NWGDzDz1UT0WgBfYObVvYj+tQDuAdABcAszP15TegWhcZx88snYs2cP9u7dW3dSvGPhwoU4+eS0GHg6tQgpM/+aYv8zAFbHPt8F4K6q0iUIbWJwcBCnnnpq3cloBDKySRAEoSQipIIgCCURIRUEQShJLUNEXUNEewHMGPzkRADPOUqOXN/v67f53tt+/SLXHmXmVyZ3NlJITSGi6bTxs3L95l+/zffe9uvbvLY07QVBEEoiQioIglASEdKICbl+a6/f5ntv+/WtXVt8pIIgCCURi1QQBKEkrRRSIvoEEf2AiB4hoq8S0SsUx11ERE8Q0Q4iut7i9dcQ0eNENEtEyqghEe0mokeJaDMRWZvy3+D6ru7/BCL6JhH9sPf3eMVxh3v3vpmISk1Yk3cvRPQyIvpy7/vvEdGSMtcrcP33EtHe2P1aW7WNiG4hop8Q0WOK74mIPtNL2yNEdJata2te/3wi2h+79z+xeO1TiOjbRLStl+c/lHJM+ftPW3+k6RuACwEs6P3/cQAfTzmmA2AngKUAhgBsAbDM0vVPB/DrAO4HsDLjuN0ATnRw/7nXd3z/f4neGlwArk97/r3vfmHpern3AuAaAH/d+/9yAF+2+Lx1rv9eAJ+1/a57534rgLMAPKb4fjWAbyCaA/gcAN+r+PrnA/iao3tfDOCs3v/HIlq+KPnsS99/Ky1SrnmpE2bexsxP2DiXw+u7XOrlYgC39f6/DcC7LZ1Xhc69xNN0B4ALyN5EnbUum8PMGwH8NOOQiwF8kSMeAPAKIlpc4fWdwczPMvPDvf+fB7AN0eobcUrffyuFNMGViGqjJGlLnSRfgGsYwD8R0abeCgBV4vL+X83MzwJRRgfwKsVxC4lomogeIKIyYqtzL0eO6VWy+wF0S1zT9PoA8J5e0/IOIjol5XtX+JDXzyWiLUT0DSJa7uICPXfNmQC+l/iq9P37sGaTE6pe6qTI9TV4MzM/Q0SvQjR36w96tXsV13d2/7rnADDSu/+lAL5FRI8y806D3x9JTsq+5L2Uul8L1/9/AG5n5l8S0dWIrOO3Wbp+Hi7vXYeHEQ29/AURrQbwfwG8zuYFiOgYAH8P4Dpm/nny65SfGN1/Y4WUK17qxPT6mud4pvf3J0T0VURNRC0htXB9Z/dPRD8mosXM/GyvCfUTxTn697+LiO5HZE0UEVKde+kfs4eIFgD4D7DXHM29PjPvi338PCLffVWUetdliQsbM99FRJ8johOZ2coYfCIaRCSiU8z8DymHlL7/VjbtKYClTohoEREd2/8fUYAsNerpCJf3fyeAtb3/1wKYZyET0fFE9LLe/ycCeDOArQWvp3Mv8TRdCuBbigrWyfUTPrnfQuTLq4o7AfxBL3p9DoD9fddLFRDRa/r+aCJahUiX9mX/SvvcBOBvAGxj5k8pDit//y4iZb5vAHYg8ols7m39aO1rAdwVO241oijfTkRNYlvXvwRRLfhLAD8GcE/y+ogivFt62+NVX9/x/XcB3Afgh72/J/T2r0S01AwA/CaAR3v3/yiAq0pec969APgoosoUABYC+EovbzwIYKnlPJd3/b/ovectAL4N4DcsXvt2AM8CONh771cBuBrA1b3vCcBNvbQ9ioyeJI6uf23s3h8A8JsWr30eomb6I7Hyvtr2/cvIJkEQhJK0smkvCIJgExFSQRCEkoiQCoIglESEVBAEoSQipIIgCCURIRVaDRHdTUT/TkRfqzstQriIkApt5xMArqg7EULYiJAKrYCIzu5NCLKwN2rscSI6g5nvA/B83ekTwqaxY+0FIQ4zP0TR5NAfA/ByAJPMXOWQW6HBiJAKbeKjiMa9vwTgj2pOi9AgpGkvtIkTAByDaKb0hTWnRWgQIqRCm5gA8D8QzT9b5TR1QsORpr3QCojoDwAcYub/TUQdAN8horcB+J8AfgPAMUS0B9EsU/fUmVYhPGT2J0EQhJJI014QBKEkIqSCIAglESEVBEEoiQipIAhCSURIBUEQSiJCKgiCUBIRUkEQhJKIkAqCIJTk/wN80LX1+xyIpwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "data_lin_sep_svm = pd.read_csv(\"./data/neural_networks/circles.csv\")\n",
    "X = data_lin_sep_svm.iloc[:,:2]\n",
    "y = data_lin_sep_svm[\"y\"]\n",
    "# This time we don't create a separate validation set but we will exploit the \"validation_split\" argument \n",
    "# of the fit method in Keras\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, test_size=0.2, random_state=1)\n",
    "sc = StandardScaler()\n",
    "X_train_scaled = sc.fit_transform(X_train)\n",
    "X_test_scaled = sc.transform(X_test)\n",
    "\n",
    "plt.figure(figsize=(5,5))\n",
    "plt.plot(X_train_scaled[y_train==0,0], X_train_scaled[y_train==0,1], \"bo\", label=\"Class 0\")\n",
    "plt.plot(X_train_scaled[y_train==1,0], X_train_scaled[y_train==1,1], \"ro\", label=\"Class 1\")\n",
    "plt.xlabel(\"x1\")\n",
    "plt.ylabel(\"x2\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Setting Up the network**. Let's review the ingredients of a Neural Network for binary classification: \n",
    "* a single output layer \n",
    "* cross-entropy for binary classification\n",
    "* sigmoid activation function for the output layer\n",
    "\n",
    "Since the classification taks is highly non-linear in this case, we use as an example a network with one hidden layers containing 15 neurons (and ReLU activation function). Let's set the network: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras.models.Sequential()\n",
    "model.add(keras.layers.Dense(15, activation=\"relu\", input_shape=X_train_scaled.shape[1:]))\n",
    "model.add(keras.layers.Dense(1, activation=\"sigmoid\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To compile the model, we use the accuracy as performance metric, and `binary_crossentropy` as loss function: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=\"binary_crossentropy\", \n",
    "              optimizer=keras.optimizers.Adam(lr=0.01, beta_1=0.9, beta_2=0.999), \n",
    "              metrics=[\"accuracy\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now we are ready to train the model: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# We use 10% of the train set as a validation set: \n",
    "history = model.fit(X_train_scaled, y_train, epochs=1000, batch_size=32,\n",
    "                    validation_split=0.1, verbose=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAE9CAYAAAAbGFuyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gc1fXw8e/Z1aoXSy5ykSvY2LhjY8AktsCEEnoCAULAmGACofyAF0IvAQIJENJDcCimgykOBAwEg4UpBnfj3rEtN/Uurbbc94/ZVV1pV5ZW65XO53n0aGd25s7Zu+XMvXNnRowxKKWUUir62CIdgFJKKaUOjSZxpZRSKkppEldKKaWilCZxpZRSKkppEldKKaWilCZxpZRSKkrFRDqAturVq5cZMmRIpMPo0iorK0lKSop0GF2e1nP4aR2Hn9Zx+K1YsaLAGNM70HNRl8SHDBnC8uXLIx1Gl5aTk0N2dnakw+jytJ7DT+s4/LSOw09EdrX0nHanK6WUUlFKk7hSSikVpTSJK6WUUlFKk7hSSikVpTSJK6WUUlFKk7hSSikVpTSJK6WUUlEqbElcRJ4TkTwRWdfC8yIifxWRbSLynYgcE65YlFJKqa4onC3xucDprTx/BjDc93c18FQYY1FKKaW6nLBdsc0Ys1hEhrSyyLnAi8YYA3wjIj1EpJ8xZn8btkHl4sXYUlNxbt3azojr2VNS8GxfBr1HggCV+VCwFXqNgKReULAFMoZBxUGITYH41EDBQf5mq4yiHZA2AGLirOdqSq0y+o6H8n1QfgD6jgNHgjW/91F1Zdir9+CpBRIyEJsXwYu3shxsdjBeSMuyli3+HpIzwe6Afaut/55aEDv0GAiuakjoCbGJkL8J0odAwTbEYUfS+uOtKIGKPPB66J1fRPHWVVC2D/qMgsJt4K7BlpiASTsSU7wbnBVW3RgDIpDSDxyJYI+Fwm3YE2PxJA6Bg+shtb+1jNcNznKIibdea/kBcDux98jAkzgI8jaC8UBssrWsuwa8Hus1pPbH5oiB8v14cYDYrHKqikgY3p+abbmYpEwozQXEqpfSXLDHgMdl1ZHNYdWZVblWGV53kzffYS0vduv5FF+Zthjr/XNVW2X4l6sr01h1EZsICRlQuse3zRgkxm69/FqXFZvYwOsiM6+A4mX/sT5vFRXW+52Sgqe8vEFAYm3b6wr9A2xzWMv7Y/T/ryvSZpVrPM2nxQ4Y7EkJeCqr8b3JDeqtDXxlNSrD5kDsBrGJVfUeFxIbh8Ql4k0ZAvu/q4+rYVGOGMRmx1vrbvx809fWkN1B3325FC/7T5A4BXtSIp6KKl+8zSUfNxGnKxNXYVnL5TgroLrY+i7sX9VyXG1hi6HuvTmU9wCav98drO/Bg8HruLX3qQW2+DiMx2Bcte2IrkkMXl8dGs8hxdQm/u9hqA7xfYrkZVcHAHsaTOf65jVL4iJyNVZrnczMTHJycgCIW76CHs88E/ZA633cweW93+Dxhx1c9qGzAQdY6pt6v7VFVTsdiHQA3UBH1LH9pQV4au0dUFLXpJ/jyIlkEpcA8wLuBhtj5gBzACZPnmyys7PxlJWx5Zpr65bp/4ffk3j88e0Oyp1fQNHDN9Cjx1piz/0NHHM5vHwBHFwH/SbC1Bvg7Sshawrk+hLd/9vUvKCFv4U1r8GEX8DqlyGpD1yz2Hruratg15cw7Vb44klrD3v8Jdae/Kb/wumPwujzqX3tNkoWLCbjqApiEjyU7kzE6xbSh1fWb+fm9dbe/79+YE3PuB8+/W3LL3DSLFjxfN1k6a5EPE4bGSMqAi+f0tdqMRshf10Kib1rSepb03odVtsp2pxM+vhEHNJ6x4rxWOWmDKghoVfre9wl262bLPQ4ov71O0scFG5KJm1wNUn9aqz3pTIfinfWr3jkKdDzSPj2X80LPfleOHKG9XjZs7DqJepajnWaTA+dDjs/D7BcAwOPg2HT4fPHKNuVgLvGTsZRzevY67ZRsC6Z1JHJEJ9E2eo8el12PrYTrrAWeO50q0di2q0w8qxW6weA+ddYPS1NY5s0CybNhL0r4YNbrHlX58CBdfDe9db0z9+EVy+kpthB2a5Eeo0ux+bwtVx+9pLVoxOqnV/AJ/c2njf+YljzOqV70vA4+pLRdzOMv5iy/76Lq8pOz5EVVmvkqs+a/TqUvvEy3uWvWZ/9Kz+yemEW/Mb6Dk68DI79ZeMVlvwd1r5FcY9xpP/sr62GWr1gLuXvv0PvsRXItZ81e77mnzPJXeAkroeLgfO/AFsLRyH/ONL6n30n5DwKl/0HEnq0uu1WbV0Iix6unz7vKatnrK3mZFv/f/QQDP3hocfTguUrVjJ5UitDmj66A3Z/AxMuhSmzQyvUGApuOp/4jBqSL7o+tM9+axY/Dps+qJ8++nzYMB+OmAEz7m15vUO17h34+q9W7+jP3whtHf/7dMpvrd+OhoZPaHG1SCbxXKDhr0IWsC/klX99HQBJP/gBydnZpJ51FmJv/56yIzOTAZcdA8tWQHoyZGaCvRQSvRBTBulJ1uO4Gus/WMs0lRZvPR/vtP4nNVguGWueFEOCrzs3thri46z5GamQmYnjyJ4kTS2uK7LX6ACJNi0e7Pb6WBJc9Y8DSU9s9HyvUS0kbz/PPki0HvY/rqT1ZX0ciV4GTC0GioMuCzDghNDK7T22vNk8R6KT5P7O+hn9s6DMBs7t9fOyhkDmSFgboF6OHAdHjrce542AzV5I7AVVBfXLZBwBRQ3K69sPDnqtQypFOwIH2z8Lho2BZV56jqoMvAxAxlAGpG6HjAyITyIlsRhGja2PKdkGtV6rLP+81qTFQaW3ecxDjrLWTwIW+erhyPGQGlf/eRh9IiR6cSQ6SRngbFzuqOOtQ0mhspfBV03qe8Rk2PoqvcZ5YNAg2LoRho6m58hX65dJTIfhzV9nr+vS4c9zrImjpliHZ3qmQpEXBo9oXjd7h8N2L56MDBxB6s1x6imkFj1ndV0HWNZxZAIDp+0jLs2Fo1+/lgvy16PJsx4fPdXqsj1U3oPwbYM6POpY6HlE28vxxzV8AgwJ4TPURs7c4tbruGcPKGjhfWpFv6k14KqCYWPbtF5AO4bC7gZ1OWgYfO+FrMHtLzuQyk2w2gspgT9TASU2+F4OCz2mSJ5i9h5wuW+U+vFAaajHw71OJ1W+O5ll/eXPZPzi0g5J4M035Euw1b4kU11ktXqh/n8wFQebz/Ov2/BHtrqo/nFtKz/6TVUVNY6lOrSE2GUlZkBCeuN5Noc1P5CGy/qXabpsSz+cGa38oCZkWH/BZAxrPaaG5bVF05j9ZTYtp+G2HAktlxffxhZloPrOGNp8XtPXGui1Q+O4pUkzPdC26soJ1OHXStktSO7vxJEU4jHpou3WWJn2JHBo/rpaqptQtXf99mrpOxiu9RqKTW48HZ9m/ZcwHSJpT8xtXDdsLXEReQ3IBnqJSC5wP+AAMMb8C1gA/BjYBlQBs0It21NsJa3EKVOwheM+tg2Tt8cNzlLfdDFUFdY/bo1/cEKgllqVb93Chkm82Opy928XrEFdwVQXN94BqCpqeVkAt7P156NdQrq19x5ofsDlG3xh4lIDL9tSsk7p23ocofxopgZo2QVKKm39Ae4xKPD6oSbNpuxt/KkIVG5SgNshNx0U2lJCjW3lex4XYGBpWxJoRye3wu1Wj0J7NY3Ln3gOubwOSIbtEZdyaOuFY+dDwtx+bU/MbVw3nKPTLwnyvAGuO5SyPQVWV2f6GcfDnmWHUkTr/MdTCzbDzhzrcfoQawT43hXWtLPBKNU9S2m2x1/kK6Nkt/XfXV0fq39HoGRXfdnlB+v3FvM3WcsWfx881tyljXcogq3TcMchVHGpjV/v4Ujs1o5TQkbzJC7SfE/cr+EXxubbK2/6Y5Pav3l50PqPamJGaHvUDcvwlxuoRdzWPfumSc//2prOj4ltW7mhCrgjEqA13fTHtKUfsKat74ZsrbWmQmiJt6VuQ/m9KdkF/Tqgi7ZZj1I7W42RbokfauLsiJ2Plj4/rX2u2qNdSfwwaYmHk3vZmwDEfHkfbO6g0w8C2fhf6w9gwCQrQa57q/lyz/4oeFk1pfDsKYGfGzAJ1r0NFb4xnuveCrydQD6+q/H07q9bX377p60/H5sMtU2Ok6f2h/wwJfFA22tN5hhrkGFTg46HXV9ZLVv/qThDfgjff2H9oDZtBR75I9j2SeNWnL/1OvhE2Lawfn7DJD7idMiaDKtfsQavLfl7/XMjz4JNvtH8Kf0Cf5HtceBxWt2tteVWGfzNGmATl2LtJCb3qV/+qDNg7Zuh/ygMnQ77VlkD/KB+O6kDrGn/j1bP4Y3XS81qXtbA42HPN6FttylHvPV/0NT6z6Q/WY44zXoft3xk1Xl8D6jx9T4F6ploqOHO2OCpsPV/kBZgwF2vEQCUpQ4nSIn1dXvUjwM/P+wkyNtgPW7pO9xUSv/gywTT1kMYLfG/B/73pLMNOh42L2jeOxTMUWdYv4sdsfOROab+ca+jrFN/Afq1PGCsXfwx+wfNhsL/exWb2KZNidUgjh6TJ082n/50HPteWMKwp+4mLquVLs32iE+zEi9Y5wcPnAJ7vgV3rdUFWFNmVbbb2XK3t3+5eF9L1l/VNpv1QcrbaL3ZvY+yfixNk+36yzBeq3XpP2fUf96wzVF//DzO92NtsEawVhdZCcpVZXWx2+zWY6+3Pp6kPtaPurPcSi6JGZDSl2Ufz+PYMUdaxzDzNlrHV+2+c9zzN0JcmnXeu9N3vnpMvHXYwHitQxGJvayy8jZB5tFW619sVhen/wfbU2u99owjrHPjK/OsOnFVW2XYY63/8anWay7cZpXR8wirTo3XSv4pfa11kjPhwHdW4vG6rR6KrGOheJdVvyJQsA3ikq36cyRa20wf0vg9y99ijWQv32e17gu3WT9CFXlWnfY80nq9/vP5C7dbrVuxW2UXbLXqduDxVhd00c76lrXXY71PFdZ2l370GlPOvMwqo8cg63UW74ReDRKs2wllewMfOw/E464vI3+zb4DbDug9on6Z0r1WHP6u7LJ91muIT4PKAut8d3c1JPb0fb7djXcsQlWyu3kZRTusHQqbw6rb3iOsnqTi76Gy0NpBamlEd/kB67vo/4H0euvLCCR/Mznr9pF90knBYy3a6bvWQYBE53FbSbwy36qblthsEJNgfSf7jTu0Omuq+Hvr82ZM8B2cltRWWr1/bU2iIcrJySE7O7vlBbxeKNxaf/2LULlqoHx/4LEUhyJvk/W593/28zbV/zaEQ+F2awcz1N6uVt4nEVlhjJkcaLWoTOL/mzGIg+9uZPiSr4lJj3AXURcU9EupOoTWc/hpHYef1nH4tZbEo/IGKJ5yq2vXnnKIAyWUUkqpLiAqk7i3sgqJESQmKg/pK6WUUh0iOpN4VTW2OL0EolJKqe4tOpN4tRNbnLbClVJKdW/RmcSdHmwJ7bwaklJKKRXlojOJu8Aep0lcKaVU9xaVSdxTa7DFaxJXSinVvUVlEve6wJYQpstFKqWUUlEiSpO4aEtcKaVUtxelSRxs8XGRDkMppZSKqOhL4sZgPDZsiZrElVJKdW/Rl8Q91s1GbAmaxJVSSnVvUZfEjdefxCN0Wz2llFLqMBF1SdzfErdrEldKKdXNRV0Sr2uJJ7XtxulKKaVUVxN1SbzumHiitsSVUkp1b1GXxI3XC4AtUVviSimlureoS+JoEldKKaWAKE7iEqfd6Uoppbq3qEvidd3p8ZrElVJKdW9Rl8Qxvpa4dqcrpZTq5qIviXsNYLQ7XSmlVLcXdUnceL2I3SB2vYuZUkqp7i3qkjjGYLMbsMVEOhKllFIqoqIviXu9iB3QlrhSSqluLvqSuDHYYgzYNIkrpZTq3qIuifuPiWOzRzoUpZRSKqKiLonXHRPX7nSllFLdXPQlca/xtcR1YJtSSqnuLeqSuDG+JG6PjXQoSimlVERFXRIHgwiaxJVSSnV70ZfEDSDaEldKKaWiL4n7W+IxmsSVUkp1b9GXxA2g3elKKaVUFCZxDKLd6UoppVQUJnF/S1xPMVNKKdXNRV8SB8RmwzowrpRSSnVfYU3iInK6iGwWkW0ickeA5weJyCIRWSUi34nIj4OXqpdcVUoppSCMSVxE7MA/gDOAo4FLROToJovdA8wzxkwELgb+GbRgA2LXVrhSSikVzpb4FGCbMWaHMaYWeB04t8kyBkj1PU4D9gUr1IC2xJVSSikgnKPDBgB7GkznAsc1WeYB4H8icgOQBJwStFTjRdxVHRSiUkopFb3CmcQD9XmbJtOXAHONMX8UkROAl0RkjDHG26ggkauBqwFGx8eBGHJycsIRswIqKiq0fjuB1nP4aR2Hn9ZxZIUziecCAxtMZ9G8u/yXwOkAxpglIhIP9ALyGi5kjJkDzAEYkxBvRCA7OztMYaucnByt306g9Rx+Wsfhp3UcWeE8Jr4MGC4iQ0UkFmvg2ntNltkNzAAQkVFAPJAftGQd16aUUkqFL4kbY9zA9cDHwEasUejrReRBETnHt9j/A2aLyBrgNeAKY0zTLvdmRIIuopRSSnV5Yb3smTFmAbCgybz7GjzeAJzY5oKj8hI1SimlVMeKvnRo9GJtSimlFERjEkdgUNMz1ZRSSqnuJwqTOEhm0wu/KaWUUt1PVCZx7NEZtlJKKdWRojIbil0vu6qUUkpFZRLHrvcSV0oppaIyiWtLXCmllIrSJI4mcaWUUkqTuFJKKRWtojKJi02PiSullFJRmcS1Ja6UUkpFaRKXGE3iSimlVEhJXETeFpEzReSwSPqip5gppZRSIbfEnwJ+DmwVkd+LyMgwxhScdqcrpZRSoSVxY8xCY8ylwDHA98AnIvK1iMwSEUc4AwxEW+JKKaVUG46Ji0hP4ArgKmAV8BespP5JWCJrjbbElVJKKUJq0orIO8BI4CXgbGPMft9Tb4jI8nAF1yJtiSullFKhJXHg78aYzwI9YYyZ3IHxhES705VSSqnQu9NHiUgP/4SIpIvIr8MUU3Dana6UUkqFnMRnG2NK/BPGmGJgdnhCCk5b4koppVToSdwmIuKfEBE7EBuekEKgF3tRSimlQj4m/jEwT0T+BRjgGuCjsEUVhLbElVJKqdCT+O3Ar4BrAQH+BzwTrqCCkZhOPzVdKaWUOuyElMSNMV6sq7Y9Fd5wQqQtcaWUUirk88SHA48CRwPx/vnGmGFhiqv1eDSJK6WUUiEPbHseqxXuBk4CXsS68EtkaBJXSimlQk7iCcaYTwExxuwyxjwAnBy+sIKI0SSulFJKhZoNa3y3Id0qItcDe4E+4QurddqdrpRSSoXeEr8JSARuBCYBvwBmhiuooDSJK6WUUsFb4r4Lu/zMGHMbUAHMCntUQYhe7EUppZQK3hI3xniASQ2v2BZxtpDvoKqUUkp1WaH2S68C3hWRN4FK/0xjzDthiSoI0RugKKWUUiEn8QygkMYj0g0QkSSOTZO4UkopFeoV2yJ+HLwhsWt3ulJKKRXqFduex2p5N2KMubLDIwqFdqcrpZRSIXenv9/gcTxwPrCv48MJjR4TV0oppULvTn+74bSIvAYsDEtEodDR6Uop1W4ul4vc3FxqamoOuYy0tDQ2btzYgVF1X/Hx8WRlZeFwhH6nzkO9aspwYNAhrttu2hJXSqn2y83NJSUlhSFDhnCoZxGXl5eTkpLSwZF1P8YYCgsLyc3NZejQoSGvF+ox8XIaHxM/gHWP8cjQ0elKKdVuNTU17UrgquOICD179iQ/P79N64XanX547WbZ9AOnlFIdQRP44eNQ3ouQDi6LyPkiktZguoeInBfCeqeLyGYR2SYid7SwzM9EZIOIrBeRV0OJx4krlMWUUkqpLi3UEWL3G2NK/RPGmBLg/tZW8F1z/R/AGcDRwCUicnSTZYYDdwInGmNGY91oJagSZ1WIYSullFJdV6hJPNBywbripwDbjDE7jDG1wOvAuU2WmQ38wxhTDGCMyQslmCLnoY+kVEopFb2Sk5NbfO77779nzJgxnRhN5IWaxJeLyJMicoSIDBORPwErgqwzANjTYDrXN6+hEcAIEflKRL4RkdODBWKAwurKYIsppZRSXV6op5jdANwLvOGb/h9wT5B1Ah2hb3rVtxis09WygSzgCxEZ4+uury9I5GrgaoCj4+NZsnIZZldBiKGrtqqoqCAnJyfSYXR5Ws/hp3XcurS0NMrLywH4w/+2s+lgRZvLMMa0OCBrZGYyt596RKvr33fffQwcOJDZs2cD8MgjjyAifP3115SUlOByubj33ns588wz69bxx9xURUUFXq+X8vJyampquPnmm1m1ahUxMTE88sgjTJs2jY0bN3Lttdficrnwer289NJL9OvXj5kzZ7Jv3z48Hg+/+c1v+OlPf9rmuugINTU1bfrMhjo6vRIIODCtFbnAwAbTWTS/ylsu8I0xxgXsFJHNWEl9WZPtzwHmAIxOSDCZQwaSPSm7jeGoUOXk5JCdnR3pMLo8refw0zpu3caNG+vO8XbEOrAfwjU4PB5Pi+s5Yh1BzyG//PLLuemmm7jlllsAePfdd/noo4+44447SE1NpaCggOOPP56LLrqobmehpTKTk5Ox2WykpKQwZ84cHA4H69evZ9OmTZx66qls2bKFl156iVtuuYVLL72U2tpaPB4PCxYsYNCgQXz88ccAlJaWRuzc9/j4eCZOnBjy8qGeJ/4JcKG/hSwi6cDrxpjTWlltGTBcRIYCe4GLgZ83WeY/wCXAXBHphdW9vqO1WAxQUqPd6Uop1ZHuP3v0Ia3X3ou9TJw4kby8PPbt20d+fj7p6en069ePm2++mcWLF2Oz2di7dy8HDx6kb9++IZf75ZdfcsMNNwAwcuRIBg8ezJYtWzjhhBP43e9+R25uLj/5yU8YPnw4Y8eO5dZbb+X222/nrLPO4oc//OEhv57OFuox8V4Nu7h9A9H6tLaCMcYNXA98DGwE5hlj1ovIgyJyjm+xj4FCEdkALAJuM8YUthqJQLmOTldKqS7jggsu4K233uKNN97g4osv5pVXXiE/P58VK1awevVqMjMz23xpWGOa3bMLgJ///Oe89957JCQkcNppp/HZZ58xYsQIVqxYwdixY7nzzjt58MEHO+JldYpQj4l7RWSQMWY3gIgMJsBdzZoyxiwAFjSZd1+Dxwa4xfcXEgOUObUlrpRSXcXFF1/M7NmzKSgo4PPPP2fevHn06dMHh8PBokWL2LVrV5vLnDZtGq+88gonn3wyW7ZsYffu3Rx11FHs2LGDYcOGceONN7Jjxw6+++47Ro4cSUZGBr/4xS9ITk5m7ty5Hf8iwyTUJH438KWIfO6bngb8KjwhBVdRqy1xpZTqKkaPHk15eTkDBgygX79+XHrppZx99tlMnjyZCRMmMHLkyDaX+etf/5prrrmGsWPHEhMTw9y5c4mLi+ONN97g5ZdfxuFw0LdvX+677z6WLVvGbbfdhs1mw+Fw8NRTT4XhVYZHqAPbPhKRY4DjsUad32yMiczwcIFKlyZxpZTqStauXVv3uFevXixZsiTgchUVLY+gHzJkCOvWrQOsAWKBWtR33nknd955Z6N5p512Gqed1toQr8NXyPf0NMYUGGPeBzYA14jIuvCF1UocQJWrOhKbVkoppQ4roY5O7wdchDW6fBzwKNao8oiodmsSV0qp7mrt2rVcdtlljebFxcXx7bffRiiiyGk1iYvIbKxknQXMA64C3jXG/LYTYgvIADUeTeJKKdVdjR07ltWrV0c6jMNCsJb4P4AlwM+NMcsBRCToqPSwEnB69NrpSimlVLAk3h+4EHhSRDKxWuOOsEcVRK1Xk7hSSinV6sA232C2p4wx04AZQCmQJyIbReSRTomwGcHtdUZm00oppdRhpNUk7hvQBoAxJtcY84QxZhJwHhCRTGoE3EaTuFJKKRXsFLPnfLcI/b2IZItIDIAxZnOkBrcd7J2AB2eLl9RTSinVdbV2P/HuKFh3+hlYtwnNAc4HvhGRd0TkahEZFP7wmhOxga2WqlpPJDavlFJK4Xa7Ix0CEMJ54saYGuAj3x++u5KdAfxdRPoaY6aEN8TGbGJDbC4qnG6S4kK9aqxSSqlWfXgHHFgbfLkmEjxusLfwW9x3LJzx+1bXv/322xk8eDC//vWvAXjggQcQERYvXkxxcTEul4uHH36Yc889N2gsFRUVnHvuuQHXe/HFF3niiScQEcaNG8dLL73EwYMHueaaa9ixw7p55lNPPUX//v0566yz6q789sQTT1BRUcEDDzxAdnY2U6dO5auvvuKcc85hxIgRPPzww9TW1tKzZ09eeeUVMjMzqaio4IYbbmD58uWICPfffz8lJSWsW7eOP/3pTwD8+9//ZuPGjTz55JMh1XNLQr3Yy/XAy8aYEmPMTuCfwD9FJLZdWz8EdrGB1FJe4yIzNb6zN6+UUqoDXXzxxdx00011SXzevHl89NFH3HzzzY3uJ37OOefU3U+8JfHx8cyfP7/Zehs2bOB3v/sdX331Fb169aKoqAiAG2+8kenTpzN//nw8Hg8VFRUUFxe3uo2SkhI+/9y6jUhxcTHffPMNIsIzzzzDY489xh//+Eceeugh0tLS6i4lW1xcTGxsLOPGjeOxxx7D4XDw/PPP8/TTT7e3+kK+AUpfYLmIrASeAz42ltp2R9BGIjbEVkuNy9vZm1ZKqa4rSIu5JdWH0f3EjTHcddddzdb77LPPuOCCC+jVqxcAGRkZAHz22We8+OKLANjtdtLS0oIm8YsuuqjucW5uLhdddBH79++ntraWoUOHArBw4UJef/31uuXS09MBOPnkk3n//fcZNWoULpeLsWPHtrG2mgvp2unGmHuA4cCzwBXAVhF5RESOaHcEbWR1p7updrk6e9NKKaXCoKPuJ97SesaYoK14v5iYGLze+kZi0+0mJSXVPb7hhhu4/vrrWbt2LU8//XTdsi1t76qrrmLu3Lk8//zzzJo1K6R4gmnLDVAMcMD35wbSgbdE5LEOiSRENqyKKXfqncyUUqoruPjii3n99dd56623uOCCCygtLT2k+4m3tN6MGTOYN28ehYWFAHXd6TNmzKi77ajH46GsrIzMzEzy8vIoLP7oh38AACAASURBVCzE6XTy/vvvt7q9AQMGAPDCCy/UzT/11FP5+9//Xjftb90fd9xx7Nmzh1dffZVLLumY24+ElMRF5EYRWQE8BnwFjDXGXAtMAn7aIZGEyGazQi7Xe4orpVSXEOh+4suXL2fy5Mm88sorId9PvKX1Ro8ezd1338306dMZP348t9xyCwB/+ctfWLRoEWPHjmXSpEmsX78eh8PBfffdx3HHHcdZZ53V6rYfeOABLrzwQn74wx/WddUD3HPPPRQXFzNmzBjGjx/PokWL6p772c9+xoknnljXxd5eEsr51iLyIPCsMabZ7pCIjDLGbOyQaEIw+OhhJvX2JO4Y8yKXTprYWZvtVnJycsjOzo50GF2e1nP4aR23buPGjYwaNapdZZS385h4d3PWWWdx8803M2PGjIDPB3pPRGSFMWZyoOVDPSZ+H9DT1yK/QUSOafBcpyVwsI6JA1TUVnbmZpVSSqlDVlJSwogRI0hISGgxgR+KUE8xuxf4GfCOb9bzIvKmMebhDoskRHZfEq9y6e1IlVKqO4rG+4n36NGDLVu2dHi5oZ5i9nNgou/CL4jI74GVQKcncX9LvNKlx8SVUqo70vuJ1wt1dPr3QMMrq8QB2zs8mhDYbXYAqt16O1KllFLdW6gtcSewXkQ+AQzwI+BLEfkrgDHmxjDF10xdd7pbW+JKKaW6t1CT+Hzfn19Ox4cSGrvNhhcvNXpMXCmlVDcXUhI3xrzgu076CN+szcaYiFwyzSZWEq/2aHe6UkpFu+TkZCoqKiIdRtQKdXR6NvAC1rFxAQaKyExjzOLwhRaYzXcY36lJXCmlVDcX6sC2PwKnGmOmG2OmAacBfwpfWC0TETCC06Pd6Uop1VUYY7jtttsYM2YMY8eO5Y033gBg//79TJs2jQkTJjBmzBi++OILPB4PV1xxRd2y/tt7dkehHhN3GGM2+yeMMVtExBGmmIIzsdR6tSWulFId5Q9L/8Cmok1tXs/j8WC32wM+NzJjJLdPuT2kct555x1Wr17NmjVrKCgo4Nhjj2XatGm8+uqrnHbaadx99914PB6qqqpYvXo1e/furbvnd0lJSZvj7ipCTeLLReRZ4CXf9KXAivCEFJwQS63XGanNK6WU6mBffvkll1xyCXa7nczMTKZPn86yZcs49thjufLKK3G5XJx33nlMmDCBYcOGsWPHDm644QbOPPNMTj311EiHHzGhJvFrgeuAG7GOiS8G/hmuoIKxmVhc2hJXSqkOE2qLuamOunZ6S/fxmDZtGosXL+aDDz7gsssu47bbbuPyyy9nzZo1fPzxx/zjH/9g3rx5PPfcc+2OIRoFPSYuInasm588aYz5iTHmfGPMn4wxEWsK23DgNrWR2rxSSqkONm3aNN544w08Hg/5+fksXryYKVOmsGvXLvr06cPs2bP55S9/ycqVKykoKMDr9fLTn/6Uhx56iJUrV0Y6/IgJ2hI3xnhEpLeIxBpzeGROu8Th1u50pZTqMs4//3yWLFnC+PHjEREee+wx+vbtywsvvMDjjz+Ow+EgOTmZF198kb179zJr1iy8Xi8Ajz76aISjj5xQu9O/B74SkfeAutuHGWOeDEdQwdglFk9kTlNXSinVgfzniIsIjz/+OI8//nij52fOnMnMmTObrdedW98NhZrE9/n+bID/4EfwG5GHiV1icRm97KpSSqnuLdQkvsEY82bDGSJyYRjiCYlDYqmi+55SoJRSSkHoF3u5M8R5nSJGYvFyWByeV0oppSKm1Za4iJwB/BgY4L9jmU8q4A5nYK1x2OIw6DFxpZRS3Vuw7vR9wHLgHBpf3KUcuDlcQQUTa4/HiLbElVJKdW+tJnFjzBpgjYi8Gqm7lgUSa4vFyGETjlJKKRURoQ5smyIiDwCDfesIYIwxw8IVWGti7XEgbrxeLzZbqIf1lVJKqa4l1CT+LFb3+QrAE75wQhNnj0PEUOlykhKXEOlwlFJKHebcbjcxMaGmvOgRajO21BjzoTEmzxhT6P8LtpKInC4im0Vkm4jc0cpyF4iIEZHJoQQTb48HoMyp54orpVS0O++885g0aRKjR49mzpw5AHz00Uccc8wxjB8/nhkzZgDWhWFmzZrF2LFjGTduHG+//TYAycnJdWW99dZbXHHFFQBcccUV3HLLLZx00kncfvvtLF26lKlTpzJx4kSmTp3K5s3WzTk9Hg+33nprXbl/+9vf+PTTTzn//PPryv3kk0/4yU9+0hnV0Sah7pYsEpHHgXeAuuudGmNavGSO75rr/wB+BOQCy0TkPWPMhibLpWDdWOXbUIOOj7GSeLlT7ymulFId4cAjj+Dc2PZbkbo9HopauBVp3KiR9L3rrqBlPPfcc2RkZFBdXc2xxx7Lueeey+zZs1m8eDFDhw6lqKgIgIceeoi0tDTWrl0LQHFxcdCyt2zZwsKFC7Hb7ZSVlbF48WJiYmJYuHAhd911F2+//TZz5sxh586drFq1ipiYGIqKikhPT+e6664jPz+f3r178/zzzzNr1qw21EznCDWJH+f737ClbICTW1lnCrDNGLMDQEReB84FNjRZ7iHgMeDWEGMhwZfEK2o1iSulVLT761//yvz58wHYs2cPc+bMYdq0aQwdOhSAjIwMABYuXMjrr79et156enrQsi+88MK6+52XlpYyc+ZMtm7diojgcrnqyr3mmmvqutv927vssst4+eWXmTVrFkuWLOHFF1/soFfccUJK4saYkw6h7AHAngbTudTvDAAgIhOBgcaY90VEk7hSSkVIKC3mQNp7K9KcnBwWLlzIkiVLSExMJDs7m/Hjx9d1dTdkjEFEms1vOK+mpvFtqpOSkuoe33vvvZx00knMnz+f77//nuzs7FbLnTVrFmeffTbx8fFceOGFh+Ux9WAXe/mzMeYm3+P/M8b8pcFzc40xV7S2eoB5dddbFxEb8CegtTL8y14NXA2QmZlJ/r48AJauXgG7ioKtrtqooqKCnJycSIfR5Wk9h5/WcevS0tIoLy9vVxkej6ddZRw4cICUlBQ8Hg8rVqzgm2++oaSkhJycHNauXcuQIUMoKioiIyOD7OxsnnzySf7whz8AVnd6eno6vXv3Zvny5QwfPpw333yT5ORkysvLcblcVFdX18VXWFhIRkYG5eXlPP300xhjKC8vZ9q0afz9739n0qRJdd3pGRkZpKSk0KdPHx566CHefffddtdVKGpqatr2mTXGtPgHrAz0ONB0gHVPAD5uMH0ncGeD6TSgAOsOad8DNVgXl5ncWrmTJk0y/1yywIyZO8a8tuYzozreokWLIh1Ct6D1HH5ax63bsGFDu8soKytr1/o1NTXm9NNPN2PHjjUXXHCBmT59ulm0aJFZsGCBmTBhghk3bpw55ZRTjDHGlJeXm8svv9yMHj3ajBs3zrz99tvGGGPefPNNM2zYMDN9+nRz3XXXmZkzZxpjjJk5c6Z5880367b19ddfm+HDh5upU6eae+65xwwePNgYY4zL5TI333yzGTVqlBk3bpz529/+VrfOa6+9Zo477rh2vca2CPSeAMtNCzkxWN+AtPA4FMuA4SIyFNgLXAz8vMHOQynQq65wkRzgVmPM8mAFJzqs08oqXTVBllRKKXU4i4uL48MPPwz43BlnnNFoOjk5mRdeeKHZchdccAEXXHBBs/lz585tNH3CCSewZcuWuumHHnoIgJiYGJ588kmefLL53bW//PJLZs+eHfR1REqwJG4TkXSsU9H8j/3JPPBwRB9jjFtErgc+9i37nDFmvYg8iLVX8d6hBp3sOze8yqXHxJVSSoXHpEmTSEpK4o9//GOkQ2lRsCSehnWBF3/ibnhKWdD7iRtjFgALmsy7r4Vls4OV55fk8CdxbYkrpZQKjxUrVgRfKMKCXTt9SCfF0SbJsVYSr3ZrEldKqfYwLYzMVp3POvzdNm2+8LjvGuoRlVKXxLU7XSmlDlV8fDyFhYWHlDxUxzLGUFhYSHx8fJvWO5ST3s4BHjiE9TpMSryVxGvcziBLKqWUaklWVha5ubnk5+cfchk1NTVtTjwqsPj4eLKystq0zqEk8Yj3u6TEJQKaxJVSqj0cDkfdVdEOVU5ODhMnTuygiFRbHcp9PCd1eBRtlOBwYLx2ajx6TFwppVT3FVISF5HHRCRVRBzAJyJSICK/CHNsLYqLsYFxUOvRlrhSSqnuK9SW+KnGmDLgLKxroI8AbgtbVEHE2ARjHDg1iSullOrGQk3iDt//HwOvGWMiesFyEUGMg1qvdqcrpZTqvkId2PZfEdkEVAO/FpHeWNc6jxgxDlze2kiGoJRSSkVUSC1xY8wdWDc0mWyMcQGVWPcGjxgbsbi82p2ulFKq+wp1YNuFgNsY4xGRe4CXgf5hjSwIG7G4jbbElVJKdV+hHhO/1xhTLiI/AE4DXgCeCl9Ywdlw4NaWuFJKqW4s1CTu8f0/E3jKGPMuEBuekEJjF22JK6WU6t5CTeJ7ReRp4GfAAhGJa8O6YWGXWDxoEldKKdV9hZqIf4Z1X/DTjTElQAYRPE8cIEbi8GoSV0op1Y2FOjq9CtgOnCYi1wN9jDH/C2tkQThscXhxRTIEpZRSKqJCHZ3+f8ArQB/f38sickM4AwvGIbEYbYkrpZTqxkK92MsvgeOMMZUAIvIHYAnwt3AFFozDFofxuvSG9koppbqtUI+JC/Uj1PE9jmjmjLXHAej105VSSnVbobbEnwe+FZH5vunzgGfDE1JoYm31STw+Rm9Ir5RSqvsJKYkbY54UkRzgB1gt8FnGmFXhDCyYeLuVuGvcNaTFpUUyFKWUUioigiZxEbEB3xljxgArwx9SaBIc9UlcKaWU6o6CHhM3xniBNSIyqBPiCVlSbAIApc6qCEeilFJKRUaox8T7AetFZCnWHcwAMMacE5aoQpDsS+LF1ZVBllRKKaW6plaTuIgcCWQCv23y1HRgb7iCCkVKnNWdXlKtLXGllFLdU7CW+J+Bu4wx3zWcKSKVwP1EcIR6amwiAGU1msSVUkp1T8GOiQ9pmsABjDHLgSFhiShEqfG+JO6sjmQYSimlVMQES+KtnYCd0JGBtFWPBCuJl9dqEldKKdU9BUviy0RkdtOZIvJLYEV4QgpNWnwSABU6Ol0ppVQ3FeyY+E3AfBG5lPqkPRmIBc4PZ2DB9E5MAaC8VpO4Ukqp7qnVJG6MOQhMFZGTgDG+2R8YYz4Le2RB9EqyknilqyLCkSillFKREeplVxcBi8IcS5ukxsdjvDFUurQlrpRSqnsK9S5mhx0RQUwcVW692ItSSqnuKWqTOICYeGrcOjpdKaVU9xTVSdxOHE6vdqcrpZTqnqI7iUsCTq+2xJVSSnVPUZ3EHRKPy6u3IlVKKdU9RXUSj7Ul4EGTuFJKqe4pqpN4vD1Rk7hSSqluK6qTeII9ESOaxJVSSnVPYU3iInK6iGwWkW0ickeA528RkQ0i8p2IfCoig9tSfoIjESNOvF5vxwWtlFJKRYmwJXERsQP/AM4AjgYuEZGjmyy2CphsjBkHvAU81pZtJMUkImL0dqRKKaW6pXC2xKcA24wxO4wxtcDrwLkNFzDGLDLG+E/0/gbIassGkmOTASioLGt/tEoppVSUCWcSHwDsaTCd65vXkl8CH7ZlAymx1u1IC6vK2xqbUkopFfVCugHKIZIA80zABUV+gXWL0+ktPH81cDVAZmYmOTk5AJQcLAJg8fIlVO/Y1+6AlaWioqKujlX4aD2Hn9Zx+GkdR1Y4k3guMLDBdBbQLNOKyCnA3cB0Y4wzUEHGmDnAHIDJkyeb7OxsALYuc5KzAfoNG0T2hOwODb47y8nJwV/HKny0nsNP6zj8tI4jK5zd6cuA4SIyVERigYuB9xouICITgaeBc4wxeW3dQHqCdUy8uFq705VSSnU/YUvixhg3cD3wMbARmGeMWS8iD4rIOb7FHgeSgTdFZLWIvNdCcQENyegJQG5pUccFrpRSSkWJcHanY4xZACxoMu++Bo9PaU/5g3tYSXxfWXF7ilFKKaWiUlRfsS01LhWAg5UlEY5EKaWU6nxRncTj7HGIcVDp0mPiSimlup+oTuIADkmkxlMR6TCUUkqpThf1STzOlkytqYx0GEoppVSni/okHm9PwkMlxgS8joxSSinVZUV9Ek929AB7JRVOd6RDUUoppTpV1Cfx9LieSEw5ZTWaxJVSSnUvUZ/EM+J6IvYqCiqqgi+slFJKdSFRn8QHpGYiYtheeCDSoSillFKdKuqT+LD0vgBsL9of4UiUUkqpztVlkviu0oMRjkQppZTqXFGfxHsn9QZgf0V+hCNRSimlOlfUJ/Ge8dZNUAqrCyIciVJKKdW5oj6Jx8fEE0MCpbV6O1KllFLdS9QncYCkmAxqTBG1bm+kQ1FKKaU6TZdI4pmJ/RBHMSt3633FlVJKdR9dIomPzRyGzVHMok15kQ5FKaWU6jRdIokPSR2I2KvZnK8j1JVSSnUfXSKJZ6VkAbCjeFeEI1FKKaU6T5dI4gOSBwBwsHofNS5PhKNRSimlOkfXSOIpVhL32otYs6ckwtEopZRSnaNLJPHU2FR6xKVjj8vj2516vrhSquN9uHY/JVW1kQ5DqUa6RBIHGJUxksTkg3y7szDSoSilupgDpTVc+8pKbnhtVaRDUaqRLpPER/YciSdmP19vP8imA2WRDkcp1YVU1roB2F1UFeFIlGqsyyTxURmj8OJGYvP4Zru2xpVSHae8xkriEuE4lGqqyyTxkRkjAUhKOcC2/IoIR6OU6krKql0A2ETTuDq8xEQ6gI4yOHUw6XHpeNJ38+2OIlweLw57l9lHUUpFUFmNlcS1Ka4ON10my9nExnH9jsPEb2FrXjnPfbkz0iGpDlTj8nDnO99RWOFsNP9AaQ2PfrgRr9dEKDLVHZRVW93p2hJXh5suk8QBju93POXuIo4eXM2jH27i+ldXYoz+uHcF763ex2tL9/DE/zY3mn/rm2t4+vMdrNLrA6gw8rfEbZrD1WGmSyXxEweciCBMHbcbgPe/209ucXWEo1IdwWDtjNW6G++UVfuu0Kc7ayqcyn1J3K09Puow06WSeN+kvkzPms4nue9x3LA0AJbs0JHqXYG/G7NpstbkrTqD0+UFoKhSL/aiDi9dKokDXDTyIopqirh8RhnDeifxm7e+40+fbMHThj3ogibHXVXk2X39mF5N2ioCaj1WEi+pclHlO2dcqcNBl0viU/tPJSs5i3lb3uA3p1mnnf3l06385KmvQ7pk4vp9pUx+eCFvr8gFwO3xNmrt7S2pptbtDU/wKihPkxzun9T3pGM9/9VO1uaWRjqMw4a/JQ6wr0QP0anDR5dL4jaxcfHIi1mZt5L4tE3M+9UJAKzZU8Kxv1vIvxfvaHX9LQfLAfhscx5VtW6OvPtDnvp8O2CNkD7x959x5ztrw/siDhM1Lg9n/e0LvjkMDkk4fUm6pZa4M0qTeKXTzZA7PuDN5Xs6bZtfby/g+4LKVpf57X83cPbfv+ykiA5//pY4wJ4iTeLq8NHlkjjAJSMv4Yi0I3hi+ROMyYrnX7+YxMi+Kbg8ht8t2Mj0xxexcMNB7nxnLS8u+R6v17BqdzEAH3x3AACPx9QNinvmC+t0tfxyq5v9w3X7W91+SVUt5//zK7aH4aIzhRXO+nNWw2xXYRXr9pZx9/zI77T4bzHb9FQyf06P1lvQ7i+1PmP+HcXO8PN/f0v2EzktPt/ZvRq3vLGa0/60uFO32VZOt4d+afEAzJq7jJnPLY3az5zqWrpkEo+1x3L7lNvJLc/lpkU3cfKoDP5z3Ym8cfXxXH7CYCqdHq56cTmvLd3Nfe+uZ9hdCzj/n1/z/Fc7WbjxIAAeY9jrS+L+1l9eeQ0Q+FxRYwy7Cq3WzScbDrJqdwl/+mRLh7+2SQ8vJPvxnA4vNxD/sb+2jCcIF/8o9KYtcf/U4dwSX7GriE99n6um/KPtbSKszS3l4jlLOiQ5GGN4/7t9zcoKZSBgpbNzj/m+s2ovmw+WM+SOD+q+Q4cbp8tLRlIsD503hgE9Evh8Sz5vr8zlha+/p6KT60uphrpkEgc4of8JPDD1AZbsX8J1n15Hjaec44b15MFzx/Dl7Sfxq2nDmq3z2/9uqHv8yYaDzJq7DLCSWHmNi1++sByACqeb2S8u5+P1B/hs00GMMfxn9V6mP57Dku2F1ATp+vUzxnCgtCbk1+T/AW7LCNnS6kNvtRf7xhB0xmk16/eVtrqzUOM7JtnSMk734dsq+ulTS+o+O035T12yi3DX/LV8s6OIjfvbfwOfb3YUcf2rq3j848bn1YeysxPJpPTtjs6/lbAxhj98tIl1e1seA1Dr8RIXY+Oy4wfzv5unEWMT7p6/jvvfW8/8VXvDGt8nGw7yzBetHwbsiqprPWw+UB7pMA57Xeayq4Gcd+R5uLwuHv32US778DIe/sHDjO89nniHnTt/PIo7zhjJgrUH6N8jnndX7+Olb3YFTBLlNW7O/+fXlFTVJ8RPNhzkkw1W62r8wB6s8V1sZPaLy+t+BHcXVbFoUx7r95VSVevhuGE92V1UxeIt+fz0mCx+89YaymrcLLo1m7JqF/9ZvZfEWDtnjevP7qIqVu8p4ajMFAoqnLy1Iperfli/4/HRuv1MHJROZmo8xhgkQO/A+n2lnPnXL7n42IE8+pOxiAiVTjcL1u4nPTGWU47OBKzDBN8XVnLskAwA9pRbg/mKKq3X6+/Cdro9XPXCcn6dfSQnHNGz3e+P37q9pZz1ty/5vxnDuflHIwIu4/S1KKtqPTz75U4+XLufN685oa4/vaNb4pVON7ExtrBfutd/Yw27Teo+ex2xy+TfAdtV2PiuW2Uh7NRVRnD0dSR2ICprPTyVs51nv9jJlt+d0ex5t8fL5gPlDOudBEBSXAyDeiayI9/qNVizp4RzxvUnLdERlvhmv2jtADb8/ncHN7+xmo/WH2Djg6eTEGuPdDhttqeoir5p8WH/DenSSRzgwhEXMiR1CHd/eTeXf3g5Jw88mctHX86E3hMQEc4c1w+AiYPSuevHo4iNsfHpxoP88oXlXD1tGJefMJgf/GER2/JaPr69psHVwiqcbjKSYimqrGXd3rK61jzAP3Pqj3v6dwAATmpyfPIfiwIfH731zTV1j695eWWj59ISHEwb0Zv/rtnHJVMGceKRPXngPatn4fVlexjaK4lteRW86Rt1D3D2+P784rhBXDTnGwDOnziAkX1TePSrau79agGXTBkIQK3HMG/5HvaVVPPF1gK+2lbAw+eN5aH3N/Dkz8bzg+G9AEiOi8HtNbg8XvYUVWMTSIi1M/O5pdx39mhG9k2h1u2l1uOlsKKWY4eks3J3MT99aglgnUWwcncxuwqruHraMAZmJHL8sAwcNlvdDlReuZOH3t9QV4drfCOoiypr2VNUxcCMRIwxuL0m4JfHP7L4QFkNn23M49bTjmr2fL+0eEbf/zFnj+/Pg+eMJj0pNuD74XegtAavMfTvkdDsuYbH8J1uD8bA9a+u5KZTrJ2VcqevJW6TumPRFTXtT2QtlRHKeIpKZ8f0ajzzxQ4mD8nAYRecbi8TB/bgdx9s5LyJAxgzIC3gOnuKO/9Wn/6zVhoOXmvokQWbyCt30jM5rm6eq8Gyb63I5a0VuUwanM7sHw5jT1EVs04cQkwH/3hXOt0kxbX9J/utFbm8t2YfL145pUPjCTf/NT4KKpwMzEjs8PJX7ylhfFZawAZQe5VU1fLDxxZx2fGDeei8MR1efkNdPokDHNv3WN455x3+teZf/HfHf1m4eyFHpB3B2UeczYkDTmRE+ghsYiM2xvrSzRiVydK7Z9AnxRrI8vysY5n1/DLOm9Cfm04ZQfYTOTx3xWTyypyMzUojKz2RbXlWt0+8w87R/VJZv6+Mq15YzrkT+vP04h0kx8Xwo6MzOXZIBnf5BoqdM74/763ZFzBmh11wNT2fqhWl1S7+6yvrtaW7eW3p7kbPP/rhpmbr/HfNvrp1gGbdgq8ttUZMF1Q4+c1b39XN9xrqXsO1rzTemWjJzOeWhrTcF1sLALjnP+sCPt9wZ+rql1bUPf7zwq38eeFWbpwxnASHnT99soXLTxjMvtJqZozMZH9pNd/uLKor3y89KZb+afEUVNaSluDgxtdWMbSX1eLy189jF4xjX0k1n2w4yC0/GsG3O4s4UFrD/WcfzWtLd/PE/6yxD+/f8AP+37w19O8Rz+lj+uL2Gh5+f2Pdth75YCO1Hi8LN+axfFcxk3sZjjnaP84C3F4rMZTVuHB7vMTYbXyxNZ9/fb6dR88fR5XLTYzNRkGFE4fdxtKdRZw9vh9uj2F7fgXHDs0gt6iafyzaVtdqLKx08ueFW7hkyiCg/j0Fqys/Jd7Bs1/uZFBGIiMyk9m4v5zPt+TXLfPyN7sorXYxcWAPBvVMJCvd+jF1e7wUVdXWfUf83B4vn2/J54QjevLwBxsbPbf0rhk88+VOnv1qJ8/OnMx/Vu1r9gP3yYaDnDthgDUew8DUI3vV7ZTdMm8NQ3om8v9OPQqP1/DJhgNc8/JK/nPdiSTH2RnWKxkREBFKqmq5a/5aZmQE76Fp2MNWVuMiNb5xi/qDtfuISV1NuefIunm9kuPYU1TNvF+dwKXPfIPLY1ixq5gVu6zP5O6iKs4e359Vu4s5eWQf4mLs1Ho8HNknxbfNWiqcbnqnxBEXE7iVaYzhL59urZvOLa7mqL4pQV9PU/6df//7HQ4uj5cYmwRMiJsPlHPX/LU8P+vYZnW7u7CKgkonxwxKb7aefyc8r7z9SXzj/jLO+MsXxNptXD1tGCcc0ZNLn/mWB84+mitOHNqusgMpqLB2DBduPBhyEv9qWwGfbszjvrOPbtO2JNqueDV58mSzfHng44uhqHRV8t7295i/dT4bi6wfmfS4dAalDiIrJYus5CxcXhcXjriQAckD6j6UHq/B5vuBaKn7uiV7iqrolxZft2du/23gawAAFvBJREFUjKGs2k1qQgzf7CjifxsOcOWJQ7HbhNV7Sjh9dF9sNuG6V1bywdr9XD1tGFf9YChTHvm0Ubn3nDmKWScOZfWeEm58bRUnj+xDjcvDmytyGZSRyO4iq1VzzKAeDO+TQmZaPCVVtby4ZBdTj+jJ1w3uuz6sVxI7Apx21CPR0ehHLjkuptsN5BGpHwXfGVp6Lw4HV08bRl5ZDYs259eNtzhnfH9sAou3FrT5imbXZh/BUznBR+YnxdqprLV6CK6ZfgT/ajKaPy7GxqCMRPqmxfPYBeOY+9X3PL14B9lZMYwdMYSyahcvLNnFlScO5cofDOGCp5bwq+nDOCozBafHy6znrR6z00f35aIpA/lXznZmTh1CvMPGlXO/JWXU3eCNZ+0sa7m9JdV8u6OQnxyTRYXTjU2snqT31+wnJT6GTQ2O5abGx2C3CcVVLpJi7Ywf2KPuu5cSF8NNPxrB+Kw0Plp3gEmD0xERThnVh3dW7W208/zPS4/BawyTB2ewr7Sa8Vk9MMZQUFFLabV1EZoRmSl1rfUN+6yxFT/+6xcAvDr7OEb3T8NhF1btLuH2t7/jb5dMZKIvgbo83rpDOiVVLvqmNd5B81u6s8hXB1WcPzGL1z/4jMdXejh7fH88XsOvpg+r29kDuPy5pSzeks9Tlx7DGWP71c2vdXuZ8shCSqpcvH3tVHonxzGoZ/16J/7+M/aWVPOvX0zi9DF9G8Ww+UA5tW4vY7Ma9+h4vIaKGnezQxu/+2AD//6i/qZYf7l4Av/3+mpOOqo3z8/quB6KvLIafv/hJs4a348r5y6nX1o8S+6c0eLyn206yJG9U7j7P2vrGhhr7j+V/9/enQfJUd0HHP/++phjZ09ppdXu6kYS9y1jg4Eo2MZgAz7AARsHQuxyxY4DsYPjI7GDXaRilxNfwUUKYxwfFMRFbCMHgiEgDAghDEgykoVA97W7OlarnT3m6v7lj+4ZRqvVSgatxKDfZ2tru3ve9rx+86Z/r9/roykd5X8wX6Iu4eI4zvOqOn+0dYxrEBeRS4DvAi5wp6p+fcTrSeAnwNnAbuBqVd041jpfbxCv1j3YzZLtS1i+czlbs1vZmt1K91A3oUat90npSaS9NBk/wxmTzyDtpcmVcrRl2pjTPIeOTAcpL0XKS1Hv15PyRq/0r9Xzm3q58vYlPH7zAma2Zpj5hQcA6GxOc/npHXzh0hP2+58d2Ry3PbaWz19yArsG8kxuSO03npTNFUn5LnsGC5WGwYqvXMyDK7uY0piib+NKJs4+lQdf7OKy0zr46A+XVv73G1eeysUnTSEbf1F6BwsEYUhdwiPpOdx073KeWhtVxsVfuIjnNvYyd3IDmaRLOuGybscg31+0lr88fyaFkvI397xQ6XG45i3TaK1PsmjNDqY0RmV55/XzeeKVXXQ0pVi4Yjv//thaAK5963Sa63ye27iHpfFOZeGn307vYIF7nt3M5ad3sGn3EM11Prc/vo5MwqMu6bK9b5h8Kaw0TMpDH7NaM7xt9gTam9J8q+qqgqvOnorvCknPZVZrhlt+vYoPnNFJR3OaDbsG6c8VK1++hOeMennWjRfN4XtxvqvNaHTY1D9+Z9VPaUzR3X/oJ04eDp4jr/lEyIakRzEMKycxvlGIv5v6Od8E4MXrD365ZU9/ju8vWsv8mRNY091/wOGxw6G9KfqMq3fjzXEAq258j6WzOU1nc5oXNu/BEakMKyQ9h/amFD39eT5wVidPvLxzv2dRVDeuql16yhTmtjXwwqY9vNSdrdwFs6MpxcT6JC8e4CTCUzobufSUdk7tbOK6uPfuH997IvVJjw27B3mlZ4D2phR3L416Gm94+0zmtTVwz7Ob+ef3n8pvVnVz26K1/OiGtyBEDQXXkf1OLD1rejMvbI6GQW/7yJnszObxHGHB8ZNxHGG4UGLPUJFd2Tyru/pZuGI789oaOH5KA/NnTiDhOqQTLrsH8jTX+dz6wGouP62Drr3D/ODJDUxtSVfK6qtXnMzZM1roaE7jOkJT2idfCli2uY9r4qHMandeN581PVmSnsOtD6ymuc5nxT+9+8gHcRFxgZeBdwFbgd8BH1bVP1Sl+RRwmqr+lYhcA3xAVa8ea72HM4iPphgU2ZzdzCObHmFT/ybyQZ6t2a1syW5hsDhYeRDHaBJOgsZkI/V+PT1DPUxrmEZrupWEk8BzPFzHxREHV1w66jtIukk8x8MTL/rrePiOj+u4lf/ZnN3MlLopTK6bzC9eWsTO4R6uPukSWlIt1Pv1+K6PIDgSHeVL/MBjQfBdn4HCAJPqosaIiBBqSKjRiWspL8WKbTvI5gpcOGd6ZTsef/xxFixYAEDPYA+LXx7gB09sZ03vau74yMW88/iZByyD4ULAiV95CFA2fv2yg5b3roE853zzZ8ybMJOHbnq1xTpab8f9ax7jsz9fjpc/kTW3RicgLdq8iM8s+jxe1+dYfPMHD9g1+cfI5or8ekUXX/rliyz78rv2GRPfO1ykMeXtk7cd/Tlu+fUqvva+U3i5O8spU5sYLgQ8s343cybXM3dymo/e9TQOSa596wySnsP2vmFmFDZy2jnzefc9H2PPtgu47qyL+NHiDYCw6OYFdDanGSqUSCdc9g4XyRVCpk+soxSE3P74Ov70hMn8atk2bnrnXHZm83T35zjvuFbueGIdZ05v4ezpLfx4yUZmtmY4d/ZEblm4igXHT+bE9gZufWA11507g5a6BAtXbOez75pH/3CRK//j6fhmJkr54dk3XzyPaRPq6GxOM6UpRXNdgn97eA3TWuq4e+km2pvSXHF6B1ec0UH/cJGV2/fyixe2sX7nIHPb6rl/+Xb+6fKT9rn6A+A/b3gLtz++juc37aYUhlxx+jS+9Wen8+X7V7Fy215e6u7n/Wd0csG8Sdx4zzIAPrXgOH72zCb6R4z5z5/RwnOb9oz6eZ4zawKZhMuzG3r5+AWzKQQhyzbvobO5jhc272FD3OvR0ZRi+94cTWmfifUJ1u8cxHOEs0/YxR/00IN4NVXlrsUbySRcLj21nStvf5pP/slxdO0dpqc/jyPRg5o+eFYnbY0pfvbMJrK5ErsPsUdjdmuG7v4cQ3EgnTGxDtXoxMbySZPHtzWwuXeocpnmSLNaM5UyGMvIXjnz2iQ8B9+RURs/ACnf2a8hu+kblx2VIH4ucIuqvjue/yKAqv5LVZrfxGmWiIgHdAOTdIxMjXcQH0sxiCpwtphlS3YLXQNdDJeGyQd5BooD9Of76S9Ev444ZAtZ+vP9lLREMSgSaICi5Eo5eoZGv274SBKk0ihJuSkCjSuVgud6ZPwMu4bjo0wnSSHM44rH1IbOSmMAIONnKtOFoMDO4V0UgjxzmucQElIKS9T79VEjIgwJiRoRjjjkSjnW7V1HU6KZ6Y3TSHtpSmGJlJeqNEw88Ui4CR7e9DAAUzOzmN7Ujud4PLE1uklIR6aDmU0zSTgJfNfHc7zKad4a/0C0U1UUVSXQgKSbJNAAz/Gi6TAqg1yQoxSWaEm14Ipb2T5FK9OuRA2GYliMGmni7vM+ZYu3L2aoOMSCaQtIudF2KUrP9h6GMkM82x0dcVzYeSFLu5Yzt/4czprasd/nFWqIKy6BBuSD/D49P77jV/JZ3r4N/Rtoz7TTkGiIxpW1RCks0ZvrZX3fes7rOI+kmyzHalAINOC2J5fg1a/huMaTKeRaeM9Jc/BdD0ecymdSCksUwyKqStJNVho15e0uf8ZBGI1np32PXDGkpz/Piu71DBSGuGH++QDc+4f72Zrt4uSmt/POeSfgOz6BBhTDIsWgSNJN8u0nfotIkRsvuIiUm2JNTy/5opBJl2ita6AhlaYYlFi8dheOCEvW7aY+5XPqBGXB6XNxHKdS3wXBddxKHfi/l9exfMcy3jvvXAqFOk7tbAKEUqAkPZentj3Nkq7FQHQjqSmZKaTcVKU+jPxOQTTsVvmpavCVvzflX9/1K/WonJ9QQ4ZLJb75yHOgPje+4zhe7ukjn2vmnFkt7MgOM62lDscRRKLPrT9XojmdiN43fr9iEPJy9wCzJ/t0D/TRl01xSkdTJbinEy5dfXlmT6qjb7jIzmyOaRPqGC6UWNOTJZN0SbjRkeOETAJQXtzWx2ChRL4YctrUFp56pY/nt3TjJrt5z/FvYebEevqGSnQ219E7WKB3oMh9yzYyfWICz/G54dw59A5Gn9OZ01u4Y8mzTEi1kHLTnDdnAho6TJtQz8s9/Ty0fhHiDtLmnsflp0xnU+8gIsJxk+pZt3OAvqECvuvSmPZ46MVu5k1pYFJ9koTn0LU3x9zJDbRkEmzYNUBLRvjV6qcIiy1MrptMT38ON7OWG86+CI8MhVJ0cy/fFeqTHp4rpH2XpOfwvyu7aWtK8fTa3aR9hxPaGzmuNUM+CGlK++zMFmhrTLJ8yx5+v20PIiWQIk5iD2GhlZOmtLBqe5Y5kxqpT/oM5AMmZpJs6etnby5L2ksRqtA7GH0u7Y0ZLj6pnec29rJh9yCFUsiaL/7DUQniVwGXqOrH4/k/B96qqp+uSrMyTrM1nl8Xp9k1Yl2fAD4B0NbWdva99947Lnk+kkKNglmgAQFB5ctbng4IKGqRJreJvcFehoIhPPHwxaekJXKaIx/mo51lvNMeaViHSUqS4XCYvEbdWYLg4BASkg/zlYBS0lJlB1QoFHB9l+FwmCaviVyYI9SQvqAPRxw8vMpOCiAX5io7d1UlJGQgGCDjZio7sqLGz2PGAYkbEHEg31HcQavXSokSgQY4OBS1SEj5evuQEiW6i9Hd9KYlphFogCsugtDoNpIP8xS1GAWqeD3l7S2r3sGW81LSEo5Ef0taigKxlm/A4lDQwj69Ak7VrRVCwsr2VRpAVesv6w+iscl6p56AoLKO8vsNhUO0++30BX2oKr74FHT/IzFBCAlxcPZLU64zDk5lO1NOioIWKGghCly4lTLLaW6fRty+7+OghGScDCEhw+Hotxktv89YvVOHyiOBRrV/v/cYa/3luvx61Tl1DIUHPjO+NHgcjakCRa+7UpePpIOVg3lzW/kXKw8YxMfz7PTRzvwaWQsPJQ2qegdwB0RH4uWuXjM+qrvTzfh5I5RzuVehrNwYG5mm3DhTjXoiysM+5SP8qt73SsOm0rCLey8UraSLzjgvXyPvknSTlWXFsIjruHgSDUGVe8B8168cnfuuH/XYuClyQY4gDHDE2a9H4Mknn+T8C86v5L8c9MMwRERwxcV1XNJemsHiYKVhUunRiHdHdW595Ul6Q6UhCsGrDaiRjZlyo3pk748jTqVcynktBIVKXkWkMtzmiEPCTUQ9Lm4qakTn+/Acb5/PaLT3GUkQkl6SXGnf8yOq05d7DEY2dKt7Ekb2LAQaUAgKLFmyhEsWXMJgcXCf+lIuB9/1ccQh1JBiUKyUq6rSkGigGBYJwgARoRTGDXlxyHgZhkvDFMJC5b33yf8o2zpyWflAQRCaU83kSjmKYZFSWGJieiLZQna/cnk9HHFIeSnyQZ62ujb6C/0EYXSAVtJSpaev3COU9tIUggIhIZ7joapRD1fV5wIwmwPfI2A8g/hWYFrV/FRg5PVU5TRb4+70JuDI37LJmGPUaEF71DQCLvufbyAi+DL2ZUvVAb1awt33+nuR6DwO3913fdXzvuPjO35lGiDt7X99flnSSZLxM2Pmr+yPSXeoaV+vpPvqtelTvCljpDy4xkTj683OqFq8FtJeeszP4bVqpvmwrq8hse8leuP9ObamW8d1/TC+t139HTBXRGaJSAK4Blg4Is1C4Pp4+irgsbHGw40xxhjzqnE7ElfVkoh8GvgN0SVmd6nqKhH5GvCcqi4Efgj8VETWEh2BXzNe+THGGGPebMb1jm2q+iDw4IhlX6mazgEfGs88GGOMMW9Wb9qnmBljjDFvdhbEjTHGmBplQdwYY4ypURbEjTHGmBplQdwYY4ypURbEjTHGmBplQdwYY4ypUeP6PPHxICI7gU1HOx9vcq3AroOmMq+XlfP4szIef1bG42+Gqk4a7YWaC+Jm/InIcwd6Yo45fKycx5+V8fizMj66rDvdGGOMqVEWxI0xxpgaZUHcjOaOo52BY4SV8/izMh5/VsZHkY2JG2OMMTXKjsSNMcaYGmVB/BgjItNEZJGIrBaRVSJyU7x8gog8IiKvxH9b4uUiIt8TkbUi8nsROevobkFtERFXRJaJyP/E87NEZGlczv8lIol4eTKeXxu/PvNo5rtWiEiziNwnIi/Fdfpcq8uHn4h8Jt5frBSRe0QkZXX5jcGC+LGnBPydqp4IvA34axE5CfgC8KiqzgUejecBLgXmxr+fAG4/8lmuaTcBq6vmvwF8Oy7nPcDH4uUfA/ao6hzg23E6c3DfBR5S1ROA04nK2uryYSQincCNwHxVPQVwgWuwuvyGYEH8GKOqXar6QjydJdrpdQLvA34cJ/sx8P54+n3ATzTyDNAsIu1HONs1SUSmAu8F7oznBbgIuC9OMrKcy+V/H/COOL05ABFpBC4EfgigqgVV7cPq8njwgLSIeEAd0IXV5TcEC+LHsLib60xgKdCmql0QBXpgcpysE9hS9W9b42Xm4L4D/D0QxvMTgT5VLcXz1WVZKef49b1xenNgs4GdwI/iIYs7RSSD1eXDSlW3Af8KbCYK3nuB57G6/IZgQfwYJSL1wH8Df6uq/WMlHWWZXdJwECJyGbBDVZ+vXjxKUj2E18zoPOAs4HZVPRMY5NWu89FYGb8G8TkF7wNmAR1AhmhoYiSry0eBBfFjkIj4RAH8blX9Rby4p9y1GP/dES/fCkyr+vepwPYjldca9nbgChHZCNxL1PX4HaIuXC9OU12WlXKOX28Ceo9khmvQVmCrqi6N5+8jCupWlw+vdwIbVHWnqhaBXwDnYXX5DcGC+DEmHpv6IbBaVb9V9dJC4Pp4+nrg/qrl18Vn9r4N2FvuqjQHpqpfVNWpqjqT6CSgx1T1WmARcFWcbGQ5l8v/qji9Hb2MQVW7gS0icny86B3AH7C6fLhtBt4mInXx/qNczlaX3wDsZi/HGBE5H3gSeJFXx2q/RDQu/nNgOtGX9kOq2ht/aW8DLgGGgBtU9bkjnvEaJiILgJtV9TIRmU10ZD4BWAZ8VFXzIpICfkp0jkIvcI2qrj9aea4VInIG0YmDCWA9cAPRwYnV5cNIRL4KXE10dcsy4ONEY99Wl48yC+LGGGNMjbLudGOMMaZGWRA3xhhjapQFcWOMMaZGWRA3xhhjapQFcWOMMaZGWRA35hgjIoGILK/6HesuZ3/sumeKyMrDtT5jzNi8gycxxrzJDKvqGUc7E8aY18+OxI0xAIjIRhH5hog8G//OiZfPEJFH42dwPyoi0+PlbSLySxFZEf+eF6/KFZEfxM+fflhE0kdto4x5k7MgbsyxJz2iO/3qqtf6VfUcojubfSdedhvRIzxPA+4Gvhcv/x7wW1U9neie5avi5XOB76vqyUAfcOU4b48xxyy7Y5sxxxgRGVDV+lGWbwQuUtX18UNyulV1oojsAtpVtRgv71LVVhHZCUxV1XzVOmYCj6jq3Hj+84CvqreO/5YZc+yxI3FjTDU9wPSB0owmXzUdYOfeGDNuLIgbY6pdXfV3STz9NNGT2ACuBZ6Kpx8FPgkgIq6INB6pTBpjItZCNubYkxaR5VXzD6lq+TKzpIgsJWrgfzhediNwl4h8DthJ9KQwgJuAO0TkY0RH3J8E7NGexhxBNiZujAEqY+LzVXXX0c6LMebQWHe6McYYU6PsSNwYY4ypUXYkbowxxtQoC+LGGGNMjbIgbowxxtQoC+LGGGNMjbIgbowxxtQoC+LGGGNMjfp/C7k0zuXWkO8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.DataFrame(history.history)[1:].plot(figsize=(8, 5))\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Cross-Entropy/Accuracy')\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model seems to have converged quite quickly! Let's assess its performance on the test set: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100/100 [==============================] - 0s 680us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.002217177735193554, 1.0]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(X_test_scaled, y_test) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which means the loss on the test set is very close to 0, and the accuracy is super! For prediction, we can obtain both the probabilities and the classes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.]], dtype=float32)"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predict probability of class 1 for the first unit of the test set:\n",
    "y_proba = model.predict(X_test_scaled[1].reshape(1,-1))\n",
    "y_proba.round(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1]])"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predict class for the first unit of the test set:\n",
    "y_pred = model.predict_classes(X_test_scaled[1].reshape(1,-1))\n",
    "y_pred "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Last, as we are in a two-dimensional case, let's plot the model decision boundary: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUwAAAFBCAYAAAAc8JaBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2de3gV5bX/v2vvJCTZ4bq5aAWSoscLKkUJ8tQLPxRrbeR4bXu0KXqEFg1qa5+2xwtPa21L66XVcmxPCz2KQkKtB4tUxXsVi1YhWES8AQWCRAQJN0OAhOz1+2Myyc7OXN657MzM3uvzPPME9p7Lmpk931nvWut9X2JmCIIgCPbEgjZAEAQhKohgCoIgKCKCKQiCoIgIpiAIgiIimIIgCIqIYAqCICgSmGASUTERrSSit4noXSK6MyhbBEEQVKCg6jCJiAAkmLmZiAoBrADwXWZ+IxCDBEEQbCgI6sCsKXVzx38LOxapohcEIbQEGsMkojgRrQGwE8ALzPxmkPYIgiBYEZiHCQDM3A5gLBENALCEiE5h5nXp6xDRDAAzACCRSIw78cQTA7BUEIRcZvXq1buYeYjdeoHFMDMhojsAHGDmX5mtU1lZyfX19b1olSAI+QARrWbmSrv1gsySD+nwLEFEJQDOB/BBUPYIgiDYEWST/GgAjxBRHJpwP8bMTwVojyAIgiVBZsnXAjgtqOMLgiA4RXr6CIIgKCKCKQiCoIgIpiAIgiIimIIgCIqIYAqCICgigikIgqCICKYgCIIiIpiCIAiKiGAKgiAoIoIpCIKgiAimIAiCIiKYgiAIiohgCoIgKCKCKQiCoIgIpiAIgiIimIIgCIqIYAqCICgigikIgqCICKYgCIIiIpiCIAiKiGAKgiAoIoIpCIKgiAimIAiCIiKYgiAIiohgCoIgKCKCKQiCoIgIpiAIgiIimIIgCIqIYAqCICgigikIgqCICKYgCIIiIpiCIAiKiGAKgiAoIoIpCIKgiAimIAiCIiKYgiAIigQmmEQ0goheJqL3iehdIvpuULYIgiCoUBDgsY8A+D4zv0VEfQGsJqIXmPm9AG0SBEEwJTAPk5m3M/NbHf/+DMD7AI4Jyh5BEAQ7QhHDJKIKAKcBeDNYS4Rcpq4OqKgAYjHtb11d0BYJUSPIJjkAgIjKADwO4GZm3m/w/QwAMwBg5MiRvWydkCvU1QEzZgAtLdr/Gxq0/wNAdXVwdgnRgpg5uIMTFQJ4CsBzzHyf3fqVlZVcX1+ffcOEnKOiQhPJTMrLgS1betsaIWwQ0WpmrrRbL8gsOQF4EMD7KmIpCF7YutXZ54JgRJAxzLMATAVwHhGt6ViqArRHyGHMojkS5RGcEGSWfAUzEzOPYeaxHcuyoOwRcpvZs4HS0u6flZZqn6swcyZQUAAQaX9nzvTfRiH8hCJLLgjZproamDdPi1kSaX/nzVNL+MycCfz+90B7u/b/9nbt/yKa+YcIpuA7YS3fqa7WEjyplPZXNTs+b56zz4XcRQRT8BW9fKehAWDuKt9xIppBCq7RsXXPMhOzz4XcRQRT8JVZs7pqHXVaWrTPderqgMGDtaYxkfZvXRT9EFy3GB172jTz9eNxZ/sOo9ctOISZI7OMGzeOhXBDxKzJTfeFSPu+tpa5qKjn94WF2nfl5cbbl5e7s0ffJ5H2t7bW/Ltk0vjYZktNjboNpaXdty0t7W6LECwA6llBgwIXQSeLCGb4sRM8s+/1dewE1wwjYayt1YTYSJiNRCwbYqlyTYTgEcEUsoad12blTVmJkL4/o++SSefHTCTMj+NWKNOXeLxL+Ky8RbcvAaH3EMEUHGElgpnr2TUvrfali4yZx2W0/6Kinp5i+jGtvNbeWqya2OJhhh8RTEEZJzE2rw+/legkk5rIJpNd/7aKLerH9Mtj9Op9pl+D9JdGMmkt+Fb3ReUlJnhHBFNQxokIOm1euk2spAuK3THN7I/FnIve5Mk9Xx6qIpqe2DLyktNfAipiKYmi3kMEU1DGSpBUBc9IXI0e+sLCnllys+PbJYr0783EpabGOCNv5yWaeXZ2TX9Ve1WQZnzvIoIpKGOVaHEaT1Tdb7ogufHYVOOmdk1js2MaYZVZd+IRqzS1rbxas/OTZrt7RDAFZcwEycybtMpYp6P60Kt4U34Jg1Wtp4oHl769WZbc6nxUm+t23qzuQUuz3R9EMPMUt8JitJ2V4KnsXyV7nUz2/oNvVJ+pC5cfx7TyiFWuiZkYZi5mFQfSbHeOCGYe4neiQOXhJjIv4lYtDtcForeallahAr8we3GpJpB0MbQqw7K6J4IzRDDzkGx0K1QRPD05ZLYPFeHtTa8oyEJyNzWjZvaKh+kfqoIpg2/kEH5Pw6CPIZlMWq/H3H1wjcx9bNmijT9pRUND7w1I4ffo604G1jAayNgOZm2QknRKS7WBQrwMiiy4QEVVw7KIh2lNNkpRnHiZXvfTWwkLP0MXbvZVW+t8oA/9PlpVAUiW3D2QJnn+kY1iZ9UmpIooqwhFbzUn/RIaNy8po/ukN7ulmR0MIph5it8eh2qSwslxamvN9xO1hIWbeKjTsiMpFco+qoIpMcwcw+00DGaoxPWSSWfHqa42j2lGbRZH1XhoepzTaH50QIs1e5l7SMg+IpiCJXZJiqIiYM4cf/YbxYSFynlkjuRuhi6yfr/0BP8QwYw42Z76IN3jMcJKAFT3G2VPSuU8jKbtyCSKL4u8RKXdHpZFYpjd8To2pVNkQAh3uO0xZdUNU7Lj/gLFGCaxWxchACorK7m+vj5oM0JDRYVxPKy8XGvK6U3BdO+mtNS9JxeLGXuURFrz0SuHDh3C/v37cfDgQRw6dKjzbyqVAjN3/iUiFBYWdltKSkqQSCSQSCRQUlICyixcDBC7+2SE0b3TKSrS7kNbW9dnRNpn5eWapxo1Tz1oiGg1M1farieCGV3sBMzNg2qFX/vbtWsXli5dijfffBPbtm1DY2MjGhsb0dTU5NwoA4gIiUQCgwYNQjKZ7FxGjBiByspKnHHGGaioqOg1UXXz4jK71ip4eSnmKyKYeYCdgPntEXrxWHfs2IElS5bg8ccfx8svv4x2g0m9CwoKMGDAAJSUlKC4uLjzbywWQywWAxGBiMDMaGtr61xaW1tx8OBBtLS04MCBAzh06JDtuQwePBjjx4/HxRdfjKlTpyKRSDi9HI6oq9NimVu3askdOy/Q7N6p4valmK+IYOYBdgLmt4epH1P1wWdmrFixAn/84x+xaNGiTpEsKCjA5MmTUVVVhYqKCgwfPhzHHHMMhgwZgljMex6yvb0dzc3N2L17N5qamtDU1IRdu3Zhw4YNWLVqFVatWoVPP/20c/0BAwZg2rRpuO6663D88cd7Pr4fePEwAf/CJPmCqmAGnshxskjSpydeZnDMFtu2beN77rmHjz/+eAbAADgej/OUKVN4/vz53NTUlF0DbEilUrx582ZesGABf/GLX+y0EQBPnDiRb7vtNl60aBGvXbuWDx8+HIiNVl1JjQZx9iMRl8+JJEjSJzdx2rRzur5T2tvb8Y9//ANvvvlm57I1bbSPo48+Gtdeey2mT5+OUaNG+XdgH6mvr8fcuXOxaNEitGRkWQoKCjBy5EgMHDgQAwcOxIABAzBw4ED07dsXiUQCZWVlKCsrw4ABAzB69GicdNJJ6NOnjy926feuoQGIx4H29q6kDtD1nZ7w0XETw/Q7QRg1xMPMQaw8xiC8g/379/PEiRO7eWgAuKysjC+55BJ+8sknua2tzX5HIXFt9u7dy4899hj/6Ec/4ssuu4yPO+44JqIe52e1FBQU8Mknn8xXXXUV19XVcSqVyrrdfly+fC8Zg3iYuYdZXCuZBA4e7F3voLm5GV/5ylewYsUKDB06FJdeeikmTJiACRMm4MQTT0Q8HlfbUchdm5aWFjQ2NmLv3r3Ys2cP9uzZg71796K5ubnbsnPnTqxbtw4bNmxA+jN1/fXX47//+79RWFgY4FnYk+2SsbAjHmaE8Tpad2/EsWbMmMEAePjw4bxhwwZX58nMOefaHDhwgFeuXMm/+tWvuE+fPgyAJ02axDt27AjaNEty7DY4BjJaUTTxOh+Ml5F/VJNEu3fv5pKSEgbAa9eu9XbCQQ5/nmVef/11PuqoozpfLCtXrgzaJFPyfZQkEcyI4mboLydzhbs9djr3338/A+AvfelL2T3hbNMLsdPGxkY+88wzGQAXFRVxbZYVSPWUjNYLSSg5EEQwI4qbOa398g5UnL1UKtVZLrRkyRLvJxyUa9OLxz18+DDPnDmTATAR8dy5cx2bqiqCKqeU796kESKYEcWtw9VbmdJnnnmms4mplAFXIQjXJgDP9u6772ZAy6b/5je/UdrGibipnlK+xyuNiIRgAngIwE4A61TWzwfBDPLtb3fszZs385AhQxgAz549O/sGZZOAYqcPPPBAp2iqXEMn4qZ6SjkcNnZNVARzIoDTRTC7E2QsyezYqVSKzz//fAbAF1xwQXZ6wPTGidtlz3rBzfrf//3fzvrO22+/3bJW04m4efUw/ZyXPWpEQjA1O1EhgmlP0AH5Rx55hAHwoEGD/CuRST+pZLJnfz8nrrXKBbKbutLL1JEOb05dXR3H43EGwDfffLOpaDrxMGtqjNedPLm7eTU1xl0ri4ryN44pgplDBN3DZ+fOnZxMJhkAP/zww9536GSe2Uxl8JL1sjqm24vnIYayZMkSLiwsZAB83XXXcXt7u9LuCwu1U8m852bimumllpYyl5WpC3E+kDOCCWAGgHoA9SNHjszCpQo/Vk2o3oh3Tps2jQHw5MmTvXf1U53oPH2x2la1ripbU1V6zKAsW7aMi4uLGQBPnTrVMJGW6YgXFRnfc6cdG/y+FFEmZwQzfclXD9PpgxCP++dxvvPOOxyLxbigoIDXr1/v/WTcVN/buVAqT3+24pY+ZFBeeuklTiQSDIC/+tWvWsaHrfTZ6eURD7MLEcwcwuuDkEy6F84pU6YwAL7hhhv8ORk3bpD+FHvpG2q1nlGhYrrXanUBfarRee2117hfv34MgKdMmcIHDx50dPn090qmA262fizW87N8rsWMhGAC+BOA7QDaAGwDMN1q/XwVTKctUaPFzcOwfPlyBrTRhz755BN/TsaN+uvemtvYhFVzPBbrbl9tbc82r5VwmmVaamocX5rVq1d3xorPP/98bm5uVr58uj5nhnhratQiIF5eqrlAJATT6ZKvgsmsnutQdbjsaG9v5wkTJjAA/slPfuLviTiNYaargdG2yaSmDEbZL5XjpdumT89oJd66TVYlSi7btu+88w4PGzaMAfDZZ5/N+/bts718di9DlRxb+ulI10gRzJzByIPQ/6/yrKvy6KOPMgAeNmwY79+/37uhmUPBG6m/0VNtNG+wyno6dh6tnRhbLVbre8iefPjhhzx8+HAGwOPHj+8xQr0bYVNx7PO5u6QIZg5i513YPfOqTk9bWxufcMIJDMBxv2clQ+22tVMDP7q/ZNrkNlBstn+PVeCbN2/mz3/+8wyAx4wZ47n21S78a/ayzZckkAhmDqKiE04dMCP0IvVRo0Zxa2urv4Y69TyNsHr6VYsT4/HuhaxuxNJq8aHbzEcffdQ50MmJJ57I27Ztc70vq1PMkqMcKUQwcxAnFSxu41Gtra08atQo9lSkbufVGXloVgWGmai2L+2K2mtr7WcTc7v4pDSffPIJn3rqqZ0vsM2bN7vaj1XrI5EwPw3xMEUwI4vZDzsW8y/W9Nvf/pYB8PHHH+9+NCIrr86p8Bg9sarxRrPUsX6xnJQZGC16l5ssK82uXbu4srKS9VGi3NbDpjvTKhVa6SWwuY4IZo5hVRlj5Yw5oaGhgcvKyhgAP/74496MVS0IdOuppYugnWiaudlW2xn1bTd6UxnV7WQhW7J3714+66yzGAAfddRRvG7dOtf7cvKeyBdEMHMMlVaoF6cmlUrxhRdeyAD48ssv926wU3fGy0k56USdLmR2Qp1M2qtLaal5SZOT66SwbXNzM5933nkMgJPJJK9evVr9OGmH8/PS5woimDmGahMqE9Xn8aGHHmIAPHDgQN6+fbt/hntJqDjJrKt6tOkqoOJqqfQQcKssVnWlJufd0tLCVVVVDID79+/Pr7/+uqNDqt6OfCopYhbBzDnceJiq1T3Lly/noqIi9m00onSceJexmPEwPEZkjkiRPvyOlcClv1WsevVkCphqKtlJts3qphYVmV6Lw4cP8xVXXMEAOJFI8Msvv+zL7VC99LmICGZI8Kv3hF2ew0gIVcsVzzjjDAbAN910U/fRiNwa77ZUx6gUx00XJ5URjNJtVKn4r6017oCduU8n8UwnL5OM/bS1tXF1dTUD4OLiYn722WeVbo1V71JVcrFHkAhmCPBSv222v3Snys4jUClDWr9+Pev9xQ8cOODdeDc9ZowMs7JBpSlt1b/c6X71C2zkjRYW2td8mjXZnb5UMvZz5MgR/va3v836jJRPPPGE9b3xcFv92j6siGCGAJ+7GWfl+HfeeScD4Kuvvtof473ELDP37WVfuldo5Aq5GcRDxTWz8hiNFMXpy8UgSJ1Kpfg73/kOA+CCggJevHix9f1hbx5i0L/pbCGCGQJUPLxsNm/svIH0KXOfe+4558Y7OWmj/di5KdnKrtuNkWZ0Q1SCf1ZNe7Mh8mtq3I8+z1338b/+678YAMfjcf7zn/9sfY88kKsTqIlghgC7t7FdJxTVIRmtsBLkv/3tbwyAhw4d2rNI3akr4SZu6bbPuN1i10Z04yb50X3SLkyQHm9xOL9RKpXi22+/nQFwLBbjFStWmJ+LB8TDDIEQqi5RE0w7D8+qlZftSar27t3L5eXlDIDvuOMO58bbrevSW3K1X5WArttz0zEb99KPxaw3k8OmRyqV4ptvvpkB8MSJE71PJ2KAxDBDIISqS9QEk9n6d+9l8HGv6BnWcePGmU+JoDpQhlVT1OvQ3naeq9u2oFNBysYAHemLT4qzb98+HjhwIAPgF1980Zd9ZiJZ8ogsURRMK9w8g37EiubOncsAuLS0lD/44AP7DYzibiqen26sH09Y0G1BrzOM2WX3fXTTfvGLXzAAPvPMM7PiZeYiIpgRwE3VjD4qmVtefPHFzvmwlYrUvfQL96mpaWpHb7YFvXiY6UPJ+TFgqQ379+/nwYMHMwB+5plnfNlnriOCGRHM6rKtxn3wohMXXHABA+Af/vCHahu4FYrMybNraoyHW3LTPA+iLehXfaldZ26fzunHP/4xA+Arr7zSl/3lOiKYESczS+6XM3LaaacxAPWBG5w0RfX5fY3GtnTqiYaRTMG2GkjS6vysXkI+ec1/+ctfGABfcsklnveVD6gKZgxCKKmuBnbtAoiMv9+61d1+9+/fDwDo27ev2gYjR6qtV1oKPPIIkEoBZWVAa6u6UW5Ppreprga2bNHOccsWYO5coKjIehsiYPbs7p/Nnq1dLyNaWoBZszybSmY/HIfU1QEVFUAspv2tq/Nlt5FFBLOXcfoDNNMrVR3LRBfMfv36Wa+oG9rQYK7aOuXlwLx5mqDU1WnbOEH1ZML29FZXAw89pJ2/GczaepnbzZtnvo2PLxDNeXJHXR0wY4Z2O5m1vzNmBH/ZA0XFDQ3LEvUmuZu8hd+5jj59+jAAbmlpcXZQuyanSlLDLL6ncjJBJ33scJLFtyuV8iFE8cQTTzAA/vd//3fX+wi6MKE3gcQww4fbH6BfuY7W1lYGtJ4gluUmbhI9qoNiZC41NWrGh/3ptaomSL9pboadcsHSpUs9C2audoM0QgQzhAT9A9yzZw8D4L59+7oz1M/F6UREQV88FdI9R7O+8lYvI6v+rw7fmrpgTpkyxfXphP0d5SeqgikxzF7E73ikUw4cOAAASCQS1iuaGZRMWsfrzMiMgZaWAgsW9IztubGpty6eCnpSqLxc05Z09GSOVXy3rMz4mrgIJvqR9DHKTZWW9sxh5RMimFkkM0dRVWWcHG1u7p1AektLCwAFwTR7Ur7+decHLS0Frr9eExGi7gkiJ0Tp6TVL2mzdCsTjzrebNUsT3HQUs+mcKdw2pP9mZ80CrrnG+63LKVTc0LAsUWqSm+UozEbz6o38xT//+U8GwKeeeqraCbjpDgl01WNmY7y6KHRitmrLWl03s7au2foW4Ygnn3ySAXBVVZWy2WHPq2UTSAwzWKyemaBiQ//4xz8YAJ9xxhnON3aSCApTXDEIrJTHLoapv3D0H0RNjdqEbhk89dRTDIC/8pWv2JpqN4ZKLsYsM1EVTGmSZwmrVpnVd9ksNTx48CAAoKSkxPnGTmoDmcNRJxkUep2lUVvWrGg9HgeamrR/t7drfxsagD/8oWc8FDAuiO/2tRbDZKNtO8gMjeqHzaShIX9vZSYimFnCKkdh9h0R8M1vZq9Q+NChQwCA4uJi5xsPGuRs/Xyvcs7sFaQH/ozENJk0VyszwWO2DCaqCKZRaNSMa68FBg8OT5+BoBDBzBJWOQozJyOV6vmZTz3lALgUTN3l1b0fJ/hpfC6RKaa7dzvfh021gkqW3Emjoa1N+wlk40UeJUQws4RVq0z/LplU25dfPeUcC2Z6m80tUeknHiR2pVFGZVmK1QFWHqaXiqx8fReKYGYRs1aZTkdI0Ra/Sg0PHz4MAOjTp4/aBk7abGaEqU4yrFgNxuGyLEulSW7WCurtF3mUEMEMCFUt8rPU0LFgen0iwlon2VuYZfAyPweMmxzJpPb5//yP9ZvXAJUmuVkraM4cc/1OJx/fhSKYAaGiRfrz4lehsKlgmj3YXp6IeDy/q5zNeufMnGn8+Wuv9WxyqDZBLLDyMAHjVlCmkCaTPevtCwvz811oKZhE1I+IjjX4fIwfByeiC4noQyLaSES3+rHPqGClRURATY02HqafetPaMUZlUfoYjlbd7mbPth/azYxUKn/FEjDvnfP73xt/Pm+e6948Rqg0ya1IF9I5c3oKpk/DbUYOU8Ekoq8D+ADA40T0LhGNT/v6Ya8HJqI4gN8B+AqA0QCuIqLRXvcbFczCVskksHCh1grzG10wu3mYVt3uqqu1+JmbpyMf22vpOA1nmJUVuQyL+DWAMKD9FDLHg25tlaRPJrcDGMfMYwFcC2AhEV3e8Z0fd+MMABuZeRMztwJ4FMAlPuw3EhjFj2prNa8SyE7xui6YhYWFXR9aVdEDwFlnOa/BzPfYJeD8hREzeRQ9vnjcepjp2P1E8gkrwYwz83YAYOaVAM4FMIuIvgPA+10AjgHwUdr/t3V8ltdkc5Rrwya5VYW9boyTGky/A69RxSrzbYRREW5RkesXj9cmeTpRGCiqt7ASzM/S45cd4jkJmhd4sg/HNvJSe9xdIppBRPVEVP/pp5/6cNhwYCaM3/2ur6GsbrS1tQHIEEyrCnsjY8xId5HzXSx13HRBTadvX9fXMmUkwC6J0kBR2cZKMGsAxNLjisz8GYALAXzLh2NvAzAi7f/DAXycuRIzz2PmSmauHDJkiA+HDQdmoUMzZ86P5o9hk9ystgRQ8yxLSzWhVCx3yQvceOZGeNh+1apVAIDPfe5z3myAdSeMfKPA7AtmfhsAiGgdES0EcA+A4o6/lQAWejz2KgD/RkSfB9AI4EoA3/C4z8jgVAD9aP7oHmY3wQS6aknS0esDjYjHtSbkyJFdbkZFhXZS+mf5+DTp+FHwD1iPnWnDX/7yFwDA192MYWqA0U8kH1Gpw5wAzRN8HZrIfQzgLK8HZuYjAG4E8ByA9wE8xszvet1vVHAqgFVV3o9pGMM0w0rRZ8zQTmDrVq3ZPm2aTC2Yjl/ZELPMuQ0NDQ1YvXo1EokEvvSlL/ljiwFhm8SzN1ARzDYABwGUQPMwNzOzLwESZl7GzMcz87HMnFcREac5gWXLvB/T1MM0wkzREwlt/nFdIJuaetacpAdd8/Gp8isbotpHMYMlS5YAAKqqqtwN5Qf725avU/CqCOYqaII5HsDZ0OolF2fVqjzArwE4nOiRI8E0i/QXF6s1N/XBPfPxqTK7drW1vVLx/fTTTwMALrvsMlfbq9w2D7NmRBu7EYYBVBp8NlVldGK/lyiNuK6K6kDmyWTPbWtrmYuKuq9XVGQ+pcAVV1zBAPixxx5TMy59OO5k0tk0ukEOLR8GzKbT6IWR64cMGcIAuKGhwdX2KrctCpN4OgEyRUU0UJ3R1kgIzfTLSFyZmS+++GIGwEuWLHFmpN1c2pmLPh1Drj1VfmA1f7kPL5YdO3YwoE2lbDn3vAUqty3X3oWqgimDbwSMarjLqCuaWdWJ2eeOmuTpqGR9y8p61pyYnZzTnkNRIzNOMnOm9VSM11/vW6Hju+9qedPRo0e77h6pUqiet7WZKqoaliWqHqbVZIdOnLdMx8xqXSPOP/98BsDPP/+8sxNQcYONphesrWUuLFRzl3MFs3NWuVY+zIj5wAMPMACePn26p1Mwm/E0cyLRKEziqQKkSR4OVKYuzXxWzJramc0dp03yyZMnuxNM1bibUXtM9WRyBdU4bzKZFbX53ve+xwD4rrvu8rQflVmWc2kKXlXBlCZ5llHJJmaOSWg0gKtRc2fOHG1cwnQKC7XPjSjt2OlBp+MsqtZAGU0vaDZfTa6O3KDaO6epqXsaeupUrYnusfRq586dAICjjjrK9T6Anr/JZcvyNCuegQhmljHThYYG83Ig1a5o1dXA/Pnd15s/37xHhl6T1+K0F4pukErPk8z6Exm5QQ1m7a/H0itdMIcNG+aXZQCsf8f5VGIbeDPbyRLFJrlZazYzLNgbzZtrrrmGAfBDDz3kbgeqKf14vOtkVGISuYST0iun4Q0FxowZwwB49erVvp6WSlQmyrcV0iQPB0atWaIuh0KnN5o3epP8wIED7nag6hW2t3d5Sbk6coNZj4E5c7Rh2bziMmSxY8cOAMDQoUO925CGSlQmL5roKqoaliWKHiZzzwC62Rs62+WJt912GwPgn//85+524LQeM1cTO3Zes2oWz+dr19zczAC4sLCQjxw54ucZM7N2WnanEtUSW4iHGR4yA+jl5cbrZTusN3DgQADAbrNEjB1OYpmAtZcU5T7mdpk8lSxeQYF5N0mXBY2bNm0CAIwaNQpxDyMdmVFdrZXbWnw3z+QAABx7SURBVJHroWkRzAAIquh3UEfB+J49e9zvpLraeHRwI8yenqj3MVeds0F/KUydqg0mnEx2TcOYGZfRxdNDyGLjxo0AgOOOO87xtqpYvQPzoXBdBDMAggrr6YLp2sPUUXEjrJ6ebIzc0Jseq0rmP/Ol0NSkTZu7cKHmpnX0uuqEWfsheBiIecOGDQCyK5hmp543syqrtNvDskQ1hhkWVqxYwQC4srLS2YZGVcx28TerdKldZ2WnvV7M+mfX1Dg7T1VUMv9Wna2z0Me+ubmZKyoqGAA/8sgjnk/RjFwteoD09BEy2bVrFwPgsrIy9YEZzJ6Q4mJzwUwmrcXOTEzicePP7Z5Iq9otN0+yimDbrWOVFcnCyBXf//73GQCPHTuW29raXO9HBZ96cYYKEcyQEZYf2eDBgxkAf/TRR2obmD3cqplfI0/PabbdTkys6kOdipAfLpTVSE36zffRTXvjjTc4FotxLBbjVatW2ZpmNa5B5ndh+d1mGxHMEBGmZsw555zDAPiFF15Q28Cq+agqdkaeXvqTaOZZqjZX/azVcuv9qZxP+nXwSYkOHDjAxx9/PAPgH/zgB7Ymmv0OVd9hudD8NkIEM0SEaezAGTNmMAC++eab1Tbw6mF69RDTj2cmMnYenRPcxBedeMw+89Of/pQB8OjRo/ngwYOW61r9Dp2Ma5yL5bUimCEiTOPorlixgmOxGAPgp556yn4DM7fEqWC69RD1JZGwdtNrarz3N62tNfcOdZUwEm0vozllHt+B19na2spHH300A+AXX3zR9vSsfodOGwy5hghmiLB6swcRI/rlL3/JAHjYsGG8a9cu+w2MjHTyhAGaEJmdpJuYppEIebmYVjZYtVudDGaaftNVjm8j+IsXL2YAfNJJJykl8cTDNEcEM0RYDcjq5BnxS1zb29s7Y5nV1dXuduLkCTMTIKOTc7ovv9wdq8y93Xw8Zl6p/rmK5+sibqOPbzpnzhylU5QYpjkimCHDSUvO6BnxO3G0YcMGLikpYQC8dOlSdydkVvuokgAxO0mjpn5hIXMs5lhQHKESN7Hyqo1GWTez2chuh3GbtWvXMgAuLS3lPXv2KJ+mXZY8kTA2QTc5F8WSWQQzEjh5RrKROLrvvvsYAH/xi19U28CogN3O5bUr+bEaBg7QnuDMqTHT3xiTJ3eJcjzuvlhd5QJbJcDMbDRbMm+ygxu8ceNGHjFiBAPg6667zt35GtDb9f9hQgQzAli18DI1KBuJo+bmZk4kEgyA169fb72yWxfXrpmt78NNc3fyZOPvnDzh6ce2azr7lQDTRdbh9U2lUjx//nzu168fA+CzzjqL9+/fr36uNjj5PRpdwijXaopgRgCVuJGdnnhtkeqDCv/oRz+yXtFLfaLdSVp1F7RarMRUBTOXShc0ox5L6TdDpX7Uynar2tQM5fn44495ypQpDIAB8KWXXuqrWDK7m+suTDXGXhDBjAiq4T4nP0wnb/y//e1vDIArKiq4vb3dfEW3Lq5ZXDJzH04TPnZipYLZMY1ij5njXbrN6qcvZrPVZfDoo4/yoEGDGAD379+fFyxY4HrOcTeXw8rsMNUYe0EEM4L4MSaF0zd+e3t7Zzxs0aJF5sa5eTJUhUU/mcw4oF1c0Eo00y+mWdbCjcB5qQ5wKOzvvPMOX3DBBax7lV/+8pfVu7S6wMl7IJuhoiAQwYwgfryt3ezjd7/7HesZ1/fee894JTdtLxVx0ecor63tmWkuLGQuKzMXL7tRk4yWdJu9NKmzKJg7duzg6667rrODQf/+/Xnu3LlZ8SozUe2xqv+exMMM8ZLrgulHPMjNGz+VSnF1dTUD4LPPPtu8ae40uq8SFNP7V5s9eWb7SCS0Y9TUdM+SqxxTf5qzKYTpN05xAvn29nb+9a9/3ZnUicfjfOONN/Knn35qc9edoXoba2utb5u+jsQwQ7rkumAye884un3j7969m4cNG8YAeO7cue6MVzXGSFzciJLRxXHi2fndvNaXzISOWbghbZ09e/Z0S+pcdNFF5t6+Ika/JacCZ6b1fnWwCgsimHlE+g82mezZslV94//5z39mvQnY2Njoj2GqYmjXW0b1LaA68pGT7i16xtyLkFuoytq1a/nYY49lADxw4ED+61//mpVLb1UBZfZCzRUP0g4RzBzAbZKnoKCrZeqkljuVSvFFF13EAHjChAm2o984OgldqKyExqz/qJXwZaIa19Sb75nlQ2bF+E7E1QGLFy/urIU97bTTeNOmTR4vuIZT59luMKaoe5B2iGBGHNU3u8qD4cQj2LFjB48cOZIB8NSpU/1NNNTW2tdOZdY4lpcb99ezcouMRi7y4wJluvKZzWwHF3rTpk08ffp01pvg3/zmN7mlpUXtOirgtKxVLwDIZVG0QgQz4qjGIlUfDCdZy3/+859cWlrKAPiuu+7y8azY/k1g9H1RkfM4g9PBPNykdV24Xhs2bOBrr72W4/E464md++67z/cMuFUvTj8ub64hghlxVLPdqprgtC7u8ccf7/R+Zs2a5b+naSY0Vk+61agRmX3cnSaSzC6QT+3R3bt389VXX91ZKhSPx/maa67hDz/80NX+7LB6L2WektO4Zi4SasEE8DUA7wJIAahU3S6fBFPVw3RSG+6UuXPndj7gVVVV/iSC7HBaF2WUgbZanI6e5DHjkUql+IknnuDy8nIGwAUFBTx9+nTeuHGju+vjAFWtz5Xicy+EXTBPAnACgFdEMI1x2xXSS5bciKeffpoHDBjAAHjAgAG8cOHC7BZQO62LcpK91rPjqhfWY1X2v/71r84kGgA+/fTTe0UonZIrxedeCLVgdh5cBNMSt61Bv7OajY2NXFVV1e3B/8Mf/uD74A/M7Nyrc9Ls1hUgy67XkSNHePbs2dynTx/Wy7QeeOCBrE9/65Z8KR2yQgRT8JVUKsUPPfRQ5yAQADiRSPC3vvUtrq+v9/dgmS6z1TznTgTT6UglLlyv7du383nnndd5jaZOncqffPKJ50uSbfKhdMiKwAUTwIsA1hksl6StYyuYAGYAqAdQP3LkyOxdsQjhRE/85uDBg1xbW8sTJ07sFAUAfOutt/paFsPMaq6PapM8FnO+f4eu1yuvvNLZW2ro0KH8/PPP+3s9skC+C6VO4IKpdHDxMB1jl+TpzabU+++/zzfddFNnYmjEiBFcW1trPUycE1Q8PKNBO4wWo+p91f0rKMrcuXO5oKCAAfCkSZP4448/9ucaeMTKfGmKdyGCmaOolBH1drD+1Vdf5bFjx3Z6m+PHj+fly5d737FqDDFTFVSnrfCYHk6lUvzKK6906wP+/e9/n48cOeLptJ1iJop2gijJni5CLZgALgOwDcBhADsAPKeynQim+gBARmSz+XXkyBGeP39+5zzZuqf10ksvuc+qZ/uJdrn/1tZWXrRoEY8bN67zXEtKSnj+/Pn+2KVIba1xREJ1lH4pJ+oi1ILpdhHBdO9hGnkbhYX+xz8/++wzvvPOOztLkQDwmWeeyc8++6xz4cx2m9GsV5HFRVm5ciVXVFR0ntuQIUP4Jz/5Ce/cudMfmzyYnvkbsJt/LiwtlDAggpmj2D0ohYXOSgqzpUV79+7ln//8592y6ueccw6/8sorznbkt1ts1CtIoYj1yJEjfN9993VOTXzCCSfwvHnzfE90qZ6u3f10M+uH37+BKCGCmcNYjWFhNlhONvqcq7B//36+++67uwnnueeey6+++qq/B1LBZVDv8NFH8/jx4zvtnz59Ore2tva6eenY3U9dbFXyYZnb5CMimDmO0/iT2z7nfjl4+/bt45/97GfdmuoTJ07k2bNn8/Lly/0vSTLCZVCvvcPe4cOH81NPPRWYeSrrZoqsatVVPsYt0xHBzHGc5ivc9DnPRghxz549fMcdd3ROw6AvRUVFfOaZZ/Ldd9/Ne/fudX8AK+zeMiYXdTPAN9xwA+/bty87dimal47Z/Uwmu9+foFoWUUMEM8dxI2ZOh3PMZpJ69+7d/Kc//YlvuOEGHjNmDBNRp3j269ePb7nlFt6+fbv3A6Vjd0IGF7UZpXx9v/m90lR18xK08/69xK7zqahdBDMP8PqDttveyjvx+yHas2cPL1myhCdNmtQpnH369OHLL7+cn3zyyexNmUHUWafZ3t7Oj116KW9GKbeDeDPK+SrU+uJZuzVP5bhW99FsAPpEwt/pmqOOCKbgGVXvxGxWB7e88cYbfNlll3VrsgPgo446iquqqviHP/wh33PPPfzggw/y0qVL+bXXXuN169ZxQ0MDNzU1WSdkDEZjby0s5PvHj+fBgwd3HGuLa8862y8xo/WzUZyeb0XtIpiCZ1TjnpmeqF+eyLvvvsu/+tWv+Nxzz+X+/fv3EFCrpbCwkPv06cN9+vTh4uJiLikp4ZKSEm4wcZs3d2xXXl7ORCnT83R6vbLtlZkJmz7estU9syLfitpVBZO0daNBZWUl19fXB21GXlFXB8yaBWzdqj0yqpSXA1u2+GcHM2Pz5s1YvXo1Nm7ciKamJuzatatz2b9/Pz777DM0Nzfjs88+Q3t7u+F+2gHEjPYP4F8bNuDYY4/F5z9PaGhwfk4VFXC1nReI3G0XxnMJEiJazcyVtiuqqGpYFvEwg8VJIXSQnkgqleJDhw5xS0sLt7S08IEDB/jAgQPc3NzM7SNGGBuc1tZ06yn67ZWpNM9VZhU2CqN4bernGpAmuaCCk5iZWc7ETawrsAysohK4sc9J3M9u/6qC5VQsgxyIOsyIYAq2uPEiamq6DwQ0ebK78qZAvZcsKYHqeXkZ5jNTfJ14/bmasPEDEUzBFjd1f5m1m0VFzrPkfnpiqvSWt+SlNjK9HFQ11GE2qIqHKdPzEhFMwZD0B1r1odQx83rM+q+b4WSYSz880cA92gxcdjhy9FLJp+a0H4hg5jFmD4ub7pHpWG3jBFUP069awLDVFJrZE49r98jqZSbClx1EMPMUK2/Kazc5vwTTzuOzs9Vp1jlsNYVWL67SUv88eUEdEcw8xcqbctvV0c4zdfMge/GCrTxDo/2GzcPU7TS7H8lkuEII+YAIZp5i5U25FQ6rIcLMBix2i50XbCUcZp5rTU34BMisj3d601tikL2HCGaeYiWKbkc4snuw/cRuWgWr49mde1iy5HZxSiIRyN5GBDNPUY0Pei0BylaT1kvzOehYpeoLSSWWLDWTvYsIZh7jpzdll7H141jp+zAapzNtBDZLgo5Vqh7fy8yfQnYQwRS64VbYrDK2ftQ3Gu0jFut5vDD1fza7lqoerniY4UMEU+jErZDU1hpPolVUpJ59thNqv7v2ZbuI203ZVqbddpUAQSek8hERTKETvweR1cuI7DwqFaFWnXPGbTPVSf9u/Xz1vvJOBN5pUi0zDOH3/PCCM0QwhU7cJkPcduHThVhFqJ14mG7qPVUGsbArJFcVeDPBlRKh8KMqmEZjqQo5xsiRzj5X3W72bKC0tPt3paXa54A26LAR6Z8b7cOM/fu1AY1VqKsDBg8GmprsbZg1C2hpMV6vpUX7XsfsmhB1Dbjb3t79OsyYoX3HrP2dMUP9PISQoaKqYVnEw3SHlxim3XZW3pOTmJ6qp6kax3TSW8guLJDuiRvt26rHTtCZe0ENSJNcSMdts9BLc9KpUNsVdKvGMVXEV0XY7QTeqveU3SJlQ+FCBFMIDC8JDT88TDvRzYyF2nmkdjWgVqJpNoWEeJjhQlUwJYYp+EpdXfeYXVMTcPAgsHChNnlWdbX19uXl1t83N9vH/6xis6WlwJw5XbZWVABTpwIlJUDM5GlYtsz6eHqs0gg9nplpg9U2QohRUdWwLOJhhh+vMTuV+KObKTB0z9Lp2KCqzWerbLxkycMPpEkuBIEf/bnTBcZtkzaIgvmwjYgkqKMqmNIkF3zFbQlTOtXVWvM9ldIWI8xKloz2YRQKsNteh0i9+VxS0vXvZBKYN88+BCFECxFMwVfsajOd4ocAe9me2V709Lhtes3nwYPubRPCiwim4CvV1ZpnpSdv4vGu4m83xdp+C7DVfo2wS0IBxoXvmQXvQm4ggin4TnV1lyC1t2ufue3hki7ARNpfP5q6mftNJoGiou7rqAqzSo8mITcgLd4ZDSorK7m+vj5oMwQFKiq6ugqmU16uxRTDSF2d5hVu3ao12WfPVhPmKJ6r0B0iWs3MlXbrBeJhEtG9RPQBEa0loiVENCAIO4TsEUWvyy5RZEa2wgZC+AiqSf4CgFOYeQyA9QBuC8gOIUtkK1kTRrIVNhDCRyCCyczPM/ORjv++AWB4EHYI2SPfvC633qkQLcKQ9JkG4BmzL4loBhHVE1H9p59+2otmCV4Qr0vIRbKW9CGiFwEcZfDVLGZe2rHOLACVAC5nBUMk6SMIQjZQTfoUZMsAZj7f6nsiugbAFACTVcRSEAQhaLImmFYQ0YUAbgHw/5jZZKxrQRCEcBFUDPO3APoCeIGI1hDRHwKyQxAEQZlAPExmPi6I4wqCIHghDFlyQRCESCCCKQiCoIgIpiAIgiIimIIgCIqIYAqCICgigikIgqCICKYPfPLJJ7jyyitx7LHHYvTo0aiqqsL69euxZcsWnHLKKVk55uHDh/Ef//EfOO644zBhwgRskYEXBSHr5J1g6nNRx2LaXzfTJqTDzLjsssswadIk/Otf/8J7772HX/ziF9ixY4cf5pry4IMPYuDAgdi4cSO+973v4ZZbbsnq8QRByDPB1CeramjQJrdyO21COi+//DIKCwtx/fXXd342duxYnHPOOd3W27JlC8455xycfvrpOP300/H6668DALZv346JEydi7NixOOWUU/D3v/8d7e3t+M///E+ccsopOPXUU3H//ff3OO7SpUtxzTXXAAC++tWv4qWXXoJ0yReE7BJIT5+gsJqsyu2wY+vWrcO4ceNs1xs6dCheeOEFFBcXY8OGDbjqqqtQX1+PRYsW4ctf/jJmzZqF9vZ2tLS0YM2aNWhsbMS6desAAHv37u2xv8bGRowYMQIAUFBQgP79+6OpqQmDBw92dyKCINiSV4IZ5LQJbW1tuPHGG7FmzRrE43GsX78eADB+/HhMmzYNbW1tuPTSSzF27FiMGjUKmzZtwk033YSLLroIF1xwQY/9GXmTRJT18xCEfCavmuTZmDbh5JNPxurVq23Xu//++zFs2DC8/fbbqK+vR2trKwBg4sSJePXVV3HMMcdg6tSpWLBgAQYOHIi3334bkyZNwu9+9zt861vf6rG/4cOH46OPPgIAHDlyBPv27cOgQYPcn4ggCLbklWBmY9qE8847D4cPH8Yf//jHzs9WrVqF5cuXd1tv3759OProoxGLxbBw4UK0d8w/29DQgKFDh+Lb3/42pk+fjrfeegu7du1CKpXCFVdcgZ/97Gd46623ehz34osvxiOPPAIAWLx4Mc477zzxMAUhy+RVk1yPU7qZStUMIsKSJUtw880346677kJxcTEqKirwm9/8ptt6M2fOxBVXXIH/+7//w7nnnotEIgEAeOWVV3DvvfeisLAQZWVlWLBgARobG3HttdcilUoBAH75y1/2OO706dMxdepUHHfccRg0aBAeffRR9ychCIISMi+5IAh5T6jnJRcEQYgiIpiCIAiKiGAKgiAoIoIpCIKgiAimIAiCIiKYgiAIiohg+kAQw7u9+uqrOP3001FQUIDFixdn5RiCIHQn/wTT5/HdghrebeTIkXj44YfxjW98I6vHEQShi/wSzCyM7xbU8G4VFRUYM2YMYrH8uoWCECR51TUyG+O7BTW8myAIvU9+CWaA47v5PbybIAi9T36157IwvltQw7sJgtD75JdgZmF8t6CGdxMEIQCYOTLLuHHj2DO1tczl5cxE2t/aWs+7bGxs5K997Ws8atQoHj16NFdVVfH69et58+bNfPLJJzMz8/r16/nUU0/lCRMm8K233sqJRIKZmR9++GE++eSTeezYsXz22Wfzpk2beM2aNXzaaafxF77wBf7CF77Ay5Yt63HMlStX8jHHHMOlpaU8aNAgHj16tOfzEIR8BUA9K2iQDO8mCELeI8O7CYIg+IwIpiAIgiIimIIgCIqIYAqCICgigikIgqCICKYgCIIigQgmEf2MiNYS0Roiep6IPheEHYIgCE4IysO8l5nHMPNYAE8B+HFAdgiCICgTiGAy8/60/yYARKd6XhCEvCWw0YqIaDaAqwHsA3BuUHYIgiCokrWukUT0IoCjDL6axcxL09a7DUAxM99hsp8ZAGZ0/PcUAOv8trUXGAxgV9BGuCSqtkfVbiC6tkfVbgA4gZn72q0UeF9yIioH8DQz205+Q0T1Kv09w0ZU7Qaia3tU7Qaia3tU7QbUbQ8qS/5vaf+9GMAHQdghCILghKBimHcR0QkAUgAaAFxvs74gCELgBCKYzHyFy03n+WpI7xFVu4Ho2h5Vu4Ho2h5VuwFF2wOPYQqCIEQF6RopCIKgSOQEM6rdKonoXiL6oMP2JUQ0IGibVCGirxHRu0SUIqLQZ0GJ6EIi+pCINhLRrUHbowoRPUREO4koUqVzRDSCiF4movc7fiffDdomVYiomIhWEtHbHbbfabl+1JrkRNRP7ylERN8BMJqZQ580IqILAPyNmY8Q0d0AwMy3BGyWEkR0ErQE3VwAP2Dm0M4TQkRxAOsBfAnANgCrAFzFzO8FapgCRDQRQDOABSpldmGBiI4GcDQzv0VEfQGsBnBpRK45AUgwczMRFQJYAeC7zPyG0fqR8zCj2q2SmZ9n5iMd/30DwPAg7XECM7/PzB8GbYciZwDYyMybmLkVwKMALgnYJiWY+VUAu4O2wynMvJ2Z3+r492cA3gdwTLBWqdExB1pzx38LOxZTTYmcYAJat0oi+ghANaI5cMc0AM8EbUSOcgyAj9L+vw0ReXhzASKqAHAagDeDtUQdIooT0RoAOwG8wMymtodSMInoRSJaZ7BcAgDMPIuZRwCoA3BjsNZ2YWd3xzqzAByBZntoULE9IpDBZ5FohUQdIioD8DiAmzNagqGGmds7Rk4bDuAMIjINhwQ2+IYVzHy+4qqLADwNwLAfem9jZzcRXQNgCoDJHLLgsYNrHna2ARiR9v/hAD4OyJa8oSP+9ziAOmb+S9D2uIGZ9xLRKwAuhMmYFaH0MK2IardKIroQwC0ALmbmlqDtyWFWAfg3Ivo8ERUBuBLAXwO2KafpSJw8COB9Zr4vaHucQERD9IoVIioBcD4sNCWKWfLHAXTrVsnMjcFaZQ8RbQTQB0BTx0dvRCG7DwBEdBmABwAMAbAXwBpm/nKwVplDRFUAfgMgDuAhZp4dsElKENGfAEyCNurPDgB3MPODgRqlABGdDeDvAN6B9lwCwO3MvCw4q9QgojEAHoH2W4kBeIyZf2q6ftQEUxAEISgi1yQXBEEIChFMQRAERUQwBUEQFBHBFARBUEQEUxAEQRERTCFnIaJniWgvET0VtC1CbiCCKeQy9wKYGrQRQu4ggilEHiIa3zHOaDERJTrGNTyFmV8C8FnQ9gm5Qyj7kguCE5h5FRH9FcDPAZQAqGXmSA3CK0QDEUwhV/gptH7khwB8J2BbhBxFmuRCrjAIQBmAvgCKA7ZFyFFEMIVcYR6AH0EbZ/TugG0RchRpkguRh4iuBnCEmRd1zOnzOhGdB+BOACcCKCOibQCmM/NzQdoqRBsZrUgQBEERaZILgiAoIoIpCIKgiAimIAiCIiKYgiAIiohgCoIgKCKCKQiCoIgIpiAIgiIimIIgCIr8fwmJ4ppq4JfUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "xx, yy = np.meshgrid(np.arange(X_train_scaled[:,0].min(), X_train_scaled[:,0].max(), 0.1),\n",
    "                     np.arange(X_train_scaled[:,1].min(), X_train_scaled[:,1].max(), 0.1))\n",
    "zz = model.predict(np.c_[xx.ravel(),yy.ravel()]).reshape(xx.shape)\n",
    "\n",
    "\n",
    "plt.figure(figsize=(5,5))\n",
    "contour = plt.contour(xx, yy, zz, levels=1, alpha=1., vmin=0, vmax=1, linewidths=2, colors='k', linestyles='-')\n",
    "plt.plot(X_train_scaled[y_train==0,0], X_train_scaled[y_train==0,1], \"bo\", label=\"Class 0\")\n",
    "plt.plot(X_train_scaled[y_train==1,0], X_train_scaled[y_train==1,1], \"ro\", label=\"Class 1\")\n",
    "plt.xlabel(\"x1\")\n",
    "plt.ylabel(\"x2\")\n",
    "plt.ylim(-3,3)\n",
    "plt.xlim(-3,3)\n",
    "plt.legend(loc=3,  frameon=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As an exercise, try to re-run this neural network with the \"sigmoid\" activation function for the hidden layer. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Multiclass Classification\n",
    "For the multiclass case, we see an example with the *iris dataset*. Unlike what done with other algorithms, however, this time we will use the dataset with all 4 original features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_iris \n",
    "X, y = load_iris(return_X_y=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the multi-class case, the output must be one-hot encoded. This can be done with the `OneHotEncoder` function of scikit-learn, or-alternatively- with the `utils.to_categorical` function of Keras: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.]], dtype=float32)"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_enc = keras.utils.to_categorical(y)\n",
    "y_enc[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now split the train/validation/test set and standardize the features:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_trainvalid, X_test, y_trainvalid, y_test = train_test_split(X, y_enc, stratify=y, test_size=0.1, random_state=1)\n",
    "X_train, X_valid, y_train, y_valid = train_test_split(X_trainvalid, y_trainvalid, stratify=y_trainvalid, \n",
    "                                                      test_size=0.1, random_state=1)\n",
    "sc = StandardScaler()\n",
    "X_train_scaled = sc.fit_transform(X_train)\n",
    "X_valid_scaled = sc.transform(X_valid)\n",
    "X_test_scaled = sc.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The network**. In the multiclass case, a typical FNN has the following architecture: \n",
    "* as many output layers as there are classes (3 in this case)\n",
    "* cross-entropy loss \n",
    "* softmax activation function for the output layer\n",
    "\n",
    "We are going to build a network with one layer containing 7 neurons, this time activated with the tanh function: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "    keras.layers.Dense(7, activation=\"tanh\", input_shape=X_train_scaled.shape[1:]),\n",
    "    keras.layers.Dense(3, activation=\"softmax\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As usual, let's compile and fit the model (using the `categorical_crossentropy` loss, and again the accuracy as metric): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=\"categorical_crossentropy\", \n",
    "              optimizer=keras.optimizers.Adam(lr=0.01, beta_1=0.9, beta_2=0.999), \n",
    "              metrics=[\"accuracy\"])\n",
    "history = model.fit(X_train_scaled, y_train, epochs=1000, batch_size=32,\n",
    "                    validation_split=0.1, verbose=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAE9CAYAAAAbGFuyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3hUVfrA8e+Zkt57oddQQuhNRYqAvVLt2HtbUXTVddV1XV3dtSvrIlgQe/kh4oqASBOpoYQaAiQhvfcp5/fHhCGRkEyAZDLwfp6Hh5nb5r2XkHfOuee+R2mtEUIIIYTnMbg7ACGEEEKcGEniQgghhIeSJC6EEEJ4KEniQgghhIeSJC6EEEJ4KEniQgghhIcyuTuA5oqIiNCdOnVydxintfLycvz9/d0dxmlPrnPLk2vc8uQat7wNGzbkaa0jG1rncUm8U6dOrF+/3t1hnNaWL1/O6NGj3R3GaU+uc8uTa9zy5Bq3PKXUgeOtk+50IYQQwkNJEhdCCCE8lCRxIYQQwkNJEhdCCCE8lCRxIYQQwkNJEhdCCCE8lCRxIYQQwkNJEhdCCCE8lCRxIYQQwkN5XBIvqSlxdwhCCCFEm+BxSTy3ItfdIQghhBBtgsclcY12dwhCCCFEmyBJXAghhPBQHpfEJYcLIYQQDh6XxKUlLoQQQjh4XBIXQgghhIPHJXFpiQshhBAOHpfEJYcLIYQQDh6XxKUlLoQQQjh4XBIXQgghhIPHJXFpiQshhBAOnpfEtSRxIYQQAjwwiQshhBDCweOSuEZLa1wIIYTAA5M4gF3b3R2CEEII0eKqLLZG10sSF0IIIdqo+b8dbHS9RyZxm278m4kQQgjhabTW7M4upbLmaI77elNGo/uYWjqoliBJXAghhKfSWlNSaSXYz+xcZrXZue3DDSzdmUO3qAC+vmskZqOBlMMljR5LkrgQQgjRSkqqLDywYDNLd+YQGehNjdXOvJuGsjW9iKU7c7hyYDzfbs7kqrdX88B5PbDaGx/I7ZHd6Xa73BMXQgjR9v26J5ez/7GUVXvzAPhgdRpLd+YAkFtaTXGlhee/T+GrTRkkxATy8uQkXprUj93ZZdz18UYCvRtva3tkErdqq7tDEEIIIRpUY7Vz3yeb2HCggLeW7SO9sJIHP91MtdXG0p05JLUPYcXMMbw2fQDPX5HIurQCNh0sYkKfGJRSXDmwHXeO7oqXycCD43s0+lke2Z0uo9OFEEK0VUtSsvluSybfbcnEoCCpXTBb0otZtPUw2zNLuG54RzqE+9Eh3A+bXfPOL/vIKqli2pD2zmM8en4CD0/oidGguLmRz2qxlrhSao5SKkcpte0465VS6jWl1F6lVLJSaqCrx5YkLoQQoq1akpLtfG3X8JdL+xDqZ2be6gNUW+30jgtyrjcaFD/cfw6rZ40lLsS33nGMBtXkZ7Vkd/pc4PxG1l8AdK/9cxvwtqsHttqlO10IIUTrK62yUF5dPwdZbHb+viiFvy9KocZqZ93+Asb3jqZzhD/dowIY0D6EAR1C2XyoCIA+ccH19vf3NhER4H1C8bRYd7rWeoVSqlMjm1wGfKAdNVTXKqVClFKxWuvDTR1bWuJCCCFaw9rUfHZklnBJUhzBvmbO//evlFRZ+Pbus+gSGcCN769j+a5c5/bVVjvphZXcPaYbr00bgF1rlFKc3S3COaCta6T/KYtPtWQd8tokvlBr3beBdQuBF7TWK2vf/ww8qrVe39gxfTv76h2bd9A5uHPDG+TshPfGQU15/eVnPwDnPd3sc3C30uXLyZz5CN2W/IQx2PHtrfj/FpL56KP4JiYSNfNhDsy4CSwWwm+7jcL587GXlzdx1MZpoOlOHHGy5Dq3PLnGLe90vsYaOJIiVe1J1n2vcHSXKwB1dB1AQz3hR7ZVzbxgvXembNBaD25onTsHtjV0Gg1+o1BK3Yajyx2fTj6s/W0tB7wONHjQ8LzfSKwpIzN2PDVeoQDEHl5C2Y4VbDUtPyWBt6bwvz6DqbSU3z74gJrERACCP12Aj91O5ZYt7HzjTXwtFgDyZ88GoPy8cWivE+uaAaipqcHLy+vkgxeNkuvc8uQat7zT6RpnV9gpqNJ0CzFgNiiWH7JgtYPZCDY7RPgq0kvt+JkVXkYI9Tawp8jG+I5mfE2KzTlWDpbaGRxtIi7g2LvVjlY5qOZ+7dmZctxV7myJvwss11p/Uvt+FzC6qe50386+evOGzfQM69nwBtu/hs9vhDvXQHRvx7L/TgSTF9zwfyd+Mi5Iv+9+yteswbtrVzp+Mh+lFCWLFlEw7wM6zv+Yw39+gtKff0YZDITffjsF8+Zhr6gAwBgaiq6pwRgYiDU3F1tRUb1jK29vVO1/FHt5OaboaKyHDzu+0tX5NzQGB9N97RpUc7/q1bF8+XJGjx59wvsL18h1bnlyjVve6XKN9+aUMfHfK7DZNRf1i2XmhJ6M/udyZl2QgF1rXly8C4OCsQlRRAX58Pn6Q1jtmvG9opl9vaORbLdrMosraRfqd0pjU0q1yZb4d8A9SqkFwDCg2JX74dDEPXGbo1WKqU5L1OR1dHkLsZWVU7pkCcbwMCo3b8Zy8CBeHTuS8dCfAKjetYuSH37AJyGB6rQ0cl58EYCQqVOxpKdTvmoVANasrGOOHTh+PKbYGOd7pRTBl11G2S+/YC0sxLdvXyqTt6KMBnwHDjqpBC6EEGea0ioLC9Y5JhqZPrQDn6w7yPfJhzEbFef1iianpApwdIc/cF4PfMwG/m9LJtVWO7ef29V5HINBnfIE3pQWS+JKqU+A0UCEUiod+AtgBtBavwMsAi4E9gIVwAxXj91o2VVrteNv49GatBi9oLqs0WNmPvY4VSkpoBQGb29sRUXYqx3HsubkgM2Gd+9ehFx+BTkvv4yursYUG+vcX1stYLcTfuON5Lz0T9KmX43yPvpF4sCMm9DV1YTdeAMlixZR+tMSjGFhxDz9F6q2bXcmcQDl5YWuqXG+b/f6aw3G7NO7t/N18KWXNnp+QgghjjqQX87n69N5Y9le57IJvaP52+V92Z9XxtrUAv52eSLdogJoF+pLlwh/wgO86BvvGJu09rFxWG26Xv1zd2jJ0enTm1ivgbtP5NiNJnHbkSRepyVu9AZbTcPbA/bKSoq//vqY5cGXXQYGg3Nd9Y4UsnccvTfhN3BgvURtDAwg9JprsJWVYc12jEJEgS2/AGNYGAY/PwJGjcIcF4chKAj/4cNRSuHTuxfht92GMSQES/ohAieeT2XyFoyBQZiioly4IkIIcWbam1NKQbmFoZ3DXN5nyY5sbv9oA7Y6dcmfu7wvkwe3w2BQvHPtINILK50J28dsZMlD52KpU/Lbv4lyqK2lbUTRTI12p1trk7WpzkALkxfaWk36nXdhST8EgCEoGGU2Y8vPo3rP3gYOBHH/eAGgwQQPEPfPlxrsuo66//5G4/dNSsI3Kcn5XhmNRD30YL1t/IcNbfQYQghxpiurtnLeKysAWPnoGPZklzF7RSqBPiYeOK9HvaIqR+zLLeOOjzbQMcyPaUPbs2ZfPveN686ADqHObUL8vAjxqz9Yz2BQeBuMLXtCJ8Ajk3ijxV6OtLiNdf4BjF7U5FZTtmwZvv37Y/Dzo3z1agAMQUf/kcNuvgk02EqK8Rt8dAxBuzffwF5ejiUrm6qUHfj0TMAUFSX3noUQwo1W7jn6fPbZ/1hWb116YSWL7j8Hi82OyaBQSmG12Zm7Kg2AD24eSrtQP24b1RVP5pFJvPGBbUeS+NFu7rwVmRSvsQCK2OefxxwXy67+AwAIuvACihZ8Sui11xI9c2aDhwwcN+5UhS6EEOIkFVdaCPY1s3xXLoHeJu4d1415qw8wrEsYd4/pxtKUHP62KIXk9CKun7OOe8d2Z1T3CM5/9Vdsds21wzu0+gC0luKRSbzJgW3KAMajp1a0LhNtgZApU/Dq1BFlMBBxzz1Ysg4T9ac/oa1WIu64vRUiF0IIcaKqrTYe/SKZbzZn8vY1A/l1Tx4ju4Vz26iu9VrUNVZHQ++BTzdTVGHh2YU7CPQ2YbNr4oJ9+NP44zyi7IE8M4nbmxjYVqcVru12LMVVhPeqIeqZvzqXR95zdExd3HPPtUicQgghTp7WGq1h4ZbDfLM5E4CHP99CeY2NO0cf2x3ePSoAs1GRmnu0emWV1cbyh0fTKeLUlTxtCzwyiVsbTeKWevfDbQUFYNOYfFr2OXEhhBCnXmmVhWve+42e0YHkl9fQIcyP60d05LnvHU8Kjeoeecw+JqOBUd0j+XlnDm9dM5DiSgtjekYRE+zT2uG3uNMviVurnSPTy9euJf+9/wJg9qlxVDaTwWhCCOER7HbNrC+3kpxeTHJ6MQDXj+jItcM7UlBeQ9fIADqEN3xv+81rBpJyuKTeqPPTkUcmcYutidHptd3p+XPmULFhAz6dIvEJz3K00k2nR41fIYTwVJU1NsxGhcl4bH3x3NJqktOLqLTYeGPpXnZmlXJxv1gWJjsKeo7qHomP2cgj5yc0+hk+ZuNpn8DBQ5O4tamBbSYvtNZUbUkm6KILibswGv63xXG/XJK4EEK4zaGCCi59YyUju0bw5jUD663bl1vGuJd/qbcsPsSXV6b0x8dsZM2+fEZ0DW/NcNs8j0ziFltj98RrwOiF5cABbMXFjqIqxoLadXJfXAghTjWtNdszS+geHYC3qfGCKHfP30hhhYXvtx7m7HUHGdcrilA/L+xac+u89XgZDdTYHKPLdz93ATa7xstk4J+Tkxo97pnKI5N4k8VejF7kvPoq4KiORkltXfIjddWFEEKcMp/+fohZX21l5sSe3D2m23G325bhuLf9xEW9+N/2bB77aisAscE+RAV6k5pXzp8v7MWYhCh8vYx4mY7tbhf1tehUpC3Bt7Ov/vDZsUzK2O5YUFlYu6L23kd1KTp2ADv/dRjsdhK2b0Nt/Qy+uQO8g8EgPxRNsVismM0e+f3Oo8h1bnlyjVuexWKl2g4Wm8ZkUAT6NHy9bRrKq63Y7ZogX8ekIdVWGzY7WGxHC3gF+5oxyPjjetSsA21yKtITpgv3QUA0RPeBbV86FnYdB36OAvi2yBFg/zPRTz6BMhqh+3gYcU+jk6CIo7IzMmgXH+/uME57cp1bnlzjlpedkcGaw3bKbTaMdsXVfTtgVIryGiu+ZiMGpaiy2PhqUwY2u2ZUj0hCwhwjyn2PHKOkCqUUBgWGAO/jf9gZ65/HXeORSdyOhg4jYNgdR5P46McgwtGNY9nuaKWbo6Md6/wjYOLf3BGqR9q7fDntRo92dxinPbnOLU+uccupqLHy7i+pdO2QwSOpFXQO9yc1r5whg84h5XAJD366hScu6sUt53ThXz+k8G5VKp/eNpyOXY4dmBbthvg9y+mWxLUdDKZjZio7wpqd7VgUHdPaoQkhxBnhqW+388WGdK7oZkZrGN0zitS8/azbX8CynY6pmOeuTmPemjQOFVRy1cB2DGsggYuT45FJ3KrtYDAeO2d4LUtWFgDmGPl+J4QQLWFnVgkAGWWO+9lDO4exel8en6w7xMF8R7nT9MJK/LyMPH1Jb64e1tFtsZ7OPHKUlx0Nygimuknc7HxpzcoGkwljuHzrE0KIU01rTXphJQAHShxJPDLQmymD25NyuITyGhvXDu8AwFUD23HjWZ1lpHkL8ciWuN3ZEj+auOsmdGt2FuaoKJSMRBdCiFPKZtcUlNdQVOGou5Fd4XjCKTLAmysHxvPMwh0AzJyQwLhe0YyQLvQW5ZFJ3IZuojs9G1OM3A8XQogTVVFj5dEvt/LTjixenTaA8b2iuXv+Rn7cnoWv2VHQxWxUWGwaL6OB2BAfzEYD/7l+MEE+JoL9zIzpGeXmszj9eWYS13ZHd3qd2cowHK0SZM3Kwrt3LzdEJoQQnm1hciaxwT6sTS3g/7Y4pv28/cMNXJQYyw/bHOONymscVTMnD27P/N8OEhnojbm2Dvr43jIWqTV5ZH+zXeEYnV63u7x2djKtNZbsbMwyMl0IIVwye8U+LntzFfNWp3HP/E1c9fYa5q1OY0CHEF6a1A9/LyPfbz3MpUlxbHjiPOd+kwa1A2BUj2OnAxWtwzNb4qh6Le+67MXF6KoqTDIyXQghXPLKT7upstjZcqgIPy8jFTU2ckqrufnszkwe3J4hncLYllnM+N7ReJuMPHt5XwZ1CKV3XBBvjfNj4ri+7j6FM5YHJnHlaImrhjsRLLXPiJvlnrgQQjjZ7ZrXlu6hT1xwvS7v3NJqqq12xvSMpEdMILeP6srV/1nL7uxSrhzoaGl3ivCnU4S/c5/rhh99XMzPrDBKnVS38bwkrsEGju70BljS0wFJ4kKIM5vFZiejsNKZfH/emcO/l+wB4IK+Mdw1uhtdo/y5ae7vaA1/vqg33aICAPj2nrMorrAQGSglUNs6z0vigE0dvzu9dMnPYDTi3bNnK0clhBBtQ2F5DRe/vpKMokoW3XcOveOC2HjQMVlUdJA3P2zLYk9OGeMSotiaUcw53SOcCRzA22QkKqjxKUVF2+CBA9vUcVvi1ampFH/9Nd5dOmPw9T1mvRBCnK6+Tz7M+rQCAN7+ZR8ZRY5iLB+sSUNrzbaMYvrEBbH2sXE8cVEv9uaU8e6KVOJDfHl9+gA3Ri5Ohke2xO0oxyNmf1Dx+3oAoh9/vLVDEkIIt1m+K4e752/EZFCsnjWWeavTuHJAPEG+ZuauTsPLZCA5vZgL+saglGJE16MFWF6b3p8QP69Gji7aMo9M4jZFg/OCV27ZgjE0FL/hw1s/KCGEcJMvNjjGAlntmn8t2UO11c5Vg9oxsms4BqWYs2o/AIntggHoExfMv6f2RykY1DHMbXGLk+dx3emKhge22crKqNy8Gd+kJJSSkZJCiNOT1poP1x7gH4t3UmVxFF3ZlVXKeb2iiA324ZN1BwHoHRuEUorHL0ygT1wQJoNibMLRCmqXD4jnsv4y17qn88CWuMKu6nSnX/0ZJSs3kzF4CADBl17ixtiEEKJlbThQyJPfbAMgq7iKlyb140B+BWMTorh7TDdmfpFMdJA3of6OLnKT0cDnd4ygvNomo81PQx6YxMEKR0en95hI2Qe/YggKIvKB+wm+6CJ3hiaEEC1iyY5sooK8+WlHNmajYniXcL7elMGoHhHU2Ox0ivBnQIdQljx07jH7+nmZ8PPyyF/3ogke+a9qV6ped3rVli34DuhP2NVXuzEqIYRoGXa75pYPHAN3u0b6M6xzOG9MH8iAZ//Hy//bDUCfuCB3hijcxOPuiTsfMaut2GYrLaV67z58k5LcGpUQQpwMrTWv/LSbHZklx6zbk1PmfL0vt5wJfaIJ9jPz8MSepBdW4mU0kBAjSfxM5HlJXEPH3QZ2Xvcs9upqqrZuBa0liQshPNq7K1J57ec9XD/nNw7mV3Dj++uYtzoNgN/25zu3O6tbONOGdADgrtHd+PDmobw0uR9eJs/7dS5Onkd2pw9baUJX1WArKKAyORkA38REN0clhBAnJjW3jBcX7wQgr6yGUS8tA2D5rlwGdAhh5Z482oX68vOfzsXLaKj3BM453WUGsTNZiyZxpdT5wKuAEXhPa/3CH9Z3AOYBIbXbzNJaL2rquAa74+99F1wIWuPVtSvGIOlKEkJ4luySKsL8vfh8QzpKKX5/fBxfbUxnS3oRkwe154lvtnHP/E0UVtRwUWIs3iYphSrqcymJK6W+BOYAP2it7S7uYwTeBMYD6cDvSqnvtNY76mz2BPCZ1vptpVRvYBHQqdHjAqo2Al1Vhf/IEYRef70rIQkhRKtatjOH1fvyCA/wJszPi8mD2zlb0YcKKhj78nIsNg3AuT0iiQz05vZzuzr3v21UF/7y3XYAzuoW0fonINo8V1vibwMzgNeUUp8Dc7XWO5vYZyiwV2udCqCUWgBcBtRN4ho40oQOBjJdCcZQ52tE2I03EjBqlCu7CSFEq7HbNY9/vZXDxVXOZQcLKvh5Zw7/uCqRLzekOxN4RIAXz1957C3BuiPOR9YplSrEES4lca31EmCJUioYmA78pJQ6BPwH+EhrbWlgt3jgUJ336cCwP2zzNPA/pdS9gD9wXtPRqHpJ3BgS4sopCCFEq9qdU8rh4iqevbwv3aMCmDZ7LW8s2wvAnz7bwsGCCqYMbsej5ydgs2uignyOOUbf+GB6xQYRF+xDeIAUahHHcvmeuFIqHLgWuA7YBHwMnA3cAIxuaJcGluk/vJ+Oo1X/slJqBPChUqrvH7vslVK3AbcB+HUIrJfE1+/cia2gwNXTEC4oKytj+fLl7g7jtCfXueW11jU+VGrncLmdoTGOX6l2rVmVYQXAmLePqioDQ2OMrMuy0SPUwO7aR8b6++Szdf0aoH4XZV2PJgGUt9mfFfk5di9X74l/BSQAHwKXaK0P1676VCm1/ji7pQPt67xvx7Hd5TcD5wNordcopXyACCCn7kZa69nAbIDADkHaoI9+Pxg5cSLG4GBXTkO4aPny5YwePdrdYZz25Dq3vJa+xlprDhZUcONLywGY/PAIMosquX7OOmx2TZCPiWkXjsFgUIw82052SRVmo4FZXyVzYWIsUwa3b/wDPID8HLuXqy3xN7TWSxtaobUefJx9fge6K6U6AxnANOCPJdUOAuOAuUqpXoAPkNtYIN6W+o15Q2Bgk8ELIcSp9uqSPXyy7iC9Yo/+Dvo5JZtNB4uw2R2/p0Z2jcBgcDQ6vEwG2of5ATB3xtDWD1icllxN4r2UUhu11kUASqlQYLrW+q3j7aC1tiql7gF+xPH42Byt9Xal1DPAeq31d8CfgP8opR7E0dV+o9b6j13u9Rhru9L9knrh3W8wqoEpSYUQoqX9a4mj3GlWSRUPje/BwuRMnvs+BYDrR3Skb3ywjCgXLc7VJH6r1vrNI2+01oVKqVuB4ybx2u0W4XhsrO6yp+q83gGc5Xq4oGpTfOzDt+E15Pzm7CqEEM1SZbHxvx3ZDOwQQrtQPypqrNz+4QZignwwKOgdF8TUIR24emgHfMwGnl+0Ey+jgelDO9ArVmpXiJbnahI3KKXUkVZy7TPgXi0X1vEduRuuTGZ3fLwQ4jRktdmpttrx967/K/G573fw0VrH/NxJ7YKZ2DeGX/fkAWBQ8M61g2gX6ugiv21UVy5JiqOixkbXyIDWPQFxxnI1if8IfKaUegdHt/cdwOIWi6oRZm1zvPByy3cIIcRp6On/28783w6y9emJ+Hub+Dklmzmr9rNqbz69YoNIOVzClvRitqQX42s2Ehfiw81nd3Em8CNig33ddAbiTOVqEn8UuB24E0dj+H/Aey0VVGNU7VNqKrKbOz5eCHGaKaqocba2P1l3kGuHd+SJb7Y5i7Q8eF53hnYO48JXf2VYl3CevqQPwX7SEyjaBleLvdhxVG17u2XDadqRe+LK26/xDYUQwgVzV6ehFAT5mHnv1/3kllVzuLiKF65MJDrYhzE9owBY+ehY50hzIdoKl4Z2K6W6K6W+UErtUEqlHvnT0sE1GMuRv00eOQGbEMJNNh4sJK+sGnAMWHvq220czK9g8bYshnQM45UpSWSVVPHuL6lMHtSOaUM7OBM4IAlctEmuZsL3gb8A/wLG4Kij7paf6CMtcczSnSWEcM2OzBKufGs1XkYDr07rD8AHaw7w+fp0Ki02nry4N2MTonht+gAOFVRw89md3RyxEK5xNYn7aq1/rh2hfgB4Win1K47E3qqc3xzk+XAhhIvW7c8HoMZm586PNzqXV1ocA2XP7xuDUopLk+LcEp8QJ8rVTFillDIAe5RS9yilrgCimtqpJSjADtglhwshav2Wms83mzKc78uqrXyy7iCW2sppK/fmER/iy+anxnNer2gArhwQD0C7UF/iQ2RUufBMrrbEHwD8gPuAZ3F0qd/QUkE1RmmN3QAWuwWTQe6LC3Gms9s1U2evBSAq0JuR3SK4/5NN/Lwzh1sTvRhYVs3yXbncfHZnQvy8ePe6QeSVVRPiZyYhNpAJvWPcfAZCnLgm27O1hV2maK3LtNbpWusZWuurtNZrWyG+Y+MBrEZHEhdCiPUHCp2vn1m4g4oaKyv3OgqyZJRpvt6UgdWuuWKgo+VtNCiig3zwNhm5bVRXOkX4uyVuIU6FJpO41toGDFJKtY2hmRpsBrDare6ORAjRBixMzsTbZODZy/qwM6uUd5bvo9rqmGRhTaaV15fuZWjnMBJipAyqOP242h+9CfhWKfU5UH5kodb6qxaJqhEKRxK32KQlLsSZRmvNCz/s5IdtWfRvH8JTl/Rm0dYsxiZEMbJ2spH/rtyPv5eRKUPa8/6qNPy97Lx4VT83Ry5Ey3A1iYcB+cDYOss00PpJ/EhLXEtLXIgzQXphBTfN/Z3LB8RjNhh4d0UqncL9+G5LJrml1eSVVTNlcHs6hPlhNCjKa2yc1S2cm8/uzMbdh7j/ogHSZS5OW65WbJvR0oE0h80oLXEhTkdaa/6xeBdl1RaeuzyRgvIa7vhoA7uzy3hx8S4AJvSO5p1rBzHkb0tYk5pPfIgvo3tGopQiPsSXgwUVDO8cTrtQPx4c5MPohGg3n5UQLcfVim3vK6Xm/PFPSwfXYCxyT1yI00JJlYUvNqQ7q6gBrEnN551f9vHR2oMcKqjgqrdXszu7jH9clUj7MMdjYPeN647BoJg2tD0Al/WP48iQnX9N7c/l/eOYIcVaxBnC1e70hXVe+wBXAJmnPpymOe+Jy+h0ITza6z/v4T+/7ueqge14eUoSAAuTDzvXj315ORab5tVp/bmsfzyX9Y/nQH4FPWMCAXhofE8m9I6hb3ywc59BHUMZ1DG0dU9ECDdytTv9y7rvlVKfAEtaJCIXWCWJC+FxKmqsHCqopGdMIFUWG19tdBRn+b/kTP58US8yiyr5Pvkw53SPIC2/nEMFlfSIDuCixFgAfMxGZwIHx6NiSe1D3HIupwuLxUJ6ejpVVVUnfIzg4GBSUlJOYVRnLh8fH9q1a4e5GWXFT7RaSnegwwnue1KOFHuR7nQhPMvL/9vNf1fu59u7z2JvThn55TU8c1kfnvp2O098s5VFW7MAGNk1gicv7k1yejEjuoZjMkp5xpaSnp5OYGAgnTp14kSfIi4tLSUwMLDpDUWjtNbk5+eTntOoc10AACAASURBVJ5O586u3w5yKYkrpUoBXWdRFo45xt1CWuJCeJ4jRVk+XHsAi81ORIA31w7ryLzVaSzamoVSMHNiT249pzMmo4Ee0ZIYWlpVVdVJJXBx6iilCA8PJzc3t1n7udqd3nb+N2mkJS6EB0hOL+LJb7czoXc01w7vyPaMYgCWpGRjNhoY3iUMg0Ex64Je/Lonl1vP6UL7MD83R33mkQTedpzIv4WrLfErgKVa6+La9yHAaK31N83+xJMkZVeFaJtKqizMW5WGyWjg9lFduPH93ykor2HLoSJe+tHxeNjEPtH8uD0bgP6197PH945mfG95DEyIE+Hqzaa/HEngAFrrItwwDamDwmZQksSFaGPeXLaXl3/azT8W7yQtv5yC8hpuPaf+vb2ZE3syLiEKpRz3voUQJ8fVgW0NJXv3TCEmz4kL0WZUWWyYDAqT0cDOw6XO5Y98kQzAJUlxrE0tILe0mnk3DaVbVCD/vXEIFpsdswxYEycgICCAsrKyBtelpaVx8cUXs23btlaOyn1cTcTrlVKvAG/iGOB2L7ChxaJqgsUk3elCuNuGA4Vc+95vAHx110j25pTRPSqAPTllrD9QiEFB96hA3r52IBabpnOd0qeSwIU4NVxN4vcCTwKf1r7/H/BEi0TUFO1I4tISF8K93li6h0qLDYBf9+SSUVTJzIk92XG4hO+TD2M0KHy9jLTzksFqnuCv/7edHZklzd7PZrNhNBobXNc7Loi/XNKn0f0fffRROnbsyF133QXA008/jVKKFStWUFhYiMVi4bnnnuOyyy5rVlxVVVXceeedrF+/HpPJxCuvvMKYMWPYvn07M2bMoKamBrvdzpdffklcXBxTpkwhPT0dm83Gk08+ydSpU5v1ee7i6uj0cmBWC8fiMovUThfCLf702RayS6r4z/WD+XVPHree05mP1h7kpx2OwWrdogI4v28M3ycf5vL+8W6OVniCadOm8cADDziT+GeffcbixYt58MEHCQoKIi8vj+HDh3PppZc2a/T2m2++CcDWrVvZuXMnEyZMYPfu3bzzzjvcf//9XHPNNdTU1GCz2Vi0aBFxcXF8//33ABQXFzd26DbF1dHpPwGTawe0oZQKBRZorSe2ZHAN0lBjklnMhGhtGw8W8uXGdAAW/H4Qq10ztHM4q/bm83ua4xnw7lEBdIkMYMtTEzCb5NElT9JUi/l4TrbYy4ABA8jJySEzM5Pc3FxCQ0OJjY3lwQcfZMWKFRgMBjIyMsjOziYmJsbl465cuZJ7770XgISEBDp27Mju3bsZMWIEf/vb30hPT+fKK6+ke/fuJCYm8vDDD/Poo49y8cUXc84555zw+bQ2V29MRRxJ4ABa60IgqmVCakJtd7q0xIU49bTWfLb+EL+l5h+zbt7qtGNeJ8YHk9TeUbs8PsSXjuGO+97Bfmb8vNwz9lV4nkmTJvHFF1/w6aefMm3aND7++GNyc3PZsGEDmzdvJjo6utmlYbXWDS6/+uqr+e677/D19WXixIksXbqUHj16sGHDBhITE3nsscd45plnTsVptQpXk7hdKeUss6qU6kj9Cm6tSga2CdF8qbllTPjXL+zPKz/uNst25fDIF8lMnb2WyhobhwoqmPT2aj5ce4BfdudyxYB4vEwG0vIruKhfLDHBPkzs42gdzbogAaNBWt+i+aZNm8aCBQv44osvmDRpEsXFxURFRWE2m1m2bBkHDhxo9jFHjRrFxx9/DMDu3bs5ePAgPXv2JDU1lS5dunDfffdx6aWXkpycTGZmJn5+flx77bU8/PDDbNy48VSfYotx9avyn4GVSqlfat+PAm5vmZCaoBU1RhnYJkRz/fN/u9idXcbXmzJ4aHyPBrdZvC3L+fqnlGx+Tslm/YFCZ8nUMQlRpBwuIbe0mmcudXS/ju4ZRfLTEwjycX3SBiHq6tOnD6WlpcTHxxMbG8s111zDJZdcwuDBg+nfvz8JCQnNPuZdd93FHXfcQWJiIiaTiblz5+Lt7c2nn37KRx99hNlsJiYmhqeeeorff/+dmTNnYjAYMJvNvP322y1wli3D1YFti5VSA4HhOIqmPai1zmvRyBphNUmxFyGaKzXX0QLfnVXa4Hq7XbN0Zy4XJcayLq2Av363nfzymnrbnNMtgs7h/piMivAAb+dySeDiZG3dutX5OiIigjVr1jS43fGeEQfo1KmT8xlxHx8f5s6de8w2jz32GI899li9ZRMnTmTixNYf4nUquPywptY6T2u9ENgB3KGUctvT9FaDkpa4EM1QbbWxN8fxyy+3rBqA/67cz8zPtzjvHW5JLyKvrJrxvaM5u1sE+eU1hPl78fY1A53HCfX3IrFdML1ig1r/JIQQx3B1dHosMBW4GugH/B2Y3oJxNcpiMkhLXIhmSMurwGp3JOviSgtl1VaeXbgDgGFdwlHA0p05GA2K0T0j6RUbRFZxFbeN6sK5PSK5rH8cY3q6ZyyrEH+0detWrrvuunrLvL29+e2339wUkfs0msSVUrfiSNbtgM+AW4BvtdZ/bYXYjstqlO50IZrjYEEFAL1ig8grq2ZXnS71hz/f4nw9qkckIX5ehPh58cltw53LX502oPWCFaIJiYmJbN682d1htAlNdae/CRiBq7XWT2itk3HjqPQjLEZpiQvRlL05pSSnO54MPVSbxPvGBVFcaSE119G1/vQlvZ3bRwV688B53Vs/UCHECWuqOz0OmAy8opSKxtEad/sIFqvRQI2tpukNhThDVVlsXPjqSmpsdr68cyQH8svx9zLSKcKfGqudlMOlmAyKa4Z3pGtUAHEhvnSNDHB32EKIZmq0JV47mO1trfUoYBxQDOQopVKUUs83dXCl1PlKqV1Kqb1KqQbLtiqlpiildiiltiul5rsStNVopNpW7cqmQpyRVuzOpcZmB+BPn21m/YFC+sQFE+Ln+A6+6VAhHcP9MBsNnNM9UhK4EB6q0SReO6ANAK11utb6n1rrQcDlQKNZVCllxNEdfwHQG5iulOr9h226A48BZ2mt+wAPuBK0TRmpsjaveo8QZ5IftmUR6GPinWsHkpZfwfbMEgZ0CCGi9rGwTQeL6CKJWwiP19Q98TlKqbVKqReUUqOVUiYArfUuFwa3DQX2aq1TtdY1wALgj9PQ3Aq8WVvGFa11jitBayXd6UIAvLh4J99syqi3LKV2FrErBsQzrlc0BgWRgd7ccW5X+sYHO7eT1rfwRAEB8nNbV6P3xLXWFyilfIDRwBXAP5VSB4HFwGKt9cFGdo8HDtV5nw4M+8M2PQCUUqtwDKB7Wmu9uKmgtTZSZZOWuDizFZTX8NbyfQAM7hRKlcVGZlEV189ZR2SgN/eM7YbZaGD1rHGE+JnxMRsJ0Rovo4Eam53z+7o+mYQQoj6r1YrJ5P75AZqMQGtdRW3SBlBKdcbRRf6GUipGaz30OLs2VET5jyPbTUB3HF8S2gG/KqX61p1spfYzbwNuA+jj7QPaSF5RHsuXL28qfHECysrK5Nq2gpO9zqszjxY8Ovsfy+qtu7uvYseGtexoYL9nRnpzqNRO0b7N1H4HOG3Jz3LjgoODKS11PG7ovewvGHK2N/sYvhqsxymZb4/qQ/WYxjttn3rqKdq3b8+tt94KwPPPP49SitWrV1NUVITFYuHJJ5/koosucu5zJOY/KisrY/r06Q3uN3/+fF5//XWUUvTp04f//Oc/5OTk8MADD5CWlgbAv/71L2JiYpgyZYrzmfPXXnuNsrIyHn/8cS688EKGDRvG2rVrufDCC+nWrRsvvvgiFouFsLAw3nvvPaKioigrK2PmzJls2rQJpRSzZs2iuLiYHTt28MILLwAwd+5cdu3axd///vd651BVVdWsn1lXi73cA3yktS7SWu8H3gLeUkp5NbJbOtC+zvt2QGYD26zVWluA/UqpXTiS+u91N9JazwZmA/T18dVaGTH5mhk9erQr4YtmWr58uVzbVnCy13n+B+uJDCxi0qB2fLLuICaDIq+shvdvHMKYBCnMAvKz3JSUlJSj04iavcDY/Jal1WbFdLz9zF54NTFN6fXXX88DDzzAQw89BMC3337L4sWLmTVrVr35xKdOneqcT/x4U5/6+vry3XffHbPfjh07eOWVV1i1ahUREREUFBQQGBjILbfcwrhx43jggQew2WyUlZVRWFiIwWBwfoa3tzcWi4XAwECMRiMVFRWsXLkSgMLCQiZPnoxSivfee4+33nqLl19+meeee46IiAi2b9/u3M7Ly4t+/frx73//G7PZzCeffMK77757zLn4+PgwYIDrdRlc/ReLAdYrpTYCc4AftUNjN6Z/B7rXttwzgGk4Kr7V9Q2OYjJzlVIROLrXU5sKRmOkWga2iTPEt5szSC+spGtkALO+SmZIpzBignz4345s7h3bjT9N6Mmj5zsmiLDY7JiNLldTFuKoC144od0q29B84lprHn/88WP2W7p0KZMmTSIiIgKAsLAwAJYuXcoHH3wAgNFoJDg4mMLCwkY/Y+rUqc7X6enpTJ06lcOHD1NTU0Pnzp0BWLJkCQsWLHBuFxoaCsDYsWNZuHAhvXr1wmKxkJiY2MyrdSxXJ0B5Qin1JDABmIGjK/0z4L9a6wY75LTW1toW/I847nfP0VpvV0o9A6zXWn9Xu26CUmoHYANmaq2Pncj4mGMbqbJVuhK6EB6tqKKG+xfUr0z1045sAG4Y0ZF7x9YvziIJXHiiI/OJZ2VlHTOfuNlsplOnTi7NJ368/bTWzlZ8U0wmE3a73fn+j5/r7+/vfH3vvffy0EMPcemll7J8+XKefvppgON+3i233MLzzz9PQkICM2bMcCmepjRnAhQNZNX+sQKhwBdKqRcb2WeR1rqH1rqr1vpvtcueqk3g1LbmH9Ja99ZaJ2qtFxzvWPWOq0wyOl14tPJqK1tzrc7JR45na0ZxvfePXZBAx3A/Ft57Nn+9rC9eJknawvOdqvnEj7ffuHHj+Oyzz8jPd7QRCwoKnMuPTDtqs9koKSkhOjqanJwc8vPzqa6uZuHChY1+Xnx8PADz5s1zLp8wYQJvvPGG8/2R1v2wYcM4dOgQ8+fPZ/r0UzP9iEu/AZRS9ymlNgAvAquARK31ncAg4KpTEklzaBPVMjpdeLCr3/uNlzdUsyW9uMH1uaXVHC6uJPkP628/tyu/zBxT71ExITxdQ/OJr1+/nsGDB/Pxxx+7PJ/48fbr06cPf/7znzn33HNJSkpy3n9/9dVXWbZsGYmJiQwaNIjt27djNpt56qmnGDZsGBdffHGjn/30008zefJkzjnnHGdXPcATTzxBYWEhffv2JSkpiWXLjg48nTJlCmeddZazi/1kqaZaAgC1XeD/1Vof83VIKdVLa51ySqJxQV8fX5344CVsS0hh03WbMBncP8T/dCODgVpep1nfA/DPyUlMGtTumPVnvbCUjKJKRveMJC2vnK/vOosam53oIJ/WDtWjyc9y41JSUujVq9dJHaP0JO+Jn2kuvvhiHnzwQcaNG9fg+ob+TZRSG7TWgxva3qWWuNb6KSC8tkV+r1JqYJ11rZbAnZ+JEUBKrwqPVGWxOV+n5ZUfs76oooaMIseYj+W7cklsF0Kov5ckcCE8WFFRET169MDX1/e4CfxEuPqI2ZPAFOCr2kXvK6U+11o/d8oiaQatHU+2VVgq8Df7N7G1EG3LgfwK5+v9+UeTuN2uMRgUv+zOrbf99KHtEUIc5YnziYeEhLB79+5TflxX+6KvBgbUFn5BKfUCsBFwSxJHOyZxKLOUEUmkW0IQ4kT9tt8xuCbaT7E/15HEv9uSyawvk3n60j488kUy4f5ezL5+MFprBncKc2e4QrQ5Mp/4Ua4ObU0D6vbleQNuq/Vk145JHMpqytwVghAuO1RQwT9/3MXhYkcX+S+7cukY7ke/SCNp+eVYbHYe/mwLFTU2HvkiGYCbzu7MoI6hksCFEI1ytSVeDWxXSv2Eo3TqeGClUuo1AK31fS0UX8OOJHGLJHHRtmUVV3Hhq79SWm1l86EiXpmaxMq9eUwb0h5dfJiKmhr+u3K/c9pQo0Hx0c3DGN5FkrcQommuJvGva/8csfzUh+I6e+098XLLsYOChGhLlqRkU1ptZWCHEFbty+Oi11aigcmD27Npo2PSvhd+2EnnCH9+fGAUhRU1MoBNCOEyVyu2zautk96jdtGu2nrnbqFrW+KlNQ0XwReirVi2M4cOYX68e91gbnx/HdszS7hnTDf6xgeTs1sR6G2itNrKned2xctkkAQuzjgBAQGUlUmv6olydXT6aGAejnvjCmivlLpBa72i5UI7Pm13JHFpiYu2aH1aAX3jgzEoxap9eUwZ3J7IQG++ufss9uaUkRDjeKbWoBRf3TWS39MKmTz42GfFhRCiKa52p78MTNBa7wJQSvUAPsFRsa3V2bWjtVJqkZa4aBssNjsfrT3At5sz2XzIMZPu2d0iqLLYnYPTzEYDvWKD6u3XPTqQ7tFSKEMIrTWPPPIIP/zwA0opnnjiCefkIlOnTqWkpASr1crbb7/NyJEjufnmm1m/fj1KKW666SYefPBBd5+CW7iaxM1HEjiA1nq3UsrcQjE1yaZMmA3elNdIS1y0De8s38fLP9V/BnTl3jwAesdKkhZt3z/W/YOdBTubvZ/NZsNoNDa4LiEsgUeHPurScb766is2b97Mli1byMvLY8iQIYwaNYr58+czceJE/vznP2Oz2aioqGDz5s1kZGSwbds2wFFI5UzlahJfr5T6L/Bh7ftrgA0tE1LT7ErhbfCT0emiVR0uriQtr4IRXcOdyzYeLEQBn204xDndI3h4Qk9sWvPjtizeXeGYVbdzRICbIhbCc6xcuZLp06djNBqJjo7m3HPP5ffff2fIkCHcdNNNWCwWLr/8cvr370+XLl1ITU3l3nvv5aKLLmLChAnuDt9tXE3idwJ3A/fhuCe+AnirpYJqih0jXgZ/SeKi1Vhtds7/968UV1q4Z0w3lIIhncK4fs465za3nN2FpPYhAEQGePPuilSuHBCP0eDaFIhCuJOrLeY/OlW10483j8eoUaNYsWIF33//Pddddx0zZ87k+uuvZ8uWLfz444+8+eabfPbZZ8yZM+ekY/BETSZxpZQRx+Qn1wKvtHxITbNixEv5ShIXrWZrRjHFlY4HMt5YtrfBbcYmRDlftw/z47fHxxHk47a7TkJ4lFGjRvHuu+9yww03UFBQwIoVK3jppZc4cOAA8fHx3HrrrZSXl7Nx40YuvPBCvLy8uOqqq+jatSs33niju8N3myaTuNbappSKVEp5aa3bxCTedqXwUX5SsU20uBd+2El6YQWJtVN/zrlxMN8nZxEf4sNrS/cypmck43vHMKhjKO3D/OrtK4+LCeG6K664gjVr1pCUlIRSihdffJGYmBjmzZvHSy+9hNlsJiAggA8++ICMjAxmzJiB3e4okvT3v//dzdG7j6vd6WnAKqXUd4BzNJnW2i0tcxsGTMqXckueOz5enCGKKmp45xdHdeGs4ipig30YmxDN2IRotNYM6BhK37hgIgO93RypEJ7ryDPiSileeuklXnrppXrrb7jhBm644YZj9tu4cWOrxNfWuZrEM2v/GIAjNz+anoi8hViVERN+lEp3ujgFMooq+XjtAR4a3wOT0cCCdQcJ8fMivfDobGPrDxQypufRyXaUUozpGdXQ4YQQotW4msR3aK0/r7tAKTW5BeJxiTIaMSofqdgmTok7P9pAcnoxYxKiiAjwZtZXWwEI8TMzqGMo5/eJ4W+LUrgkKc7NkQohRH2uJvHHgM9dWNYqjGYTJgIot5RjsVswG2TwkGgerTVKOUaN78pyfBnceKCQAJ+j/yWKKixcMSCea4Z14ILEGNqF+jV4LCGEcJdGk7hS6gLgQiD+yIxltYIAa0sG1hhlMGHSjmdvi6uLifCNcFcowgNprRn/rxV0DPPj1ekDqLY6BsfszSnDZFQE+ZgoqXL8eA/rHIZSShK4EKJNaqolngmsBy6lfnGXUsBtNe5MZhOG2iReWFUoSVw0S2ZxFXtzytibU8aGA4V1lldSVGEhqX0I94zpxsLkw3SNlEItQoi2q9EkrrXeAmxRSs1356xl9WlMJhPK7g9AUfWZW25PnJjdWUfHUvy4PQuAPnFBpOVVkF1SxW2jujCsSzjDuoQf7xBCCNEmGFzcbqhS6iel1G6lVKpSar9SKrVFIzseBV5mI9gcSbygqsAtYYi2L7Ookklvr2bV3vqPIu7KPprEF2/LwstkYGjnMDKKKrHaNUM6h7V2qEIIcUJcHdj2Xxzd5xsAW8uF4xovswmb1RsUFFVJS1w07PP16aw/UMiT32xj4X1nc/Pc9RwsqCCjqJKIAG9qrDYKymvoFRvEqO6RvL8qjfgQX87tHtn0wYUQHsVqtWIyuZryPIerLfFirfUPWuscrXX+kT8tGlkjvIxGbFZHNazC6sImthZniuIKC2NfXs7fvt8BwJpURws8q6SKH7ZmsSY1n4yiSgB6RAcwprZM6sX9YhndM5JXp/Xnq7tGYpBa50K0qssvv5xBgwbRp08fZs+eDcDixYsZOHAgSUlJjBs3DnAUhpkxYwaJiYn069ePL7/8EoCAgKNjV7744gtnGdYbb7yRhx56iDFjxvDoo4+ybt06Ro4cyYABAxg5ciS7djkm57TZbDz88MPO477++uv8/PPPXHHFFc7j/vTTT1x55ZWtcTmaxdWvJcuUUi8BXwHVRxZqrd1SMsfLbKLUYiDQL1DuiQun1fvySM0tJzV3P49f2IvtmSWYjYqKGhvv/LKP6CBvFtw2gqe+3ca9Y7vTKzaQsQlRTOwTg1KKy/rHu/sUhHCbrOefpzql+VORWm02Co4zFal3rwRiHn+8yWPMmTOHsLAwKisrGTJkCJdddhm33norK1asoHPnzhQUOG6bPvvsswQHB7N1q6OWQ2Fh04243bt3s2TJEoxGIyUlJaxYsQKTycSSJUt4/PHH+fLLL5k9ezb79+9n06ZNmEwmCgoKCA0N5e677yY3N5fIyEjef/99ZsyY0Ywr0zpcTeLDav8eXGeZBsae2nBc42U2U1VmI8QnhMIqaYkLh7T8oxXWHv96K6VVVu4e05U3l+1jT04Zl/WPo3OEPx/ePMy5nSRuIdzvtdde4+uvvwbg0KFDzJ49m1GjRtG5c2cAwsIc41SWLFnCggULnPuFhoY2eezJkyc75zsvLi7mhhtuYM+ePSilsFgszuPecccdzu72I5933XXX8dFHHzFjxgzWrFnDBx98cIrO+NRxKYlrrce0dCDN4WM2UGWxEOMdKkn8DFdjtbPjcAn924dwsMBZ1p9P1h1iXEIUt43qyq6sMpakZHPFAEnYQhyPKy3mhpzsVKTLly9nyZIlrFmzBj8/P0aPHk1SUpKzq7uuukWa6qq7rKqqqt46f39/5+snn3ySMWPG8PXXX5OWlsbo0aMbPe6MGTO45JJL8PHxYfLkyW3ynnqj98SVUv+u8/r+P6yb20IxNU7VtsQtdkJ9QqU7/Qw366tkLn9zFUt2ZLMzqxRTnfvZL1zVj2BfM69PH8BXd41ktNQ6F6LNKS4uJjQ0FD8/P3bu3MnatWuprq7ml19+Yf/+/QDO7vQJEybwxhtvOPc90p0eHR1NSkoKdrvd2aI/3mfFxzu+zM+dO9e5fMKECbzzzjtYrdZ6nxcXF0dcXBzPPfdcm53utKmBbaPqvP7jNDL9TnEsLvMym6i22gj3DSevUmYyO1OtTyvgq40ZAMxZtZ9tGcXcMLITAHHBPs7ZxXy9jAzs0HS3mxCi9Z1//vlYrVb69evHk08+yfDhw4mMjGT27NlceeWVJCUlMXXqVACeeOIJCgsL6du3L0lJSSxbtgyAF154gYsvvpixY8cSGxt73M965JFHeOyxxzjrrLOw2Y4+aHXLLbfQoUMH+vXrR1JSEvPnz3euu+aaa2jfvj29e/duoStwcprqG1DHee1W3mYjVRY70X7R5FXmSf30M9SirY5nvK8cEM+C3w8BMKhjKFMGt5fpQYXwEN7e3vzwww8NrrvgggvqvQ8ICGDevHnHbDdp0iQmTZp0zPK6rW2AESNGsHv3buf7Z599FgCTycQrr7zCK68cO7v2ypUrufXWW5s8D3dpqiVuUEqFKqXC67wOU0qFAQ0PR2wFPmYTVRYb0X7RaDS5FbnuCkW0oqziKp5buINOs77ngzVprE3NZ0inUGfrG6BrZAA9YwIJ8/dyW5xCiNPDoEGDSE5O5tprr3V3KMfVVEs8GEeBlyOt8LqPlLllPnGFI4lb7ZpIX8c9zuyKbOICZJrI00V+WTUhfl6k5pZhMChn/fL7F2zit/2Oe1VPfbsdgLvHdKVXbJBz347hMlGJEOLU2LBhQ9MbuVlTtdM7tVIczeJjdnQghHg7knhWeZY7wxGnUEWNlUHPLWHq4PZ8ut7RRb79rxPx8zKyI7OEixJjuWpQPDfNXQ9Av3YhADw8oQdrUwvwMbutg0gIj3S8kdmi9Wnd/LaxqxXbnJRSTzdj2/OVUruUUnuVUrMa2W6SUkorpQYfb5ujG4Oft+O7R6DJMUFFdnm2qyGJNu5IS/tIAgd495d9ZBRVUlptZUTXcMYmRDvX9WsXDMA9Y7vz0S3DEEK4zsfHh/z8/BNKHuLU0lqTn5+Pj49Ps/Y7kYfeLgWebmojpZQReBMYD6QDvyulvtNa7/jDdoHAfcBvrgYQWJvEtc0Hf7M/WRXSEj8daK15bmG9Hw8SYgL55PdDRAU5frCPjDLvFRtEyuESYoKa9wMvhDiqXbt2pKenk5t74uOKqqqqmp14RMN8fHxo165ds/Y5kSTuar/LUGCv1joVQCm1gP9v787jqyrvxI9/vnfPHgIkkIQtJLJvgmxFxWpba620U62KTq0/q7ad/sbu08Vxaus43cY6VrvY1rZ2Ea3WFjtuiImKIquALAIBIkkIJCRkQdjmFAAAIABJREFUT27u8swf5+QmxCRcJTc3N/m+MS/uec5zzv3eh2u+5zznOc+BVcDeHvW+B/wQ+Gq0AaTaSbzZHyQnOUfPxIeJyvo2DtW08Ilz83liewWTRydz04opfO3xXTz48mEmZCUxY7w1qcRjty6ltSOk3YBKnQW32x2ZFe29KikpYcGCBQMUkXq33ksSXxhlvTygvNtyBV3TtwIgIguACcaYf4hI9EncZyfx9iDjU8dT2VwZ7aZqCHrpQA3zJ2SypczqSr9pxRRuuaCA3Ewfh2usWdiO1rVy8/lTIkk7zecmzae3FSqlRraokriI/BC4C2gDnhWRecAXjTF/7G+zXsoiF15ExAH8BPh0FO9/C3ALwOxkL/t37wBg47YduNwujjQfobi4WM/KBkhzczMlJSWD8l6bjwf52Q4/s8c4cQgkueD4/m04RKgC2oNd1+oKOU5JSfWgxDUYBrOdRypt49jTNo6vaM/EP2iM+bqIfBzrjPoqoBjoL4lXABO6LecDx7otpwGzgRI7+Y4D1orIFcaYrd13ZIx5EHgQYE6qz6x831J4rYTJhdMZl7yMlza/xNylcxmdNDrKj6P6U1JSEplT+Gw9trWcx7dW8PBNiyMjx9sDIbwuBzVNfu588HXAz+6T1uxJl8zI4f0XnT6+8RfjqsgflczsvIwBiWmoGMh2Vr3TNo49beP4ijaJd/ZbXgY8Yoypi+KsdwtQJCJTgErgGmB150pjTAMwpnNZREqAr/ZM4L3pfk28IGciAEebjmoSH2KMMXz98V0A/HV7JauXTKS22c/Cu17gzitm8eiWcspqW/jF9efyj11VTB+Xxj8vm/yO/Vw6u+9pFJVSaiSLNok/JSJvYXWnf15ExgLt/W1gjAmKyBeA57Bmd3vIGLNHRL4LbDXGrH0vAXvSw13XxP1BJqZbSfztxrdZkK2DK4aSPccaI69fPlDD6iUTWbfXGoR499P78AfDfO1D07h09nhN1Eop9R5E+yjSb4jID4BGY0xIRFqwRpqfabungad7lN3RR92V0cQiDsHrcuJxOmj2B8lNnYRTnBxtPBrN5moQvVpqPZzm4unZbC6rwxjDZnvwmj8YBuDDs8fFLT6llEp0UU32IiJXAUE7gd+OdS08rvOcpvpcNLcHcTvc5KbmcrRJk/hQU1nfRrrPxcppY6lr6eBYQzt7KhtJ9ljXxsekepgyJuUMe1FKKdWXaLvT/90Y8xcRWQF8CPgx8HN63DI2mFK9Lpr91rNfJ6ZP1DPxIeLJNyo4b3IW+aOSOVbfTm5mEnPsqVEf3ljG/hNNfPGSIpZPHUOyx6l3FCil1FmIdtrVzgevfgT4uTHm70CcHhNl/dJP9bpoareS+KS0SRxtOqpTB8ZZaXUzX3p0Jyt+UMyJxnZKq5vIy0xidm46HpeDX750GICPzBnP4ilZw260uVJKDbZok3iliPwS+CTwtIh438W2A8s+cbPOxAOAdSbeEmihrr0uLiGNdAdPNLH96KnIoDWAJXevp6y2lSUFWbicDv5lZSEicMflMynKSYtjtEopNXxE253+SeBS4MfGmHoRGQ98LXZhnVmqz0V1kzVAfmKa3mYWLy3+IB/4ycunlTkdQihsSPO6uGlFAQC3XVLEZ1cW4HXpU8aUUmqgRHU2bYxpBQ4BH7JvG8s2xjwf08j61NWd3tzedU0crNvM1OB6s7LhtOXzJo/igdXWrX4XTc/G6ei65q0JXCmlBla0067eBtwM/NUu+qOIPGiM+WnMIjuDVF/XwLbc1Fxc4tIkPkjufeEAuysbuP0jM9l/vAmAjd98P6+V1jInP4OCMSnc9bHZ/NO5eXGOVCmlhrdou9NvApYYY1oA7HvGNwJxS+LpPjcNbQGMMbgdbiamT+RQ/aF4hTOi3Lf+IGEDNU1+po9LJzPZzbh0H59Y2PUIveuXTopjhEopNTJEOzhN6Bqhjv06TvcGWW87Ns1LIGSob7UGtxWNKqK0vjQ+IQ1Dh2ua+e2rRwiHTx/xX1rdTNjAuHQfOysaeHRrOQsnjtJbxZRSKg6iTeK/BTaJyHdE5DvA68BvYhZVf+xckZ3mBaC6yQ9AYWYhFU0VtAZa4xLWcBI2hovveYk7n9rL64drI+WPbjnKJfe8BMA9n5xHZ94+v2hMb7tRSikVY9FOu3qP/YCSFVhp9EZjzBuxDOxMOpN4TZOfaePSKMoswmA43HCY2WNmxzO0hLe5KkTnLffP7TnO3AmZPPjyYe5bfxCAf146ieWFY3jk5qW8XdvClQsn9LM3pZRSsXLGJG4/93uXMWY2sD32IZ2JdfqXne4DiNxmVjiqEICDpw5qEj9Law93MHN8OqleFzvK61mz+Wgkgf/55iUsn2qdeS8tGM3SAr2lTyml4uWMSdwYExaRnSIy0RgzZOY27dmdnp+aj8/p0+viZ+nt2haONRs+szKf2hY/DxQfYmdFAxOykvivj8+NJHCllFLxF+3o9PHAHhHZDLR0FhpjrohJVFFI8bpI9jipbrSSuNPhpCCzgIOnDsYrpIS17e06fvzcAe5cNYsNB60nj10yIwd/MMQDxYfIy0ziD/9vCZP1YSVKKTWk9JvERaQQyAHu7LHqQqAyVkH1r2sUdHaaN9KdDtbgto3HNsYjqIT02JZyNh6uZe+xRvafaOJnxaXUNPvJTRUmjk4G4IUvX0D+qGR8bp2oRSmlhpoznYnfC3zLGLOre6H9PPH/IB4j1LvdyZSd5ot0pwOcM+oc1h5aS317PZm+zEEPLZF0BMN8/YnT/lnZfrSeY/VtfHBS19eiMFvnOVdKqaHqTLeYTe6ZwAGMMVuByTGJ6ExGF0ZeZqd7OdF4+pk4wMF67VI/k2d2V0VeXzEvl+uXTuRoXSvBsGHm6Pg820YppdS7c6bf1r5+1iUNZCBRc3SdJRaMSaG8rpX2gDUPTWcS18FtvatubI88rvX3r5VRMCaFQ3dfxn3XLuD8orGRegUZ2nWulFKJ4ExJfIuI3NyzUERuArbFJqToFeakETZQVmuNtctOzibdk07pKU3iPf338/tZfPd6pnzzaXaU17OjvJ7L5oyPPKDkwnPGkpeZxFUL80l26+xrSimVCM50TfyLwJMich1dSXsR4AE+HsvAolE4NhWAgyeamT4uHRGhMLNQu9O7Mcawu7KRn5V0zSv/8MYywgYKxnaNNve5nbzy9YtwOISSkpLBD1QppdS71u+ZuDHmhDFmOdbo9DL7505jzDJjzPHYh9e/grEpOMSaz7tT0agiSk+VRrqNR7r/fbOKj96/gVDY8Nsbz2NCVhJ/3W7dWNDzljGHQ8/AlVIqkUT7PPFiY8xP7Z8XYx1UtHxuJxOykjlY3RQpK8osoinQxInWE3GMbPDVNvsp+vbTlOyvjpQZY/jVy4cB+PDscSwrGM2iSVkATBmTwvRxOvJcKaUSWbSTvQxZM8ens6uiIbI8LWsaAHtO7mFcyrh4hTXoXjpQQyBk+M2GI6yclg1Yt4ztrGjgu6tm8allkwH498tnMmVMCjcsn0yyJ+H/+ZVSakRL+HuJzpucRcWpNo7VtwEwc/RMPA4Pb1TH9fksg67zQCbV25WYf/vqEdJ8Lj5xbtdzvrNSPPzrxUVkJLkHPUallFIDK+GT+OIpVvfwlrI6ADxOD9NHT2d37e54hhUznbfTdTrZbE12U3HKOog5Wmc9irW22c8zu49zzXkTSPHqGbdSSg1HCZ/EZ9hP29p0pK6rLGsG++v2EzbhOEY28L771F5m3PFs5IDlf3dVseiuF/jTprcjPRFHTrZQXtfKt558k1DY8NF5ufEMWSmlVAwlfBJ3OoSFk0axpVsSn541neZAM5XNcZrePUaK91djDPxk3QEaWgN8w5429a5/7GNvVSMArR0hzv9hMc/tsQb2zc7NiFu8SimlYivhkzhYXeoHq5uptbuWp2dNB+CturfiGdaAMsZEpph97VAtz+05TpM/yNc+NI02u4v94unZkfrZaV4eu3WZ3jamlFLD2LBI4sumjgZg4+FawLpX3ClO9tXui2dYA+pkcwetHSGuXTwBgLU7jwFw2ZzxJNlPGPviJedE6r/0tYsi4wWUUkoNT8NixNPcvAzSvC5eLa3l8rm5eJ1epmRMYf+p/fEObUBUN7bz0xetqWQ/OGscT2yvZEOp9dzv3Ewf6758AS8dqGF2XjpPfn45ToeQ5NH5z5VSargbFknc5XSwpCCL1w6djJRNz5rO5qrNcYxq4Nz6x228cbQegAUTMllWMJqXDtSwtCALr8tJ/qhkrlsyyVo/cVQ8Q1VKKTWIhkV3OsDyqWN4u7aVilPWLVbTs6ZT3VZNbVttnCN774wxfGftnkgCXzRpFJnJHn505VzuX72ANbcsi3OESiml4mlYnIkDrCgaA8CGgye5ZvFEZmTNAGB/3X6W5y2PZ2jvyd5jjXzylxtp9gcBKPnqSiaNTgYgO93H5XP11jGllBrphs2ZeFF2KnmZSbywz5o7vHP61X11iTW4bf2+E8y783kuu++VSAJ/8SsXMnlMCiI60lwppVSXYZPERYQPzMxhQ2kNbR0hMrwZ5Kbksr8usQa3rdlSTkNbILI8LSeNAvuRq0oppVR3MU3iInKpiOwXkVIR+UYv678sIntFZJeIrBeRSWfzfh+YmUN7IMwrB2sA67r4UD4Tf630JH/edDRyf3uLP8jLB2q45rwJPHPb+Ry468M89f9XxDlKpZRSQ1XMkriIOIEHgA8DM4FrRWRmj2pvAIuMMXOBx4Efns17Lp6SRbrPxfN7rdnK5mfPp6yxjOrW6jNsOThCYcOWsjrqWjpo8QdZ/etNfOvJN1l41wtc/tNXKN5fjT8Y5mML8pgxPh2Py4HHNWw6S5RSSg2wWGaIxUCpMeawMaYDWAOs6l7Bfk55q734OpDPWXA7HVwyI4fn9hynPRBiWa41entT1aaz2e2A+fyftnHVLzbyvX/sjfQWdNpd2cj9L5YyOsXDeZN1khallFJnFsskngeUd1uusMv6chPwzNm+6ccW5NHUHqT4rWqKMotIcaews2bn2e72rDW0BVhn9xCs23uC4rdq8LocVpf5F6wu87eON3HD8sk4dapUpZRSUYjlLWa9ZSLTa0WR64FFwIV9rL8FuAUgJyeHkpKSPt80bAwZXuFX63aSVOsjz5nHa0deo6S9720Gw86aIGEDnzzHzWMHAjy6tZx5Y528tuFlwsYwJkloCxpmSgUlJfF9cEtzc3O/bawGhrZz7Gkbx562cXzFMolXABO6LecDx3pWEpFLgG8DFxpj/L3tyBjzIPAgwKJFi8zKlSv7feOrWvby+41lzF+8nAvf2sVDux9i8fsWk+xOfk8f5Gz4gyFONnew541KYD+3r76IJ767jlDYcMeVi1k4yeo6f/X8EG0dITKTPYMeY08lJSWcqY3V2dN2jj1t49jTNo6vWCbxLUCRiEwBKoFrgNXdK4jIAuCXwKXGmAEbffaxBXn8esMR/rGrinl58wiZEHtq93DeuPMG6i2iUlrdxKd+s5njje1MyEomf1QS6T43f7xpCduPnookcACvy4nXpfOdK6WUil7MrokbY4LAF4DngH3AY8aYPSLyXRG5wq72IyAV+IuI7BCRtQPx3rNy0zknJ5W/vVHJ/Oz5AGw/sX0gdt2v7z/zFl9/fCeN7dZ93rf/bTfHGtoJG3i7tpVrF08ErKeu/ctFhTGPRyml1PAW02lXjTFPA0/3KLuj2+tLYvG+IsLHFuTxw2f309DspjCzkO3VsU3idS0d/OKlQ4A1QO13Ny5m05E6bru4iE8tm0RNs5/p49JjGoNSSqmRZdjehLxqvjUQ/m87KlmYs5Ad1TsIhoMD+h7GGE61dABEbhn75KJ8dlU0cN/6gxgDF5wzltGpXk3gSimlBtywTeJ5mUksLcji8W0VLMheSGuwlV01u856v20dITYeqiUQCvPQq2Us+N461u09wYaDJ8lIcvPty2aSmezmd6+V4XM7mJefMQCfRimllHqnYZvEAW5aUcDRulaqjk3E5XBRXF58VvszxnDdr1/n2l+9zqNbynlsi3Ub/F+3V/Bq6UmWTx1NRrKbn167AKdDuGH5ZFzOYd3ESiml4mhYZ5hLZmSzZEoWPys+xrnZiyguL8aYXm9V79MfNpbxQHEpAJuP1LHdfrb333dUcqC6CYBndh/nWEN75HGo5xeNZed/fJBvfnjGwH0YpZRSqodhncRFhNs/MpO6lg6CTTN5u/FtjjQciXr7E43t/Pvf9/Cj5/bTHgjxnaf2MirZzeolE9lSdgpj4NPLJwMwLt3H5XO6nvGd6h02j2pXSik1RA3rJA4wJz+D65ZM5OUdOQC8WP5i1NvurmyIvH619CT7qhq5+YICrlxoTfHucTn4/EVTeeTmpTz/5QvISHYPbPBKKaVUP0bE6eIdH53JjvJ6Kjom8PyR9Xxmzmei2m5nRVcSf2TzUQDmT8jk3ImjeOoLK8hMdpOd5iM7zReTuJVSSqn+jIgk7nU5+c+Pz+G6v8xin+dZ3jxezpxxXTPC7iyvRwTm5meypayOz/1xO0XZqew51sC8CZlUnmrlhX3WhHKzcq3R5nN01LlSSqk4G/bd6Z3mT8jkvy69FoDr/vwg31m7h6O1rTS2B1j1wKtccf+rHDnZwreffJOTzX72VjXS2B7kynPzmD9hFGDdtpaRpF3mSimlhoYRcSbe6YqZ5/LQ/hmUhzfxyOZlPLL5KMunjo6sv+jHJQD84vpzWVE0luMN7RRmpzIuI4kX9p3gwmlj4xS5Ukop9U4j5ky806fnrKbDUcX/fDqdqWNTKd5fQ066l6e+sILlU0fzrcumc+ns8aR6XRRmpwLwgZk5rP/KhXxv1ew4R6+UUkp1GVFn4gCXTr6UH235Ec+VP8ETn/sxrx+pZcKoJAqz0/jzzUv73G7q2NRBjFIppZQ6sxF3Ju5z+bjynCt5sfxFav1VXDQtm8LstHiHpZRSSr1rIy6JA6yevhqHOPjdnt/FOxSllFLqPRuRSTwnJYdPFH2CJw48QVlDWbzDUUoppd6TEZnEAT4777N4nB7+e9t/v+v51JVSSqmhYMQm8TFJY7h13q2UlJfw+MHH4x2OUkop9a6N2CQO8OlZn2bxuMXcveluth7fGu9wlFJKqXdlRCdxhzi4Z+U9jE8Zz+2v3k5roDXeISmllFJRG9FJHCDDm8H33vc9jjUf455t98Q7HKWUUipqIz6JAyzMWcj1M6/n0f2P8sudv4x3OEoppVRURtyMbX350sIvcar9FPfvuJ8MbwbXTL8m3iEppZRS/dIkbnM73Ny5/E4aOxr5z03/SWl9Kf+2+N9wO/SpZUoppYYm7U7vxuP0cN9F93Hj7Bt5dP+j3LruVura6+IdllJKKdUrTeI9OB1Ovrzwy9y94m52Vu/kwkcv5NZ1t9ISaIl3aEoppdRpNIn34aNTP8rDlz3M1dOuZlPVJlY8soJvb/i23oamlFJqyNBr4v2YNXoWs0bP4uKJF/P7Pb9n7aG1FB8tZlXhKm6eezNZvqx4h6iUUmoE0yQehWW5y1iWu4xtJ7ax5q01rHlrDcXlxcwdM5cPTv4gF0+8GBGJd5hKKaVGGE3i78LCnIUszFnIjuodfH/z99lwbAPPlD2Dy+FiRd4KVk9fzawxs0j3pMc7VKWUUiOAJvH3YH72fNZcvgZ/yM+at9aw9cRWtp3YRkl5CS6Hi2XjlzF37Fx8Th//dM4/aVJXSikVE5rEz4LX6eWGWTdww6wbqG+v5+XKlzlQd4B1b6/jlcpXAHhgxwOMTx3PnDFzmDd2HnmpecwaPYtMX2aco1dKKZXoNIkPkExfJldMvQKmwlcWfYVAOMDB+oM8degpSk+VUlxezNpDawFwiYtkdzLnjDqHqZlTyU/NJ9OXSU5yDjOyZmiCV0opFRVN4jEgInicnsjodgBjDFUtVVQ2V7Lx2EYa/A3sq9vH00eepqmjqWtbhOzkbLJ8WaR70slKyuLc7HPJ9GbiD/kJmzAFmQUUZRbhcrjwOD3x+phKKaXiTJP4IBERclNzyU3N5bxx5522rsHfQFljGW3BNnZU76CiqYJT/lM0dTSxqWoTzxx5ps/9FmYWkuHNIBQOkenNZGL6RLKTsxmbNJaxyWOZkDaBVHcqXqcXt1OnkFVKqeFEk/gQkOHNYN7YeQAsHb/0tHWhcIja9lrq/fUcbzlOfmo+RxqOUNZYRoO/gUMNh2j0N3Ky7SRHm46yoXIDQRN8x3s4xUlOcg4uhwunw4nL4aIwo5BMXyY+pw+fy4fBkOxKpr2tnfQT6fhDfvJT80nzpOFz+fA6vXornVJKDSGaxIc4p8NJdnI22cnZnDPqHAAKMgv6rG+MoSnQRE1rDSdaT1DeWE5ToInatloaOxoJhoOETIiWQAvbqrfRHmynPdhOR7jjtP387NmfvTMWcZLqScXtcONz+kjzpJHsTibNk4bX6cXr9OJxevA4PLQGWwmbMNNGTaM12Mrk9MmRg4HmjmYmZ0wm1Z1q1Xd6cDvcOEQnEFRKqXcjpklcRC4F/gdwAr82xny/x3ov8DCwEKgFrjbGlMUypuFOREj3pJPuSWdq5lTIjW67QChAQ0cDboebx4ofY9bcWbjERWVzJS2BFtpD7bQGWjnlP0VbsI1wOExLsIWWQAtVzVX4Q378IT8doQ4C4QAGg0MckcF80XCJ1UsQMiHcDjfJrmQc4iDJlYTBkOpOJdWTSoo7Bac4CYVDFGQW4BQngXCAuvY60j3pZHgzSPekW70O4oz0PLjE9Y6yYDgYqQ/WgUqmLxOXuHCIQ3sflFJDWsySuIg4gQeADwAVwBYRWWuM2dut2k3AKWNMoYhcA/wAuDpWMam+uZ1uxiSNAaDIV8Ty3OVnvU9jDNWt1aR50ihvKqc12Bp5kMyJlhN0hDsiST8QCtAR7iAUDhE2YcKEafA3ANDc0YzP5aMl0EJTRxNVzVWRM/1Xj72KMSZyKaA50IzBnHXsnRzi6Er+Diduhxun2AcF9gFB5+vu9XoeNETq2D+BUIA3q95k3YZ1JLmScDvceJwe/CE/guB2uhGEJFcSLod1QOEUJ05x4nF6TitziAOnw9n12q7nFCcOhwMHDkImhIhQ01qDy+EixZ1CkisJn9OHw+Hoc1unOCPt0NlTEgpb+3KIA0EiAyyNMZFyYwzBcJBkd7L2sCgVQ7E8E18MlBpjDgOIyBpgFdA9ia8CvmO/fhy4X0TEGDNwv4VV3IgIOSk5AEzLmhbz9wubMCEToj3YTiAcIBQOETKhyCWEYDh42uuQCSEINW01BEJW70HIhDjVfspKQiZIa6CVoAkSCp++fSAc6NqPvS5gApHXHaEOWsOt1jY9tu8cs5Aqqbx+7PXIwUxHuAOf0weAP+THYCXCROcUJ4JEEnxnUo+U4QDpWo78EXlHWWe9zgOIzl6S7vsX7DIR2lrbuPdv91r76GXfwGn76u09OvfVa7w9tnPgeHfxRvvZ+4i3v8/e+cchfcTb43NF9mWv676NQxyn9Uh13/5ww2GO7D5yWllv9brH2l3PWPqs12N9z7Le6vWlezx9rTvtffr5bN13EVX9vmLquT7KDsBYJvE8oLzbcgWwpK86xpigiDQAo4GTMYxLDVOdCcLtSYxR+CUlJaxcubLP9cZYBxWdByedByUdoY7IAUTnurAJR5Yj9e1tOi9tGGPI8GbgFCctgRZagi3WvsIhwoQjvSBBY+/X3jZswhiM1eNhDA6HIxJf2IQjvSlApBckbMJ4nV5aAi0Ew8HI9mHCke06yzrrR97D3kf3+p377lz/jr/7WFcdqGZs5tjT3xOD9Z85La5IWT/7DIfDhAj1vq/OOib8zni7tV/35Wg+e2/xRvPZT9tXP599QHqutp39LtR7E8sk3ttxRM9vSzR1EJFbgFsAcnJyKCkpOevgVN+am5u1jQdBPNq5meZ3lAmC0/4z3DQnNZNKqvWbRoc2nFH3A6aef3c/KIjUx9DS0kJKSso76nWvY7/oc1338tNem3eW91U38v5RduT2dvDSX1mv63p5rzPto7cz8v4OpAyGT/GpPtfHMolXABO6LecDx/qoUyEiLiADqOu5I2PMg8CDAIsWLTL9nb2os3emM0Q1MLSdY0/bOPa0jWOvvyQeyxEnW4AiEZkiIh7gGqDnUOW1wA326yuBF/V6uFJKKRWdmJ2J29e4vwA8h3WL2UPGmD0i8l1gqzFmLfAb4A8iUop1Bn5NrOJRSimlhpuY3idujHkaeLpH2R3dXrcDV8UyBqWUUmq40hs4lVJKqQSlSVwppZRKUJrElVJKqQSlSVwppZRKUJrElVJKqQSlSVwppZRKUJrElVJKqQQliTZBmojUAG/HO45hbgz6EJrBoO0ce9rGsadtHHuTjDFje1uRcElcxZ6IbDXGLIp3HMOdtnPsaRvHnrZxfGl3ulJKKZWgNIkrpZRSCUqTuOrNg/EOYITQdo49bePY0zaOI70mrpRSSiUoPRNXSimlEpQm8RFGRCaISLGI7BORPSJym12eJSLrROSg/fcou1xE5D4RKRWRXSJybnw/QWIREaeIvCEi/7CXp4jIJrudHxURj13utZdL7fWT4xl3ohCRTBF5XETesr/Ty/S7PPBE5Ev274vdIvKIiPj0uzw0aBIfeYLAV4wxM4ClwL+IyEzgG8B6Y0wRsN5eBvgwUGT/3AL8fPBDTmi3Afu6Lf8A+IndzqeAm+zym4BTxphC4Cd2PXVm/wM8a4yZDszDamv9Lg8gEckD/hVYZIyZDTiBa9Dv8pCgSXyEMcZUGWO226+bsH7p5QGrgN/b1X4PfMx+vQp42FheBzJFZPwgh52QRCQf+Ajwa3tZgPcDj9tVerZzZ/s/Dlxs11d9EJF04ALgNwDGmA5jTD36XY4FF5AkIi4gGahCv8tDgibxEczu5loAbAJyjDFVYCVHMzGxAAADvUlEQVR6INuulgeUd9uswi5TZ3Yv8HUgbC+PBuqNMUF7uXtbRtrZXt9g11d9KwBqgN/alyx+LSIp6Hd5QBljKoEfA0exkncDsA39Lg8JmsRHKBFJBZ4AvmiMaeyvai9lekvDGYjI5UC1MWZb9+Jeqpoo1qneuYBzgZ8bYxYALXR1nfdG2/g9sMcUrAKmALlACtaliZ70uxwHmsRHIBFxYyXwPxlj/moXn+jsWrT/rrbLK4AJ3TbPB44NVqwJ7H3AFSJSBqzB6nq8F6sL12XX6d6WkXa212cAdYMZcAKqACqMMZvs5cexkrp+lwfWJcARY0yNMSYA/BVYjn6XhwRN4iOMfW3qN8A+Y8w93VatBW6wX98A/L1b+afskb1LgYbOrkrVN2PMN40x+caYyViDgF40xlwHFANX2tV6tnNn+19p19ezl34YY44D5SIyzS66GNiLfpcH2lFgqYgk278/OttZv8tDgE72MsKIyArgFeBNuq7VfgvruvhjwESs/2mvMsbU2f/T3g9cCrQCNxpjtg564AlMRFYCXzXGXC4iBVhn5lnAG8D1xhi/iPiAP2CNUagDrjHGHI5XzIlCROZjDRz0AIeBG7FOTvS7PIBE5E7gaqy7W94APoN17Vu/y3GmSVwppZRKUNqdrpRSSiUoTeJKKaVUgtIkrpRSSiUoTeJKKaVUgtIkrpRSSiUoTeJKjTAiEhKRHd1++pvl7N3ue7KI7B6o/Sml+uc6cxWl1DDTZoyZH+8glFJnT8/ElVIAiEiZiPxARDbbP4V2+SQRWW8/g3u9iEy0y3NE5EkR2Wn/LLd35RSRX9nPn35eRJLi9qGUGuY0iSs18iT16E6/utu6RmPMYqyZze61y+7HeoTnXOBPwH12+X3AS8aYeVhzlu+xy4uAB4wxs4B64BMx/jxKjVg6Y5tSI4yINBtjUnspLwPeb4w5bD8k57gxZrSInATGG2MCdnmVMWaMiNQA+cYYf7d9TAbWGWOK7OV/A9zGmLti/8mUGnn0TFwp1Z3p43VfdXrj7/Y6hI69USpmNIkrpbq7utvfG+3Xr2E9iQ3gOmCD/Xo98DkAEXGKSPpgBamUsugRslIjT5KI7Oi2/KwxpvM2M6+IbMI6wL/WLvtX4CER+RpQg/WkMIDbgAdF5CasM+7PAfpoT6UGkV4TV0oBkWvii4wxJ+Mdi1IqOtqdrpRSSiUoPRNXSimlEpSeiSullFIJSpO4UkoplaA0iSullFIJSpO4UkoplaA0iSullFIJSpO4UkoplaD+D2QEbtB5P4bGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.DataFrame(history.history)[1:].plot(figsize=(8, 5))\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Cross-Entropy/Accuracy')\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model does not perform bad, but it can be improved. For example, you can see that the validation accuracy is about 90% and that the validation cross-entropy degrades with the epoch (maybe an early stopping with tweaking of some other hyperparameters might help in this case; try as an exercise).\n",
    "\n",
    "Next, let's assess the performance on the test set: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "15/15 [==============================] - 0s 133us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[8.136499673128128e-05, 1.0]"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(X_test_scaled, y_test) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Last, the predicted probabilities for each class (for the first test unit):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 1., 0.]], dtype=float32)"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predict probability of class 1 for the first unit of the test set:\n",
    "y_proba = model.predict(X_test_scaled[1].reshape(1,-1))\n",
    "y_proba.round(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "...and the same unit's predicted class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1], dtype=int64)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predict class for the first unit of the test set:\n",
    "y_pred = model.predict_classes(X_test_scaled[1].reshape(1,-1))\n",
    "y_pred "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Interfacing Keras with scikit-learn for network tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is possible to interface Keras with scikit-learn to tune the model with cross-validation. Here, we will see an example with the Heart dataset, therefore we are in a binary classification case. However, the implementation for multiclass classification and regression is straightforward. \n",
    "\n",
    "We first load the data: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_train = pd.read_csv(\"./data/heart_data/heart_train_processed.csv\")\n",
    "data_test = pd.read_csv(\"./data/heart_data/heart_test_processed.csv\")\n",
    "X_train = data_train.drop(\"y\", axis=1)\n",
    "X_test = data_test.drop(\"y\", axis=1)\n",
    "y_train = data_train[\"y\"]\n",
    "y_test = data_test[\"y\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To start with, we need to wrap the Keras model in scikit-learn. First, we create a function that takes as arguments the hyperparameters that we want to tune (plus the number of neurons in the input layer), and that returns a compiled Keras model. In this case, we use the SGD optimizer, but of course you can use the ADAM or any other optimizer you desire (in addition, you can also decide to tune the type of activation function -here set as ReLU- of the hidden layers, as well as the  $l_1$ and $l_2$ regularizers, by adding these hyperparameters to the arguments of the function): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "def build_binary_model(n_hidden=1, n_neurons=7, learning_rate=1e-3, input_shape=[19]):\n",
    "    model = keras.models.Sequential()\n",
    "    # This is an alternative way to declare an input layer: \n",
    "    model.add(keras.layers.InputLayer(input_shape=input_shape))\n",
    "    # Add as many hidden layers as the ones specified by the model:\n",
    "    for layer in range(n_hidden):\n",
    "        model.add(keras.layers.Dense(n_neurons, activation=\"relu\"))\n",
    "    model.add(keras.layers.Dense(1, activation=\"sigmoid\"))\n",
    "    optimizer = keras.optimizers.SGD(lr=learning_rate)\n",
    "    model.compile(loss=\"binary_crossentropy\", optimizer=optimizer, metrics=[\"accuracy\"])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's wrap the function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "keras_binary_classifier = keras.wrappers.scikit_learn.KerasClassifier(build_binary_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And this is it! We can now use `keras_multi_classifier` as if it was a normal scikit-learn object, with the usual `fit` and `predict` methods. The last arguments of this function can be arguments of the Keras library; in this case, we use the `validation_split` parameter with 5% of the training data, as well as an early stopping callback: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "kmc = keras_binary_classifier.fit(X_train, y_train, epochs=700, validation_split=0.05,\n",
    "                                 callbacks=[keras.callbacks.EarlyStopping(patience=10)], verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the usual `score` method of scikit-learn, we can assess the performance of the model on the test data: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "61/61 [==============================] - 0s 82us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.7704917788505554"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras_binary_classifier.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Tuning with cross-validation**. We are now going to perform tuning with cross-validation. Given the large number of hyperparameters that need to be optimized, we will use random search, rather than Grid Search (which is nonetheless still possible, if you prefer manually specify a grid of value). As it is highly unlikely to find the best setting for a Neural Network with just grid/randomized search methods, a good tip would be to use these methodologies to find a good initial setting for (some of) the hyperparameters, and then manually refine these (and other) hyperparameter, with a validation set. Moreover, for large datasets tuning a Neural Network with cross-validation is not recommended given the large computational costs involved in this operation. \n",
    "\n",
    "We are going to perform the random tuning in the following way:\n",
    "* we will make the algorithm search between 0 and 3 hidden layer (if the best number of hidden layer was 0, we can interpret the result as a sign of linear relationships between the features and the target on this dataset)\n",
    "* we will choose between 1 and 15 neurons per hidden layer\n",
    "* we will use a [reciprocal distribution](https://en.wikipedia.org/wiki/Reciprocal_distribution) for the learning rate of SGD; this distribution will range between 1e-4 and 0.99, giving larger probabilities to smaller values\n",
    "* we will vary the number of epochs (equal to 500, 1000, 2000, or 3000)\n",
    "\n",
    "The search will occur across 10 iterations (of course, in more intensive analyses you should increase such number). Let's initialize the random grid: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.stats import reciprocal    # Reciprocal distribution in scipy.stats\n",
    "from sklearn.model_selection import RandomizedSearchCV\n",
    "param_distribs = {\n",
    "    \"n_hidden\": [0, 1, 2, 3],\n",
    "    \"n_neurons\": np.arange(1, 15),\n",
    "    \"learning_rate\": reciprocal(1e-4, 9.9e-1),\n",
    "    \"epochs\": [500, 1000, 2000, 3000]\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are ready to perform cross-validation (we will use 3-fold CV): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 3 folds for each of 10 candidates, totalling 30 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 2 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  30 out of  30 | elapsed:  4.0min finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "RandomizedSearchCV(cv=3, error_score='raise-deprecating',\n",
       "          estimator=<keras.wrappers.scikit_learn.KerasClassifier object at 0x0000018AAC375308>,\n",
       "          fit_params=None, iid=False, n_iter=10, n_jobs=-1,\n",
       "          param_distributions={'n_hidden': [0, 1, 2, 3], 'n_neurons': array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14]), 'learning_rate': <scipy.stats._distn_infrastructure.rv_frozen object at 0x0000018AAD8ABE48>, 'epochs': [500, 1000, 2000, 3000]},\n",
       "          pre_dispatch='2*n_jobs', random_state=2, refit=True,\n",
       "          return_train_score='warn', scoring=None, verbose=2)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rnd_search_cv = RandomizedSearchCV(keras_binary_classifier, param_distribs, n_iter=10, cv=3, n_jobs=-1, iid=False, \n",
    "                                  random_state=2, verbose=2)\n",
    "rnd_search_cv.fit(X_train, y_train, verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best set of hyperparameters was: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'epochs': 3000,\n",
       " 'learning_rate': 0.0010786090301759563,\n",
       " 'n_hidden': 1,\n",
       " 'n_neurons': 11}"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rnd_search_cv.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Keep in mind that increasing the number of ranomized search iterations might yield a different (and more reliable) result. The CV accuracy of this model is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8511316974957784"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rnd_search_cv.best_score_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the accuracy of the best model on the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "61/61 [==============================] - 0s 311us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.8360655903816223"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rnd_search_cv.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Last, you can save the Keras model in a separate object, with the `best_estimator_.model` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = rnd_search_cv.best_estimator_.model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise**. As an exercise, try to find the best set of the following hyperparameters (either with Grid-Search, or with random-search), on top of the ones already explored in this last section: \n",
    "* $l_2$ regularizer\n",
    "* activation function for the hidden layers\n",
    "* momentum \n",
    "* number of epochs\n",
    "* number of batch_size (you can simply add the 'batch_size' key to the dictionary of hyperparameters grid)\n",
    "\n",
    "If you use randomized search, increase the number of randomized search iterations to search across a larger number of parameter sets (and have a nice walk while you wait for the results...). "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}