SYDE 556 Lecture 5 Dynamics.ipynb

{
 "metadata": {
  "name": ""
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# SYDE 556/750: Simulating Neurobiological Systems\n",
      "\n",
      "Accompanying Readings: Chapter 8\n",
      "\n",
      "Terry Stewart\n",
      "\n",
      "## Dynamics\n",
      "\n",
      "- Everything we've looked at so far has been feed-forward functions\n",
      "    - There's some pattern of activity in one group of neurons representing $x$\n",
      "    - We want that to cause some pattern of activity in another group of neurons to represent $y=f(x)$\n",
      "    - These can be chained together to make more complex systems $z=q(f(x)+g(y))$\n",
      "- these are all feed-forward networks\n",
      "    - what about recurrent networks?\n",
      "\n",
      "<img src=\"files/lecture5/recnet1.gif\">\n",
      "\n",
      "- What happens when we connect a neural group back to itself?\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Recurrent functions\n",
      "\n",
      "- What if we do exactly what we've done so far in the past, but instead of connecting one group of neurons to another, we just connect it back to itself\n",
      "    - Instead of $y=f(x)$\n",
      "    - We get $x=f(x)$ (???)\n",
      "- This is clearly non-sensical\n",
      "    - For example, if we do $f(x)=x+1$ then we'd have $x=x+1$, or $x-x=1$, or $0=1$\n",
      "    - What would happen if we built this?\n",
      "    - What about $f(x)=-x$?  or $f(x)=x^2$?\n",
      "   "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Try it out\n",
      "\n",
      "- Let's try implementing this.\n",
      "- Hmm, that's a bit difficult with what we've done so far\n",
      "    - All of our code so far is set where we figure out what happens to one group of neurons, then take all that data and feed it to the next group of neurons\n",
      "    - Much easier to write code that way\n",
      "    - But difficult when we want to have recurrent connections\n",
      "- So let's switch to other software\n",
      "    - [Nengo](http://nengo.ca)\n",
      "- Start with $x=x+1$    \n",
      "    \n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nef\n",
      "net = nef.Network('Recurrent Test 1')\n",
      "net.make('A', neurons=1000, dimensions=1)\n",
      "def feedback(x):\n",
      "    return x[0]+1\n",
      "net.connect('A', 'A', func=feedback, pstc=0.1)\n",
      "net.add_to_nengo()\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<img src=\"files/lecture5/recurrent1.png\">\n",
      "\n",
      "- That sort of makes sense\n",
      "    - $x$ increases quickly, then hits an upper bound\n",
      "- How quickly?\n",
      "    - What parameters of the system affect this?\n",
      "- What are the precise dynamics?\n",
      "\n",
      "- What about $f(x)=-x$?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nef\n",
      "net = nef.Network('Recurrent Test 1')\n",
      "net.make('A', neurons=100, dimensions=1)\n",
      "def feedback(x):\n",
      "    return -x[0]\n",
      "net.connect('A', 'A', func=feedback)\n",
      "net.add_to_nengo()\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<img src=\"files/lecture5/recurrent2.png\">\n",
      "\n",
      "- That also makes sense.  What if we nudge it away from zero?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nef\n",
      "net = nef.Network('Recurrent Test 1')\n",
      "net.make('A', neurons=100, dimensions=1)\n",
      "def feedback(x):\n",
      "    return -x[0]\n",
      "net.connect('A', 'A', func=feedback)\n",
      "\n",
      "net.make_input('input', [0])\n",
      "net.connect('input', 'A')\n",
      "\n",
      "net.add_to_nengo()\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<img src=\"files/lecture5/recurrent3.png\">\n",
      "\n",
      "- With an input of 1, $x=0.5$\n",
      "- With an input of -1, $x=-0.5$\n",
      "- With an input of 0, it goes back to $x=0$\n",
      "- Does this make sense?\n",
      "    - Why / Why not?\n",
      "    - And why that particular timing?\n",
      "    \n",
      "- What about $f(x)=x^2$?    "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nef\n",
      "net = nef.Network('Recurrent Test 1')\n",
      "net.make('A', neurons=100, dimensions=1)\n",
      "def feedback(x):\n",
      "    return x[0]*x[0]\n",
      "net.connect('A', 'A', func=feedback)\n",
      "\n",
      "net.make_input('input', [0])\n",
      "net.connect('input', 'A')\n",
      "\n",
      "net.add_to_nengo()\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<img src=\"files/lecture5/recurrent4.png\">\n",
      "\n",
      "- Well that's weird\n",
      "- Stable at $x=0$ with no input but something very strange happens around $x=1$ with no input\n",
      "- Why is this happening?\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Making sense of dynamics\n",
      "\n",
      "- Let's go back to something simple\n",
      "- Just a single feed-forward neural population\n",
      "    - encode $x$ into current, compute spikes, decode filtered spikes into $\\hat{x}$\n",
      "- Instead of a constant input, let's change the input\n",
      "    - Change it suddenly from zero to one to get a sense of what's happening with changes"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import syde556\n",
      "\n",
      "dt = 0.001\n",
      "\n",
      "t = numpy.arange(1000)*dt\n",
      "x = numpy.zeros(1000)\n",
      "x[300:]=1\n",
      "\n",
      "A = syde556.Ensemble(neurons=80, dimensions=1, max_rate=(50,100))\n",
      "\n",
      "decoder = A.compute_decoder(x, x, noise=0.2)\n",
      "\n",
      "spikes_A, xhat = A.simulate_spikes(x, decoder, tau=0.1)\n",
      "\n",
      "import pylab\n",
      "pylab.plot(t, x, label='$x$')\n",
      "pylab.plot(t, xhat, label='$\\hat{x}$')\n",
      "pylab.legend(loc='best')\n",
      "pylab.ylim(-0.2, 1.2)\n",
      "pylab.show()\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVPX+x/HXsCMggiiymQooGAIqhruomVvaZl2sW6RG\nZPkzW225JnpvLt1781beunbLNhWX1LBEMs1xTTE3UtAQQVkEQUBUlGXm/P7gNkWDhswwA8zn+XjM\n48HMfOecz3yFt585Z845KkVRFIQQQlgMK3MXIIQQwrQk+IUQwsJI8AshhIWR4BdCCAsjwS+EEBZG\ngl8IISyMwcE/depUPD096dWrV73Pr1y5krCwMEJDQxk0aBCpqamGrlIIIYQBDA7+KVOmkJycfMPn\nu3Xrxq5du0hNTWXOnDk8+eSThq5SCCGEAQwO/iFDhuDm5nbD5wcMGICrqysAkZGR5ObmGrpKIYQQ\nBjDpNv6PP/6YcePGmXKVQgghfsfGVCvasWMHy5cvZ+/evaZapRBCiHqYJPhTU1OJjY0lOTm53s1C\n4eHhHDt2zBSlCCFEqxEWFsbRo0dv+XVNvqnn3Llz3H///axYsYKAgIB6xxw7dgxFUeSmKMydO9fs\nNTSXm8yFzIXMxc1vjW2YDe74J0+ezM6dOykuLsbPz4958+ZRXV0NQFxcHPPnz6e0tJTp06cDYGtr\nS0pKiqGrFUII0UgGB39CQsJNn//oo4/46KOPDF2NEEIII5Ejd5uZqKgoc5fQbMhc/Erm4lcyF4ZT\nKYpi9guxqFQqmkEZQgjRojQ2O032dc7GcHd3p7S01NxlGI2bmxslJSXmLkMIYeGadcff2j4JtLb3\nI4Qwr8ZmimzjF0IICyPBL4QQFkaCXwghLIwEvxBCWBgJfiGEsDAS/EIIYWEk+IUQwsJI8AshhIVp\n1kfuNoRKZZzlNPa4qk2bNmFtbc3u3bvp1asXycnJvP766wQFBRmnMCGEMDI5ctcA586do6qqioCA\nAPr27cv27dvZs2cPI0aMoE2bNnrjm/v7EUK0LK3yXD3NXefOnQEoLCzExcWFdu3acffdd5u5KiGE\nuDnZxm+AkydPcuzYMZKSkhg6dCgA33zzjZmrEkKIm5OO3wBbt27l8uXLeHl5cf36dTZu3IiPj4+5\nyxJCiJuSbfwm1NrejxDCvOTsnEIIIRpEgl8IISyMBL8QQlgYCX4hhLAwEvxCCGFhDAr+qVOn4unp\nSa9evW44ZubMmQQGBhIWFsaRI0cMWZ0QQggjMCj4p0yZQnJy8g2fT0pK4vTp02RkZPDhhx8yffp0\nQ1YnhBDCCAwK/iFDhuDm5nbD5zdt2kRMTAwAkZGRlJWVUVhYaMgqhRBCGKhJj9zNy8vDz89Pd9/X\n15fc3Fw8PT2bcrUmd+3aNVQqFQ4ODuYuxSgUBTQac1chLEVmaSZt7drSwalDg8Znl2VjZ22Ht4u3\n7jGtosVKVdvHVmuqAbC1tm1UPYqioKDolldwpYBLlZfo0b5Ho5Z3qwqvFHLo/I+MCxx/wzF7c/Yy\nwHdAo9fR5Kds+P1RZaobnEc5Pj5e93NUVBRRUVFNWJXxaDQaXe0LFy7Eyqrl7y9/6SV4+21oBW9F\nNAPKbWq41BlVWbd6n9c83x9qHLB+N6d2fLsz4J6B6szoG4zvB07FWL1dgOqqJ4qqBu1fbOHkfVht\nXIH23sfAIx2uuWOV9AFYV4NNBarcQfXX13kXyu0JqA7ORFUcjDbifZSxM7Ba+S2qM3eheXA6BH2F\n9V9rs0yxuwwOZajK/XTvT7ltJ1a75v7v/g6Ubtuw2vFm7X0UlJEvozoxGVVBHxS0KH3+i+rMKFRl\n3VCCv0Q76UFUGxKwOhGNtu8GlHFPo9oZj2rfS2B7DcX/W1TZw+HyKbRWyTBoMaT+udH/Jk0a/D4+\nPuTk5Oju5+bm3vBcNr8N/pZErVbzwgsvoNVq2bFjByNHjjR3SQYrLYX//hemTTN3JaI5UmerCfYI\nxtO59pP7b7ttgMqaSmq0NTjZOQGgmjec2zvczvGnjwPwz33/xEplxbQ+0/j54s9Ef+lKZmkm1dW1\nwTpx9bN88/M3ZM7MpJtbN6o0Vew9t5essiweC3uMwPecyS4rJnjhSPZO3cvuc7uZkADOod/x2J9f\nIqusgi2n0wCY9uG75F3OIykjiZiwGLad2camyZt4ZMMjfDzxYwb6DWTmli95L+U/KBH/QZmr8PJ3\nZ/k2MxTHl99g/xN3MTXRjU+Owq6sfWxI38DWzK38dOEn3h/3PgHuAaw5sYaPj3yMx7gP6OTcCVcH\nV3ad3cWyxd3YeHIjLw96mWGf/oPxj5/k68lfsyH9Kx5Y+xQKsGDEAgBe+x6U+ydT8MOdJPxUw8xk\nUIbF89zscvzd/Xkm6RkUYPag2YzoOoLRKxbjPi6JktmN+zc0+Fw92dnZTJgwgZ9++knvuaSkJJYu\nXUpSUhL79+9n1qxZ7N+/X78IOVdPszJ1KgwaJMFvaTakb2Cg30DcHNworyzXbXrZdGoT69LW8ck9\nn2BjZYNqXu2n9rSn0wjuEMzg5YNJL07HWmVN1rNZ3L/2frZmbuX5/s+zYOQCHN6s3QQa3imcPp36\n8PXPX1NUUYRvW19yy3N16/9g/Ac8k/QMWkV7wxpVqFBQWDJ6Cc99+xz3Bt2Lt7M3/u7+3N39bqI+\njWJw58GsS1vH4M6DOX7hOH29+lKpqWTPuT265UR4R/Bj/o+6+zPvmMl/D/8X9eNqXt3+Ko/0eoSX\nvnuJDm06kH85n8tVl29YU7BHMOnF6QD09+3P/lz9jJsaPpVtWduYGj6VlT+txNPZkz3n9tDGtg0V\n1RW6cYP8BtHPux8KCk9FPMWQT4bwwoAX2Hl2JzuydlCpqWS0/2hKr5eSkpcC8fpbVRrCoA/zkydP\nZuDAgZw6dQo/Pz+WL1/OsmXLWLZsGQDjxo2jW7duBAQEEBcXx/vvv2/I6pqlTZs2sXnzZl555RVW\nrlzJo48+ysmTJ81dlsGMdWUz0fwUXS1Cq2jRaOvuyHlg7QPM+X4OzyQ9Q8d/dOSzo59Req2UJfuX\nsDF9Ix8e+pCCKwW68T3f78m+nH0UXCmg5FoJRRVFjPpiFFszt/JU36d4e//bTEmcgq1V7bb24xeO\ns+r4Kooqiph5x0xd6NtY2TA+cDzTN0/Xhf6hJw+hjlHzlyF/AeD7x76n4IUC3abiP4f+mSNxR/jq\n5Fe8/+P7hHqG0r19d9o5tCP5dDJrJq0h+ZFkxgeOZ3vWdhaNXEQ/7354OXsB8O6Yd/nP+P/wcK+H\nebDng7w86GVeH/I6kR9F8n3W97jau/JU36c4dfEUl6suMzlkMnbWdrw75l36evUl7enaTxSr7l9F\nenE64Z3CmRwymc/v/RyAkV1rP/kvGrkIgLb2bVn34DreTXmXjJIMxgaM5dysc7rQf77/81x65RL2\nNvb868C/6N6+O0EeQXg5ezF/53zG+I/h+l+uA/Bt5rf06dSHE0+faPTvQIs/O+cv3YehlLm3Pg2t\n9QpcU6bAkCG1nb9oeQ7mHcTVwZXu7bsDsPvsboI8gtiauZVHQh9BNU+Fo40j12qu8WDPB0k+nazr\naMcEjKFGW8Oec3u4XlMbNJ2cO/G34X/jia+fAMDZzpnzL5xn+GfDdV3z6f87zfkr5xnyyRAAsp7N\n4mTxScauHMuE7hNY9+A6bK1t+fDQh8zZMYcND21g6KdDSX8mnSCPIL7L/I67Vtylew/5z+fj5VIb\n0hkXMwhwD6h3/+CUxClcrbrKfyf8F1cHV97Y8QZ/3fVXDjxxgDt87iC9KJ2nk55myyNbcLBx4Fr1\nNWq0NbjYu+gtK+NiBt2Xducfo/7BtD7TaOfQjmpNNbbWtrq/29/W8Mtj69LWMS5wHM52zgDkXMrB\n28Wb4xeOE9g+kPSidLq5dcPN0Q2tomX32d307NCTDk4dKLlWwsnik/Tx6oODjQOrj69m8vrJuvpj\nvorh82OfUzq7lHYO7Viftp5J6yYxd9hc4qPiG58pSjNwozKaSXl/qKCgQBk2bNgfjmsp7+fxxxVl\n+XJzVyFu5ItjXyjpRem6+09/87Sy4tgK3X3redYK8ShXq64qiqIoxKMQj2I1z0r5LvM73X3i0Y39\n5ea2yE3p9I9Oys/FPysH8w4qwUuDFeJR8srzlEW7FynEo0z9aqqiKIpy+uJpxWWBi25dWq1WeeP7\nNxTiUa5XX1cURVHOlJxRLlderlO/VqtVtFqt8tq215Sqmqo6j1+pvFLnsVuVV56nPLDmAeVixcVG\nvb6yprLR6zaGGk2NkngyUdFqtYqiKMqxgmPK69tfrzNGo9Xofm5sprT4jt+cTp48SWVlJYcPHyYr\nK4v58+fzzTff3PDyi839/fzi8cdh2LDazl80P6p5KoZ3Gc73Md+jKApW862wVlmz4U8bGBswFp+3\nfXCycyK7LJudj+9k2KfDWHX/KlQqFZPXT9YtZ/eU3WSXZfPoxkdJfSqV/Mv5fPDjBySeSqTqL1XY\nWtuyJWML41aNo/IvldhZ2/H5sc8Zdtswbmt3m245P+b/SIR3hDmmwuLJNXfNoLVegUtRZBt/cxK7\nKZbYvrF0dOrIfWvuA2BH9g4ySzKxUlnR0akj4wLHcc/qe3SvORx3mPk75zPs02EATO41GUVRsFZZ\nc7rkNAP9BjK482AG+Q0i0ieSwPaB9PLsxZ3d7uTniz/rvgM/ousIYvvEYmdtB8BjYY/p1Seh3/JI\nx29CLeX9xMTA8OG1nb8wPXW2mt6devPQlw/xzph3CP53MPcH389DPR8ien00UPstkdzLuWzN3Eps\nn1juDbqX8at+PeBHmatQWVPJ6uOrebjXw40+mEk0b9LxC6ORjt+01qetx9rKmvOXzzO933SGfzZc\n91zwv4MB2HV2F8UVxbrH50bN5cWtLwIwuPNgxgWOQ5mrUK2p5kDeAQDsbeyJCY8x4TsRLYUEvxBm\nNmndJN3PgzsPxt7ankpNpe6xhSMX4u3izZNfP0nmzEw8nTxxsnNi7YNr2XNuD3f43KEba2tty+DO\ng01av2h5JPiFHun4m9b4VePp2q4rlyovsfjOxdhb23Mk7ggfHvqQQcsHUampJOvZLGysbPBt66t7\nXX3b1yXkRWNI8AvRxE4Vn2J9+nr25exj9aTVJGUk6Z5bkboCgOAOwcSEx/CvA//iuf7P0aVdFzNV\nKyyBBL/QIx2/YRRFYdfZXUT6RmJnbcewT4dReLUQL2cvXBa64GznTNnsMnad3cUjGx5h7rDak3uF\neYZxT497eGXwK2Z+B6K1k+AX9ZLgb7wv077koS8fAiCqSxSVmkoKXijAxd4FpwVOONo4Ym1lzfCu\nw8l/IV/3OpVKxVfRX5mrbGFB5MS7Qk8L+MZps3Km9AyKojB5/WQKrhToQh9qv5pZdr2Mjk4daWPb\nBmWuQsGLBTdZmhBNr1l3/G5ubjc8f39LdLOrlTU3rWjam5RW0eL/rj/+bv5klmay+vhqAC68eIEO\nTh24Z/U9bDq1qc7v8W9PYSyEOTTr38CSkpLaq+G0kltJSYm5p7RBpOO/MUVR8H3bV3fq3aKrRUDt\nVaR+65dTGv9ylkYhmpNm3fEL85Cduzd2tfoqeZfziP4yGq2iJac8hy7turDhoQ14u3izOWMzXdt1\n1Y3/vzv+j4d7PWzGioXQJ8EvRAMVXinkStUVAM5eOqt7fNhtw+jt1RuAqb3rnstapVLh0cbDdEUK\n0QAS/EKPdPx1LT+ynHt63EOnf3Zi9qDZ9Gjfg0d6PcKEHhPwd/Ov99zuQjRnEvxC3EDptVJqtDVM\n2zSNtf5rAVi8dzHTI6YzZ9gcM1cnRONJ8As90vHDuUvnuO1ft7F07FKg9nJ32x7dxh0+d0iHL1q8\nZv2tHiFMTVEUVqau5MF1DwIwY8sMXhv8GpV/qWRkt5ES+qJVkI5f6LHkjr/sehl/3vhnAC6+fJFH\nNz7KzMiZuguRCNEaSMcvLN6a42uYkDCBsutl/HThJ1ztXdn26DbcHd3Z/PBmPJ09zV2iEEYlHb/Q\nY0kd/+XKy7qrWrkt/vXI6pHd5MAr0XpJxy/qZSnBn1ueC8BLA18CYNHIRZybdc6cJQnR5AwO/uTk\nZIKCgggMDGTx4sV6zxcXFzNmzBjCw8MJCQnh008/NXSVoom19lM2bDuzjdzyXLSKlumbp+Pl7MXi\nOxez8/GdzB48Gz9XP3OXKESTMij4NRoNM2bMIDk5mbS0NBISEkhPT68zZunSpfTu3ZujR4+iVqt5\n4YUXqKmpMaho0fRaa8ev0WoY9cUo7vriLrZkbGHn2Z1MDpmMSqVi6G1DzV2eECZhUPCnpKQQEBBA\nly5dsLW1JTo6msTExDpjvLy8KC8vB6C8vJz27dtjYyO7Fpqz1tzxF1cU42jjSNn1Mu5OuJvHwx/n\nn6P/ae6yhDApg4I/Ly8PP79fPxb7+vqSl5dXZ0xsbCwnTpzA29ubsLAw3nnnHUNWKUygNe/cvXD1\nAl3dupL/Qj6dXTsz2n+0uUsSwuQMar0bcq78BQsWEB4ejlqtJjMzk1GjRnHs2DFcXOoeCBMfH6/7\nOSoqiqioKENKE0JnacpSEo4nsGfKHo5fOE6HNrWnTD476+wfvFKI5kWtVqNWqw1ejkHB7+PjQ05O\nju5+Tk4Ovr6+dcbs27eP119/HQB/f3+6du3KqVOniIiIqDPut8EvzKs1dfyVNZXMVc+l5FoJ4cvC\nSS1MJbxTuLnLEqJRft8Uz5s3r1HLMSj4IyIiyMjIIDs7G29vb9asWUNCQkKdMUFBQWzbto1BgwZR\nWFjIqVOn6NatmyGrFaLBcstzaWvfltznctmcsZl2Du3wbev7xy8UohUzKPhtbGxYunQpo0ePRqPR\nMG3aNIKDg1m2bBkAcXFxvPbaa0yZMoWwsDC0Wi1vvfUW7u7uRileNI3W1PHnXc7Dx8UHR1tHJvWc\nZO5yhGgWVIpi/u9wqFQqmkEZ4n/uvRdiYuC++8xdya27XHmZtovacv3169jb2KOap8LV3pWyV8rM\nXZoQRtfY7JTvVQo9Lbnjzymv3efk8KYDz/R7BoA1k9aYsyQhmh05ZYOoV0sN/vOXz+vOpPnvg/8G\n4M5ud5qzJCGaHen4hZ6WvNWt4EoBDwQ/wD/u+gcH8w6iUqmwtrI2d1lCNCsS/KJeLbHjv1p1lZS8\nFHzb+uLt4s09QfeYuyQhmiUJfqGnJXb8xRXFdPh77YFZ/xn/HzNXI0TzJtv4hZ6WtnO39FopL2x9\nQXc/pGOIGasRovmTjl+0aOWV5bi/VXtcSNFLRbR3bN+gU4kIYckk+IWeltTxZ5VmATC482A82niY\nuRohWgYJftEiVdZU8tJ3L5F4KpF+3v3YPWW3uUsSosWQ4Bd6WkLHf/j8Yd5LeQ+AoqtFZq5GiJZF\ngl+0SIfPHyYmLIYRXUcwqtsoc5cjRIsi3+oRepp7x//slmeZsWUGd3a7k8fCHsPLxcvcJQnRokjw\nixYl/3I+76a8y99H/V3OtilEI8mmHqGnOXf8mSWZDPAdwIsDXzR3KUK0WNLxi3o1x+DPLc9l6KdD\naefQztylCNGiSfALPc3xlA3/+fE/+C3xAyCub5yZqxGiZZNNPaJeza3jX5G6gtmDZnNPj3sY4DfA\n3OUI0aJJ8As9za3jv1hxkb05e/nm4W9kM48QRiCbeoSe5rRzN/9yPh5/rz0Vg4S+EMYhwS+atcyS\nTACuvnbVzJUI0XpI8As9zaXjVxSFIwVHmNRzEm1s25i7HCFaDQl+0SxVa6oZs3IMzyY/S8c2Hc1d\njhCtiuzcFXrM3fHnlefhu8SXUM9QqudUY6WS/kQIYzL4Lyo5OZmgoCACAwNZvHhxvWPUajW9e/cm\nJCSEqKgoQ1cpWrkDeQcAeHHAi9hY2UjwC2FkBnX8Go2GGTNmsG3bNnx8fOjXrx8TJ04kODhYN6as\nrIxnnnmGb7/9Fl9fX4qLiw0uWjQtc3f8R84fYc7QOTwa9qj5ihCiFTOolUpJSSEgIIAuXbpga2tL\ndHQ0iYmJdcasWrWKBx54AF9fXwA8POQqSS2BOYP/0PlDhHcKN18BQrRyBgV/Xl4efn5+uvu+vr7k\n5eXVGZORkUFJSQnDhw8nIiKCL774wpBVChMw5wFcZ8vO8kPuDwy9baj5ihCilTNoU09DLmpdXV3N\n4cOH2b59OxUVFQwYMID+/fsTGBhYZ1x8fLzu56ioKNkXYGbm6vhPl5wmvFO4XD9XiHqo1WrUarXB\nyzEo+H18fMjJydHdz8nJ0W3S+YWfnx8eHh44Ojri6OjI0KFDOXbs2E2DX5iXuTr+0yWn+euuv+Js\n52yeAoRo5n7fFM+bN69RyzFoU09ERAQZGRlkZ2dTVVXFmjVrmDhxYp0x99xzD3v27EGj0VBRUcGB\nAwfo2bOnIasVJmDqjr9KU8Wg5YMoqijiodsfMu3KhbAwBnX8NjY2LF26lNGjR6PRaJg2bRrBwcEs\nW7YMgLi4OIKCghgzZgyhoaFYWVkRGxsrwd/MmaPjj/06loF+A9n4p42mX7kQFkalKOY/F6NKpaIZ\nlCH+Z+hQ+OtfYdiwpl+XVtHyz33/ZOGeheQ8l4OTnVPTr1SIVqKx2SlH7gqzSi1MJX5nPF9P/lpC\nXwgTkUMihR5THsBVdLWI/r79GdF1hGlWKISQ4BfmVVxRTIc2HcxdhhAWRYJf6DFlx19cUSzf2RfC\nxCT4hVntPrebQPfAPx4ohDAaCX6hxxQdv6IovLrtVQ7kHeCJPk807cqEEHXIt3pEvZo6+Hed3cWi\nvYtQx6jl2zxCmJh0/EKPKQ6pSMlLYXrEdIZ1McHBAkKIOqTjF/Vqyo4/5P0QThSd4Iv75EytQpiD\ndPxCT1N2/GfLznKi6AQhHUOI6hLVdCsSQtyQdPxCT1Pu3M0oySCqSxQ7YnY0zQqEEH9IOn5hUnnl\nefi4+Ji7DCEsmgS/0NOUHf+CPQvkgC0hzEw29QiT0Wg1/HzxZ76e/LW5SxHCoknHL/Q0Vce/YPcC\n/N386d6+u/EXLoRoMOn4hUlcun6Jt/a9xaEnD5m7FCEsnnT8Qk9TdPzfZ33PQL+B0u0L0QxI8It6\nGTv4t5zewmj/0cZdqBCiUWRTj9BjzAO4yivLmb55Oqt+WsXJZ04ab8FCiEaTjl/Uy1gd/9GCo6z6\naRVzhs6hh0cP4yxUCGEQCX6hx5gdf8m1EiZ0n8D84fONt1AhhEEk+EW9jNXxl1wroX2b9sZZmBDC\nKCT4hR5jdvwXKy7S3lGCX4jmxODgT05OJigoiMDAQBYvXnzDcQcPHsTGxoYNGzYYukrRxIz5dU71\nWTWdXTsbZ2FCCKMwKPg1Gg0zZswgOTmZtLQ0EhISSE9Pr3fc7NmzGTNmDIoprvIhmoX3D75PUkYS\ncX3jzF2KEOI3DAr+lJQUAgIC6NKlC7a2tkRHR5OYmKg37r333mPSpEl06NDBkNUJEzFWx7/2xFre\nHPEm9jb2hi9MCGE0BgV/Xl4efn5+uvu+vr7k5eXpjUlMTGT69OkAqJr6Yq6iWajSVHG04CixfWLN\nXYoQ4ncMOoCrISE+a9YsFi1ahEqlQlGUG27qiY+P1/0cFRVFVFSUIaUJAxij41+0ZxH9fPrRwUk+\n5QlhLGq1GrVabfByDAp+Hx8fcnJydPdzcnLw9fWtM+bQoUNER0cDUFxczJYtW7C1tWXixIl1xv02\n+EXLpigKX6R+QcIDCeYuRYhW5fdN8bx58xq1HIOCPyIigoyMDLKzs/H29mbNmjUkJNT9Yz9z5ozu\n5ylTpjBhwgS90BfNiyEdv1bRMnfHXMory+nr1de4hQkhjMKg4LexsWHp0qWMHj0ajUbDtGnTCA4O\nZtmyZQDExcm3OVqqxgb/3nN7+dvuv7Fw5ELZnyNEM6VSmsH3K3/Z/i+ah169YOVKCA299deuO7GO\n1SdWs/6h9cYvTAhRR2OzU47cFfVqbLNeeLUQTydP4xYjhDAqCX6hx5APX4VXJPiFaO4k+IUeQ3bu\npuSn0Muzl3ELEkIYlQS/MJpr1dfYl7OPkV1HmrsUIcRNSPALPY3t+Hed3UWYZxiuDq7GL0oIYTQS\n/MJotmZulevqCtECSPALPY3p+C9WXGTFTysYEzCmaYoSQhiNBL8wis0ZmwnzDCPCO8LcpQgh/oAE\nv9Bzqx3/hasXmLNjDvcH3y9H6wrRAkjwC4N9cPADXO1diQmLMXcpQogGMOhcPaJ1utWOf23aWj6a\n8BGOto5NV5QQwmik4xf1amjwZ1zMoPRaKZG+kU1bkBDCaCT4hZ5bOWXDzxd/JqxTGFYq+VUSoqWQ\nv1ZRr4Z2/BeuXpBz8wjRwkjwCz230vFfuHqBjk4dm64YIYTRSfALPbeyc3df7j58XHyatiAhhFFJ\n8ItGq9ZU8+3pb3k8/HFzlyKEuAUS/EJPQzv+0yWn8XP1k5OyCdHCSPCLRksvTqdnh57mLkMIcYsk\n+IWehnb8Jy6coKeHBL8QLY0Ev2iUtSfW8ob6DYZ1GWbuUoQQt0iCX+hpSMe/ZP8S5kfNl/PvC9EC\nSfCLet0s+E+XnOZM6RleHfKqnI1TiBbI4OBPTk4mKCiIwMBAFi9erPf8ypUrCQsLIzQ0lEGDBpGa\nmmroKkUT+6MDuBJ+SuBPt/8JGys5x58QLZFBwa/RaJgxYwbJycmkpaWRkJBAenp6nTHdunVj165d\npKamMmfOHJ588kmDChamcbNGfve53bKJR4gWzKDgT0lJISAggC5dumBra0t0dDSJiYl1xgwYMABX\n19rveUdGRpKbm2vIKoUJ3Kzjr9JUcTD/oFxpS4gWzKDgz8vLw8/PT3ff19eXvLy8G47/+OOPGTdu\nnCGrFCZws527KXkpBLgH4OksJ2YToqUyaCPtrezY27FjB8uXL2fv3r31Ph8fH6/7OSoqiqioKENK\nE00k51IO/m7+5i5DCIukVqtRq9UGL8eg4Pfx8SEnJ0d3PycnB19fX71xqampxMbGkpycjJubW73L\n+m3wC/O6Wcd//sp5Ojl3Mm1BQghAvymeN29eo5Zj0KaeiIgIMjIyyM7OpqqqijVr1jBx4sQ6Y86d\nO8f999//hvWQAAAOpUlEQVTPihUrCAgIMGR1ohk4f/k8Xs5e5i5DCGEAgzp+Gxsbli5dyujRo9Fo\nNEybNo3g4GCWLVsGQFxcHPPnz6e0tJTp06cDYGtrS0pKiuGViyZzs47/0PlDPD/gedMWJIQwKpWi\n3MplN5qoCJWKZlCG+B9fX/jhB/jNfnsAtIoWl4Uu5D+fL2fkFKIZaGx2ypG7Qs+NOv6zZWdxd3SX\n0BeihZPgFw12IO8A4Z3CzV2GEMJAEvxCT30df+GVQl7b/hoxYTHmKUoIYTQS/KJevw/+bzO/pbdX\nbyb1nGSegoQQRiPBL/TUt6/oaMFRIrzkNA1CtAYS/KJev+/4d5/bzUC/geYpRghhVBL8Qs/vO/6s\n0iyyy7Lp79vfPAUJIYxKgl/o+f3O3e1Z2xkbMBZ7G3vzFSWEMBoJfvGHDuQeINIn0txlCCGMRIJf\n6Pl9x5+Sn8IdPneYryAhhFFJ8IubulJ1hdMlpwn1DDV3KUIII5HgF3p+2/EfLTjK7R1ul+37QrQi\nEvziprLLsvF3lwuvCNGaSPALPb/t+PPK8/Bx8TFvQUIIo5LgF/X6Jfhzy3Ml+IVoZST4hZ7fHsCl\nPquWA7eEaGUk+EW9VCpIL0rnwtUL8lVOIVoZCX6h55eOP35nPM/0ewZrK2vzFiSEMCoJflEvlQq+\nz/qeqb2nmrsUIYSRSfALPYoC5VXlVFRXyI5dIVohCX6hR1Eg/0oOnV07o6rv4rtCiBZNgl/U61RJ\nGt3cupm7DCFEE5DgF3oUBb7J3MDE7hPNXYoQogkYHPzJyckEBQURGBjI4sWL6x0zc+ZMAgMDCQsL\n48iRI4austUruFLAlaorZlu/otLwXXYS9wXfZ7YahBBNx6Dg12g0zJgxg+TkZNLS0khISCA9Pb3O\nmKSkJE6fPk1GRgYffvgh06dPN6jglqC4ophjBcf0Ht+YvpEFuxdwsvjkDV97peoKXv/04s7P76RG\nW9PgdZZdL2Py+sm8tv01vkz7ssGv02g1FF0t4ljBMc5dOsf1muto3NPwdOpER6eODV6OEKLlsDHk\nxSkpKQQEBNClSxcAoqOjSUxMJDg4WDdm06ZNxMTEABAZGUlZWRmFhYV4enoasuoGKb1WirOdM7bW\ntgCcKj5F2fUyvFy8+D7re8YFjtOFm6IoqLPVXLh6gRFdR/Bj/o/4ufoR0jEEqD11gYONAx5tPAD4\n9Oin9PXqSy/PXrrXHzp/iAW7F3Ag7wD5l/M5M/MMXd26svr4an7I+YHPUz/nLv+7WPXTKt4b+x7D\nuw7Xq/lA7gFu73A7BVcK6Pj3jrRv054BvgOY2GMi/X37887+d3B1cGVm5Eza2rfljR1vcLrkNJml\nmaTkpXBntzv55OgnBHkEca36Gg42DuzN2cvOsztJLUwlrSiNMQFjWDhyIX/b9Tc2ntyIVtHi0cYD\ne2t7iiqK0IwYQB9POWhLiNbKoODPy8vDz89Pd9/X15cDBw784Zjc3Fy94A+Z/XSd+yqsCK2Yhasm\noM7jWfaJtK8Oo622i+6xaio4Z5+EW01Prlrn4lndHzulLR94ugPQ78p8LltncdLxE91rnDV+vKR9\nl3tLd6GgcMrhM444LcJJ602RzcNYYYdGdR2vqqG01XTllONndKy+A5+qEZyxX88lmwycNX6oFBsu\n22ThoPXgulUxPSueZOC1TZx0/JQ+b4/DTuvKBbsDtKsJIuLKB7Q5dydF7e5mxOcjCL06C0WlxVqx\np8TmOLaKC4W2++lZEcfgipe5bJ1NjeoaJ86tZtPh17lsnU3QtamU2fzI298l4KTxptj2CLdXPI2r\nZixPVN6HbaEzKe396fVBL11NXa/fhzV2nHZII+B6NN9mrGNrxnbCKl6kp2o6Qdem4l4YgjV25Npt\n42u/UUR6v9+4XwohRLNnUPA39Kt+yu+u3l3v646c1f3Y0T8QpavCnrbRPN9uL7Yqe85W/0hm9W6S\nrz6vGxdidzdDHKZzuPIzMqt3c0l7HgAXlSfd7YZDJYTbPUCNeyqdrQMYZ7+HjtaB2KmcsMKaRaV9\n+aijS21NWPGYy2dE2D9MTs0hvG1CSa74G94uIZzXnGC07XaOVH6Jg0qLoulFT7uXqVIqcLe+jUrl\nCt1sB+Jq5YWtygGAUI0Ha66cYaDDEwTYDsHZykNXdz/2c+D652Q47KRCW0JH62CCrCeSX/MTPa3i\nuMtzNlYqa6D2P71h9EKjzKOGSuxVTpyvSeOj8gdxslKIdvqGrp51z6XzouZ7FLS0s/LhmnIJF6sO\n/3tm9f/+PVahoGCl0t/SF8KdjHYsIm6gWwP+ZYUQpqRWq1Gr1QYvR6X8PpVvwf79+4mPjyc5ORmA\nhQsXYmVlxezZs3VjnnrqKaKiooiOjgYgKCiInTt31un4VSqV3n8OiqIwad0kNqRv4IneT7Dyp5U8\nFvYYYwPG0te7Lwt2L8Dd0Z03d7/JmIAxrJ20lvNXzuPb1pfjF46zI2sH47uP122qqU/+5XwO5R8i\n1DOU29rd1thpEEIIs6gvOxv0OkOCv6amhh49erB9+3a8vb254447SEhIqLONPykpiaVLl5KUlMT+\n/fuZNWsW+/fvb1DxVZoqor+MppNzJ+YPn6/bvv4LRVFYl7aOe4Puxc7arrFvQwghWiSzBD/Ali1b\nmDVrFhqNhmnTpvHqq6+ybNkyAOLi4gB03/xxcnLik08+oU+fPkYpXgghLJnZgt8YJPiFEOLWNTY7\n5chdIYSwMBL8QghhYST4hRDCwkjwCyGEhZHgF0IICyPBL4QQFkaCXwghLIwEvxBCWBgJfiGEsDAS\n/EIIYWEk+IUQwsJI8AshhIWR4BdCCAsjwS+EEBZGgl8IISyMBL8QQlgYCX4hhLAwEvxCCGFhJPiF\nEMLCSPALIYSFkeAXQggLI8EvhBAWptHBX1JSwqhRo+jevTt33XUXZWVlemNycnIYPnw4t99+OyEh\nIbz77rsGFSuEEMJwjQ7+RYsWMWrUKH7++WdGjhzJokWL9MbY2tqyZMkSTpw4wf79+/n3v/9Nenq6\nQQW3dmq12twlNBsyF7+SufiVzIXhGh38mzZtIiYmBoCYmBi++uorvTGdOnUiPDwcAGdnZ4KDg8nP\nz2/sKi2C/FL/SubiVzIXv5K5MFyjg7+wsBBPT08APD09KSwsvOn47Oxsjhw5QmRkZGNXKYQQwghs\nbvbkqFGjKCgo0Hv8zTffrHNfpVKhUqluuJwrV64wadIk3nnnHZydnRtZqhBCCKNQGqlHjx7K+fPn\nFUVRlPz8fKVHjx71jquqqlLuuusuZcmSJTdclr+/vwLITW5yk5vcbuHm7+/fqPxWKYqi0Agvv/wy\n7du3Z/bs2SxatIiysjK9HbyKohATE0P79u1ZsmRJY1YjhBDCyBod/CUlJTz00EOcO3eOLl26sHbt\nWtq1a0d+fj6xsbFs3ryZPXv2MHToUEJDQ3WbghYuXMiYMWOM+iaEEEI0XKODXwghRMtk0iN3k5OT\nCQoKIjAwkMWLF9c7ZubMmQQGBhIWFsaRI0dMWZ5J/dFcrFy5krCwMEJDQxk0aBCpqalmqNI0GvJ7\nAXDw4EFsbGzYsGGDCaszrYbMhVqtpnfv3oSEhBAVFWXaAk3oj+aiuLiYMWPGEB4eTkhICJ9++qnp\nizSBqVOn4unpSa9evW445pZzs1F7BhqhpqZG8ff3V7KyspSqqiolLCxMSUtLqzNm8+bNytixYxVF\nUZT9+/crkZGRpirPpBoyF/v27VPKysoURVGULVu2WPRc/DJu+PDhyvjx45Uvv/zSDJU2vYbMRWlp\nqdKzZ08lJydHURRFKSoqMkepTa4hczF37lzllVdeURSldh7c3d2V6upqc5TbpHbt2qUcPnxYCQkJ\nqff5xuSmyTr+lJQUAgIC6NKlC7a2tkRHR5OYmFhnzG8PCouMjKSsrOwPjw9oiRoyFwMGDMDV1RWo\nnYvc3FxzlNrkGjIXAO+99x6TJk2iQ4cOZqjSNBoyF6tWreKBBx7A19cXAA8PD3OU2uQaMhdeXl6U\nl5cDUF5eTvv27bGxuek31FukIUOG4ObmdsPnG5ObJgv+vLw8/Pz8dPd9fX3Jy8v7wzGtMfAaMhe/\n9fHHHzNu3DhTlGZyDf29SExMZPr06QA3PWakJWvIXGRkZFBSUsLw4cOJiIjgiy++MHWZJtGQuYiN\njeXEiRN4e3sTFhbGO++8Y+oym4XG5KbJ/nts6B+r8rt9za3xj/xW3tOOHTtYvnw5e/fubcKKzKch\nczFr1iwWLVqESqVCURS935HWoiFzUV1dzeHDh9m+fTsVFRUMGDCA/v37ExgYaIIKTachc7FgwQLC\nw8NRq9VkZmYyatQojh07houLiwkqbF5uNTdNFvw+Pj7k5OTo7ufk5Og+rt5oTG5uLj4+PqYq0WQa\nMhcAqampxMbGkpycfNOPei1ZQ+bi0KFDREdHA7U79LZs2YKtrS0TJ040aa1NrSFz4efnh4eHB46O\njjg6OjJ06FCOHTvW6oK/IXOxb98+Xn/9dQD8/f3p2rUrp06dIiIiwqS1mlujctNoeyD+QHV1tdKt\nWzclKytLqays/MOduz/88EOr3aHZkLk4e/as4u/vr/zwww9mqtI0GjIXv/X4448r69evN2GFptOQ\nuUhPT1dGjhyp1NTUKFevXlVCQkKUEydOmKniptOQuXjuueeU+Ph4RVEUpaCgQPHx8VEuXrxojnKb\nXFZWVoN27jY0N03W8dvY2LB06VJGjx6NRqNh2rRpBAcHs2zZMgDi4uIYN24cSUlJBAQE4OTkxCef\nfGKq8kyqIXMxf/58SktLddu1bW1tSUlJMWfZTaIhc2EpGjIXQUFBjBkzhtDQUKysrIiNjaVnz55m\nrtz4GjIXr732GlOmTCEsLAytVstbb72Fu7u7mSs3vsmTJ7Nz506Ki4vx8/Nj3rx5VFdXA43PTTmA\nSwghLIxcelEIISyMBL8QQlgYCX4hhLAwEvxCCGFhJPiFEMLCSPALIYSFkeAXQggLI8EvhBAW5v8B\nIdr1gTNnP0cAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x7d27128>"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- This was supposed to compute $f(x)=x$\n",
      "- For a constant input, that works\n",
      "    - But we get something else when there's a change in the input\n",
      "- What is this difference?\n",
      "    - What affects it?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import syde556\n",
      "\n",
      "dt = 0.001\n",
      "\n",
      "t = numpy.arange(1000)*dt\n",
      "x = numpy.zeros(1000)\n",
      "x[300:]=1\n",
      "\n",
      "A = syde556.Ensemble(neurons=80, dimensions=1, max_rate=(50,100))\n",
      "\n",
      "decoder = A.compute_decoder(x, x, noise=0.2)\n",
      "\n",
      "spikes_A, xhat = A.simulate_spikes(x, decoder, tau=0.03)\n",
      "\n",
      "import pylab\n",
      "pylab.plot(t, x, label='$x$')\n",
      "pylab.plot(t, xhat, label='$\\hat{x}$')\n",
      "pylab.legend(loc='best')\n",
      "pylab.ylim(-0.2, 1.2)\n",
      "pylab.show()\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtcVHX+x/HXwAwMF7krImCgoKAoanjLNMy8VrbbxbTd\n1lUzs6xsu9evUrc13doum26rbWWZumY3LY0urpiXFE3Fu+IFuQnK/Q4zw/n9Mc2BcUCR4SZ8no+H\nDzkzZ875znfgPZ/5nu85o1EURUEIIUS74dDSDRBCCNG8JPiFEKKdkeAXQoh2RoJfCCHaGQl+IYRo\nZyT4hRCinbE7+KdPn46/vz99+vSp9f5Vq1YRHR1N3759GTZsGAcPHrR3l0IIIexgd/BPmzaNuLi4\nOu/v1q0bP//8MwcPHuSll17iwQcftHeXQggh7GB38A8fPhxvb+867x86dCienp4ADB48mLS0NHt3\nKYQQwg7NOsb/wQcfMGHChObcpRBCiEtom2tHW7Zs4cMPP2THjh3NtUshhBC1aJbgP3jwIDNnziQu\nLq7WYaF+/fqRmJjYHE0RQog2Izo6mgMHDlz145p8qCclJYU777yTTz/9lLCwsFrXSUxMRFEU+aco\nvPLKKy3ehtbyT/pC+kL64vL/Glow213xT5kyha1bt5KdnU1wcDDz58/HYDAAMGvWLBYsWEBeXh6z\nZ88GQKfTkZCQYO9uhRBCNJDdwb9mzZrL3v+f//yH//znP/buRgghRCORM3dbmdjY2JZuQqshfVFN\n+qKa9IX9NIqitPgXsWg0GlpBM4QQ4prS0OxstumcDeHj40NeXl5LN6PReHt7k5ub29LNEEK0c626\n4m9rnwTa2vMRQrSshmaKjPELIUQ7I8EvhBDtjAS/EEK0MxL8QgjRzkjwCyFEOyPBL4QQ7YwEvxBC\ntDMS/EII0c606jN360OjaZztNPS8qg0bNuDo6Mi2bdvo06cPcXFxvPjii0RERDROw4QQopHJmbt2\nSElJobKykrCwMK6//no2b97M9u3bufnmm3F1dbVZv7U/HyHEtaVNXquntevatSsAWVlZdOjQAS8v\nL2677bYWbpUQQlyejPHb4fjx4yQmJrJp0yZGjBgBwLffftvCrRJCiMuTit8OP/zwA0VFRQQEBFBe\nXs5XX31FYGBgSzdLCCEuS8b4m1Fbez5CiJYlV+cUQghRLxL8QgjRzkjwCyFEOyPBL9q9dUfWcTbv\nbIMem1aYZvf+88vz7d5GS9qdtpsLJRcabXufHfmM+OT4Rtuevc7ln2PbuW1Ntv2c0hx+OP1Dk22/\nNhL8ot2b9PkkXo5/uUGPDX4rmMMXDtu1f+/F3hzIPKAu7z+/32qbZYYyvjj6hV37qKmoosiuSQbh\n74bz8YGP1eUhHwzh1tW3Wq2zN2MvWcVZtT7+RPYJskuzrdpTYaxQl+/9/F7u+uyuBrfPVGWiqKJI\nXb5z7Z3c/PHNVuv89/B/eW/Pe3VuI6c0R/35owMfMWLFCHX5VO4p+i/r3+D2gbkPLDYlbWLsp2Mp\nM5TZtc2rYVfwT58+HX9/f/r06VPnOo899hjh4eFER0ezf/9+e3YnBACVpspG36aDpuF/CttTttu9\n/5pBOGD5APq810d9ns9vfp67191d52Ove/s6fs34td778ljkwfv73m9wW0/lnmJL8har2y7d/8D3\nBzJ9w/RaHx+xNIIxK8eoy13f7sqfvv6Tuqx10JJbllvv9pzJO2P1xvPsT8/ischDXY5Pjrdp7xPf\nP8HDmx6udXsJ6Qn4ve6nLuscdAAUVhQCEHcqjgOZB6hSqurdRouzeWdZc2gNEUsj1O1pHcyz6s8X\nn7/q7TWUXcE/bdo04uLi6rx/06ZNnDp1iqSkJJYvX87s2bPt2Z0QADi/6mxTAVv+iC6nSqnijZ1v\nWFVzpipTrevWZ3uWx87eONuqwgRslutiMBkArCpei91puwF4Z/c7AHx+9HP1vptW3ERSThIAKQUp\nfHPyG/W+65dfz/7z5iJr1cFVtQ7D7Du/74ptS8pJotxYzr7z+2z6wxKGAK46VxQU9blYWPrgYslF\nfjz9o3q7k6MT+zOri8D88nz1uQIEuAcAWIV5Uk4SaYVp6v81xSfHsylpk7pc800UwN3J3ea5eTh7\n2NxmYWmL5TUpM5or8dSCVKD6Nas5RKcoCt+f+r7ObVos3rGY+768T31OAIYq8/bOF1kH/8rElTZv\nLqYqE58e/JQ+79VdbNeHXcE/fPhwvL2967x/w4YNTJ06FYDBgweTn59PVlbtH/+EuBoHsw6qP2cW\nZ+K5yPOKwxeFFYU8/ePT+L3ux1u/vAVAubEcgLyyPHW9vLI8PBd5qn/gd392txryBpNB3U+FqTqs\ng98KVrc18P2BeCzyIL0wvdZ23Lr6VjKLMwEoqCgAIKcsR92+RXFlsdXjJq2bpP6cXphOSkEKGUUZ\n6rLFvvP7+P60OYT++NUfeeqHp2zacLH0otXye3ve40zeGavbeizpwYubX+T65dczfb25ereEXVFl\n9RubJUQfj3vc6vHlxnKe++k5hn4wlDGfVlf4ff37qs912d5lQHW4gvmNBODYxWPqbTetuIlu73Sj\nx5IeTFwzUb39o/0fEXfKXHwqisKkdZP4ONE8DGV5zTo4d7B5/h2cbG+zsLwWGUUZlBnK1EC2VOSW\n173mp5Ls0mwmrJ5gU0ikFKTwv7P/U5e7e3dXfz6dd1rtB4D0ourXMKMogz99/Sf198Ri67mt3P/V\n/Ry+cNiuoaEmHeNPT08nODhYXQ4KCiItzf6DYa1NWVkZ5eXlLd2MRqMoYDS2zn8l5eY/usKKYvW2\n/FJzQKbmn7dZ/3Dmcf6VsAyjESoN1dXTl8e+wmiE4t9et+zSHIxGqKg04fN3HwBySgooKivji2Nf\nkF6QhcGg4PSqEw4LHLhQlKu2BcwBnpKXgdFoHt8GSMo5W+tz2JS0iZUHVmE0QnaxOUizirMxGiF2\nxUgAbg4ZRUZhFkZj9euiddCq2yg1lJJZdJHAN81nipcYyhi3cjyHM4+b21NWpD525cGVJGWb2xJ3\n8sffnv+XxJ38kczCbCoNVTy86WFe3PwSH+//VN0HwKlcczjtTtuN0Qinc87hqnMlPjkeg0HBaAQP\nJ3Pwbz6z2eqxZcZyFu9YrAac5T7Nb7FzvvAiD218CAAnByf1fhetOfgzijLUffTz769Wxo6a6n6Y\nvmE6646uU1+vr45/pfbXtuSdGI3gpjNX/EezTqqP6/Bbm41GMBgUKipN6n25pebX5GxuCg9seJAP\nD3z42/bzzX1faf6dySrKUR+TVZRDlVLFhaJcjEb4x8636fluTx7YMJNRn4yq8dyrL5aQXpiJ0Qjl\nBvPzuvfze3ki7kmMRvg13XzM51xeutXvTk5J9aeM6etn0FBNfnD30ipMU8d1lOfNm6f+i4+Pb+pm\nNRqTycS8efN45ZVXqKq6+jG/1ujpp8HJCfT6lv3nrFdwvu6A1W0eAeZK9Z1lueptvfuXABDy53k2\n2+izLJJHvnsIZ/dSAoKqq+ntqdtw7phGl67m8P7lQA56Pbh6V1eygWF5eHUxV3+h0ak4X28OGBQN\nnWPX0znIengmMuYCen318s13nVLb4XT7EzhHf6Xe/+xfM9HNGE3vG82h+Ozik+j1sDNtBwBbVg9g\n5hNZ5vXLvNHseRhDUqy6vfPZpdw/q3pIY81nFXx/Jo6+938CwKIPj1m1peft36LXw/g11ZX3hKc/\nI+CtjuhvNFfd/92+kz9vuB+n8S/g9HvzsOyGk+sBSDtvQK+HQWPOUnpkJFn5BehDEtHr4WQSaL55\nn5MXknF2rVT3e/iYdTHkrFfQ62FPormPQyOrQ8yyfb0eDux1AcyVtd47H70e4r7sqK67d2sn8+9H\nL+uhFf/uWRhL3dTlkStH4Bx0jL1xPQDovaQvzr7mPo3/ybwP524JOC10QP+aFuewnTgHH+bduE1Q\n5cgt9x1hzWe/vcYmHVOm56HXw6uLzLeNGJOLs2c+ur9piL7BPJIReOsn6J6I4Kkfn+Bk7kl+PGOe\nrWN5bs+8WKq278mXM9Hr4dG5BrjYC4C3d7+JXg93TDIXMzeOy8Dp9r/g3DENvR4m/W4vbAG2wH8X\n/kxDNWnwBwYGkpqaqi6npaXVeS2bmsEfGxvblM1qVPHx8Tz55JM88cQTbNmy5coPuAbk5cH777d8\ndb8zOYGqB/tb3ZZ0xnzA847J+eptT65YBcDv7su22YZF1fNunDlnHUQr4reQdLYcV50rHbuaK7Xk\njOrgH/WPR9h3zBz8//0uBeWuewF4YuhcXn83l7MplQR5BLHw5oUAfPF9Jjd/VB2sgx7+t/opQBny\nNlWT7uSJTU8D8MCcAuj2E8++txWtg5YhdxzEaIQ+nfrw+ODHef0Vfx5/wVwRunpUsPndu4kZnsea\nxHWUlFfg5FbGiwurh01H3Woeqrrxvu146b0IGrQHoxGCPYL5XcTvmPzUL5zOPmf1/G+dbB5GeH7h\nb5/CvZMBUG58DSXm31br+nQ0YDTCq/8+xlNTI0FXrr42waFlnP1pNP6e3nx1KI70fHO7gkOt+zu3\nuND86azjMQLcA9i2pzr4tR45VBqqMBrhphurjx8cPGPuv3v/UP0me+ut5tf28Xe/Z0z36v7ekpBF\nYEfr8fzvdqQyacpvb/jaCn48cITk3DQGxZrfNB/+90p13Q+/TkIzux/4JjEidBiPvHKMpx8INT8X\n784sftv8O/f4X8zPq+q+Cfzft+Yhw6qpsQC43zof/Kpn7FgkZhzBYFB47qUS9bZpc8yf6N5408A9\nN/UGwEXrwoZjm9TftVeXHUUZ8hZVc4PJKszhvf92JereKFa/u5rrH+pss5/6atLgnzhxIp98Yq5A\ndu3ahZeXF/7+/k25y2Y3atQoOnXqROfOnRk1alRLN6dRtJbLCdX26dBywK3mgbXXd74OoB7ETMxM\npLCiEM18DcEeweoY9J6MPepjHh/8ONml2ZQbywlwDyCvPA9FUazGrn888yP9lvUDUMe/f/jjD3jp\nvcgrz6PCWIGzozPPDHuG23vcTlZxFj+e+W0oZdKX7E7fzT3r7mF+/Hx1m2/88gZQPcb+2vbXCPMJ\n45e0X3h719uUGEqYM2gOnd07k1ViDtBKUyWBHoHszdjLpM8nceNHN1JpquRETnXAWMalt6VsY3Lv\nyZwvOs/aw2spN5Yzqdck1hxeQ8g7IVZ9eTWzo3LLcik1lHL44mGiOkUR6hWq3lduLMdF54Kvqy93\n/PcOOv/DHEg1j4EAfHrwUzq8Zh5bv87rOqvX0Fhl5F97/mXTrvUn1rM9ZbvVwW/LwdtyYzkTe5jH\n+50cncgqzsLNyVzxfzHpCyZHTSarOMuqHbvSdhH8VjAJ6Qk2bbxQcgGTYh6jD/MOI688D0cHRwAc\nHRzV9laYKtRZYP/Z/x+r51jXTJ+o96J4e9fblBqqK37L62swGbjO8zrAfLyk5oHw5fuWqz+fLz5P\ncWUxY7qNIaZLjF3nf9gV/FOmTOGGG27gxIkTBAcH8+GHH7Js2TKWLTN/dJwwYQLdunUjLCyMWbNm\n8a9//cue3bVKGzZsYOPGjTz33HOsWrWK+++/n+PHj7d0s+zWWN9sZg/LH9fTPzzNPevuAapDoebB\nWIuskiyS85Ppt6yfeuAtoyiDTm6dADiUdUhdt5NbJy6UXKDCWEEH5w64aF345+5/8v6vtU9zPJ13\nmlCvUEZ3H4233pv88nwqTBU4a51xdHCkr39fzhVUV9R3RNwBwMmck8zbOs9meykFKerPlul8r21/\nzRyiWhf83f3JLM6kSqnCVGVSZ7pA9TGEzWc309G1I1GdoqwONN7a41ZMionJX0ymzFjG9V2ur/U5\nWWY3Ldy+sNb7L/Xqz6+SW5ZLR7eO7Jm5B09nT8B8YNZF64Kfa/UUyHt732s1m0iDhv2Z+wn1CuXs\n42cJ8Qohvzyfm667iS1Tt/DIwEdYe2QtVUqVepAc4MX/vcjwj4ZTYapgzV1rSH48maTcJLKKsyg3\nlqPX6ll22zKmRE0hqyQLN505+PVaPQHuAWSVZFFhrGDeTfMYEDCA3em76enbU92+5eB4VKcoTuac\nVG/vH9CfvPI8tS1dOnThaPZRjlw4QkFFAV06dFEfP7zrcPVxJYbqiv5Sx7OP1x78VQactc7kP5tP\nmbHMahZScn6y+nNmcSYllSW4ObnhpfeyK/jtuizzmjVrrrjOkiVL7NnFFWnmN05CKa9cfZmbkpJC\nr169CAsL4+WXX+a5557D09NT/YKWa1VrqfgtVd5HBz5SK9pKUyV+rn5WyxYXSy6qs31C3zFXpCbF\nhJfeC6ieRQHQ0bUjx7KP8fbut3F2dMbX1Ze538+ttR0Lb17I5rOb8XExH/T10nuxdM9SgjyCcHZ0\nBsDfzZ+t57aqj3HQOGB62UTk0ki1QlVeUYhYEsGJnBNW0yktz7OrZ1eS85NxcnRSK/4KYwVOjk61\nTknMLcvlhRtf4Lkbn6PLm13U20dcV32yUXFlMYEdah9ePXLxSK231zQ0aCi/pP2i7q/MYA55Hxcf\nKk2VFFcWqwFcM/inRk9l7ZG16nJsSCzHso8xpvsYQrxC8HfzJ6MoQ31sV8+uLN2zlD9++UerqZ41\n+8jHxYeunl3JLcul8z8608O3B7d0u4X7+tzHhZILpBemqxW/pT3ZpdlUmCoY1nUYTo5OvLf3PYYE\nDVE/LVnegIcEDrGq3qP9o1lzeA3lxnJeHfkq0Z2juX3N7Xx9/GsABgUOUqeVDgkawraUbbjqXCk1\nlDI7Zjbv7TWfHPbU0KfUT3nL9y3nnl73ML3fdP7Q9w+M+mQUG09uxGAyoHPQ0cG5AyWVJegczUNd\n1wdcT4mhhBV3rOCpH5/idO5pEjISGNF1hN3Bf82fuau8ojTKv4bo2rUrYWFhNt/AVdvXLl5rWqri\nP3LhiPpmbqm2nByd1PsrTZWE+YRRWFGI5yJPtWqN9Ivklm63sPLgSqvt+br4EuYTBphPPPLWe7Nj\n+g46uXXi04Of8kniJxirjFaVVU0b79uIt4s3p/NO4+1inrps+f/5zc/jrP0t+N39rSpGMIf/LaG3\nWN0W4hVisw+do47EhxIpM5SZh4+0zvi7+ZNVnEWlqRInR6c6J0X4uPjg7uSuvnksGrXIZo66ZXpk\nTcmPJ1NuLMdbXz0d29/Nn27e3azWe/928yeg3Q/s5sczP1JqKMVF54JGoyGgQwBHLx7FVeeKzlGH\nn0t18LvqXNU33JNzThLtH83JnJPqbT18e6jnCei1evXT3ZrD5mIy7g9xPH/j8+r2Np/djLOjMxqN\nhvv6mOfBn8s/h15rPpLczbsbp/NOq8/VydEJb703eWXVQ3Kh3qGkFqbS2b16bPx03mmeueEZ5o+s\nHo7bPm07Pi4+5JWZK/6ADgGE+4Rb9cuNwTeqPw8JGgJAhJ/5e7ZHdxut3ldqKOXhmIfVft2bsZdJ\nvScxNGgoALetuQ1DlQGdow4HjQNhPmHqkOSQoCEce+QYg4MGMzV6Kg9tfIhNSZtwc3JD56hTn3tD\nXPPB35La6jdwtWTFvzt9929tUNTgt8yfPpN3hhs/uhEXrQu+Lr4UVhRyNt88bHDk4SP09e9rdZIT\nwM/TfuaDiR+wZPwSMoszCfUO5YbgG9ThH7Ce0777gd2kPZHGxJ7mseMJ4RPw0nuRnJ+sVvz+btXH\nqSxvSl09u3LoQvVQkkVAB/MQzcb7NgLwyMBH6OjaUf2kkPRoEt//8Xuu87yOs/lnqTCZK3xfV/Pz\nG/bhMKvjDhaWIPFw9kCj0ahte3zI4zbr1nzTsAR9sKd5mnVMlxh+vP9Hzjx2hrS/pLHijhVA9Xzz\n3p16c+ThIwzsMhBnR2f2ZOzBRWueEdOlQxe2Jm9VQ9HX1Vfdj6vOVe3jcN9w/N39yS7NVoeHunt3\nZ0fqDrJLs9Fr9TYnsPXw7aHO97ewvMlaQrPCVKG2JcwnjNN5p6vPsTBW4O3izcXSixRUFOCsdVaP\nS3R278yWqVu4t/e9lBpKGdN9jNXvw7Cuw/B28VY/Lei1eps37NiQWN671VzVW4I/2MPcpzXbHd05\nmqW3LuXpG8wH9c/mn2VI0BBcdC6svnM1YB7is5wQN6b7GL49ac6QvPLq4cxp/aapP3f1NI8o1Gzz\n1ZLgt8MPP/zAt99+aw6p376Bq1Onhr8YrYWiNG/FX/MklRkbzHOTI5dG2lwDJzEzETCPiW/6g/lM\nzWEfDsNN54ZGo2HuEPNQzYvDX1T/+Hp17IWrzpWefj25UHJBHU/v6FY9PfBCyQXeGWc+OzbcJ5xA\nj0B1PagOeh+9OVwtYQ6oAW6p9i5lGf4YHDgYgNt73s6Fpy/goqsOrCCPIDz1njg7OqsVvoPGAT9X\nvzqHYw7PNveNpbq3fAqxvBE5ahxtHtPVsyv/HP9PoPr4iUaj4ZZutxDqHYrWQauGydM3PM2gwEFq\nH2o0GjVwLW3v0qELO1J3qG9CNYd6agZ/zT701JuD39/dnyMXj5BelI5eq7f5pGFSTPTwNU/DnHfT\nPKC6rzVU/3Jaqt7u3t05lXuKoxePAuZxc2+9N18d/4rDFw5b7SPAPYDYkFj1eXRy62T1eoP5zaHM\naD55S6/Vq286FsYqI3dG3qlub8/MPbw7/l2g+lPdSyNe4sHrHwTgoZiHiOoUZdUHk6Mmq9uzDO9Y\ntgnWExgsB5mh+nfN8ubdEPLVi3Z47LHHWroJ17zvkr5jwuoJKK8oVpcFOJFzgkMXDjElagoajYYv\njn5BYpY5+EsMJVZBa/mjrDnE8f7t71udieqt96bMWKZWVjVDqdxYzkMxD/F43OPqH6WleoPqPzBL\nVV3zsZYxZcsQxqWMVeY5pTWrYTD/UV96fSA/Vz/yyvPU2wM9Amu9fssbo99QQ8uybu+OvTmefVxd\n1mv1VgcaLcOZiqKoBya/vvdrNdwtOrt3xt/Nn1kxs5gVM8vqPkugWarsAPcAvjz2Jb+P+L3a/pr9\nUrOfLMMrlj7s6Fr9xqvX6pkcNZnfR/6eER+NYHLUZEK8QtA6aMl5JoeLJReZt3We+jr36tjL6rGW\n7RZWFFJYUcibY95kdLfRVtdQ6uTWCT9XP9x0bmpbnh32LNP7TaePf/XlDyxvMg4aByL9ItmfuV/d\nR++OvTly8QhvjX2LiT0n4ujgSOX/VaLRaIjpEmPVz4DNmbXfTPmGU7mn1GXLsNXqQ6ttfi8HdhnI\n44NtP705OTqpb4hBHkE299eXBL+w0ZwVv+VgbJVSZXNp5LzyPPzd/Pnwjg/ROmhZkmCeKHDpQa1L\n/8BCvUIZFDjIKtQsFbGlsrKcsj938FwevP5BnBydrP5oF9+ymGeHPQugHhy1vCk4OTrx4vAX+du2\nv132mi9gPkho+Whe0/Zp220O2FreRCy6enZVZ/AAHHvEfAmDmm96loPbMV1i+OJY9fWLLMFvGbqx\n0Gg03BxqvlKlZeZRTZ56T84+XvslqtXgr1Hxpxam4u9uruZr9rerzpVHBz2Kl7P5DdGyjiXwa75J\nWIar9Fo9CTMTrPbp4+KjztSxfIoZGTqSNXetYcoXU9S21BzOGn7dcBwdHOndqTeezp4UVBTQ0bUj\nGo2GAQED1Mo/wi/C5pOa5fcEzG9WezL2qMF/+OHDlBvL1WMNUP37VJtLZ/iEeIXYDBktnbDUHPy/\nbcfSP+/f/j7RnaNttvm7iN+pP98XdR+rWV3n/i9HhnpEi7JU8RdLLnKh5AIDuwxU7ztfdF79eH9X\n5F3qTB7LVE7Lgcea17Qper6I6f1trwppqTQtlZXlD9dZ60xkx0ib9Z21zuqQjiVcak7Fe3TQo0D1\n5QrqMvy64Zybe87m9mFdh9n8YV96ELaTq7n6e2nES4BtUN3X5z51Bo9l7NxiwcgF/N/w/2Nqv6mX\nbV9tLM/3Utd5meeaWyp+y6ciy74j/CI49egpdZ3YkFg+uOMDoHqox1LR1hw6qe3gc02Wg901PzVZ\nPrVYfj8Asp82z56yXBzOz9WPj39nvm6PZajk52k/09OvejpnTXF/iGNG/+rLIFh+R2r2uV6rr/NA\ne01r715rdXC6Lpa+Kyg3X7PJ8ntaczjRIvvpbD664yN1+dYet9qsU19S8QsbzVXxG6uMbEnegpvO\njYyiDD45+AmBHoEcvXiUEkMJGUUZarV1S7db+Oe4f/JY3GNqxf/AgAeY+c1MqyGi2qY9QvUfWM3w\nrm25Lul/SbcazrH8gdbcX4RfBBN7TLSaIXI1LJWthSXoFoxcUOv6q+5cpf5s+TRi8fDA2i85bA/L\nSUaWN4a7e91NiaGEsd3Hqut09+nOnZF32nx6sQR+zfA+89gZm4Pxdcl8yvpiZYMDB3N/3/utxrl9\nXX3ZMnWL1bz68eHj+e4P39VrH2PDxlotXywxH/S3vMlcjUm9J115JarfXCxTfh0dHNk1Y1etB24v\nHS60hwS/aDHHLh7D09mTSL9IDmYd5NODnxLiFcIt3W5h/Yn1ZJVkqYHmqnPl0cGP8uQPT1pdydFR\n46he4fJyLH9gqYWpVrfXNTZ/qUv/+C0fzWu+cViGYhrq0rC0vLnUR81pmU0l0COQG4JvUI8j6Bx1\nPDDgAZv1vphk+6UxOkcdi0YtspoRFeodytPDnm5QW5y1znzy+09sbo8NibVadnJ0YlzYuAbto7mk\nzE2xeq0HBw1u8n1K8AsbzVXx55Xn0dGtI94u3hzLPoarzpVvpnxDr469+PjAx0zfMN3q0gBgno1j\nOdsSoI9/H6tvr7oSyzXVATL+kmF3FVXzbF17+bpYt8UyE6g+xnQfw4/3/3jlFe2gddCyY/qOBj/+\n2RufbcTWNL1196yrdSptY7Nndk5DSfCLWjV18P91619Zd3Qd13ldh4/eh+T8ZAYFDlKnvIV6mwP/\n0oNh26dttxrT/+n+n2yuW18Xfzd/9TR5qH0c9WqsunOV2t7GcOkZtsO6Dqv3yYWODo7c0u2WK68o\n6i3Qo/YzntsCObgrbDTlCVymKhN7M/by1q63OHThEJ7Onvi4+LAtZZvVl2NYrqdy6cfeUO9Qq+l3\nvq6+6kHz4Tx6AAAfvklEQVTHK2nMkAbzwdVLTzKyhwS3aC6tuuL39vau1xH0a8Xlvq2stWmqbv/m\n5Df8fu3v1WUvvRed3DqRUZRhVbkHdAho8KU06rLunnU23zzVmgy/bnijP2chatOqgz83t/5fuCwa\nT2NX/McuHiPCLwKNRqOe0GThpfdSxzgv/c7Wxubt4m01T1uI9kqGeoSNxjy4W1xZTK9/9eLQhUNU\nmiqtvn8UzGd/WmY01DypRwjRdFp1xS+ufSeyzZe/zS3LJe5UnHq5WosOzh0YFTqKbdO22VwBUQjR\nNCT4hY3GrPgtlxS4WHLRZpjnrsi7GNt9LBqNhhu73ljbw4UQTUCGekSTsgT/7vTd3PflfercdG+9\nN59P+ly9hosQovlIxS9sNEXFv/nsZsD8zUwxXWLU670IIZqfBL9oUoYq80wdy9m1IV4hzB44uyWb\nJES7J0M9wkZTVPwWrf26KUK0BxL8oklVmirV66gHuAe0qRPyhLhWSfALG41Z8RtMBvp17gdQ78vj\nCiGalozxi1rZE/yW7421/NzHvw97H9x7hUcJIZqL3RV/XFwcERERhIeHs3jxYpv7s7OzGTduHP36\n9SMqKooVK1bYu0vRxOy9ZIPzq87sO78PMAe/5VuvhBCtg13BbzKZmDNnDnFxcRw9epQ1a9Zw7Jj1\nl1EsWbKE/v37c+DAAeLj43nyyScxGo11bFG0FvYO9SSkm7871VBlUKt/IUTrYFfwJyQkEBYWRkhI\nCDqdjsmTJ7N+/XqrdQICAigsNH81XmFhIb6+vmi1MsLUmtlT8VcpVQDsStvF4QuHefS7RyX4hWhl\n7Erg9PR0goOrvz0mKCiI3bt3W60zc+ZMbr75Zrp06UJRURGfffaZPbsUzcCeg7sllSWA+UzdQ1mH\nANRZPUKI1sGu4K/P1LyFCxfSr18/4uPjOX36NKNHjyYxMZEOHTpYrTdv3jz159jYWGJjY+1pmmgh\nBRUFuGhdOF90njd3vQlASmFKC7dKiLYhPj6e+Ph4u7djV/AHBgaSmlr9HaapqakEBQVZrbNz505e\nfPFFALp3705oaCgnTpwgJibGar2awS9alj0Vf2FFIV09u3Ku4Bx7M8wzeY5nH2/E1gnRfl1aFM+f\nP79B27FrjD8mJoakpCSSk5OprKxk7dq1TJw40WqdiIgIfvrpJwCysrI4ceIE3bp1s2e3ohUrrCjE\nU+9JgLv5+2wnR01mRv8ZLdwqIURNdlX8Wq2WJUuWMHbsWEwmEzNmzCAyMpJly5YBMGvWLF544QWm\nTZtGdHQ0VVVV/P3vf8fHx6dRGi+ahj0Vf0F5AZ7OnmgdtJzNP8uKO1bgrHVu3AYKIexi9/Sa8ePH\nM378eKvbZs2apf7s5+fHN998Y+9uxDWisKIQD2cPPPWeOGgcZEaPEK2QzKsUNuwd4/dw9sDdyR03\nnZtcm0eIVkiu1SNq1ZC8TspJ4oFvHsDT2ZMuHbrgqnNt/IYJIewmwS9sNPQErpQC87TNtKI0AtwD\nJPiFaKUk+EWtGlLxmxQTAOPDxtOlQxfcnNwauVVCiMYgY/zCRkMr/nJjObf1uI3p/aeTV5bHo4Me\nbdyGCSEahVT8wkZDD+6WG8vRa/UAeLt48+D1DzZyy4QQjUGCXzSamsEvhGi9JPiFjYZW/BXGCvSO\nEvxCtHYS/KLRSMUvxLVBDu4KG1db8R/IPIC7k7sEvxDXCKn4hd2uX349/Zf1p9xYLtflEeIaIMEv\nbFxtxd/RtSPFlcVS8QtxjZDgF3YpNZSSVZIFyBi/ENcKCX5h42oq/o0nN6o/55fn46aTs3WFaO0k\n+EWt6hv8xiojYB7uScpNwt3JvQlbJYRoDBL8wsbVXLLhYulFHhn4CKHeoZzIOSHX5xHiGiDTOUWt\n6lvxZ5dm09G1IwnpCQBS8QtxDZCKX9i4moo/rTCNLh26EOQRBEjwC3EtkOAXNq7m4O7Ri0eJ8Ivg\n+CPHAeQa/EJcAyT4RYOdyj3FiZwT9PXvq47tWw72CiFaLwl+YaO+FX9KQQr9OvfDU+8JQNwf4ujf\nuX8Tt04IYS85uCsarNxYjovWRV0eGza2BVsjhKgvqfiFjfpW/HKmrhDXJruDPy4ujoiICMLDw1m8\neHGt68THx9O/f3+ioqKIjY21d5eilZDgF+LaZNdQj8lkYs6cOfz0008EBgYycOBAJk6cSGRkpLpO\nfn4+jzzyCN9//z1BQUFkZ2fb3WjRtKTiF6Jts6viT0hIICwsjJCQEHQ6HZMnT2b9+vVW66xevZq7\n7rqLoCDzPG8/Pz97dimaiQS/EG2XXcGfnp5OcHCwuhwUFER6errVOklJSeTm5jJy5EhiYmJYuXKl\nPbsUzaC+J3BJ8AtxbbJrqEdTj7LQYDCwb98+Nm/eTGlpKUOHDmXIkCGEh4dbrTdv3jz159jYWDkW\n0MKk4hei9YmPjyc+Pt7u7dgV/IGBgaSmpqrLqamp6pCORXBwMH5+fri4uODi4sKIESNITEy8bPCL\nliUVvxCt06VF8fz58xu0HbuGemJiYkhKSiI5OZnKykrWrl3LxIkTrda544472L59OyaTidLSUnbv\n3k2vXr3s2a1oBlLxC9F22VXxa7ValixZwtixYzGZTMyYMYPIyEiWLVsGwKxZs4iIiGDcuHH07dsX\nBwcHZs6cKcHfytWn4lcUhfPF57nO87qmb5AQolFpFOVqrsXYRI3QaGgFzRC/GTEC/vpXuOmmutf5\n+vjX/H7t7/lmyjfc1uO25mucEELV0OyUM3dFg1SaKgHUyzELIa4dEvzCRn1O4DKYDGgdtER1imqe\nRgkhGo0Ev2iQwopCHuj/AFoHuc6fENcaCX5hoz4Vf2FFIR7OHs3TICFEo5LgFw0iwS/EtUuCX9io\nT8V/vvg8vq6+zdMgIUSjkuAXtbpc8GcWZ/LlsS+JDYlttvYIIRqPBL+wcaVpwbvSdjEgYAARfhHN\n0yAhRKOS4Be1ulzFn16YTk/fns3XGCFEo5LgFzauVPEv37ecQI/A5mmMEKLRSfALG5c7uGswGTiY\ndZCJPSfWvoIQotWT4BdXJaUgha6eXenr37elmyKEaCAJfmHjchV/cn4yoV6hzdsgIUSjkuAXVyWz\nOJPO7p1buhlCCDtI8Asbl6v4//jVH+v1lZtCiNZLgl9ctRPZJ1q6CUIIO0jwCxt1VfymKhMAn/z+\nk2ZukRCiMUnwi1rVFvwlhhLcndzlGvxCXOMk+IWNuk7gKq4sxt3JvXkbI4RodBL8ola1VfzFlcV0\ncOrQ/I0RQjQqCX5hQyp+Ido2CX5Rq9oq/sKKQgl+IdoACX5ho66KP6MoQy7OJkQbYHfwx8XFERER\nQXh4OIsXL65zvT179qDVavnyyy/t3aVoYnVN50wpSCHYI7j5GySEaFR2Bb/JZGLOnDnExcVx9OhR\n1qxZw7Fjx2pd79lnn2XcuHEoV7rmr2i1LBdoE0Jc2+wK/oSEBMLCwggJCUGn0zF58mTWr19vs967\n777L3XffTceOHe3ZnWgmdVX8qYWpUvEL0QbYFfzp6ekEB1cHQVBQEOnp6TbrrF+/ntmzZwPIdV6u\nYVLxC9E2aO15cH1CfO7cuSxatAiNRoOiKHUO9cybN0/9OTY2ltjYWHuaJuxQZ8VfkEqwp1T8QrSU\n+Ph44uPj7d6OXcEfGBhIamqqupyamkpQUJDVOr/++iuTJ08GIDs7m++++w6dTsfEidbf4FQz+EXr\ns/nMZsqMZfi6+LZ0U4Roty4tiufPn9+g7dgV/DExMSQlJZGcnEyXLl1Yu3Yta9assVrnzJkz6s/T\npk3j9ttvtwl90brUVvHfsvIWQIbqhGgL7Ap+rVbLkiVLGDt2LCaTiRkzZhAZGcmyZcsAmDVrVqM0\nUjS/2vL9dxG/a/6GCCEanUZpBfMrLeP/onXo0wdWrYK+v32tbpVShXaBFtPLJqn4hWhFGpqdcuau\nqFXNfK8wVuCsdZbQF6KNkOAXNi4tIMqMZei1+pZpjBCi0UnwCxuXHtwtN5bjonVpuQYJIRqVBL+4\nojKDVPxCtCUS/MJGbRW/BL8QbYcEv7iiMmMZLjoZ6hGirZDgFzak4heibZPgF1dUZiiTg7tCtCES\n/MKGVPxCtG0S/OKKZIxfiLZFgl/YkIpfiLZNgl/USk7gEqLtkuAXNmwu2SAncAnRpkjwi1pJxS9E\n2yXBL2zIRdqEaNsk+IWNWi/SJrN6hGgzJPjFFckYvxBtiwS/sHFpxX8m/4x8yboQbYgEv7isksoS\n/nf2f9wZeWdLN0UI0Ugk+IWNmhX/gcwD9O7YGzcnt5ZtlBCi0Ujwi8vKKcuhs3vnlm6GEKIRSfAL\nGzUr/qKKItyd3Fu2QUKIRiXBL2plCf7iymI6OHVo2cYIIRqV3cEfFxdHREQE4eHhLF682Ob+VatW\nER0dTd++fRk2bBgHDx60d5eiidU8gau4slgqfiHaGK09DzaZTMyZM4effvqJwMBABg4cyMSJE4mM\njFTX6datGz///DOenp7ExcXx4IMPsmvXLrsbLppWzYpfgl+ItsWuij8hIYGwsDBCQkLQ6XRMnjyZ\n9evXW60zdOhQPD09ARg8eDBpaWn27FI0g5oVf1GljPEL0dbYFfzp6ekEBwery0FBQaSnp9e5/gcf\nfMCECRPs2aVoBjUP7hZXFtPBWcb4hWhL7Brq0dQ8vfMKtmzZwocffsiOHTtqvX/evHnqz7GxscTG\nxtrTNNFIZKhHiNYjPj6e+Ph4u7djV/AHBgaSmpqqLqemphIUFGSz3sGDB5k5cyZxcXF4e3vXuq2a\nwS9altV0ThnqEaLVuLQonj9/foO2Y9dQT0xMDElJSSQnJ1NZWcnatWuZOHGi1TopKSnceeedfPrp\np4SFhdmzO9ECZDqnEG2PXRW/VqtlyZIljB07FpPJxIwZM4iMjGTZsmUAzJo1iwULFpCXl8fs2bMB\n0Ol0JCQk2N9y0WQuHeOXil+ItkWjKJd+7UYLNEKjoRU0Q/wmKAh++QWCg6H3v3rz2d2f0btT75Zu\nlhDiEg3NTjlzV9iQSzYI0bZJ8Is6lRnKyC3LxVPv2dJNEUI0Igl+YcNS8W9L2Ub/gP546b1auklC\niEYkwS9qpdFAfnk+Ae4BLd0UIUQjk+AXNizHigrKC/Bw9mjZxgghGp0Ev6iVRgOFFYUS/EK0QRL8\nwoal4i+sKMTTWQ7sCtHWSPALG5aDuwUVMtQjRFskwS/qdLH0Ir6uvi3dDCFEI5PgFzYsFf/p3NN0\n9+7e0s0RQjQyCX5RK0VROJlzkjAfubCeEG2NBL+woSiQWpyMi84Ff3f/lm6OEKKRSfCLWu3L2s2g\nwEEt3QwhRBOQ4Bc2FAUOXNzDoC4S/EK0RRL8olbZZVkEedh+m5oQ4tonwS9sKAqUGkrkcsxCtFES\n/KJWJcZi3JzcWroZQogmIMEvbFgqfjedBL8QbZEEv6iVDPUI0XZJ8AsbigKlxhIZ6hGijZLgv4z2\n8AXwy39dznt73rO6TVGgxFAsQz1CtFES/HV4ecvLOCxwILUgtc51ckpzSC9Mb8ZWNb4XNr/Aw5se\ntrpNQaFQrswpRJvV5oN/T/oeKowVV/WYl/73En/9+a8ALN2z1OZ+yyeBRdsXEfJOCJ8kfqLel1Oa\nw6R1k/j+1Pd2tBoqTZW8sfMNDmQeuOpPHh8f+JiPD3xMUUURSTlJl13XMlc/ryxPva3KOQcXnasM\n9QjRRtkd/HFxcURERBAeHs7ixYtrXeexxx4jPDyc6Oho9u/fb+8ua1VpqrS5zVRlYtB/BvH2rrfZ\nm7EX37/7cqHkwhW39a+9/wLglZteYfGOxaw+tFq97+jFozgscODLY1+SXJDMlKgprDiwQr3/ZM5J\ndqTuYNyqcWw8udFqu3lleRzKOkSVUsW2c9soqSxR77tQcgHNfA3HLh4D4NbVt/Lqz68ycc1EFmxd\ncFX98FjcYzz5w5NELo2kx5Ie/OHLP/C3n/9GSkGKbR8pJrp06GL1HBSPcwR16FrvfQohri12Bb/J\nZGLOnDnExcVx9OhR1qxZw7Fjx6zW2bRpE6dOnSIpKYnly5cze/ZsuxpcmyqlCudXnXln1ztWQzOZ\nxZkAvLTlJeJOxZFblsuaQ2sA2Jq8lYyiDKtPA3sz9qKZr6GgvID1k9czL3YeC29eSGJmorrO4QuH\nAVhzeA2fH/2ce3vfy6/nf1Wr8uzSbPp37g/An9f/2aqdb+96mwHLBzDy45GMWDGChzc9TG5ZLgAn\nsk8AkJiVqC5/e9+3vD3ubQ5kHeCVLa9wwwc38ETcE+SU5nA8+zj7z+/HYDIAsOHEBjTzNTy66VF8\nXHz4923/Jr3IPAy1+cxmDl04xJxNc2z6Lq8sj2n9prHp1CbO5Z8z96drFv6una/6dRBCXBvsCv6E\nhATCwsIICQlBp9MxefJk1q9fb7XOhg0bmDp1KgCDBw8mPz+frKyseu+jSqni53M/q9WzoihWQx/x\nyfHqOPvc7+cyYPkA9b6UghQGdhnItH7TeCX+Ffxc/Vh/Yj1Tv55K7MexBL4ZiP5ven468xMA285t\nA8xVsK+L+QtI/N39uVBa/Snh48SPuSH4Bj4/+jk9fHswMnQkOgcdXx3/CjAHv5+rH9/94TvCfMI4\nl38OU5UJgBM5J5gQPoGfz/1MbEgsGUUZPLLpERRFYeXBlQDqG1epoZQevj3o7N6ZzOJMFm5fyC9p\nv/D27rfxe92PyKWRDFg+gIB/BBD4ZiB3/PcObutxG8v3LcdB48Ddve5m858280D/B5gfO5+nbniK\ntMI09Xk8svER/P7uR1ZJFjP6z8DHxYe7191t7mNdMW4ylVOINktrz4PT09MJDg5Wl4OCgti9e/cV\n10lLS8Pf3/pyv1HPWh9gBNBX+ZHo9gZGTRkA0y/kssXjAc7qv2RY4du4VnXmR6/JBFQOByfzY7JL\ns/nznEwUqtjisQAPUwzZh6ai+PyH0AuPkliwjgSHE+AI0SVP4YCOJ5atZ3jRLWztcIJAx1GkO29m\nyeu+rDLBOSd/El1TeeCHQnRKBzb5b+J3udu4gyq8LkTw1OOuRLq8wl9WruCn/Dv5sOOTXFdxO/zU\nnySvMiLfHIynKYxOhoEcdFvLjYXvgscGnA88RifDINb59mfA7uc4oV9PF9NI/u+HV0lYezN5PgW8\n/LQvxY7FHPA5DRodf76YQYH2FBe1e9nu8Rjexl5cf34BP3iZA7tg7zhuVu6BAg0PPwxwMzpuJhHY\n6ZDBcZ+M326Hr7wPcn3pMnyMUbz+YiherOTXTi6M+ssnVGLCXSfBL0RbZVfwazSaeq136cHJWh+3\n/5z6Y6fu4XQK68FnxY+iUKXe3iE8EUPJGTDCDo+59NSNor/DPexnHVqcucv9TbaWLaU0dB0uGi8c\nynL5o+cKPBz8GYXpt628zP9K3+KLkr/wYMjrnKyM592CW3D0SkMH3Oh0J2nGbgyJCEGngY7GMLbl\nH+IDZ086OYaDCW7pMczqOXSr+iPP5MylMnwdFYV5TAyYTajOn0EcwKCU80v5RyQbdkEFjAu5gwtF\nXzG8WwzejoF4Va7mY+39mKpKeM7/B1YVPUCqy7t4GQLpG+WISQlmf8H1KFQxsHdHoCMwlO0XH8PR\nqZxh3Qbwg3m0iAHBvejhNLLW18CkdOKT7Czig8Zxb4elZOXuYFjnj/Fz7PbbGk48VPENG3UvM/K+\nwXi6yYFdIVqb+Ph44uPj7d+QYodffvlFGTt2rLq8cOFCZdGiRVbrzJo1S1mzZo263LNnTyUzM9Nq\nnbqa4bPYR2EeCvNQRn08Shn36Til8xudlRPZJ9Tbj144qry27TXl9R2vK4qiKIu3L1amfD5FGbBs\ngPLCTy/Uul2jyaicLzqvLh84f0C57q3rlBs+uEHZdm5brY+58cMbFeahRL8XXev9r259VRn/6XjF\nfaG7UlVVZXP/jpQdCvNsn2dOaY4S/s9w5c2dbyqKoihrD69Vwv4Zpizbu6zW/Vi8seMN5eMDHytG\nk1F56JuHlDO5Zy67vqIoSm5prtpvTn91UsoN5Vb3Xyy5qN7/zA/PXHF7QoiW1dAItyv4DQaD0q1b\nN+Xs2bNKRUWFEh0drRw9etRqnY0bNyrjx49XFMX8RjF48GDbRtTReEvYKoqiFFcUK72X9lZ8Fvso\nlcZK5R87/6EwD6WkssTqMZ8d/kwNr1UHV9XreZQbyhXdAp0S+I9A5UT2idqfq8mgmKpMtYa6oijK\noaxDCvNQ7l13b537SS1IrVd7mtLOlJ2XDfUJqyYozENZEL+gGVslhGiIhga/5rcHN9h3333H3Llz\nMZlMzJgxg+eff55ly5YBMGvWLAB15o+bmxsfffQRAwYMsNqGRqOpda56ZnEmqQWpDAwcCJjH750c\nnfBw9qDCWMH54vOEeIVYPabCWMGO1B106dCFnr496z0cNWblGEK8Qlg6YSk6R93VdgNgPiDrqnNt\n0GNbi5zSHPxe9+MfY/7BX4b+paWbI4S4jLqy84qPszf4G0NDGy8aX5VSheMCR5bftpyZ189s6eYI\nIS5Dgl80mm3ntjEwcCB6rb6lmyKEuAwJfiGEaGcamp1t/lo9QgghrEnwCyFEOyPBL4QQ7YwEvxBC\ntDMS/EII0c5I8AshRDsjwS+EEO2MBL8QQrQzEvxCCNHOSPALIUQ7I8EvhBDtjAS/EEK0MxL8QgjR\nzkjwCyFEOyPBL4QQ7YwEvxBCtDMS/EII0c5I8AshRDsjwS+EEO2MBL8QQrQzDQ7+3NxcRo8eTY8e\nPRgzZgz5+fk266SmpjJy5Eh69+5NVFQU//znP+1qrBBCCPs1OPgXLVrE6NGjOXnyJKNGjWLRokU2\n6+h0Ot566y2OHDnCrl27WLp0KceOHbOrwW1dfHx8Szeh1ZC+qCZ9UU36wn4NDv4NGzYwdepUAKZO\nncrXX39ts07nzp3p168fAO7u7kRGRpKRkdHQXbYL8ktdTfqimvRFNekL+zU4+LOysvD39wfA39+f\nrKysy66fnJzM/v37GTx4cEN3KYQQohFoL3fn6NGjyczMtLn9b3/7m9WyRqNBo9HUuZ3i4mLuvvtu\n3nnnHdzd3RvYVCGEEI1CaaCePXsq58+fVxRFUTIyMpSePXvWul5lZaUyZswY5a233qpzW927d1cA\n+Sf/5J/8k39X8a979+4Nym+NoigKDfDMM8/g6+vLs88+y6JFi8jPz7c5wKsoClOnTsXX15e33nqr\nIbsRQgjRyBoc/Lm5uUyaNImUlBRCQkL47LPP8PLyIiMjg5kzZ7Jx40a2b9/OiBEj6Nu3rzoU9Npr\nrzFu3LhGfRJCCCHqr8HBL4QQ4trUrGfuxsXFERERQXh4OIsXL651nccee4zw8HCio6PZv39/czav\nWV2pL1atWkV0dDR9+/Zl2LBhHDx4sAVa2Tzq83sBsGfPHrRaLV9++WUztq551acv4uPj6d+/P1FR\nUcTGxjZvA5vRlfoiOzubcePG0a9fP6KiolixYkXzN7IZTJ8+HX9/f/r06VPnOledmw06MtAARqNR\n6d69u3L27FmlsrJSiY6OVo4ePWq1zsaNG5Xx48criqIou3btUgYPHtxczWtW9emLnTt3Kvn5+Yqi\nKMp3333XrvvCst7IkSOVW2+9Vfn8889boKVNrz59kZeXp/Tq1UtJTU1VFEVRLl682BJNbXL16YtX\nXnlFee655xRFMfeDj4+PYjAYWqK5Ternn39W9u3bp0RFRdV6f0Nys9kq/oSEBMLCwggJCUGn0zF5\n8mTWr19vtU7Nk8IGDx5Mfn7+Fc8PuBbVpy+GDh2Kp6cnYO6LtLS0lmhqk6tPXwC8++673H333XTs\n2LEFWtk86tMXq1ev5q677iIoKAgAPz+/lmhqk6tPXwQEBFBYWAhAYWEhvr6+aLWXnaF+TRo+fDje\n3t513t+Q3Gy24E9PTyc4OFhdDgoKIj09/YrrtMXAq09f1PTBBx8wYcKE5mhas6vv78X69euZPXs2\nwGXPGbmW1acvkpKSyM3NZeTIkcTExLBy5crmbmazqE9fzJw5kyNHjtClSxeio6N55513mruZrUJD\ncrPZ3h7r+8eqXHKsuS3+kV/Nc9qyZQsffvghO3bsaMIWtZz69MXcuXNZtGgRGo0GRVFsfkfaivr0\nhcFgYN++fWzevJnS0lKGDh3KkCFDCA8Pb4YWNp/69MXChQvp168f8fHxnD59mtGjR5OYmEiHDh2a\noYWty9XmZrMFf2BgIKmpqepyamqq+nG1rnXS0tIIDAxsriY2m/r0BcDBgweZOXMmcXFxl/2ody2r\nT1/8+uuvTJ48GTAf0Pvuu+/Q6XRMnDixWdva1OrTF8HBwfj5+eHi4oKLiwsjRowgMTGxzQV/ffpi\n586dvPjiiwB0796d0NBQTpw4QUxMTLO2taU1KDcb7QjEFRgMBqVbt27K2bNnlYqKiise3P3ll1/a\n7AHN+vTFuXPnlO7duyu//PJLC7WyedSnL2r685//rHzxxRfN2MLmU5++OHbsmDJq1CjFaDQqJSUl\nSlRUlHLkyJEWanHTqU9fPPHEE8q8efMURVGUzMxMJTAwUMnJyWmJ5ja5s2fP1uvgbn1zs9kqfq1W\ny5IlSxg7diwmk4kZM2YQGRnJsmXLAJg1axYTJkxg06ZNhIWF4ebmxkcffdRczWtW9emLBQsWkJeX\np45r63Q6EhISWrLZTaI+fdFe1KcvIiIiGDduHH379sXBwYGZM2fSq1evFm5546tPX7zwwgtMmzaN\n6Ohoqqqq+Pvf/46Pj08Lt7zxTZkyha1bt5KdnU1wcDDz58/HYDAADc9NOYFLCCHaGfnqRSGEaGck\n+IUQop2R4BdCiHZGgl8IIdoZCX4hhGhnJPiFEKKdkeAXQoh2RoJfCCHamf8HtR0GzsYOb4kAAAAA\nSUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x7e53908>"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- The time constant of the post-synaptic filter\n",
      "- We're not getting $f(x)=x$\n",
      "- Instead we're getting $f(x(t))=x(t)*h(t)$\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import syde556\n",
      "\n",
      "dt = 0.001\n",
      "\n",
      "t = numpy.arange(1000)*dt\n",
      "x = numpy.zeros(1000)\n",
      "x[300:]=1\n",
      "\n",
      "h = syde556.psc_filter(t-0.5, 0.1)\n",
      "fx = syde556.apply_filter(numpy.array(x)[:,None], h)*dt\n",
      "\n",
      "A = syde556.Ensemble(neurons=80, dimensions=1, max_rate=(50,100))\n",
      "\n",
      "decoder = A.compute_decoder(x, x, noise=0.2)\n",
      "\n",
      "spikes_A, xhat = A.simulate_spikes(x, decoder, tau=0.1)\n",
      "\n",
      "import pylab\n",
      "pylab.plot(t, x, label='$x(t)$')\n",
      "pylab.plot(t, xhat, label='$\\hat{x}(t)$')\n",
      "pylab.plot(t, fx, label='$x(t)*h(t)$')\n",
      "pylab.legend(loc='lower right')\n",
      "pylab.ylim(-0.2, 1.2)\n",
      "pylab.show()\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlclOX+//HXwKAisoOALKKCLCrgFnrcMDUXSnP5KlZm\n2lHyezyZddpsES1Tq1+eyvqmLVbmlqZpLpSWmGYuKUK5hbkBKoqIyg4z1+8PzpmkAUVmmEH4PB+P\necjcc819fbhk3lzcq0YppRBCCNFg2Fi7ACGEEJYlwS+EEA2MBL8QQjQwEvxCCNHASPALIUQDI8Ev\nhBANjMnBP3HiRLy8vOjQoUOlry9btozIyEgiIiLo0aMHqamppnYphBDCBCYH/4QJE0hMTKzy9dat\nW/Pjjz+SmprKSy+9xOTJk03tUgghhAlMDv5evXrh6upa5evdu3fH2dkZgOjoaDIyMkztUgghhAks\nuo3/448/ZsiQIZbsUgghxF9oLdXR9u3b+eSTT/jpp58s1aUQQohKWCT4U1NTmTRpEomJiZVuFoqK\niiIlJcUSpQghRL0RGRnJoUOHbvt9tb6p5+zZs4wYMYIvvviCoKCgStukpKSglJKHUsycOdPqNdSV\nh4yFjIWMxc0fNZ0wmzzjHzt2LDt27CA7Oxt/f39mzZpFaWkpAPHx8cyePZsrV64wZcoUAOzs7Ni3\nb5+p3QohhKghk4N/xYoVN339o48+4qOPPjK1GyGEEGYiZ+7WMTExMdYuoc6QsfiTjMWfZCxMp1FK\nWf1GLBqNhjpQhhBC3FFqmp0y4xdCiAZGgl8IIRoYCX4hhGhgJPiFEKKBkeAXQogGRoJfCCEaGAl+\nIYRoYCT4hRCigZHgF0KIBkaCXwghGhgJfiGEaGAk+IUQooGR4BdCiAZGgl8IIRoYCX4hhGhgJPiF\nEKKBkeAXQogGRoJfCCEaGAl+IYRoYCT4hRCigZHgF0KIBsak4J84cSJeXl506NChyjaPP/44wcHB\nREZGkpycbEp3QgghzMCk4J8wYQKJiYlVvr5582ZOnDhBWloaixcvZsqUKaZ0J4QQwgxMCv5evXrh\n6upa5esbNmxg/PjxAERHR5Obm0tWVpYpXQohGqDj2cc5e/WstcuoUnFZMXd/djdl+rIar0Ov9Gas\n6Oa0tbnyzMxM/P39Dc/9/PzIyMjAy8urNrsVJlIKdDprVyHuFIWlhVwrvoZXM/N+rvVKj42mfG76\n7z3voJRi4eD3AbiUfwktNrg2dgagRFdC0unt3NNmoOH9Z6+exd/Jn/zSfDRocGjkYNb6UrJSCHYL\npqldU7KuZ/HjH9vZe2ov0b7RAJy8cpITOWkVaqpMxrUMgha2wkajoeD5kpu2LdWVci7vHC2dW5pU\ne60GP4BSqsJzjUZTabuEhATD1zExMcTExNRiVeJmnn4a3noLbGTXv6iE0hZAmT0ayj/LujH/A623\nYTu3qEI7jdLjxDVcyDU8nLhKM5WHA/k4+CfikN8UxxxfHChfZnit0Xkc3FJwuNiGxvoyXmqShZ2m\nhPzCZTSiBE9NEVoFZdgC0AjFPRo9KFtD/wEaHShbmmn0gPrzNY0e1H9/uBVoFCgNUHk2VSVSoyt/\nj7KhBVCkAV7pbVhPa42e1jf2W2Fw/jO7Vzb4ocrfi4LZTW5oozN672707EDddq1/VavB7+vrS3p6\nuuF5RkYGvr6+lba9MfiFdV25Ah9+CI8+au1KRG25XnydgtKC6s/Si4vh0iVKz5/j3neieS54An0d\n2sHFi2z5+QCFB4vpf1dnnAr1kJtLYfYFmhSWomnWDFxcwMWFa/a2pOku0jm4Nzg48Nav6yhqbMPT\ng18DB3+e2T2b5wa9gou7L1+e2cwbKSnMH/YE/cNieeCbR9ia8SO74rcS4tMe7TwHdDaARof+ZT02\ns8uDfNG97zOm3Rg+Sf6EJ797ktf7z2X/uf2sPrKay89c5I+cP7jro7t4vuczxHeO5/5V93PowiFA\nUTCjgO4fdyclK4Wlw5fyUMRDzNw+k9k/zi4fgheLaWTbiFJdKY1ebQRA75a92PHIDn5O/5meS3ri\nZu/G2SfOsvXkVoatHAZAxvQz3LfiPpIvJHPp6Uu427tjM9uGpnZNyZ+Rz+IDi0k6ncTF/ItMi56G\nr5MvnRd3BmDFyC/o5NMJPyc/mmibkJGylFfWP0L69LO4NnHllcbNavT/X6vBP3ToUBYuXEhcXBx7\n9uzBxcVFNvPcAf7yR5qohyZ9M4lVh1ehXtJBdjZkZHDtxGGaXczFJvMcZGSUPzIzKb1wDm1RCRpP\nT3B35al8yDmygvzu43HwC+RAYGNUxygu+7VlUv9n0Ds70WJxEH063EsJOgYHDeaxLo+xaM+/eWbb\nM6wcOZR72tzDZ5/tJTUrlZgHeuHaxJU3Lv+TN049xpxWc3jh0v9BC/i+USb9W7Uizb6AlkGdWXh8\nKe+0fgeltYH/bBPfm7kXp8ZOPNfjOfZl7uPs1bPM2TkH58bO7M7YTVFZ+V8i7q+708mnE61cWjF3\n11x+OPUDQW5B+Dv5883v3zBl0xRSslIAGLduHL0CehlCH2DwssH8cOoH3h70tmHZHzl/AJBblEt3\nv+78lP4Tnx76lILSAgBsNbbsTt9Nia6Eri264vmGJ11bdKWJtgktHFvw8cGPuV58He9m3oR7hrPj\nzA5CPUIBiPKO4vuT3zP2q7EA9ArohaeDJwCPfP0I35/6vsb//xr1120xt2Hs2LHs2LGD7OxsvLy8\nmDVrFqWlpQDEx8cDMHXqVBITE3FwcGDJkiV06tTJuAiNxmiTkLCeiROhZ8/yf8Wd63j2cUI8QqC4\nmNWb3uDYno285PcAnDjBgV2rcTh7gZC8xmgcHbno2oi9mnMEtuuBQ+tQUrTZrL2+jxH9/8nEvTNo\n7N6cC09n8dvF3xj55Uia2jXlkchHmNZtGkHvBPFsj2f5KPkjYoNjOXThEOuOrSPILYgTOScAuK/t\nfUR4RTBn5xwARoSNIPNaJoVlhaRmpVao26OpB9kF2YyPHM/xy8f5YvgXDFg6gM+Hf87D6x7mkahH\nmJk0k7zn8xi6cihRXlEkX0hmwcAFDF42mI4+HXFt4srEjhN54KsH8GrmxcSoiTzx7RMADA8dTkJM\nAiO/HImXgxdz7p5DU7umjP96PJcLL/Pvgf9m3k/zOJ59nGJdMR2ad+D+0Pt55cdXDDUm9ElgwZ4F\naG20LBuxjNSsVH45/wvDQoYRvzGevJI8ZsfMxr2pO3sz97L+2HrWjF7DgKUDAGjt2pq1o9cyZPkQ\nHu34KDYaG/q16seT3z1JbHAseqVneOhwOi02zsvJnSaz5ugacgpzIMF4c3p1mDTjX7FixS3bLFy4\n0JQuhBXI7+A704lzhwk4X0Cj42nk7N/JsS0f0KrYn0bnsujl2gjHZnnoBkRh2zaEHQ4hfHztAt88\n/wveXq3xeq18x+f4yCDsbDR8lLwe7OGL1BnYNLVBX3CRiesnMj5yPM0dmvP3jn9n7bG1TOs2jdyi\nXAYHD+bJ757k5JWTZBdkMzJsJF8d/QqtjZbfp/5O63da883v3+Dv5E/6tXTWHl0LwOYHNjNs5TBK\n9eUTxsX3LmbyxsnM6DmDl/u8TJM5TQh6NwjHRo5E+0bzZPcn+eeWfwLg0MiBHv49+CL1C7r6diXS\nO5KHIh7ijd1v8OWoL+kb2BdbG1tSs1K5t+29huA/eP4g7TzbcTH/IievnKSNWxt8mvlwNPsoUP5L\naVT4KMPmnNQpqaRfTa8Q/N38upH7XC4PrX2IQcsGARDfOZ572txDXkkeAPZ29vRu2ZsXfngBhaJf\nq34kxyfTcVFHLuRdINI7kibaJuw/t5/+rfpzl+9dHMs+RiuXVgxoPYAIrwhDf/kz8un+cXdSs1Ip\n1hXTtUVXvv3j2xr/rMjuO1GpKvbBi7oiPx927YIFC+DBByEsDN+W7bk4rD98/TXXKebzSPh0zv9A\nfj5j5nVi8DhYNDECpk9nU5gtjSM68sLeOaRfLd8Pd1/b+9hyYgs5RTkVutIrPWPajWHJoSV8lvIZ\n7vbujAofxQ+nfuDXrF+5UnQFLwcvon2juVxwmTHtxjCzz0wCnAMo05fRyrUV34z9Bi8HLxIfSiT7\n6WwSHyw//yfCK4Kz089yatopVv/Pau5pcw8AEzpOoLG2saGGXyb/gp2tHcNDhwPlvzAAegb05FTu\nKZo3bQ7A2Pblm0Xcm7qj0WgYEjQEgECXQGw0NsR3jueFXi9ga2NLG9c26JUeX0dfbG1sua/tfQA0\n1jbGztaORzs+Su+WvQHwdy4/OnHdmHVcevoSA4PKj9R5uc/LtG/enlkxs5gWPQ2Pph6GmvNK8gjz\nCCO3KJdAl0A0Gg1R3lE887dnmBA1AYBo32gSTyTi2NiRxtrGdGnRhdVHVuPn5IetjS2Xnr7EtnHb\naGrXlEPxhzg97TQLBi5g5aiV/DTxp5r//Kg6oI6UIf5j/HilPvnE2lWI/zp5+Q+VcXCHUh9/rNSE\nCUrfvr3S29sr1bWrUlOmqC0zRqu3Fz+qGr2I8njdQyml1OrDq1XTOU0VCagL1y8oj9c91Noja1WL\n/9dCfX30a+U2303tSd+jHF9zVGuPrFW9PumllFLqsW8eUySgnt/2vFp/bL368MCHigRUXnGeIgFF\nAur1Xa8rpZTqvaS3uvuzu9WY1WOUUkrN+XGOIgH1e/bvSimlJm2YpEio+rP94YEPVZmuzGh5Vl6W\n4evk88nqj5w/Krz+1u63VEFJgVJKqeKyYkUCaub2mUoppXR6nSIBtTdjr1JKqff2vVdlDePXjVcB\nCwIMzzOvZarQhaGG53q9vkL7vz6vavmF6xdU8vlkdb34ulJKqQPnDqhfMn+p9L0zts1QJKB2ntmp\nlFJq5vaZigRU6oXUStv/VU2zs04krgR/3fLww0otWWLtKhowvV6pY8eU+uADpcaOVemOqCxnrVJj\nx6r8f7+hBkxzU41eRB29dFT9mvWrIZDtZtspTYJGrT2yVkV9EKUmfD1BNX+juXp1x6vKZZ6L0uv1\nqveS3ob2lwsuq7Frxir3+e5q9OrRSimllqUuUySgVv66UilVHsIBCwKUXq9X/7f//9QbP72hisuK\nlVJKvfnTm4oE1JOJTyqllPo5/WdFAio7P1sppVRRaZE6mXOy1oersLTQUJNSSp3JPWMI4/ySfLUs\ndVml7yvVlapSXWmt13czlwsuq11ndhmeHzh3QJGAulJ4pVrvr2l2mrRz11xk527dMn489O0Ljzxi\n7UoajrLrV9FsT8L22+9g82YoLYW770bfuxdtUyfzhxt8NeYrRn450vAelyYu5BblVlhPbHAsm9I2\nAXDy8ZNsP72d+I3xdGnRhZ8f/Zl39r7DtMRpAOhf1pNblMukbyYxLmIcw0KHkVeSx/0r7+e9Ie+V\n7ximfOdhZeff6JUe29m2TO82nbcGvkWZvoyRX45k7ei12NpUcuy6qJbsguwKm4xupqbZWesncIk7\nj1Kyjd8iLl+GdetQX31FSdJWTgQ6UTpwAN8+3Y0xY16htVsbPjr4IVevePBAm3tYfWS14a0bx27k\n3hX3ApAcn8zTW59m28ltjAofxaa0Tfw25TdaubbC1saWMn0Zjo0cAZjceTKeTT0p05eh0WhwtXdl\nzeg1hvU2a9SMbQ9vq1BmVSdd2mhsOD71OD7NfADQ2mhZH7ferEPUEFU39E0hwS+EJeXkULRqOdeX\nL8Ez9QTccw9zWmfwRpSOa02uMDjoOltObOGFhSsNR8AA9G/Vn2mJ02jl0oom2iZ09+8OwJr/WUOU\ndxTP9niWuwPvZnjocDKuZdCueTsAApwDKHqhiBJd+aUAmmibMLbDWLN9O23d25ptXcJyJPiFEZnx\nm5cqK2PpW+N5YH8x2q3b2NqyiM9Dipm3PAUcHHjp3SBD24LSAl7s9SLbT2/np/Tyoza+fehbWrm0\n4nrJdYaGDOWLEV8Y2vs6lZ8J3791f/q37g/Ai71frNB/Y23jCkfICCGHcwpRCzKvZcLZs/DSS1z2\nciTs7eWciPCFU6cYOqKY4uH30Wf1EH69+CsDWg/g+NTjjAofxY4zO+gZ0JP1cet5vufzANzT5h6C\n3Mp/Ody4TV//sp5uft2s8v2JO5sEvzAiM34TKMUfG5fyU3c/yiI7wLVrzH4mmrsmw/tdFElXU9Da\naJnbby6Z1zNZdXgV3s28aevelsFBg4HyY9Ddm7rzcp+X+fah8pN0NBoNbvZuFJYVGrqqatu7ELci\nwS8qJZlym3Q6WLUKoqNp/r//YmdLmP7BcNS//81v3jYsGLiAd/e9S9/P+gLQrnk7Xuz1Iit/W8nl\nwssA3B96PwD+TuUnCzXRNjGc0ARwatop1o5ea+FvTNRHso1fGJEja2/DfwL/ygtP4ewdiM2LL/Ke\n06/kXjnGF6mf4bUziEsFl+gb2Nfwlv/erKNvq768uvNVIpqXn5rvZu+Gmln14Ds1dqrd70U0GBL8\nolIy478FvR5WrUKXMBMbz+aM7nWB/pOm8UzP+3h+9jDeG/Iegc6B7E7fzcX8izR3aE7+jHzm75rP\nBwc+AODuVnffNOiFqC1yApcw8sADEBtbfgkY8adj2cfKL5m7axfF06aSU5DNQ3dl8vbrqXT4oHzW\nvnT4UsZ/PZ6SF0vIKczB600vtDZail4sMtxNqqoTooS4XXIClzAb2blrrExfRuwrYRz4ox8uv6bx\n2chAHnNMQdnAe/vfJ9AlkLh2cYxbNw4AWxtbPB086dyiM7+c+8UQ+iA7ZYX1yc5dIW6lpIT8mS+w\n90N4X7+PksOpbOvuhbIpP0HqgwMfcDr3NP+46x8AvDv4XcNbd07YyYl/nrBW5UJUSmb8wojM+G+w\naxdMnoxq4caIZwLYY3ueu3OPcrnwMi/1fokJURPIL83nwLkD+Dn5sWrUKkaG/Xk9nSbaJrRxa2PF\nb0AIYxL8QlRCn5/HZ4O8eTCtKY0Wvs+P7e1wTv6I6R5xzNs1j7TLabwx4A1aubYCoH3z9gCMbjfa\nmmULUS2yqUcYafAz/oMH0XfqRJPcfPr8yw01ciTn8s7TolkLnvrbU+SX5pN+LR13e3drVypEjUjw\nC/Ffej3MnQuDBnE4fjgvTw4iy66EF394kaOXjtLCsQXNHZrzxoA3AMtcRVGI2iCbeoSRBjnjv3yZ\nC/f3x6nUhqa//ELUkpYAfDX6K/6x+R9cyLvAonsXARDlHUXxi8U0sm1kzYqFqDGZ8YsGb+fatznb\n1otlukO0HZ5BcQsvoPz+qiPCRrDn0T109O5IbHCs4T0S+uJOJjN+YaRBzfiXLqXzP2cw/h4da9oB\nRRfZcHwDLZ1bGq6d09KlJQfjD1q3TiHMSIJfVKreB79SkJAAn3/OG3NiWZNdfnerhD4JjF4jR+aI\n+s3kTT2JiYmEhoYSHBzM/PnzjV7Pzs5m0KBBREVF0b59ez799FNTuxS1rL5fPePTvYu4OGIQJCZS\nvGsHr+as44noJwB4rudzfH7/54xtb767VAlR15h0rR6dTkdISAjbtm3D19eXrl27smLFCsLCwgxt\nEhISKC4uZu7cuWRnZxMSEkJWVhZa7Z9/bMi1euqWUaMgLq7833rn2jW2Rzpz2R46JB5kQ/o23v/l\nfU4+fpJfL/5KhFeEtSsUotqscq2effv2ERQURGBgIABxcXGsX7++QvD7+PiQmpoKwLVr13B3d68Q\n+qLuqbe/g3NyYNAgzno14clhTbD7cjBZ+Vn8zf9vaDQaCX3RYJi0qSczMxN/f3/Dcz8/PzIzMyu0\nmTRpEocPH6ZFixZERkby9ttvm9KlsIB6uXM3KwtiYijt2Z3H7lW8OegtsvKzAGhsK/ejFQ2LSVPv\n6lxl8LXXXiMqKoqkpCT++OMPBgwYQEpKCo6OjhXaJSQkGL6OiYkhJibGlNKE+NP58+WhP3YMjXgF\nH3sfHox4kIkbJjK63WjeG/KetSsUolqSkpJISkoyeT0mBb+vry/p6emG5+np6fj5+VVos3v3bl54\n4QUA2rRpQ6tWrTh+/DhdunSp0O7G4BfWVZ9m/LqLWVzr2ZkTg6PJHdcLvoDzeedpZNuILQ9uIco7\nSs7AFXeMv06KZ82aVaP1mLSpp0uXLqSlpXH69GlKSkpYtWoVQ4cOrdAmNDSUbdu2AZCVlcXx48dp\n3bq1Kd0KUT25uZT078v7vue5y/Nr4r6KA2Bev3kADAoahHczb2tWKIRVmBT8Wq2WhQsXMnDgQMLD\nwxkzZgxhYWEsWrSIRYvKT2+fMWMGv/zyC5GRkfTv35/XX38dNzc3sxQvake9mPHn5cGQISxuepQX\n74Z/df8XOYU59PDvwbM9n7V2dUJYldx6URi5/34YPx6GD7d2JTVUVgb33Qc+PgRFJPFq/9eIax+H\nZlb5bzO5z62oL+TWi8Js7ugZv1KoqVPJK75G2b8/w+aznnT07ghA4oOJnLl6xsoFCmF9EvyiUndq\n8Ge+PJ3srxfRawJcX+CFg50DLk1cABgYNNDK1QlRN0jwCyN37Fa3r77CafFnRD8I15uUL8ovzce5\nibN16xKijpHgF5W602b86rff0Dz2GCteHc0A7xL+edc/yS/JZ8DSATTRNrF2eULUKRL8wsidNuM/\neyaVoj6ReM99h6MBJ2nv1JZOPp1QSpH0SJK1yxOizpEbsQgjd9TOXb2ek/f25Ls2ML/NBX44/YNh\n045Go6GbXzcrFyhE3SMzfnFHK3ttDo2u5+P48WL+uWUyAA52DlauSoi6TWb8wsgdM+P/+Wc077zN\nE+O9GB45BjsbOwBsbWytXJgQdZvM+MWd6epVePBBTs97ntLSL3Bq7MTkzpN5KOIh7vK9y9rVCVGn\nyYxfGKnrM/65P75GxoNDYdAgTsVE4mZffgmQhUMW0s2vGzYa+bEW4mbkEyLuOEf+/QK5+37k7ohk\ncgpzDMEvhKgeCX5hpE7P+M+dY8FWGx4aAduz9jBmzRiuFV+zdlVC3FEk+MWdQyl47DE+7GLDurkn\nGRVeflPguf3mWrkwIe4sEvzCSJ2d8S9fjv7UKV7tDYEugTzV/Skc7BwMF2ETQlSPBL+oVJ0L/gsX\n4MknufzeGzg6uqPRaIj2jWbv3/dW6xagQog/SfALI3Xxkg1FU6egJkzgZGtX/J39gfIzc9s1b2fl\nyoS480jwi0rVqUn0999zYdvX+Dp8QLePu+Hv5G/tioS4o0nwCyN1asZfXEzRY3/nufuacF5/FYCY\nwBjr1iTEHU7O3BVG6tTO3bfeIsOzMT4PxHOu57MUlBbQxq2NtasS4o4mwS/qrjNn4M03Wfn/huHn\n7I+Po4+1KxKiXpBNPcJInZnxP/ss+qn/4LhzKZ4OntauRoh6Q4Jf1Ekntizj2vebcSx7hS9Sv8Cj\nqYe1SxKi3pBNPcKItWf8JWXFnH/sIeb+DQoalS8LcQ+xXkFC1DMmz/gTExMJDQ0lODiY+fPnV9om\nKSmJjh070r59e2JiYkztUtRzWcsW41mixXPKv9g+fjt7/75XdugKYUYmzfh1Oh1Tp05l27Zt+Pr6\n0rVrV4YOHUpYWJihTW5uLv/4xz/49ttv8fPzIzs72+SiRe2y6oy/tBT32W+wIC6Ytwa9YaUihKjf\nTJrx79u3j6CgIAIDA7GzsyMuLo7169dXaLN8+XJGjhyJn58fAB4esq32TmC14P/oI/K93fi9Sysr\nFSBE/WdS8GdmZuLv/+dZlH5+fmRmZlZok5aWRk5ODn379qVLly4sXbrUlC6FBVjtBK6iIpgzh73/\nez9uTd2tVIQQ9Z9Jm3qqc3Gs0tJSDh48yPfff09BQQHdu3enW7duBAcHV2iXkJBg+DomJkb2BViZ\nVWb8ixdDp06caOOC6xVXKxQgRN2WlJREUlKSyesxKfh9fX1JT083PE9PTzds0vkvf39/PDw8sLe3\nx97ent69e5OSknLT4BfWZZUZf2EhRa/N5szShRzP3kGoR6gVihCibvvrpHjWrFk1Wo9Jm3q6dOlC\nWloap0+fpqSkhFWrVjF06NAKbYYNG8auXbvQ6XQUFBSwd+9ewsPDTelWWIClZ/xl77/HFtfLhO4e\ny/LfltO7ZW/LFiBEA2LSjF+r1bJw4UIGDhyITqfj0UcfJSwsjEWLFgEQHx9PaGgogwYNIiIiAhsb\nGyZNmiTBX8dZfMZfUEDx3FdY/88ourTQMrTtUDr6yM1VhKgtGqWsfy1GjUZDHShD/Efv3vDKK9Cn\nj4U6fOstNn06g4xP3ia+S7yFOhXizlfT7JRLNgjrKimBt95ibm8b4trHWbsaIRoECX5hxKIncK1Y\nwRmfpvzkWYhTYycLdSpEwybBL6xHr6d43hz+HpYGVO/wYCGE6ST4hRGLzfi3bKHYVnGpewSnp522\nQIdCCJCrcwprmj+fQ+MG0N7rGi1dWlq7GiEaDJnxCyOWmPEX70wiJy2VV5ofxc/J79ZvEEKYjcz4\nRaVqO/hz5r7M3Kir9A0aQHxnOYRTCEuS4BdGav2UisxMXHb9gvPnjzOj14xa7kwI8VeyqUdUqlZn\n/IsXs7d3G7xaBN+6rRDC7CT4hZFanfGXlMDixay724fmDs1rsSMhRFUk+IWRWt25+9VXFLRtzTvX\nt0rwC2ElEvzCotTCd3nAZzcAAc4BVq5GiIZJgl8YqbUZ/8GDnD38Mxvbwulpp2nt2roWOhFC3IoE\nv7AY9f77LOoMh6b+KidsCWFFEvzCSK3M+PPyUF+t4cu7mtK+eXszr1wIcTsk+IVF5C37lI3Nr3Le\n0dqVCCEk+IWR2pjx6z5azCcd4buHvjPvioUQt02CX1TKrMF//DiNTp2lybAR9AjoYcYVCyFqQoJf\nGDH7CVyffMKRQZ3xdG5h5hULIWpCgl9Uymwz/tJS1OefsaWXDz6OPmZaqRDCFBL8wog5Z/yFG9ay\n2y6Lly4sI9Ir0nwrFkLUmAS/qJS5Zvx5H/8fn0bBU92f4u5Wd5tnpUIIk0jwCyNmm/FfvYrzzn3o\nRwznzXvexN7O3kwrFkKYwuTgT0xMJDQ0lODgYObPn19lu/3796PValm7dq2pXYpaZq7DOXcumM72\nVhpcfFrqXvO4AAAbW0lEQVSZvjIhhNmYFPw6nY6pU6eSmJjIkSNHWLFiBUePHq203bPPPsugQYNQ\ntX6XD1FX2K78ko9CC2jXvJ21SxFC3MCk4N+3bx9BQUEEBgZiZ2dHXFwc69evN2r37rvvMmrUKDw9\nPU3pTliIWWb858/TIb0Ir9ETiWsfZ5a6hBDmYVLwZ2Zm4u/vb3ju5+dHZmamUZv169czZcoUADS1\nfTNXUTd8+SWJ7ZrwVL8XaWrX1NrVCCFuYNI9d6sT4k888QTz5s1Do9GglKpyU09CQoLh65iYGGJi\nYkwpTZjAHDN+tXw5n4WXMERutiKE2SQlJZGUlGTyekwKfl9fX9LT0w3P09PT8fPzq9DmwIEDxMWV\n/6mfnZ3Nli1bsLOzY+jQoRXa3Rj84g534gTqzGl+vFeLQyMHa1cjRL3x10nxrFmzarQek4K/S5cu\npKWlcfr0aVq0aMGqVatYsWJFhTYnT540fD1hwgTuu+8+o9AXdYvJM/4VK7gwuDfFmg1mq0kIYT4m\nBb9Wq2XhwoUMHDgQnU7Ho48+SlhYGIsWLQIgPj7eLEUKyzMp+NesYXpMNt38upmtHiGE+WhUHTi+\n8r/b/0Xd0KEDLFsGERE1ePOJE+h69sDriVKynrmErY2t2esTQpSraXaaNOMX9VeNZ/xffUVK91YM\nDgmW0BeijpLgF0ZM+eOrcNUy5nQ5y2u9PjNfQUIIs5Jr9QgjNd25ezIlibzjv+I+eAQhHiHmL0wI\nYRYy4xdmc2zxa2jv8ueJnv+ydilCiJuQGb8wUtMZf9D2FNTw4YR7hpu/KCGE2UjwC/O4cAGfs1co\njulp7UqEELcgwS+M1GjG/8037GnnjKuzd63UJIQwHwl+YR4bN/JtmB3uTd2tXYkQ4hYk+IWR257x\nFxai++F7vmldSkvnlrVWlxDCPOSoHmG67dvJDvGjY3iUXJStnnNzc+PKlSvWLqPBcXV1JScnx2zr\nk+AXRm53xq/fsIE33Y4T4j6m9ooSdcKVK1fk8ipWYO77mMimHlGpav+cKUXphnUc7OzDcz2fq9Wa\nhBDmIcEvjNzWhC4lhWIttOh6N/Z29rVWkxDCfCT4RaWqO+NXGzbwSYuL+Di2qN2ChBBmI9v4hZHb\nmfEXfr2GbzrA8u5P1V5BQgizkhm/MFLtnbvZ2dge/50OI6bg1cyr1usSQpiHzPhFjeVvXs+ulnoG\ntxtm7VKEELdBZvzCSHVn/LkbVvFtG+jbqm/tFyWEmZw6darS5efPn6egoMDC1ViHBL+oGaVw3fkL\n+nsG0Mi2kbWrEaJaTp48yZ49eyp9zdPTk9dff93CFVmHBL8wUq0Z/+HDlNlqKGkVYJGahDCHRYsW\nMXbs2Epf02q1xMbG8vnnn1u4KsuT4Bc1cm3Dapa3yMHV3s3apQhRLSkpKfj5+d20TdeuXdm2bZuF\nKrIeCX5hpDoz/ryNa/k2CFztXS1TlBAm2rhxI3fffbfR8n79+lFWVmZ47unpyYkTJyxZmsVJ8ItK\n3TT4CwtxSz5O1wef4cnuT1qsJiFMsX//fsLDK94dLjMzE6UUWu2fBzhGRkZy4MABS5dnUSYHf2Ji\nIqGhoQQHBzN//nyj15ctW0ZkZCQRERH06NGD1NRUU7sUtexWJ3DpdiRxoHkZY3pMxkYjcwdRkUZj\nnkdNbdiwgU2bNvHcc8+xbNkyxo0bx7FjxygoKKhwsbOtW7cyffp0vL29Wbp0qWG5q6srGRkZpgxB\nnWfSp1an0zF16lQSExM5cuQIK1as4OjRoxXatG7dmh9//JHU1FReeuklJk+ebFLBwjJu9sHL/WY1\n+9u50MatjeUKEncMpczzqImzZ88SHh5ObGwsW7duJTY2lri4OAICAtDpdBXaDhgwAK1Wy1NPPcW4\nceMMy+3t7SkpKTFlCOo8k07g2rdvH0FBQQQGBgIQFxfH+vXrCQsLM7Tp3r274evo6Oh6/5u0PrjV\nh06/bStXx3exTDFC3IaAgPKjzLKysnB0dMTFxYXY2FiACptzAJRSJCcn07lz5wrLr169iptb/T5o\nwaQZf2ZmJv7+/obnfn5+ZGZmVtn+448/ZsiQIaZ0KSzgpjt3c3JodvYCrQaMtmhNQlTHsWPHSElJ\nYfPmzfTu3Rso36kL4O3tTV5enqHtkSNHDJPUlStXGpafP3+eoKAgC1ZteSbN+G/n5gDbt2/nk08+\n4aeffqr09YSEBMPXMTExxMTEmFKaqC07dpDcuimhLSKsXYkQRr777juuX7+Oj48PRUVFrFu3Dl9f\nXwD69OnDvn37DEf2uLu74+zszIoVKyrkzaFDh/j73/9ujfJvKSkpiaSkJJPXY1Lw+/r6kp6ebnie\nnp5e6XGyqampTJo0icTERFxdKz/878bgF9Z1sxl/2fdbSfQv4mmPsMobCGFFjz/+eJWvjRgxgjff\nfNMQ/N7e3ixZsqRCm6KiIpycnGjSpEmt1llTf50Uz5o1q0brMWlTT5cuXUhLS+P06dOUlJSwatUq\nhg4dWqHN2bNnGTFiBF988UW9//OpISjcuoVzXUJwbOxo7VKEuC0uLi54eHiQnZ1dZZuVK1cSHx9v\nwaqsw6Tg12q1LFy4kIEDBxIeHs6YMWMICwtj0aJFLFq0CIDZs2dz5coVpkyZQseOHbnrrrvMUrio\nPVXO+C9eRJt5joC+91u8JiHMYdq0aaxbt67S19LT03F1dSUkJMTCVVmeRtWBOydrNBq5gXMd4ucH\nP/8MN+y3L/fll2xNeJiWO1Np697WKrUJ65LPqnVUNe41/f+Qs2+Ekapm/PofvufbgBLauMrx+0Lc\nyST4RbXpf/iBAyFO2NrYWrsUIYQJJPiFkUpn/OfOwaWLZAV5W6UmIYT5yK0XRaWMgv/HH8nuHI6X\nU908zE0IUX0y4xdGKttXpHbuJMm/jKFthxq/KIS4o8iMX1Tqxhl/qa6UUxs+ZcGgAjZ02Gi9ooQQ\nZiEzfmHkrzP+pJT1+F0qYttb2Xg187JOUUIIs5HgF0b+unO36McfSG/rjaOju/WKEkKYjQS/uCX7\nvQe53FmuzSNEfSHBL4z8dcbvk/IHpd2jrVeQEGZ06tSpSpefP3+egoICC1djHRL84uaKi2l1Modm\nvftbuxIhqq2wsJCioiKj5SdPnmTPnj2VvsfT05PXX3+9tkurEyT4hZEKM/6DB/ndQ0OAf3ur1iRE\ndel0OhISEpg5cyZ6vb7Ca4sWLWLs2LGVvk+r1RIbG8vnn39uiTKtSoJf3FRR0jZ2B2jwaOph7VKE\nqJakpCSeeuoppk+fzvbt2w3LU1JSKr1fyI26du3Ktm3bartEq5PgF0ZunPGXJH3PH+18butua0JY\nU79+/WjevDne3t7069fPsHzjxo2Gm7D8tX1ZWZnhuaenJydOnLBIrdYiwS8qpdEAStF43wFyO7Wz\ndjlCmGz//v2Eh4dXWJaZmYlSqsKN2CMjIzlw4ICly7MoOXNXGDGcwHXiBPmNbQhq39uq9Yg7i2aW\nef46VDNrdt3/DRs2YGtry86dO+nQoQOJiYm88MILFBQUVPjLdevWrXz44Yd4e3uzdOlSxo0bB4Cr\nqyu///67Wb6HukqCX1RKowH27OF4G2daurS0djniDlLTwDaHs2fPEh4eTlBQEC+//DLPPfccLi4u\nBAQEoNPpKrQdMGAAS5Ys4amnnqJz586G5fb29pSUlFi6dIuS4BdGDDP+vXtJCWxKK3s5Y1fcGQIC\nAgDIysrC0dERFxcXYmNjASpszgFQSpGcnFwh9AGuXr2Km5ubZQq2EtnGLyr13xn/fn8Nbvb1+0Mg\n6o9jx46RkpLC5s2b6d27fBPlxo3lFxb09vYmLy/P0PbIkSOEhZWfkb5y5UrD8vPnzxMUFGTBqi1P\ngl8YUQooLISjR9ntXijBL+4Y3333HRs3bkQpRVFREevWraN58+YA9OnTh3379hnauru74+zszIoV\nK+jTp49h+aFDh+jRo4fFa7ck2dQjjCgF2tSD6MJCOVV8GE8HT2uXJES1PP7441W+NmLECN58803D\nIZ3e3t4sWbKkQpuioiKcnJxo0qR+33BIZvyiUra//Mwapwzu8r0Lp8ZO1i5HCJO5uLjg4eFBdnZ2\nlW1WrlxJfHy8BauyDgl+YUQpSP9+GcfbOLN9/PZbv0GIO8S0adNYt25dpa+lp6fj6upKSEiIhauy\nPJODPzExkdDQUIKDg5k/f36lbR5//HGCg4OJjIwkOTnZ1C5FLVM2JTgcSmHc5PewtbG1djlCmI1G\no2HSpEmVvubv78+wYcMsXJF1mBT8Op2OqVOnkpiYyJEjR1ixYgVHjx6t0Gbz5s2cOHGCtLQ0Fi9e\nzJQpU0wquL7T6XW8v/99Dpyr/pmDSim2pG3hevH1m7YrKC0gpzCHUl3pTdt5um+hWZkNrbrIFTmF\nqI9M2rm7b98+goKCCAwMBCAuLo7169cbDpGC8rPoxo8fD0B0dDS5ublkZWXh5VX9W/gdyz7GJ8mf\n8Hj04/g5GV9kSSlFWk4ay1KX0dSuKRfzLzK582RCPCr+yXal8Aqfp5Rfec/O1g6nxk6MbjeaRraN\nOHDuALHLYxkXMY6M6xl08u7EtG7T0Ol1rPxtJZ4OnkR4RRDgHEBBaQGDlw2muUNz/J38aeveFo+m\nHoS4h7D++HqGhgzFp5kPP2f8jE8zHz5O/pjNaZsJcgti6fClvLP3HVyauPBw5MN8nvI5rVxbEeZR\nPmYf/PIBG9M2cr34OlvHbaWrb1eOXjrK5rTNfH/qe45fPo53M2/iO8cT4RVB0ukklqYu5eD5g7jb\nuzMsZBijwkfh3tSdTj6dSLmQwqELh8gvzefl7S+jUNhr7RkWMoxAl0Aa2Taiq29XercsP/Rt55md\nRPnPItMlAFe5Po8Q9ZJGqb/eYbX61qxZw7fffsuHH34IwBdffMHevXt59913DW3uu+8+nn/+ef72\nt78B0L9/f+bPn1/hpAmNRsOKqIrXg2mq96bQ5hI52t8APQ46f8o0+fiW9sNe78FV25MU2JzjivYo\nTXXe5Nlm4KJri61qjF5TylXbNBx0/jTVe2OnmpFne5YizWW0yp7GyhWFjiKbHMo0Bdjrm3PV9gQu\nuhD0lGKv9+Kq7e+U2lzHTu8IlAdgsU027mVRgIY8m3RcdCH/Wc9l9JSQb5tBE70HpZo8SjX52NII\nG6XFXu9F89K7yNEeJkf7G066VgBctz1FU30LNMqGUpt8ijVXcNWF4loWTp5NBle1aTiVteaS3QHs\nlAON9C4U2WbjW3w32XbJ6CnFhkZ4lnakqd6bUk0+ebbpXLL7BR0l2Os90WkKaaz3wE41pam+BS66\nYApssrhucxalKUOh44r2KIr/ntWooeM5DS3+ZzIB/36/pj8aop7SaDSYEBmihqoa95r+f5g046/u\nFRv/Wlhl71t+wz0TQjw8sHM5ilbTiCC7h2luG4y9xpmkwnf5Rbea5rbBXFcXaWHbAWeb4eg1TWhj\nG04Tmz+PPtGVHeRs6S9klu3DzTYAP20UzWw88LWNQKOx/U9denL0Z8jW/YEt7fC1G2C4LKUHcFWX\nSbHKw802EK2mMZd1pzhbdoBcXTqhjUbjrq364mWl+kIA7GzsDct86I3Pf8dCo0GnSrDBzjAepfpC\nQ/vGqoyLJd+SSQG+ttNorm0LSqFHh41GiyP3GvXZ+D8PJ/0I8tQl8vSXCLBth9bmz0PTFGBPe+xv\neJ+XUoAehUKDDYRp8H9mRJXfmxDCOpKSkkhKSjJ5PSYFv6+vL+np6Ybn6enpRte7/mubjIwMfH19\njda14ehvt+yvu24SZ3LP8OvFX4nyjqK1a2sTqq/7+lL1MclCiIYnJiaGmJgYw/NZs2bVaD0m7dzt\n0qULaWlpnD59mpKSElatWsXQoUMrtBk6dKjhjjZ79uzBxcXltrbv36iRbSOC3YMZETai3oe+EELU\nFpNm/FqtloULFzJw4EB0Oh2PPvooYWFhLFq0CID4+HiGDBnC5s2bCQoKwsHBwehMOSGEEJZl0s5d\nsxUhO4yEuCPIZ9U6zL1zV87cFUKIBkaCXwjRoJw6darS5efPn6egoKDe9VsZCX4hRINx8uRJ9uzZ\nU+lrnp6evP766/Wq36pI8AshGoxFixYxduzYSl/TarXExsYajkKsD/1WRYJfCNEgpKSkGJ1n9Fdd\nu3Zl27Ztt73u48eP07NnTz777DOL9ltTEvxCiAZh48aNhpuw3Khfv36UlZUZnnt6enLixInbWndI\nSAharbbCyVU36/evfda035qS4BdCNAj79+8nPDy8wrLMzEyUUhVuxB4ZGcmBA5VfHTc7O5tr164Z\nLS8oKODcuXO0bNnylv1W1uet+jU3CX4hhHlpNOZ51NCGDRvYtGkTzz33HMuWLWPcuHEcO3aMgoKC\nCtcJ27p1K9OnT8fb25ulS5calru6upKRkVHpurdv315pOO/atQs3NzcSExN5++23WbhwoeG1G/ut\nqs9b9WtuEvxCCPNSyjyPGjh79izh4eHExsaydetWYmNjiYuLIyAgAJ1OV6HtgAED0Gq1PPXUU4wb\nN86w3N7enpKSkgptL1y4wJgxYygoKODy5cuMGjWKvLw8w+s//PADI0eOZNCgQURHR7Njxw7Dazf2\nW1WfVfVbW+Rm60KIeiMgIACArKwsHB0dcXFxITY2FsBo04pSiuTk5AqXiAe4evUqbm5uFZZ5e3uz\naNEiRo0ahYuLC0uXLsXe/s9r3CYlJfHNN98AsG3btgrb9G/st6o+q+q3tsiMXwhRbxw7doyUlBQ2\nb95M797lNxfauHEjUB7eN87Sjxw5Yrhp1MqVKw3Lz58/T1BQUIX1nj9/nv/93/9lwoQJPPTQQzz8\n8MNcv15+x7urV69SWlqKp6cnAKtXr+aBBx5g06ZNRv1W1WdV/dYWCX4hRL3x3XffsXHjRpRSFBUV\nsW7dOpo3bw5Anz592Ldvn6Gtu7s7zs7OrFixgj59+hiWHzp0iB49elRYr4+PD8uXL8fe3h43NzdW\nr16No6MjAMnJyRWuSty2bVu++eYbOnXqZNRvVX1W1W9tkYu0CSGq7U7+rObm5vLmm2/y6quvVtmm\nqKiIGTNm8NZbb9WpfuUibUIIUQMuLi54eHiQnZ1dZZuVK1cSHx9fL/q9GQl+IUSDMW3aNNatW1fp\na+np6bi6uhISElJv+q2KbOoRQlSbfFatQzb1CCGEMIkEvxBCNDAS/EII0cBI8AshRAMjwS+EEA2M\nXKtHCFFtrq6uFa5wKSzD1dXVrOur8Yw/JyeHAQMG0LZtW+655x5yc3ON2qSnp9O3b1/atWtH+/bt\neeedd0wqVghhXTk5OSil5GHhR05Ojln/H2sc/PPmzWPAgAH8/vvv9OvXj3nz5hm1sbOzY8GCBRw+\nfJg9e/bw3nvvcfToUZMKru+SkpKsXUKdIWPxJxmLP8lYmK7Gwb9hwwbGjx8PwPjx4/n666+N2nh7\nexMVFQVAs2bNCAsL49y5czXtskGQH+o/yVj8ScbiTzIWpqtx8GdlZeHl5QWAl5cXWVlZN21/+vRp\nkpOTiY6OrmmXQgghzOCmO3cHDBjAhQsXjJbPmTOnwnONRnPTHT55eXmMGjWKt99+m2bNmtWwVCGE\nEGahaigkJESdP39eKaXUuXPnVEhISKXtSkpK1D333KMWLFhQ5bratGmjAHnIQx7ykMdtPNq0aVOj\n/K7xRdqeeeYZ3N3defbZZ5k3bx65ublGO3iVUowfPx53d3cWLFhQk26EEEKYWY2DPycnh9GjR3P2\n7FkCAwP58ssvcXFx4dy5c0yaNIlNmzaxa9cuevfuTUREhGFT0Ny5cxk0aJBZvwkhhBDVVycuyyyE\nEMJyLHrJhsTEREJDQwkODmb+/PmVtnn88ccJDg4mMjKS5ORkS5ZnUbcai2XLlhEZGUlERAQ9evQg\nNTXVClVaRnV+LgD279+PVqtl7dq1FqzOsqozFklJSXTs2JH27dsTExNj2QIt6FZjkZ2dzaBBg4iK\niqJ9+/Z8+umnli/SAiZOnIiXlxcdOnSoss1t52aN9gzUQFlZmWrTpo06deqUKikpUZGRkerIkSMV\n2mzatEkNHjxYKaXUnj17VHR0tKXKs6jqjMXu3btVbm6uUkqpLVu2NOix+G+7vn37qtjYWLVmzRor\nVFr7qjMWV65cUeHh4So9PV0ppdSlS5esUWqtq85YzJw5Uz333HNKqfJxcHNzU6WlpdYot1b9+OOP\n6uDBg6p9+/aVvl6T3LTYjH/fvn0EBQURGBiInZ0dcXFxrF+/vkKbG08Ki46OJjc395bnB9yJqjMW\n3bt3x9nZGSgfi4yMDGuUWuuqMxYA7777LqNGjcLT09MKVVpGdcZi+fLljBw5Ej8/PwA8PDysUWqt\nq85Y+Pj4cO3aNQCuXbuGu7s7Wm39u/xYr169bnqtnprkpsWCPzMzE39/f8NzPz8/MjMzb9mmPgZe\ndcbiRh9//DFDhgyxRGkWV92fi/Xr1zNlyhSAenuRsOqMRVpaGjk5OfTt25cuXbqwdOlSS5dpEdUZ\ni0mTJnH48GFatGhBZGQkb7/9tqXLrBNqkpsW+/VY3Q+r+su+5vr4Ib+d72n79u188skn/PTTT7VY\nkfVUZyyeeOIJ5s2bZ7i/6F9/RuqL6oxFaWkpBw8e5Pvvv6egoIDu3bvTrVs3goODLVCh5VRnLF57\n7TWioqJISkrijz/+YMCAAaSkpODo6GiBCuuW281NiwW/r68v6enphufp6emGP1erapORkYGvr6+l\nSrSY6owFQGpqKpMmTSIxMdHsl2WtK6ozFgcOHCAuLg4o36G3ZcsW7OzsGDp0qEVrrW3VGQt/f388\nPDywt7fH3t6e3r17k5KSUu+CvzpjsXv3bl544QUA2rRpQ6tWrTh+/DhdunSxaK3WVqPcNNseiFso\nLS1VrVu3VqdOnVLFxcW33Ln7888/19sdmtUZizNnzqg2bdqon3/+2UpVWkZ1xuJGjzzyiPrqq68s\nWKHlVGcsjh49qvr166fKyspUfn6+at++vTp8+LCVKq491RmL6dOnq4SEBKWUUhcuXFC+vr7q8uXL\n1ii31p06dapaO3erm5sWm/FrtVoWLlzIwIED0el0PProo4SFhbFo0SIA4uPjGTJkCJs3byYoKAgH\nBweWLFliqfIsqjpjMXv2bK5cuWLYrm1nZ8e+ffusWXatqM5YNBTVGYvQ0FAGDRpEREQENjY2TJo0\nifDwcCtXbn7VGYsZM2YwYcIEIiMj0ev1vP7667i5uVm5cvMbO3YsO3bsIDs7G39/f2bNmkVpaSlQ\n89yUE7iEEKKBkXvuCiFEAyPBL4QQDYwEvxBCNDAS/EII0cBI8AshRAMjwS+EEA2MBL8QQjQwEvxC\nCNHA/H+CBBYXlivWyAAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x7e576a0>"
       ]
      }
     ],
     "prompt_number": 28
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Recurrent connections\n",
      "\n",
      "- So a connection actually approximates $f(x(t))*h(t)$\n",
      "- So what does a recurrent connection do?\n",
      "\n",
      "$x(t) = f(x(t))*h(t)$\n",
      "\n",
      "- where $$\n",
      "h(t) = \\begin{cases}\n",
      "    e^{-t/\\tau} &\\mbox{if } t > 0 \\\\ \n",
      "    0 &\\mbox{otherwise} \n",
      "    \\end{cases}\n",
      "$$\n",
      "\n",
      "- How can we work with this?\n",
      "\n",
      "- General rule of thumb: convolutions are annoying, so let's get rid of them\n",
      "- We could do a Fourier transform\n",
      "   - $X(\\omega)=F(\\omega)H(\\omega)$\n",
      "   - but $h(t)$ is this really nice exponential and is zero for values of t<0, so there's another transform that might be a bit easier\n",
      "- Laplace transform\n",
      "   - $X(s)=F(s)H(s)$\n",
      "   - $H(s)={1 \\over {1+s\\tau}}$\n",
      "- rearranging:\n",
      "\n",
      "$X=F {1 \\over {1+s\\tau}}$\n",
      "\n",
      "$X(1+s\\tau) = F$\n",
      "\n",
      "$X + Xs\\tau = F$\n",
      "\n",
      "$sX = {1 \\over \\tau} (F-X)$\n",
      "\n",
      "- convert back into the time domain:\n",
      "\n",
      "${dx \\over dt} = {1 \\over \\tau} (f(x)-x)$\n",
      "\n",
      "\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Dynamics\n",
      "\n",
      "- This says that if we do a recurrent connection, we actually end up implementing a differential equation\n",
      "\n",
      "- So what happened with $f(x)=x+1$?\n",
      "    - ${dx \\over dt} = {1 \\over \\tau} (x+1-x)$\n",
      "    - ${dx \\over dt} = {1 \\over \\tau}$\n",
      "- what about $f(x)=-x$?\n",
      "    - ${dx \\over dt} = {1 \\over \\tau} (-x-x)$\n",
      "    - ${dx \\over dt} = {-2x \\over \\tau}$\n",
      "- and $f(x)=x^2$?    \n",
      "    - ${dx \\over dt} = {1 \\over \\tau} (x^2-x)$\n",
      "\n",
      "- What if there's some differential equation we really want to implement?\n",
      "    - we want ${dx \\over dt} = f(x)$\n",
      "    - so we do a recurrent connection of $f'(x)=\\tau f(x)+x$\n",
      "    - the resulting model will end up implementing ${dx \\over dt} = {1 \\over \\tau} (\\tau f(x)+x-x)=f(x)$\n",
      "        "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Inputs\n",
      "\n",
      "- What happens if there's an input as well?\n",
      "    - We'll call the input $u$ from another population, and it is also computing some function $g(u)$\n",
      "    - $x(t) = f(x(t))*h(t)+g(u(t))*h(t)$\n",
      "- Follow the same derivation steps\n",
      "\n",
      "${dx \\over dt} = {1 \\over \\tau} (f(x)-x + g(u))$\n",
      "\n",
      "- So if you have some input that you want added to ${dx \\over dt}$, you need to scale it by $\\tau$\n",
      "\n",
      "- This lets us do any differential equation of the form ${dx \\over dt}=f(x)+g(u)$\n",
      "    "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Applications\n",
      "\n",
      "### Eye control\n",
      "\n",
      "- Part of the brainstem called the nuclei prepositus hypoglossi\n",
      "- Input is eye velocity $v$\n",
      "- Output is eye position $p$\n",
      "- ${dp \\over dt}=v$\n",
      "    - This is an integrator ($p$ is the integral of $v$)\n",
      "\n",
      "- So, to get it in the standard form ${dx \\over dt}=f(x)+g(u)$ we have:\n",
      "    - $f(p)=0$\n",
      "    - $g(v)=v$\n",
      "- So that means we need a recurrent function of $f'(p)=\\tau(0)+p = p$ and $g'(v)=\\tau v$\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nef\n",
      "\n",
      "tau=0.1\n",
      "\n",
      "net = nef.Network('Neural Integrator')\n",
      "net.make('velocity', neurons=100, dimensions=1)\n",
      "net.make('position', neurons=100, dimensions=1)\n",
      "def recurrent(x):\n",
      "    return x[0]\n",
      "net.connect('position', 'position', func=recurrent, pstc=tau)\n",
      "net.connect('velocity', 'position', transform=[[tau]])\n",
      "\n",
      "net.make_input('input', [0])\n",
      "net.connect('input', 'velocity')\n",
      "\n",
      "net.add_to_nengo()\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<img src=\"files/lecture5/integrator.png\">\n",
      "\n",
      "- But, in order to be a perfect integrator, we'd need exactly $f'(p)=p$\n",
      "    - We won't get exactly that\n",
      "    - Neural implementations are always approximations\n",
      "- Two forms of error:\n",
      "    - $E_{distortion}$, the decoding error\n",
      "    - $E_{noise}$, the random noise error\n",
      "- What will they do?\n",
      "\n",
      "<img src=\"files/lecture5/integrator_error.png\">\n",
      "\n",
      "- What affects this?\n",
      "    \n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import syde556\n",
      "reload(syde556)\n",
      "\n",
      "x = numpy.linspace(-1, 1, 100)\n",
      "\n",
      "A = syde556.Ensemble(neurons=200, dimensions=1, seed=1, tau_rc=0.02)\n",
      "decoder = A.compute_decoder(x, x, noise=0.1)\n",
      "activity, xhat = A.simulate_rate(x, decoder)\n",
      "\n",
      "import pylab\n",
      "pylab.axhline(0, color='k')\n",
      "pylab.plot(x, x-xhat, label='$x-\\hat{x}$')\n",
      "pylab.legend(loc='best')\n",
      "pylab.xlabel('$x$')\n",
      "pylab.show()\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEPCAYAAACzwehFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt4FOXdPvB7CUFRKSQgCyTRkJMhgDGI4KFgLAYMlLy0\nVsFTU6stxVLra6uk+rNiWzX05FuNKNq+Fl9rpIocqmkKBSOI0KhEBRIkgQSSQKIQosgpEJ7fH18n\n2Wx2d2ZnZ8/357pyQXZnZh+W3bnnOY5NKaVARETkQZ9gF4CIiEIfw4KIiHQxLIiISBfDgoiIdDEs\niIhIF8OCiIh0+RwW5eXlyMzMRHp6OhYtWuRym7vvvhvp6enIzs5GVVVV1+Pf//73YbfbMXbs2B7b\nt7W1IS8vDxkZGZg6dSra29t9LSYREfnAp7Do7OzE/PnzUV5ejurqapSWlqKmpqbHNmVlZairq0Nt\nbS2ee+45zJs3r+u522+/HeXl5b2OW1xcjLy8POzatQtTpkxBcXGxL8UkIiIf+RQWlZWVSEtLQ3Jy\nMmJjYzFnzhysWrWqxzarV69GYWEhAGDixIlob29HS0sLAGDSpEmIi4vrdVzHfQoLC7Fy5UpfiklE\nRD7yKSyam5uRlJTU9XtiYiKam5u93sZZa2sr7HY7AMBut6O1tdWXYhIRkY98CgubzWZoO+cVRYzu\np23rzfZERGS9vr7snJCQgMbGxq7fGxsbkZiY6HGbpqYmJCQkeDyu3W5HS0sLhg0bhgMHDmDo0KG9\ntklLS8Pu3bt9KT4RUdRJTU1FXV2d1/v5VLMYP348amtr0dDQgI6ODixbtgwFBQU9tikoKMCLL74I\nANiyZQsGDRrU1cTkTkFBAZYuXQoAWLp0KWbNmtVrm927d0MpxR+Lfh5++OGglyGSfvh+8v0M1R+z\nF9k+hUXfvn1RUlKCadOmISsrC7Nnz8aoUaOwZMkSLFmyBAAwffp0pKSkIC0tDXPnzsXixYu79r/p\npptw5ZVXYteuXUhKSsILL7wAACgqKsLatWuRkZGB9evXo6ioyJdiEhGRj3xqhgKA/Px85Ofn93hs\n7ty5PX4vKSlxuW9paanLx+Pj4/Hvf//b16IREZFFOIObAAC5ubnBLkJE4ftpLb6fwWdTSoXlzY9s\nNhvCtOhEREFj9tzpczMUEVGoiI+Px+HDh4NdjJAQFxeHtrY2y47HmgURRQyeF7q5ey/MvkfssyAi\nIl0MCyIi0sWwICIiXQwLIqIwcvz4cZw4cSLgr8vRUEREYaKzsxMLFy4EADz++OPo0ydw1/scDUVE\nESPSzwvr1q3D2LFjcebMGezYsQNTpkxxu63Vo6EYFkQUMXhe6GZ1WLAZiogoQFavXo2YmBhs3LgR\nY8eORXl5OR588EFkZmYGZH9fsGZBRBFD77xg1X3UzJx69u3bh46ODqSlpeHSSy/FunXrsGnTJlxz\nzTUoKSnB8ePHXe5XWFiI5ORkl/u/8847+MY3voFzzjmn135shvoKw4KInIXDeaG1tRWzZ89GRUWF\nX/fnDG4iojC0c+dOfPTRRygrK8PkyZMBAG+88UbA9vcV+yyIiAJgzZo1OHLkCIYPH44TJ05gxYoV\nureYtnJ/X7EZiogiBs8L3dgMRUREAcewICIiXQwLIiLSxbAgIiJdDAsiItLFsCAiIl0MCyIi0sVJ\neUQUMeLi4mCzagGoMBcXF2fp8TgpjyLOq68CU6YA8fHBLglR6OGkPKKv3H8/sHFjsEtBFFkYFhRR\njhwBGhrkh4isw7CgiLJ9u/zJsCCyFsOCIsq2bcCgQQwLIqsxLCLA8uXAr38d7FKEhu3bgfx8hgWR\n1RgWEaCiAvj442CXIjRs2wbMnMmwILIa51lEgG3bgD6MfSgl70VuLnD6NNDeLk1SROQ7nmLCnFLS\n9NLWFuySBF9Li/w5bBiQnAzs3RvU4hBFFIZFmGtpAQ4fZlgAUqsYOxaw2SQs2BRFZB2GRZjbtg0Y\nN45hAXSHBcCwILKaz2FRXl6OzMxMpKenY9GiRS63ufvuu5Geno7s7GxUVVXp7rtw4UIkJiYiJycH\nOTk5KC8v97WYEWvbNuDyy4GODuDkyWCXJrgYFkT+41NYdHZ2Yv78+SgvL0d1dTVKS0tRU1PTY5uy\nsjLU1dWhtrYWzz33HObNm6e7r81mw7333ouqqipUVVXhuuuu86WYEW37djlBxsezdsGwIPIfn8Ki\nsrISaWlpSE5ORmxsLObMmYNVq1b12Gb16tUoLCwEAEycOBHt7e1oaWnR3ZeLBBqjnSCjPSw6O4Ga\nGmD0aPmdYUFkLZ/Corm5GUlJSV2/JyYmorm52dA2+/fv97jvU089hezsbNxxxx1ob2/3pZgRSztB\njhkDDB4c3WFRVyejoAYMkN8ZFkTW8mmehdF1472tJcybNw+//OUvAQAPPfQQfvazn+Evf/lLr+0W\nLlzY9ffc3Fzk5uZ69TrhbvduYOhQOUFGe83CsQkKkPeDcy2IgIqKClRUVPh8HJ/CIiEhAY2NjV2/\nNzY2IjEx0eM2TU1NSExMxKlTp9zuO3To0K7H77zzTsycOdPl6zuGRTRyPEHGxwOHDgW3PMG0bZvU\nsDTa8Nm9e82FxdKlwI03Av37W1ZEoqBwvpB+5JFHTB3Hp2ao8ePHo7a2Fg0NDejo6MCyZctQUFDQ\nY5uCggK8+OKLAIAtW7Zg0KBBsNvtHvc9cOBA1/4rVqzAWMdLRuqyfXv3CZI1i541C8B8U1RbG3DH\nHXJMIhI+1Sz69u2LkpISTJs2DZ2dnbjjjjswatQoLFmyBAAwd+5cTJ8+HWVlZUhLS8O5556LF154\nweO+ALBgwQJ8+OGHsNlsGDlyZNfxqKdt24Drr5e/R3tYbN/eezFFs2FRVib9QY2NwIQJVpSOKPzx\ntqph7KKLZMXZMWOAZ58FPvxQ/ow2x45JB/8XXwCxsd2P/+EPQFMT8MQT3h3vxhuBdeuAhx4C7rnH\n2rISBRtvqxpljh8H9u2TwACiu8+iqgrIyuoZFIC5msXJk8CaNcDtt0vNgogEwyJM1dQA6endJ8ho\nboZ6+21g8uTej5sJi7fekrkal14qtRIKD++9B3z2WbBLEdkYFmHKefRPtIfF1Vf3ftzMyrOrVgH/\n9V9AUlJk1CxaWoC8vGCXwr/27QOuvRb4ahwNefDll+b3ZViEKVfzCqIxLE6dAjZvBiZN6v1cfLw8\n//nnxo6lFLB6tYRFYmJkhMXOncDGjfJvi0RnzkiT4YUXAjt2BLs0oe/dd83vy7AIU9qaUJponcG9\ndavUIAYP7v2c41wLIz74ADjvPOkHSkgAWltlYl8427tX+mEOHw52SfyjpET670pKGBZG7Nplfl+G\nRZj65BMgM7P79/POA06ckNVno0lFhesmKI03/RZaExQgfUFDhnTfUClcaUG5f39wy+EPO3cCv/qV\nTKC8+GKgujpya1BWYVhEmZMngQMH5ESosdmisynKXX+FxmxYAJHRb6GFhcM814igFFBYKGGRni6z\n9L/2Nem/IPc++cT8vgyLMFRfLyeyvk5TKqMtLE6fBjZtcj0SSpOcDOzZo3+s+nqpRVx+efdjkdBv\nsXevrB8WaTWLpia5CPjqjgcAZBTb9u1BK1JYYM0iytTVAWlpvR+PtrD48EPpW3BYSqyXvDygtBQ4\neNDzsZYvl1pFTEz3Y5FSs7jiisgLi5oamVvjuJbp6NHst/DkxAnfapgMizBUVydVb2eDB0fXxLy3\n3wb0Fhq++GLg5puB++/3vN1rrwE33NDzsXAPizNnpPyXXx55zVA1NcBXqwN1YVh4tnt3z6ZrbzEs\nwlBtLWsWgH5/heZXvwLWrpUhpK7s2ycBfM01PR9PSgrviXmtrdKOn5oamTULhoV3PvkEyMgwvz/D\nIgyxGUoW+tu40VhYDBgg60PNm+d6tNhrr0kTlPNyIeHeZ7F3r8w/GD48OsIiK0seP3MmOGUKdbt2\ndS8PZAbDIgwxLGRS4tChcnc8I66/HrjgAteLCr76au8mKCD8m6G0sBgxIvSaofT6kPS4CouBA6Up\nlndIdG3XLtYsokpHB9Dc7LrtMZr6LN56y1itQmOzycSt3/1OTjSaxkb5Ek2Z0nuf4cPlpBauc1cc\naxYHDoTOHASl5GKnqsrc/ocOyfDxESN6P8emKPfYDBVlGhpkBFC/fr2fi5aaxfHjwP/8D3Drrd7t\nl5ICPPYYMGeOjAwBukdBOTdBATI02W4Pvatyo7Sw6N9ffkLls/Hpp7IEy+LF5vavqZEJqa7u6syw\ncI81iyjjrgkKiJ6w+OMfgcsu8zy/wp0f/EC+MNroqFdfBb7zHffbh3O/xd693TXQUGqKqq8HRo6U\nviIzy5C4aoLSMCxca2uT2pjRZltXfLpTHgVetIfF/v3S71BZaW5/mw147jkgJ0dOODU1smKpO+Hc\nb9HQIDULoLuT23Gl4mCpr5ewj4mRpTq8vcGUNsfCldGjgSef9L2MZinlusYTbLW1cpHkS9lYswgz\n7uZYANERFg8+KLWDlBTzx4iLA15+GfjJT4CCAtdNeppwDQulupuhAKlZhMqIqD17pGZx113SFOXt\n6CVPNYusLGmb7+z0vZxmfP3rcjESanztrwBYswg7tbXA1Kmun4v0Du4PPgD+9S9ZQM5XV14J/PWv\nMmnPk6QkY8uFhJrDh4E+fWTNJCD0mqEuuwy46irpS1m3zrt7bngKi/POk1Fye/a4v6jyl9OnZRXk\nPXvk/f7mNwP7+p742l8BsGYRdjw1Qw0YENkrz/7858Cvfy0Tzaxw6636YRGufRaOtQogtOZa1NdL\nzdBm665dGHX0qEw2HDnS/TbB6reor5f3eeVKucfGe+8Fvgzu+DrHAmBYhBxP1edTp+TE5e6LYrNJ\nE0sk3rvgyBHpp/judwP7uuHaDOUcFqHYDAUAt9wiM/GNvsc7d0qNwXENL2fBCoudO2WU1sSJwJ//\nLKPsQqVWyppFhGlq6r7pjiv79smVy1lnuT9GpPZbfPSRdM66GuLqT5ESFtpci2A7fVpC64IL5Pfz\nzpMa3jPPGNvfUxOUZsyY4IYFIEExb54sNRNsZ85I87WvzXIMixDy85/LGPSPP3b9vLs1oRzFx0dm\nv8XWrcC4cYF/XbtdamonT1p3zM8+A/7wB+uO50qo1iwaG2X4puOggp/8BHj+eeDYMf39jYTF5MnA\nmjWB/x44hgUAfOtbwDvvBLYMrjQ3S9Otr823DAuLnDolo3TM3lmtokLuJX377e7X5PfUX6GJ1Nur\nBissYmLkqry52bpjvvKKXBj4cm8BPe5qFsGexe3YBKVJT5eVcV96SX9/I2GRnCxzZxYtMl1MU5zv\nXpmVJd/FYN9t0Yr+CoBhYZnf/EbaKcvLvd/39Gng7rvlanPCBN/CIlKboYIVFoD1TVHLl8s8D6NN\nL2Y4h0X//sA55/jvs9HWBvz3f8vJ0lMgaRPynN1zj8zK1wszI2EBAL/8JfCXv7gPeaWkJnP0qP6x\njHKuWfTpI/cS2bTJutcww4r+CoBhYYnNm4ElS+QDWlHh/f7PPAOcf74sdjdmjCyS54qnORaaSAyL\n48fl3x6sCWVWhsWnn8pNm15+GXjxRWtPVo6cwwLwT1NUR4dMkszMlJF47e3dt3J1RRsJ5ewb35Dl\nVdaudb/vqVOyv5ET34gRwJ139uwzOHUK+PGP5bt21llSC09Ntaa2dfCg9A2cf37Px7/+9eCHhRVz\nLACGhc+OHOnuoJszR0Z2eOOzz+QD/eSTMpppzBi58byriUrRWrPYtk2q0Z469v0pKUkWLrQiMFat\nAqZNk5PrVVfJXfysdvQo8OWXve8g6BwWBw/6fhvShQuBFSvkIumZZ4BLL5XBCO64aoYC5LOv1S7c\nqauTocxnn22sbAsWAK+/LlfW7e1Afr4E2Ycfyvf2+HHpO6mtNXY8T7RahfMM6auuCn5Y7NhhrDam\nh2Hho3vukZvmfOtb8mE5dszzlZWzN96Q/UePlt8HDpQTvvMyy6dPy2N6M5cjcWJeMJugAOC226ST\ne9w4aQ9/4AHzx1q+XGqQgFzlPv209f0I+/ZJwPVx+nY7j4j67W+9X2rD0enTMrFxyZLu5Teys+Vk\n7I67ZihA7mi4dWvPVYEdGW2C0sTHS9PYT34ikzBHj5awTkjovvC44gppGfDVzp2u+wUuu0wC2Ujn\nvb98/LH8v/iKYeGDDz+UarN2jwSbTUZieFO7qK6W9mtHY8b0vuLbu1euFPWuqiKxZhHssBg7Vq5Q\nP/1UrqLNrj3U3g68+y4wfbr8npcnNYAtW6wrK+C6CQroWbPo7JQO5fffN3+zoDVrZAis4wn8kkv0\nw8LdBc/ZZwNz5wJ/+pPr53fv9n74509/Kq85b54c13l+hpVh4dhfoenfXyZ+ml3LbPFi4Be/MH9B\n0doqzW8JCeb2d8Sw8MH27VLNHDCg+7Grr/YuLHbs6L0omquwePtt+WDr8RQW7e3SnBJugh0WGptN\nToYnTsgX0Fv/+IfUIs87T37v00dOYk8/bW05HVebdeQ4i3vdOjmBxMebH5X1wgvA977X87FLLnHf\nDPXll9L842nl01tvBcrKXD+3b5/rEPTk3HOlzf4nP3H9/BVXWBPW7sICkHOEmSG0SknALV8O/PCH\n5ta72rZNLnasWNyQYeGD3bulg8xRbq73YaE1QWnGju3dyf2vf0lbtx5PYfH730u1PJx0dEjtS29Z\njkCx2aSp8PPPvd/39de7m6A03/se8Oab0ndllfp69zULrRnqxRdlNvzEicB//uP9axw6JLXqOXN6\nPp6a2n2/CmcNDRJink5cycky1NRVGO/b1z2ZzxueXu+SS6TP4sgR74/ryFNYmO3k/vhj+fx/8IH0\n19x2m/cXKR9/bN13h2HhA1cdzllZcgXf1KS//5EjcpJwbsN1rll0dgL//rf7BQQduZuUd/SotC3X\n1QV/rL03qqvl/Tn33GCXpNugQfJ/7I0vvwTWrwdmzuz5eHy8fJl97Wh2tHu364EQWjPUkSPSVzZn\njgzVNtNEUloqzWnaQoWamBi52HE1sdRd57aj2Fipebj6/pgNC0/OOkva831Zx+nECSmvu+a1K6+U\npi5vawbLlgE33igtF2VlEsDeLnfDsAgRrmoWffoY77eoqZFOMed21MxMOalrCwK+/740ISQm6h9T\n6+B2DoT//V8pV//+7pcTCUWh0gTlaNAg79ff+uc/ZeJZXFzv5+x2a/9P3A2x1jq4ly+X5tLzzzcf\nFq6aoDTuOrk9dW47uvBC14NE/BEWgO/9FnV18u9ytxTN0KHy480SJEoBf/+7hAUg39vXXwc2bvTc\nJ+SMYREiXIUFYLzfwlUTFCAfjAsu6B7SZ7QJCpAp/aNH91xO4vRpubvcfffJFWddnbFjhYJQDIu4\nOO9rFn/7W/cX39mwYdaFhVLy/+vqc6mFxdKl3Veo48ZJ7U27zawRH38sTU2u7lsOuO/k9tS57chV\nWHz5pYwoGjLEeDmN8jUsPDVBabxtiqqqkv9Lx8/+WWfJKr1GB1icPi1lc3WOMYNhYdKRI/IBHj68\n93NG+y2qq93/Rzr2W3gTFjabVF9//3tgwwZ5bPlyqZVcfjnDwgreNkPt2ydXhM7t+xq73bolIVpb\nZVSRc/MQ0D2L+6OPuu+10L+/1G69uVr961+BwkL3K7+66+Q20gwFuA6Lxka5gPLHXei0Tm6zzbPO\ny3y44m0n99//Dsye3fvf+8Mfyog8I31ctbXS9KgNqPCVz2FRXl6OzMxMpKenY5GbxVjuvvtupKen\nIzs7G1VVVbr7trW1IS8vDxkZGZg6dSravb2MC4Ddu7vX5Hc2dqz8Z+qt8ulqJJRG67dob5cruUmT\njJftwgvlC33TTXIS+t3vpFYBhFdYdHbKv/2SS4Jdkp68DYtnn5XOSXf9LlY2Q+lN3BwxQk5CjhMc\nXXVy/9//uW+e+sc/3NeSAPn8V1f37oz1pRnKX01QgIwK69/f/PfC3RwLR96EhXMTlKMhQ2SQxJIl\n+sexsgkK8DEsOjs7MX/+fJSXl6O6uhqlpaWocZpRU1ZWhrq6OtTW1uK5557DvHnzdPctLi5GXl4e\ndu3ahSlTpqC4uNiXYvqFuyYoQPotJk3Sr124a4YCusNi3bruO4p547rrZLmDSZOkBqRdSYZLWJw6\nJaNthg+X0UehxJs+i5MnZY2irz72LlnZDKW3JExBAfCjH/V8zLnf4sgRmZ/gap2zzk45cXtaPuLc\nc2VS4CefdD+mlPGwSE7uPSnVn2EB+NYUZaQZ6qKLpKnP+d/lyvvvy9In7ibS/fSnMlteb2RUSIVF\nZWUl0tLSkJycjNjYWMyZMwerVq3qsc3q1atRWFgIAJg4cSLa29vR0tLicV/HfQoLC7Fy5UpfiukX\neldwU6cCr77q/nl3I6E0Wlh40wTl7Je/lA/cww93z+ZNTZWgC1XPPAOMHy8Bcc89ssBiqPGmz+LV\nV+UL6+nK08pmKL1l7B9/vPdJyDksnn9eTkSuTmwHDsgILr3JodnZPZuiPvtMajNGgj/QNQvAfFgo\nZaxmYbNJ87SReU5arcJdk9vYsfJ6r73m+TghFRbNzc1ISkrq+j0xMRHNTss8uttm//79bvdtbW2F\n3W4HANjtdrSG4PAdTzULQJYa37xZOqpccTcSSpOWJsMc33jDfFjExMgH6qabeh63tjY0h8/W1EjA\nPfGEnFx27nQ/mSqYvGmGevppWdbDk0A2Q7mSmSmvf+iQ1IT++EfgoYdch4W72eHOnDu5jdYqAAmF\npqaeM8tDNSz275ealKtRbs6uuUY/LJSS7+wNN3je7qc/dT/TXaNNyLOKT2FhM9jbpAycmZRSLo9n\ns9kMv04g6YXFOefINP2HHnL9vKcmKECqoVqYWLEImGbwYPkz1JYEUUpqEg8+KE1noTSvwpnRZqit\nW2WJbK0J0J2hQ2V0kRUBbmRlYmcxMVKbe+89GbU1ejTw7W+7DgttYp0ex07uEyekdnv11cbK07+/\n1EAca1v+DoucHJnJ/uWX3u1XUSHrPxlxzTWyvaf/5127ZBSTXo3gm9+UCyp3Eyo//1wWijQy+sww\n5YPNmzeradOmdf3+2GOPqeLi4h7bzJ07V5WWlnb9ftFFF6mWlhaP+1500UXqwIEDSiml9u/fry66\n6KJerw0gyD8NChips02/r7a73MVzv1VAkc7+LyrgeT+U/T0FTAiB99DxZ6YCdiigbwiURe8nXwFl\nBrb7swJ+YfCYbQqIs6Bs7QoYZGK/xxWwUAE1CshV8tk9oYAYp+1+oYBiA8cboYDWr47zpgJecXEs\nTz//UT2/N3UKSPPz/+smBeR5uc8bCrjZi+2bFZDi4fn5Sj43Ro51jwL+5ua5qxSwxe2+ZvgUFqdO\nnVIpKSmqvr5enTx5UmVnZ6vq6uoe27z55psqPz9fKSXhMnHiRN1977vvvq7gePzxx9WCBQt6F9zg\nP/j4cdP/PLdOnFCqXz+lTp3S3/bPf1bqmmt6P56fr9TKlZ73/fBDpXbuNFdGT2bPVuqll6w/rlnH\njyuVkqLUmjXBLokxmzYpdfnlnrc5fVqp885TqqXF2DEvukipHTt8K1drq1Lx8eb2ff11peLilJow\nQakzZ+SxhASl9u7tud0PfqDU4sX6xztzRqkhQ5SaPFmp73xHqY4O78pzww1Kvfyy/L2zU75vx455\ndwxv/fa3St1xh/HtDx5U6mtfU+rIEeP73HyzUs8/7/75mTOVcri29qi9Xf7Pmpp6P/f000rdeafr\n/cyGhU/NUH379kVJSQmmTZuGrKwszJ49G6NGjcKSJUuw5KuxXdOnT0dKSgrS0tIwd+5cLF682OO+\nAFBUVIS1a9ciIyMD69evR1FRkanytbXJaJoHHzS/sqYrDQ0y2qNvX/1tv/tdGSO+bl3Px/WaoQDp\nJLTidojOQm1E1BNPSLU7Ly/YJTHGSJ/Fnj3S5PdV15suK/otzPRXaCZMkKa1oqLujlVXo5KM9lnY\nbNK0NWSI3OjJ3exmdxw7uVtb5T33dkSgt266SWZJG52g+NprMurQm3kMnvotTp2SEZTXXmvsWAMH\nArfcIivTOrO6cxuAyYgJAUaK/tBDclVz1VVypWLVlcmbbyo1darx7f/2N6Uuu0ypkyfl9y++UKp/\nf7n6DIYXXlDq1luD89qOTp9W6rnn5Gp49+5gl8a45mal7HbP27z+ulLf/KbxY954o/ErSneWLlXq\nllvM7//KK3IVr7nlFjmmo4wM4zWgL74w/xl/6imlfvQj+fuWLUqNH2/uON665hqlXnvN2LZXX63f\nOuCsrk6p4cO7a2+ONmxQatw47473ySdKnX9+73PblVcq9dZbrvcxe9qP2Bnchw9L4i5aJIvwxcZK\nqn/6qe/H9vYKbs4cmfhz883SeaU3EsrfQqFmsXmzTAZbulT+fyztiPMzI0Nnt2/37jawVgyf9aVm\nAchkPccbJjnXLJTybpnwAQPMf8Ydaxb+7tx2dMst0smvp7lZRhtdd513x09JkRYJV8vCr1ljbLFQ\nRxkZUit8+eXuxxobpWZh5UgoIIKX+/jTn2SFz5QUGRP+0ktSLXY3OskbeiOhnPXpA7zyiqz8evvt\n8iGzar0WM4IdFs88A3znO7Jc+saNvW/+FOrOPltOnJ6aK7wdtmhFM5TeHAtvOYdFa6s0uQRipNqF\nF3a/diDD4vrrpclYb7TbsmXArFne3+rXZnPfFLV2rbmm2J/+VG5Hq5Qc47LLZAi6NvLRKhEZFu3t\nQEmJ9FVobDbg3nuBlSvN3UTEkbdhAciHavlyGT9+//3BDQu7Xe4/bOaeDL5qbZXAXr9eruJCcFS0\nLptNv9/CTM3Cij4Lb4fNepKcLPMjNEb7K6yg1Sy02kygwmLQIDlhL1/e8/Hjx3v+/vLLPecvecNV\nWBw+LEukXHWV98e79lrpk731Vlmzq7S0e3kfK0VkWDz5JDBjRu+rrJQUWRvH1xuom63un3MOsHq1\nVBu9WevJajZb8GZyP/CALG3tj477QIqLc3/1efKknGS9+TcOG+ZbM5RS/q9ZGJ1jYYWBA4F+/WSi\nYCDDApDFvpWnAAAYYUlEQVSLmJdekr8rBTz1lKzmfPXVsojfzp1y0XfNNeaO72q+xfr1EhTe1lQA\n+T4/8IDMu3j/ffPl0mNgPE94+fxz+c99913Xz3/723LVMHmyueN3dsqXxmwb+4ABcm+DYNOaogK5\nout778m/fefOwL2mv3iqWezcKZ8Pb774vtYs2trkpBEfb/4Yzi64QGYonz4t7eyBrFkA3bWLQIfF\n9Omyrtru3bI8SmWljF6sqgJ++1uZbHjnnb71x5x7rjRVaiOWzPRXOLrlFvnxp4irWbz+uly1u6uO\nX3+9bGN2KG1zs7QF+nsYn78Fut/izBlZ5+mxx+QqLdx5Cgtvm6AA38NCq1VY2azXr5/MLtdW8Alk\nzQIIXlicdZacJy67TEL43XelI3n2bBmYsXGj732f8+dL89FTT8mQ2TVrQn/oeMSFRVmZrKzpTlaW\ndNK9/7654/s64iRUBDosXnpJAsPb20KGKn+Exaefmr+Isbq/QuPYFBWMmkV1tSzBMXRo4F4XkMEX\nDz8scymc51FceqncZdAX994rHen/+IeszXXyZHD7MY2IqLA4dUqGYeoNZ9Oaosww07kdigIdFo89\nJjdk6hMhnzhPfRZmFnA76yxpmvD2dq0af13EOIZFoGsWyclyD4ikpMAPhBg1SkYZ+fPzOnasrCr9\n5JMyxD/UB3tEyFdXbN4sbcXDhnne7vrrJSzMLNzGsPDegQPS+WZmpEeosrpmAfjWFOXvsFAqODWL\nd98NbBNUoNlsMhjnttuCXRJ9ERUWZWXSOaUnJ6f7Lmze2rMnvCaQuZOQIFexR4/6/7U2bpR7EEdK\nrQJwHxZffOH5PiWe+BIWVo+E0mhhceiQTGwN5I2oLrxQ7vsSyWERTiLo6ysjbYyEhc1mvikq0FVx\nf+nTR6raZgLTWxs3BneosD+4a4bSbpVrZqSML8Nn/VXj1cIi0LUKoPv1GBahIWLCoqlJfiZMMLb9\nt78NON3Uz5BICQtATuAbNvj/dTZsMD9UOVS5q1mYbYICzNcsjhwBjh3zTyewY1gE+nM/ZIjMTWJY\nhIaICYvycrmjnNErupwcuUewN6NPjh2TL6bRlURD3dVX698n3FeHD0vTXbgt6aHHU1iYXZPHbFho\nJ3J/dJAmJclci7q6wNcsbDZ5TYcbalIQRUxYlJUB+fnGtz/nHPnCHzhgfJ+9e+UqJ1La3idNkg5E\nX5c/8WTTJuDyy71fojrUuQuLbdt8q1mYaYbyZ21Xm2vx7rvBqVG/8ELk1UrDVUSc9jo6ZLq8t/eq\nHjmy59o3eiKpCQqQseIJCT3vlWy1SGyCAtz3WfjSDDVsmLmaRUODuQ51o5KT5f8x0DULQFYmNrME\nBlkvIsJi0yZZh8fbNttoDwtATuT+7LfYsCHyOrcB1zWLTz+VWtrw4eaOabYZyt+fy+RkCcZI++yT\ndyIiLP75T++aoDTOq2rqicSw8Ge/xdGjcqU9caJ/jh9MAwdKWDjO1dGaoMz2HYRiMxTQfexI++yT\ndyIiLDZsAL7xDe/3Y81CahYbN/bs6D9+XNbP8tWWLcAll4T/Olqu9OsnzSOO81S2b/dtyQa7XeZo\neLvkR329/8PivPOk6Y2iV9iHxcmTckV36aXe78uwkCXb4+NlfoDmiSfkrn6+dnxH4vwKR879Fkbu\nq+6J2SU//P25TEnx32grCh9hHxZVVdJfYebuXSNH9r4hvSeRGBaA1C60pqjWVuCPf5TRYt4EqSuR\n2rmtce638DUsAM/9FgsXykAOR198IXfsGzLEt9f1ZPJkWVCPolvYh8WWLTI004wLLpChs6dO6W97\n7Jh8MSNljoWjq6/u7uR++GFZGXbiRLlXuFkdHXL/iiuvtKaMocgxLJSyLizc9VusWye3zXTkzzkW\nmpiY8L9ZFfku7MPiP/8x34EaGyvDFRsb9beNtDkWjrSaxfbt0lfx//6fLAXiS1h8/LE0XwRyLaFA\ni4vrDov9+6Ufw9elqz0Nn21slJq0o0it7VLoCftTny81C8B4v0UkfymTk4Gzz5Z7Cj/wgPRh+BoW\ndXVyw5hINmhQd/+CFbUKwH0zVGen3ISoqqrnCKxI/lxSaAnrsGhpkduo+nLTF4aFmDxZ2r7vukt+\n9zUs6uv9O1EsFDg2Q1kZFq6aoVpapF+is7PnqgOR/rmk0BHWYaE1QfnSNOR8U3p3Iv1LuWAB8Pe/\nS1MK0B0WZu75AUTOUu6e+CMsEhNlQUxn2q1Fc3J6NkX5e9gskSYiwsIXrFmIMWN6LvY3eLAEhzdr\nZzmKhpqF49BZq8IiJUWWG3fW2Og6LPy91AeRJqzDwtf+CoBh4YkvTVH19dFTs1BK7hVtRVikproO\ni337ZPVVV2ERbZ9LCo6wDov33zd+/wp3GBbumQ2L06elKSXS70OghUVTk8xSHzzY92MOHw58+aUs\nhe/IVc3i889liLIVr0ukJ6zDQpt97OsxDh+WJS7cieQ5Fp6YDYumJnmvIn21UG3o7I4d5leadWaz\nuW6K0moW6emyJEh7e2DmWBBpwjosrFigrk8fuWLz1MkdyXMsPDEbFnv2REc7ujZ01qr+Co2rpiit\nZhETA1x8sSwrH421XQqesD79+dpfodFb9iNav5RmwyIa+iuA7mYof4TFnj09H9NqFoA0RW3dGr2f\nSwoOhgX0+y2i9UuZlCRt567uCOdJNIyEAvwXFs7NUMePSzOodr8Wrd8iWj+XFBxhHRZm73XsTO++\nFtH6pbTZgMxM72sX0dIMNXCghKlVI6E0zs1QTU1yR0OtGVQLC86xoEAK67Do29ea47Bm4Z6Zpqho\naYbq0wcYMAD42teklmEV57DQ+is0o0fL8zU10fu5pMAL67Cwil5YaKNOopGZsIiWmgUgIWFlrQKQ\nz1pzc/dqyI79FYCMMsvMBD75JHo/lxR4DAu47+D+/HOZ+FdXF71fSm/D4uhRaV8fNsx/ZQolcXHW\nh0W/fjLfYu9e+d25ZgFIU9S553KOBQWO6bBoa2tDXl4eMjIyMHXqVLS76QUtLy9HZmYm0tPTsWjR\nIt39Gxoa0L9/f+Tk5CAnJwd3aSvb+dGQIXLHvS++kCGJP/yhrNGTkADMnw/cckv0zbHQeBsWWpNd\ntAwz9kfNAujZFOVcswAkLEaO5BwLChzTX+ni4mLk5eVh165dmDJlCoqLi3tt09nZifnz56O8vBzV\n1dUoLS1FzVdnHk/7p6WloaqqClVVVVi8eLHZIhpms8kX78orgYIC4MIL5WZAX3whs8T/9KfoOfk5\nS02VJhFPkxYdRcMCgo7uvReYMcP64zqGhauaxXXXAbfdZv3rErlj+hS4evVqFBYWAgAKCwuxcuXK\nXttUVlYiLS0NycnJiI2NxZw5c7Bq1SrD+wfSAw8Av/mNnOwefFBOeNEaEI5iY+W9qK01tn20DJvV\nzJwpTUZWc65ZOIdFejpw//3Wvy6RO6ZPh62trbB/1TZjt9vR6uKOLc3NzUhyqD8nJiaiublZd//6\n+nrk5OQgNzcX77zzjtkieuWmm4BZs6wbYRVJUlKM36s82moW/qKFhVKum6GIAs3jqTEvLw8tLu7E\n8uijj/b43Wazweai8dT5MaWU2+20x0eMGIHGxkbExcVh69atmDVrFnbs2IEBAwbo/2vILwYPBg4d\nMrZtfb3c05t8k5IiwdveLhcwX/tasEtE0c5jWKx1vju8A7vdjpaWFgwbNgwHDhzAUG16qYOEhAQ0\nOtzguqmpCQkJCR7379evH/p9dQeecePGITU1FbW1tRg3blyv4y9cuLDr77m5ucjNzfX0zyGT4uOB\ntjZj20bTsFl/0pb82LuXtQryTUVFBSoqKnw+julGl4KCAixduhQLFizA0qVLMWvWrF7bjB8/HrW1\ntWhoaMCIESOwbNkylJaWetz/4MGDiIuLQ0xMDPbs2YPa2lqkuGnXcAwL8h+jNQuloq/Pwl8GDpT7\nor//fuQv9U7+5Xwh/cgjj5g6juk+i6KiIqxduxYZGRlYv349ioqKAAD79+/HjK+Gh/Tt2xclJSWY\nNm0asrKyMHv2bIwaNcrj/hs2bEB2djZycnJwww03YMmSJRhk5fRY8prRsDh4UCaMDRzo/zJFg9RU\noKKCNQsKDTalzN5lObhsNhvCtOhh5+9/l5/XXvO83X/+I/NS3nsvMOWKdDffDLz9NvDjH8toPSIr\nmD13cnAo6YqPN1azYBOUtVJTgf37WbOg0MCwIF2DBxvr4OawWWulpsqf7LOgUMCwIF1G+yxYs7CW\nFhasWVAoYFiQLoZFcGhh8dVoc6KgYliQrnPOkWGxx4553o5hYa3hw4E1a2SEGVGwcTQUGTJiBFBZ\nKavxutLZKUtmf/45T25EoYyjociv9JqimpqA889nUBBFKoYFGaIXFmyCIopsDAsyhGFBFN0YFmSI\n3mKCDAuiyMawIENYsyCKbgwLMoRhQRTdGBZkiF5Y8D4WRJGNYUGGeFpM8Phx6c8YMSKwZSKiwGFY\nkCGeFhPU7uYWExPYMhFR4DAsyBBPzVDsryCKfAwLMoRhQRTdGBZkSFwccPgwcOZM7+cYFkSRj2FB\nhvTrJ6vPfvFF7+cYFkSRj2FBhrlrimJYEEU+hgUZxrAgil4MCzLMVVh8/jnQ0QEMGRKcMhFRYDAs\nyDBXiwlqtQqbLThlIqLAYFiQYa5qFmyCIooODAsyjGFBFL0YFmQYw4IoejEsyDBXiwkyLIiiA8OC\nDHO1mCDDgig6MCzIMOdmqI4OWXGWYUEU+RgWZJhzWGzaBIweDQwYELwyEVFgMCzIMOc+i7IyYPr0\n4JWHiAKHYUGGDRwIHDsGnDolvzMsiKJH32AXgMJHnz6yVHlbm9xK9eBB4NJLg10qIgoEhgV5Reu3\nePtt4LrrJECIKPLxq05e0fot2ARFFF0YFuSVwYOB5mapWUydGuzSEFGgmA6LtrY25OXlISMjA1On\nTkV7e7vL7crLy5GZmYn09HQsWrSo6/FXX30Vo0ePRkxMDLZu3dpjn8cffxzp6enIzMzEmjVrzBaR\n/GDwYGDFCiA7W/oviCg6mA6L4uJi5OXlYdeuXZgyZQqKi4t7bdPZ2Yn58+ejvLwc1dXVKC0tRU1N\nDQBg7NixWLFiBSZPntxjn+rqaixbtgzV1dUoLy/HXXfdhTOubvxMQTF4MLBqFZugiKKN6bBYvXo1\nCgsLAQCFhYVYuXJlr20qKyuRlpaG5ORkxMbGYs6cOVi1ahUAIDMzExkZGb32WbVqFW666SbExsYi\nOTkZaWlpqKysNFtMslh8PHDyJMOCKNqYDovW1lbY7XYAgN1uR2tra69tmpubkZSU1PV7YmIimpub\nPR53//79SExM9GofCpzBg4ERI4CLLw52SYgokDwOnc3Ly0NLS0uvxx999NEev9tsNthc3CrN1WNm\nuDvOwoULu/6em5uL3NxcS16P3JswAViwgHfGIwoXFRUVqKio8Pk4HsNi7dq1bp+z2+1oaWnBsGHD\ncODAAQwdOrTXNgkJCWhsbOz6vbGxsUetwRXnfZqampCQkOByW8ewoMAYN05+iCg8OF9IP/LII6aO\nY7oZqqCgAEuXLgUALF26FLNmzeq1zfjx41FbW4uGhgZ0dHRg2bJlKCgo6LWdUqrHcV955RV0dHSg\nvr4etbW1mDBhgtliEhGRBUyHRVFREdauXYuMjAysX78eRUVFAKTPYcaMGQCAvn37oqSkBNOmTUNW\nVhZmz56NUaNGAQBWrFiBpKQkbNmyBTNmzEB+fj4AICsrCzfeeCOysrKQn5+PxYsXW9acRURE5tiU\n42V9GLHZbAjTohMRBY3ZcydncBMRkS6GBRER6WJYEBGRLoYFERHpYlgQEZEuhgUREeliWBARkS6G\nBRER6WJYEBGRLoYFERHpYlgQEZEuhgUREeliWBARkS6GBRER6WJYEBGRLoYFERHpYlgQEZEuhgUR\nEeliWBARkS6GBRER6WJYEBGRLoYFERHpYlgQEZEuhgUREeliWBARkS6GBRER6WJYEBGRLoYFERHp\nYlgQEZEuhgUREeliWBARkS6GBRER6WJYEBGRLoYFERHpYlgQEZEu02HR1taGvLw8ZGRkYOrUqWhv\nb3e5XXl5OTIzM5Geno5FixZ1Pf7qq69i9OjRiImJwdatW7seb2hoQP/+/ZGTk4OcnBzcddddZotI\nREQWMR0WxcXFyMvLw65duzBlyhQUFxf32qazsxPz589HeXk5qqurUVpaipqaGgDA2LFjsWLFCkye\nPLnXfmlpaaiqqkJVVRUWL15stojkhYqKimAXIaLw/bQW38/gMx0Wq1evRmFhIQCgsLAQK1eu7LVN\nZWUl0tLSkJycjNjYWMyZMwerVq0CAGRmZiIjI8Psy5PF+GW0Ft9Pa/H9DD7TYdHa2gq73Q4AsNvt\naG1t7bVNc3MzkpKSun5PTExEc3Oz7rHr6+uRk5OD3NxcvPPOO2aLSEREFunr6cm8vDy0tLT0evzR\nRx/t8bvNZoPNZuu1navH9IwYMQKNjY2Ii4vD1q1bMWvWLOzYsQMDBgzw+lhERGQNj2Gxdu1at8/Z\n7Xa0tLRg2LBhOHDgAIYOHdprm4SEBDQ2Nnb93tjYiMTERI8F6tevH/r16wcAGDduHFJTU1FbW4tx\n48b12C41NdVUGJF7jzzySLCLEFH4flqL76c1UlNTTe3nMSw8KSgowNKlS7FgwQIsXboUs2bN6rXN\n+PHjUVtbi4aGBowYMQLLli1DaWlpr+2UUl1/P3jwIOLi4hATE4M9e/agtrYWKSkpvfapq6szW3Qi\nIvKS6T6LoqIirF27FhkZGVi/fj2KiooAAPv378eMGTMAAH379kVJSQmmTZuGrKwszJ49G6NGjQIA\nrFixAklJSdiyZQtmzJiB/Px8AMDbb7+N7Oxs5OTk4IYbbsCSJUswaNAgX/+dRETkA5tyvKwnIiJy\nIWxmcLubxOfM3SRA6snopMrk5GRcfPHFyMnJwYQJEwJcytBn5PN29913Iz09HdnZ2aiqqgpwCcOH\n3ntZUVGBgQMHdk3Y/c1vfhOEUoaH73//+7Db7Rg7dqzbbbz+XKowUVNToz755BOVm5urPvjgA5fb\nnD59WqWmpqr6+nrV0dGhsrOzVXV1dYBLGh7uu+8+tWjRIqWUUsXFxWrBggUut0tOTlaHDh0KZNHC\nhpHP25tvvqny8/OVUkpt2bJFTZw4MRhFDXlG3su33npLzZw5M0glDC8bNmxQW7duVWPGjHH5vJnP\nZdjULIxM4vM0CZB6MjKpUqPYUumSkc+b4/s8ceJEtLe3u5yTFO2Mfnf5WTRm0qRJiIuLc/u8mc9l\n2ISFEWYnAUYjI5MqAZkrc+2112L8+PF4/vnnA1nEkGfk8+Zqm6ampoCVMVwYeS9tNhveffddZGdn\nY/r06aiurg50MSOGmc+l6aGz/uBuEuBjjz2GmTNn6u7PeRc9+TqpEgA2bdqE4cOH47PPPkNeXh4y\nMzMxadIkv5Q33Bj9vDlfDfNz2puR92TcuHFobGzEOeecg3/+85+YNWsWdu3aFYDSRSZvP5chFRae\nJgEaYWYSYCTzdVIlAAwfPhwAcP755+Nb3/oWKisrGRZfMfJ5c96mqakJCQkJAStjuDDyXjqu4pCf\nn4+77roLbW1tiI+PD1g5I4WZz2VYNkO5a7d0nATY0dGBZcuWoaCgIMClCw/apEoAbidVHjt2DEeO\nHAEAHD16FGvWrPE4uiLaGPm8FRQU4MUXXwQAbNmyBYMGDepq/qNuRt7L1tbWru9+ZWUllFIMCpNM\nfS6t6Xv3v9dff10lJiaqs88+W9ntdnXdddcppZRqbm5W06dP79qurKxMZWRkqNTUVPXYY48Fq7gh\n79ChQ2rKlCkqPT1d5eXlqcOHDyuler6fu3fvVtnZ2So7O1uNHj2a76cLrj5vzz77rHr22We7tvnx\nj3+sUlNT1cUXX+x2JB/pv5clJSVq9OjRKjs7W11xxRVq8+bNwSxuSJszZ44aPny4io2NVYmJieov\nf/mLz59LTsojIiJdYdkMRUREgcWwICIiXQwLIiLSxbAgIiJdDAsiItLFsCAiIl0MCyIi0sWwICIi\nXQwLIiLSFVILCRKFs87OTixbtgx79uxBUlISKisr8bOf/QwpKSnBLhqRz1izILLIRx99hOuvvx4p\nKSk4c+YMbrjhhq5Ve4nCHcOCyCLjxo3DWWedhc2bNyM3Nxe5ubno379/sItFZAmGBZFF3nvvPRw8\neBDbt2/HyJEjsXHjxmAXicgy7LMgskh5eTnsdjuuuuoqrFixAkOGDAl2kYgswyXKiYhIF5uhiIhI\nF8OCiIh0MSyIiEgXw4KIiHQxLIiISBfDgoiIdDEsiIhIF8OCiIh0/X9i+MtObswIbwAAAABJRU5E\nrkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x7d36518>"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- So we can think of the distortion error as introducing a bunch of local attractors into the representation\n",
      "    - There will be a tendency to drift towards one of these even if the input is zero\n",
      "- What will random noise do?\n",
      "    - Push the representation back and forth\n",
      "    - What if it is small?\n",
      "    - What if it is large?\n",
      "- So what will changing the post-synaptic time constant $\\tau$ do?\n",
      "    - How does that interact with noise?"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- But real eyes aren't perfect integrators\n",
      "    - If you get someone to look at someting, then turn off the lights but tell them to keep looking in the same direction, their eye will drift back to centre\n",
      "    - How do we implement that?\n",
      "    \n",
      "${dp \\over dt}=v - {p \\over \\tau_c}$\n",
      "\n",
      "- $\\tau_c$ is the time constant of that return to centre\n",
      "    \n",
      "- $f'(p)=\\tau ({-p \\over \\tau_c})+p$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nef\n",
      "\n",
      "tau=0.1\n",
      "tau_centre=1.0\n",
      "net = nef.Network('Neural Integrator')\n",
      "net.make('velocity', neurons=100, dimensions=1)\n",
      "net.make('position', neurons=100, dimensions=1)\n",
      "def recurrent(x):\n",
      "    return (1-tau/tau_centre)*x[0]\n",
      "net.connect('position', 'position', func=recurrent, pstc=tau)\n",
      "net.connect('velocity', 'position', transform=[[tau]])\n",
      "\n",
      "net.make_input('input', [0])\n",
      "net.connect('input', 'velocity')\n",
      "\n",
      "net.add_to_nengo()\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- Humans (a) and Goldfish (b)\n",
      "- Humans have more neurons doing this than goldfish (~70 vs ~20)\n",
      "- They also have slower decay.\n",
      "- Why do these fit together?\n",
      "\n",
      "<img src=\"files/lecture5/integrator_decay.png\">\n",
      "\n",
      "- With fewer neurons, the stable points on the integrator are more attractive, and so a gradual decay is more likely to get stuck\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Controlled Integrator\n",
      "\n",
      "- What if we want an integrator where we can adjust the decay?\n",
      "- Separate input telling us what the decay constant $d$ should be\n",
      "\n",
      "${dp \\over dt}= v - p d$\n",
      "\n",
      "- So there are two inputs: $v$ and $d$\n",
      "\n",
      "- Can we write this as ${dx \\over dt}=f(x)+g(u)$?\n",
      "- We need to compute a nonlinear function of an input ($d$) and the state variable ($p$)\n",
      "- How can we do this?\n",
      "    - change of variables\n",
      "    - let's have the state variable be $[p, d]$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nef\n",
      "\n",
      "tau=0.1\n",
      "tau_centre=1.0\n",
      "net = nef.Network('Controlled Neural Integrator')\n",
      "net.make('decay', neurons=100, dimensions=1)\n",
      "net.make('velocity', neurons=100, dimensions=1)\n",
      "net.make('position', neurons=200, dimensions=2)\n",
      "def recurrent(x):\n",
      "    return x[0]-x[1]*x[0], 0\n",
      "net.connect('position', 'position', func=recurrent, pstc=tau)\n",
      "net.connect('velocity', 'position', transform=[tau, 0])\n",
      "net.connect('decay', 'position', transform=[0, 1])\n",
      "\n",
      "net.make_input('input', [0])\n",
      "net.connect('input', 'velocity')\n",
      "\n",
      "net.make_input('input_decay', [0])\n",
      "net.connect('input_decay', 'decay')\n",
      "\n",
      "net.add_to_nengo()\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<img src=\"files/lecture5/controlled_integrator.png\">"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Other fun functions\n",
      "\n",
      "- Oscillator\n",
      "    - ${dx \\over dt}=[x_1, -x_0]$\n",
      "    \n",
      "\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nef\n",
      "net = nef.Network('Linear Oscillator')\n",
      "net.make('A', neurons=100, dimensions=2)\n",
      "def feedback(x):\n",
      "    return x[0]+x[1], -x[0]+x[1]\n",
      "net.connect('A', 'A', func=feedback, pstc=0.01)\n",
      "\n",
      "net.make_input('input', [0])\n",
      "net.connect('input', 'A')\n",
      "\n",
      "net.add_to_nengo()\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<img src=\"files/lecture5/oscillator.png\">\n",
      "\n",
      "- Lorenz Attractor\n",
      "\n",
      "    - ${dx \\over dt}=[10x_1-10x_0, -x_0 x_2-x_1, x_0 x_1 - {8 \\over 3}(x_2+28)-28]$\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "tau = 0.1\n",
      "sigma = 10\n",
      "beta = 8.0/3\n",
      "rho = 28\n",
      "\n",
      "import nef\n",
      "net = nef.Network('Lorenz attractor', seed=6)\n",
      "net.make('A', 2000, 3, radius=60)\n",
      "\n",
      "def feedback(x):\n",
      "    dx0 = -sigma * x[0] + sigma * x[1]\n",
      "    dx1 = -x[0] * x[2] - x[1]\n",
      "    dx2 = x[0] * x[1] - beta * (x[2] + rho) - rho\n",
      "    return [dx0 * tau + x[0], \n",
      "            dx1 * tau + x[1], \n",
      "            dx2 * tau + x[2]]\n",
      "net.connect('A', 'A', func=feedback, pstc=tau)\n",
      "\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<img src=\"files/lecture5/lorenz.png\">\n",
      "\n",
      "- Note: This is not the original Lorenz attractor.  \n",
      "    - The original is ${dx \\over dt}=[10x_1-10x_0, x_0 (28-x_2)-x_1, x_0 x_1 - {8 \\over 3}(x_2)]$\n",
      "    - Why change it to ${dx \\over dt}=[10x_1-10x_0, -x_0 x_2-x_1, x_0 x_1 - {8 \\over 3}(x_2+28)-28]$?\n",
      "    - What's being changed here?"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Oscillators with different paths\n",
      "\n",
      "- Since we can implement any function, we're not limited to linear oscillators\n",
      "- What about a \"square\" oscillator?\n",
      "    - Instead of the value going in a circle, it traces out a square\n",
      "\n",
      "$$\n",
      "{{dx} \\over {dt}} = \\begin{cases}\n",
      "    [r, 0] &\\mbox{if } |x_1|>|x_0| \\wedge x_1>0 \\\\ \n",
      "    [-r, 0] &\\mbox{if } |x_1|>|x_0| \\wedge x_1<0 \\\\ \n",
      "    [0, -r] &\\mbox{if } |x_1|<|x_0| \\wedge x_0>0 \\\\ \n",
      "    [0, r] &\\mbox{if } |x_1|<|x_0| \\wedge x_0<0 \\\\ \n",
      "    \\end{cases}\n",
      "$$    "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nef\n",
      "net = nef.Network('Square Oscillator')\n",
      "net.make('A', neurons=1000, dimensions=2)\n",
      "\n",
      "tau = 0.02\n",
      "r=4\n",
      "def feedback(x):    \n",
      "    if abs(x[1])>abs(x[0]):\n",
      "        if x[1]>0: dx=[r, 0]\n",
      "        else: dx=[-r, 0]\n",
      "    else:\n",
      "        if x[0]>0: dx=[0, -r]\n",
      "        else: dx=[0, r]\n",
      "    return [tau*dx[0]+x[0], tau*dx[1]+x[1]]        \n",
      "net.connect('A', 'A', func=feedback, pstc=tau)\n",
      "\n",
      "net.make_input('input', [0,0])\n",
      "net.connect('input', 'A')\n",
      "\n",
      "net.add_to_nengo()\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<img src=\"files/lecture5/osc_square.png\">\n",
      "\n",
      "- Does this do what you expect?\n",
      "- How is it affected by:\n",
      "    - Number of neurons?\n",
      "    - Post-synaptic time constant?\n",
      "    - Decoding filter time constant?\n",
      "    - Speed of oscillation (``r``)?"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- What about this shape?\n",
      "\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nef\n",
      "net = nef.Network('Heart Oscillator')\n",
      "net.make('A', neurons=1000, dimensions=2)\n",
      "\n",
      "tau = 0.02\n",
      "r=4\n",
      "def feedback(x):    \n",
      "    return [-tau*r*x[1]+x[0], tau*r*x[0]+x[1]]\n",
      "net.connect('A', 'A', func=feedback, pstc=tau)\n",
      "\n",
      "from math import atan2, sin, cos, sqrt\n",
      "def heart(x):\n",
      "    theta = atan2(x[1], x[0])\n",
      "    r = 2 - 2 * sin(theta) + sin(theta)*sqrt(abs(cos(theta)))/(sin(theta)+1.4)\n",
      "    return [-r*cos(theta), r*sin(theta)]\n",
      "\n",
      "net.make('B', neurons=10, dimensions=2)\n",
      "net.connect('A', 'B', func=heart, pstc=tau)\n",
      "net.make_input('input', [0,0])\n",
      "net.connect('input', 'A')\n",
      "\n",
      "net.add_to_nengo()\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<img src=\"files/lecture5/osc_heart.png\">\n",
      "\n",
      "- We are doing things differently here\n",
      "- The actual $x$ value is a normal circle oscillator\n",
      "- The heart shape is a function of $x$\n",
      "    - But that's just a different decoder\n",
      "- Would it be possible to do an oscillator where $x$ followed this shape?\n",
      "    - How could we tell them apart in terms of neural behaviour?\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Controlled Oscillator\n",
      "\n",
      "- ${dx \\over dt}=[x_1 x_2, -x_0 x_2]$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nef\n",
      "net = nef.Network('Controlled Oscillator')\n",
      "net.make('A', neurons=500, dimensions=3, radius=1.7)\n",
      "\n",
      "tau = 0.1\n",
      "scale = 10\n",
      "def feedback(x):\n",
      "    return x[1]*x[2]*scale*tau+1.1*x[0], -x[0]*x[2]*scale*tau+1.1*x[1], 0\n",
      "net.connect('A', 'A', func=feedback, pstc=tau)\n",
      "\n",
      "net.make_input('input', [0,0,0])\n",
      "net.connect('input', 'A')\n",
      "\n",
      "net.add_to_nengo()\n",
      "net.view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": []
    }
   ],
   "metadata": {}
  }
 ]
}