07_numbers/Perils_of_Floating_Point.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Perils of Floating Point (Bruce M Bush)\n",
    "http://www.lahey.com/float.htm\n",
    "\n",
    "Examples from Bruce Bush's notes on floating point numbers. Python tends to hide FP problems fairly well and few \"strange\" results appear:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Binary Floating-Point\n",
    "\n",
    "No \"weird\" results in Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = 0.01\n",
    "x * 100. == 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = 0.00000000000000000001\n",
    "x * 100000000000000000000. == 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inexactness\n",
    "\n",
    "No \"weird\" results in Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = 7777\n",
    "y = 7\n",
    "y1 = 1/y\n",
    "z = x/y\n",
    "z1 = x * y1\n",
    "z == z1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Insignificant digits "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.20000000000004547\n"
     ]
    }
   ],
   "source": [
    "y = 1000.2\n",
    "A = y - 1000.0\n",
    "print(A)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A == 0.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Give the most attention to:\n",
    "\n",
    "1. subtractions of numbers that are nearly equal,\n",
    "2. additions of numbers whose magnitudes are nearly equal, but whose signs are opposite, and\n",
    "3. additions and subtractions of numbers that differ greatly in magnitude. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Crazy conversions\n",
    "The following example *might* show `i=2132` because 21.33 is not exactly representable in binary and is slightly less than 21.33."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2133.0 2133\n"
     ]
    }
   ],
   "source": [
    "x = 21.33\n",
    "y = x * 100\n",
    "i = int(y)\n",
    "print(y, i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "int(1.999999999999999)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "int(1.9999999999999999)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Programming with the Perils\n",
    "(Bruce M Bush)\n",
    "\n",
    "There are no easy answers. It is the nature of binary floating-point to behave the way I have described. In order to take advantage of the power of computer floating-point, you need to know its limitations and work within them. Keeping the following things in mind when programming floating-point arithmetic should help a lot:\n",
    "\n",
    "1. Only about 7 decimal digits are representable in single-precision IEEE format, and about 16 in double-precision IEEE format.\n",
    "2. Every time numbers are transferred from external decimal to internal binary or vice-versa, precision can be lost.\n",
    "3. Always use safe comparisons.\n",
    "4. Beware of additions and subtractions that can quickly erode the true significance in a result. The computer doesn't know what bits are truely significant. \n",
    "5. Conversions between data types can be tricky. Conversions to double-precision don't increase the number of truely significant bits. Conversions to integer always truncate toward zero, even if the floating-point number is printed as a larger integer.\n",
    "6. Don't expect identical results from two different floating-point implementations.\n",
    "\n",
    "I hope that I have given you a little more awareness of what is happening in the internals of floating-point arithmetic, and that some of the strange results you have seen make a little more sense.\n",
    "\n",
    "While some of the \"perils\" can be avoided, many just need to be understood and accepted. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}