GAN's.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### __Generative Models and GANs__\n",
    "![title](https://miro.medium.com/max/1250/1*vBmmy3bnFF4wfruuCwhiGA.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Simply they are a class of Unsupervised Machine Learning Models which are used to generate some Data . It uses the Joint Probability Distribution over the observations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Generative Adversarial Networks belong to the set of generative models. It means that they are able to produce / to generate (we’ll see how) new content\n",
    "- Naturally, this ability to generate new content makes GANs look a little bit “magic”, at least at first sight"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Generative model Vs Discriminative model__\n",
    "The discriminative models learn the conditional probability distribution $P(y \\mid x)$ i.e. the decision boundary between classes e.g. logistic regression, neural network. The generative models learn the joint probability distribution $P(x, y)$  i.e. the distribution of individual classes e.g. naive Bayes, Gaussian discriminant analysis (GDA).\n",
    "![title](https://kharshit.github.io/img/discriminative_generative.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GANs were invented by Ian Goodfellow et al. in 2014 in Generative Adversarial Nets. __It has two networks, competing with each other:__\n",
    "\n",
    "__Generator try to fool the discriminator by generating real-looking images\n",
    "Discriminator try to distinguish real images from the fake ones generated by the Generator__\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](https://miro.medium.com/max/875/1*XKanAdkjQbg1eDDMF2-4ow.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](https://kharshit.github.io/img/gan.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Training\n",
    "__Training involves alternating between training the discriminator and the generator.__ The real loss and fake loss are used to calculate the discriminator losses as follows.\n",
    "![title](https://kharshit.github.io/img/gan_MNIST.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GANs take a long time to train. On a single GPU a GAN might take hours, and on a single CPU more than a day. While difficult to tune and therefore to use, GANs have stimulated a lot of interesting research and writing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__BatchNorm__:\n",
    "<br>\n",
    "- Construct different mini-batches for real and fake, i.e. each mini-batch needs to contain only all real images or all generated images.\n",
    "- when batchnorm is not an option use instance normalization (for each sample, subtract mean and divide by standard deviation).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](https://github.com/soumith/ganhacks/raw/master/images/batchmix.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__The two networks can be considered black boxes representing some arbitrarily complicated function that is applied to either noise or real data__. The input to the generator is some random noise, which produces a fake image, and the input to the discriminator is both fake samples as well as samples from our real data set. The discriminator then makes a binary decision about whether the image supplied, z, is a real image D(z)=1, or a fake image, D(z)=0.\n",
    "![title](https://miro.medium.com/max/875/1*3sl6xiqg_5vV2e-3jUsm0A.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In summary, the game follows with:\n",
    "\n",
    "- The generator trying to maximize the probability of making the discriminator mistakes its inputs as real.\n",
    "- And the discriminator guiding the generator to produce more realistic images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}