Simple neural network implementation, based on KMNIST (japanese ideograms) classification. net.py contains all the necessary functions. loader.py load the kmnist dataset and applies basic data transformations.
- backpropagation algorithm with SGD
- basic sigmoid activation function
- quadratic cost function
- random gaussian weight inizialization
- weight inizialization
- L2 regularization
- cross entropy cost function
- learning rate scheduler?
- momentum based SGD?
- animation showing the change in bias and weights
- basic train evaluation tools: time, accuracy
- basic hyperparamenter tuning with graph equalizing time, random initializaions
- shows vanishing gradient problem
The program saves mid-training weights and biases for visualization purposes. The animation shows how the network change its parameters with gradient descent. It is possible to show the vanishing gradient problem by inspecting the first layer.
