Deep-learning-from-scratch

This repository consists of basic implementations of deep learning models. These are homework assigments for an online computer science degree I am doing with Georgia Tech. I am not supposed to share my code. This intro is mainly to showcase the experience I have gained.

1. Two networks with a simple SGD optimizer from scratch:

• a simple softmax regression
• a two-layer multi-layer perceptron

2. A simple CNN architecture from scratch as well as CNNs with PyTorch.

• Convnet Modules implemented are 2D convolution, 2D Max Pooling, ReLU, and Linear. For each module, a forward pass (computing forwarding results) and a backward pass (computing gradients) are implemented.
• Pytorch Vanilla Convolutional Neural Network with a convolution layer, a ReLU activation, a max-pooling layer, followed by a fully connected layer for classification.

3. Data wrangling

• Class-Balanced Focal Loss

4. Natural language processing with Pytorch

• RNN and LSTM
• Seq2seq
• Transformer