Deep Learning Reading List

This repository is a compilation of all the important readings related to Deep Learning.

Philosophy

• It won't be an exhaustive list
• This list would focus on the principles rather than the catching up with latest models.

Image Classification

Initially, this subfield is focusing on improving the accuracy on the imagenet-1000 challenge. Recently, the papers here focuses on two things:

• These models as a feature extractor to other area such as object detection, segmentation, GAN, etc.
• Making the models smaller without harming the accuracy

Gradient-Based Learning Applied to Document Recognition, IEEE 1998 paper

ImageNet Classification with Deep Convolutional Neural Networks, NIPS 2012 paper

Very Deep Convolutional Neural Networks for Large-Scale Image Recognition, ICLR 2015 paper
Key Insight: It's all about deep neural network architecture. This is one of the classic image classification architecture wherein many of our recent algorithm uses it as a backbone.

Deep Residual Learning for Image Recognition, CVPR 2016 paper
Key Insight: Going deep requires us to consider the vanishing gradient problem. One way to alleviate it is to use a residual block that harnesses skip connection to propagate the gradients to the lower layers.

Identity Mappings in Deep Residual Networks, ECCV 2016 paper

Densely Connected Convolutional Network, CVPR 2017 paper

Xception: Deep Learning With Depthwise Separable Convolutions, CVPR 2017 paper

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications, Arxiv 2017 paper

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices, CVPR 2018 oaoer

Squeeze and Excitation Network, CVPR 2018 web paper

FishNet: A Versatile Backbone for Image, Region, and Pixel Level Prediction, NIPS 2018 web paper

Neural Network Modules

Rectifier Nonlinearities Improve Neural Network Acoustic Models, ICML 2013 paper

Dropout: A Simple Way to Prevent Neural Networks from Overfitting, JMLR 2014 paper
Key Insight: It's all about the concept of regularization. On how we can use it to make our ML algorithm be more robust on generalizing on our test set.

Adam: A Method for Stochastic Optimization, ICLR 2015 paper
Key Insight: It's about understanding the behaviours on how we optimize the weights of the network. We would understand the direction of the research on this field of optimization.

Batch Normalization, ICML 2015 web paper

Spatial Transformer Network, NIPS 2015 web paper

Deformable Convolutional Networks, ICCV 2017 website paper

Group Normalization, ECCV 2018 paper

Deformable ConvNets v2: More Deformable, Better Results, CVPR 2019 paper

Object Detection

Region-based convolutional networks for accurate object detection and segmentation, TPAMI 2015

Spatial pyramid pooling in deep convolutional networks for visual recognition, TPAMI 2015 paper

Fast rcnn, ICCV 2016 paper

Faster r-cnn: Towards real-time object detection with region proposal networks, NIPS 2016 paper

SSD: single shot multibox detector, ECCV 2016 paper

Speed Accuracy trade-offs for modern convolutional object detection network, CVPR 2017 paper

Feature Pyramid Networks for Object Detection CVPR 2017 paper
Key Insight: Applying the principle of U-net for object detection

Focal Loss for Dense Object Detection, ICCV 2017 paper

Cascade R-CNN: Delving Into High Quality Object Detection, ECCV 2018 web paper

Multi-Object Tracker

Tracking without bells and whistles, ICCV 2019 paper

Instance Segmentation

U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical image computing and computer-assisted intervention 2015 paper web

Mask R-CNN, ICCV 2017 web paper

Hybrid Task Cascade for instance segmentation, CVPR 2019 paper

Layer Visualization

Visualizing and Understanding Convolutional Networks, ECCV 2014 paper

Sequence Models

On the Properties of Neural Machine Translation: Encoder–Decoder Approaches, Arxiv 2014 paper

Empirical evaluation of gated recurrent neural networks on sequence modeling, Arxiv 2014 paper

Sequence to Sequence Learning with Neural Networks, NIPS 2014 website paper

Neural Machine Translation by Jointly Learning to Align and Translate, ICLR 2015 paper slide

An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition, ICDAR 2015 paper

Generative Advserial Network

Generative Adversarial Nets, NIPS 2014 paper

Unsupervised Representational Learning with Deep Convolutional Generative Adversarial Networks, ICLR 2016 paper

Unpaired Image-To-Image Translation Using Cycle-Consistent Adversarial Networks, ICCV 2017 web paper

Image-to-Image Translation with Conditional Adversarial Networks, CVPR 2017 paper

High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs website paper

Motion Transfer

Everybody Dance Now, ICCV 2019 website paper