Figma RCNN(Fine-grained Multi-Attribute RCNN)

Person Detection and Multi-Attributes Recognition with only one Jointly-Trained Holistic CNN Model
The master branch works with tensorpack 0.9
It is a part of the Figma RCNN project developed by Junlin Gu, Graduate student at UESTC.

Main References

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Human Attribute Recognition by Deep Hierarchical Contexts
• An All-In-One Convolutional Neural Network for Face Analysis

Dependencies

• Python 3.3+; OpenCV
• TensorFlow ≥ 1.6
• Tensorpack ≥ 0.9
• pycocotools: pip install 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'
• Pre-trained ImageNet ResNet model from tensorpack model zoo
• COCO data. It needs to have the following directory structure:

COCO/DIR/
  annotations/
    instances_train201?.json
    instances_val201?.json
  train201?/
    COCO_train201?_*.jpg
  val201?/
    COCO_val201?_*.jpg

• Wider Attibutes data. It needs to have the following directory structure:

wider attibutes/
  Anno/
    wider_attribute_trainval.json
    wider_attribute_test.json
  train/
    0--Parade/
      0_Parade_marchingband_?.jpg

    1--Handshaking/
      1_Handshaking_Handshaking_?.jpg
    ...
  test/
    0--Parade/
      0_Parade_marchingband_?.jpg
    1--Handshaking/
      1_Handshaking_Handshaking_?.jpg
    ...

Usage

Installation

Setup Docker Environment on Ubuntu host(recommended)
Download the image file and install the Docker environment, please click here to view my blog

Train:

To train on a single machine:

./attr_train.py --config \
    BACKBONE.WEIGHTS=/path/to/COCO-R50C4-MaskRCNN-Standard.npz

Options can be changed by either the command line or the tensorpack_config.py file (recommended).

Inference:

To predict on an image (needs DISPLAY to show the outputs):

./demo_cam.py 
--image
/path/to/input.jpg
--cam
0
--obj_model
all-in-one
--obj_ckpt
/root/to/checkpoint
--obj_config
DATA.BASEDIR=/path/to/COCO/DIR

The trained models can be downloaded in the [Baidu Cloud] (Waiting upload).

Results

The models' detection branch are trained on COCO trainval35k and evaluated on COCO minival2014 using mAP@IoU=0.50:0.95. attributes branch are trained on Wider Attribute trainval and evaluated on Wider Attribute test using mAP. The models are fine-tuned from ResNet pre-trained R50C4 models in tensorpack model zoo

Performance in Person Detection can be approximately reproduced.

| Backbone  | mAP (box) | Detectron mAP (box) |Configurations                            |
| R50-C4    | 33.3      | 32.8                |TRAIN.LR_SCHEDULE=[120000, 240000, 280000]|

Performance in Person Attributes Recognition can be approximately reproduced.

|      Atrributes   | AP (positive/negative)     |
|        male       |        0.9503              |
|      longhair     |        0.8598              |
|      sunglass     |        0.7342              |
|        hat        |        0.9477              |
|       tshirt      |        0.7892              |
|     longsleeve    |        0.9602              |
|       formal      |        0.8084              |
|       shorts      |        0.9105              |
|       jeans       |        0.7461              |
|     longpants     |        0.9711              |
|       skirt       |        0.8454              |
|     facemask      |        0.7372              |
|       logo        |        0.8912              |
|      stripe       |        0.6159              |
|        mAP        |        0.8405              |

Some examples

Here are some visualization results of the figma rcnn model.