Code

[Feature]: MAE official#221

Code

Merged

fangyixiao18 merged 151 commits intoopen-mmlab:dev_v0.7.0open-mmlab/mmselfsup:dev_v0.7.0from YuanLiuuuuuu:feature/mae_officialYuanLiuuuuuu/mmselfsup:feature/mae_officialCopy head branch name to clipboardMar 3, 2022

+1,226-18

Commits

Commits on Mar 2, 2022

• [Feature]: MAE single image pre-training

  YuanLiuuuuuu

  committed
  bd2832c

  Copy full SHA for bd2832c

• [Fix]: Fix config

  

  LIU Yuan

  authored and

  YuanLiuuuuuu

  committed
  a4abe91

  Copy full SHA for a4abe91

• [Fix]: Fix dataset link

  YuanLiuuuuuu

  committed
  3bdaee7

  Copy full SHA for 3bdaee7

• [Feature]: Add run

  YuanLiuuuuuu

  committed
  9fa79de

  Copy full SHA for 9fa79de

• [Refactor]: Delete spot

  YuanLiuuuuuu

  committed
  ff5eb75

  Copy full SHA for ff5eb75

• [Feature]: ignore nohup output file

  YuanLiuuuuuu

  committed
  7635f8d

  Copy full SHA for 7635f8d

• [Feature]: Add auto script to generate run cmd

  YuanLiuuuuuu

  committed
  249c67e

  Copy full SHA for 249c67e

• [Refactor]: Refactor mae config file

  YuanLiuuuuuu

  committed
  85456a0

  Copy full SHA for 85456a0

• [Feature]: sz20 settings

  YuanLiuuuuuu

  committed
  00a3ccd

  Copy full SHA for 00a3ccd

• [Feature]: Add auto resume

  YuanLiuuuuuu

  committed
  54ef7de

  Copy full SHA for 54ef7de

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  69ba6cd

  Copy full SHA for 69ba6cd

• [Feature]: Make git ignore txt

  YuanLiuuuuuu

  committed
  df01dc3

  Copy full SHA for df01dc3

• [Refactor]: Delete gpus in script

  YuanLiuuuuuu

  committed
  a81aedd

  Copy full SHA for a81aedd

• [Fix]: Make generate_cmd to add --async

  YuanLiuuuuuu

  committed
  bcf24dc

  Copy full SHA for bcf24dc

• [Feature]: Initial version of Vit fine-tune

  YuanLiuuuuuu

  committed
  e32845a

  Copy full SHA for e32845a

• [Fix]: Add 1424 specific settings

  YuanLiuuuuuu

  committed
  cddc03e

  Copy full SHA for cddc03e

• [Fix]: Fix missing file client bug for 1424

  YuanLiuuuuuu

  committed
  d147286

  Copy full SHA for d147286

• [Feature]: 1424 customized settings

  YuanLiuuuuuu

  committed
  d543567

  Copy full SHA for d543567

• [Fix]: Make drop in eval to False

  YuanLiuuuuuu

  committed
  e4fbbac

  Copy full SHA for e4fbbac

• [Feature]: Change the finetune and pre-training settings

  YuanLiuuuuuu

  committed
  f34e1a4

  Copy full SHA for f34e1a4

• [Feature]: Add debug setting

  YuanLiuuuuuu

  committed
  0ff72bd

  Copy full SHA for 0ff72bd

• [Refactor]: Refactor the model

  YuanLiuuuuuu

  committed
  c5d08c6

  Copy full SHA for c5d08c6

• [Feature]: Customized settings

  YuanLiuuuuuu

  committed
  d07ba7b

  Copy full SHA for d07ba7b

• [Feature]: Add A100 settings

  YuanLiuuuuuu

  committed
  cc12ab6

  Copy full SHA for cc12ab6

• [Fix]: Change mae to imagenet

  YuanLiuuuuuu

  committed
  35c590a

  Copy full SHA for 35c590a

• [Feature]: Change mae pretrain num workers to 32

  YuanLiuuuuuu

  committed
  0fda414

  Copy full SHA for 0fda414

• [Feature]: Change num workers to 16

  YuanLiuuuuuu

  committed
  f4e24f4

  Copy full SHA for f4e24f4

• [Feature]: Add A100 setting for pre_release ft version

  YuanLiuuuuuu

  committed
  3b21c8b

  Copy full SHA for 3b21c8b

• [Feature]: Add img_norm_cfg

  YuanLiuuuuuu

  committed
  02eaeb4

  Copy full SHA for 02eaeb4

• [Fix]: Fix mae cls test missing logits bug

  YuanLiuuuuuu

  committed
  6e7f077

  Copy full SHA for 6e7f077

• [Fix]: Fix mae cls head bias initialize to zero

  YuanLiuuuuuu

  committed
  735b118

  Copy full SHA for 735b118

• [Feature]: Rename mae config name

  YuanLiuuuuuu

  committed
  daf29b5

  Copy full SHA for daf29b5

• [Feature]: Add MAE README.md

  YuanLiuuuuuu

  committed
  964a46e

  Copy full SHA for 964a46e

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  53f3cef

  Copy full SHA for 53f3cef

• [Feature]: Fix typo

  YuanLiuuuuuu

  committed
  18037f5

  Copy full SHA for 18037f5

• [Fix]: Fix typo

  YuanLiuuuuuu

  committed
  0cc25ea

  Copy full SHA for 0cc25ea

• [Feature]: Fix invalid link

  YuanLiuuuuuu

  committed
  54ac07d

  Copy full SHA for 54ac07d

• [Fix]: Fix finetune config file name

  YuanLiuuuuuu

  committed
  2250554

  Copy full SHA for 2250554

• [Feature]: Official pretrain v1

  YuanLiuuuuuu

  committed
  bbb792a

  Copy full SHA for bbb792a

• [Feature]: Change log interval to 100

  YuanLiuuuuuu

  committed
  6cddae6

  Copy full SHA for 6cddae6

• [Feature]: pretrain 1600 epochs

  YuanLiuuuuuu

  committed
  d4ac05e

  Copy full SHA for d4ac05e

• [Fix]: Change encoder num head to 12

  YuanLiuuuuuu

  committed
  f65083b

  Copy full SHA for f65083b

• [Feature]: Mix precision

  YuanLiuuuuuu

  committed
  0356ac1

  Copy full SHA for 0356ac1

• [Feature]: Add default value to random masking

  YuanLiuuuuuu

  committed
  951ae49

  Copy full SHA for 951ae49

• [Feature]: Official MAE finetune

  YuanLiuuuuuu

  committed
  dec4e12

  Copy full SHA for dec4e12

• [Feature]: Finetune img per gpu 32

  YuanLiuuuuuu

  committed
  c67a398

  Copy full SHA for c67a398

• [Feature]: Add multi machine training for lincls

  YuanLiuuuuuu

  committed
  e1509b8

  Copy full SHA for e1509b8

• [Fix]: Fix lincls master port master addr

  YuanLiuuuuuu

  committed
  850afeb

  Copy full SHA for 850afeb

• [Feature]: Change img per gpu to 128

  YuanLiuuuuuu

  committed
  8faf83b

  Copy full SHA for 8faf83b

• [Feature]: Add linear eval and Refactor

  YuanLiuuuuuu

  committed
  230e30e

  Copy full SHA for 230e30e

• [Fix]: Fix debug mode

  YuanLiuuuuuu

  committed
  6f000f4

  Copy full SHA for 6f000f4

• [Fix]: Delete MAE dataset in __init__.py

  YuanLiuuuuuu

  committed
  ecf499b

  Copy full SHA for ecf499b

• [Feature]: normalize pixel for mae

  YuanLiuuuuuu

  committed
  df106df

  Copy full SHA for df106df

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  a79dbd2

  Copy full SHA for a79dbd2

• [Feature]: LARS for linear eval

  YuanLiuuuuuu

  committed
  d8cf7f7

  Copy full SHA for d8cf7f7

• [Feature]: Add lars for mae linear eval

  YuanLiuuuuuu

  committed
  f27b98b

  Copy full SHA for f27b98b

• [Feature]: Change mae linear lars num workers to 32

  YuanLiuuuuuu

  committed
  f85888e

  Copy full SHA for f85888e

• [Feature]: Change mae linear lars num workers to 8

  YuanLiuuuuuu

  committed
  ce96371

  Copy full SHA for ce96371

• [Feature]: log every 25 iter for mae linear eval lars

  YuanLiuuuuuu

  committed
  7a458c0

  Copy full SHA for 7a458c0

• [Feature]: Add 1600 epoch and 800 epoch pretraining

  YuanLiuuuuuu

  committed
  c29c969

  Copy full SHA for c29c969

• [Fix]: Change linear eval to 902

  YuanLiuuuuuu

  committed
  f38b023

  Copy full SHA for f38b023

• [Fix]: Add random flip to linear eval

  YuanLiuuuuuu

  committed
  8b265f5

  Copy full SHA for 8b265f5

• [Fix]: delete fp16 in mae

  YuanLiuuuuuu

  committed
  05b5d1a

  Copy full SHA for 05b5d1a

• [Refactor]: Change backbone to mmcls

  YuanLiuuuuuu

  committed
  1a0c391

  Copy full SHA for 1a0c391

• [Fix]: Align finetune settings

  YuanLiuuuuuu

  committed
  4cc9675

  Copy full SHA for 4cc9675

• [Fix]: replace timm trunc_normal with mmcv trunc_normal

  YuanLiuuuuuu

  committed
  8e5224f

  Copy full SHA for 8e5224f

• [Fix]: Change finetune layer_decay to 0.65

  YuanLiuuuuuu

  committed
  3007827

  Copy full SHA for 3007827

• [Fix]: Delete pretrain last norm when global_pooling

  YuanLiuuuuuu

  committed
  fa993d4

  Copy full SHA for fa993d4

• [Fix]: set requires_grad of norm1 to False

  YuanLiuuuuuu

  committed
  e41873a

  Copy full SHA for e41873a

• [Fix]: delete norm1

  YuanLiuuuuuu

  committed
  a93bf4d

  Copy full SHA for a93bf4d

• [Fix]: Fix docstring bug

  YuanLiuuuuuu

  committed
  e2b5a46

  Copy full SHA for e2b5a46

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  bfcede7

  Copy full SHA for bfcede7

• [Fix]: Add external link

  YuanLiuuuuuu

  committed
  2292097

  Copy full SHA for 2292097

• [Fix]: Delete auto_resume and reformat config readme.

  YuanLiuuuuuu

  committed
  06eb87f

  Copy full SHA for 06eb87f

• [Fix]: Fix pytest bug

  YuanLiuuuuuu

  committed
  92cb93a

  Copy full SHA for 92cb93a

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  eb1f1e7

  Copy full SHA for eb1f1e7

• [Refactor]: Rename filename

  YuanLiuuuuuu

  committed
  0e0b329

  Copy full SHA for 0e0b329

• [Feature]: Add docstring

  YuanLiuuuuuu

  committed
  a6bcb17

  Copy full SHA for a6bcb17

• [Fix]: Rename config file name

  YuanLiuuuuuu

  committed
  819d757

  Copy full SHA for 819d757

• [Fix]: Fix name inconsistency bug

  YuanLiuuuuuu

  committed
  4daf8ea

  Copy full SHA for 4daf8ea

• [Fix]: Change the default value of persistent_worker in builder to True

  YuanLiuuuuuu

  committed
  4431e22

  Copy full SHA for 4431e22

• [Fix]: Change the default value of CPUS_PER_TASK to 5

  YuanLiuuuuuu

  committed
  3da7da1

  Copy full SHA for 3da7da1

• [Fix]: Add a blank line to line136 in tools/train.py

  YuanLiuuuuuu

  committed
  740aaa3

  Copy full SHA for 740aaa3

• [Fix]: Fix MAE algorithm docstring format and add paper name and url

  YuanLiuuuuuu

  committed
  ad0a0cd

  Copy full SHA for ad0a0cd

• [Feature]: Add MAE paper name and link, and store mae teaser on github

  YuanLiuuuuuu

  committed
  32c3034

  Copy full SHA for 32c3034

• [Refactor]: Delete mae.png

  YuanLiuuuuuu

  committed
  8546415

  Copy full SHA for 8546415

• [Fix]: Fix config file name”

  YuanLiuuuuuu

  committed
  431837c

  Copy full SHA for 431837c

• [Fix]: Fix name bug

  YuanLiuuuuuu

  committed
  9bffe84

  Copy full SHA for 9bffe84

• [Refactor]: Change default GPUS to 8

  YuanLiuuuuuu

  committed
  67fbd40

  Copy full SHA for 67fbd40

• [Fix]: Abandon change to drop_last

  YuanLiuuuuuu

  committed
  ebbe995

  Copy full SHA for ebbe995

• [Fix]: Fix docstring in mae algorithm

  YuanLiuuuuuu

  committed
  7b81385

  Copy full SHA for 7b81385

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  c37c9a3

  Copy full SHA for c37c9a3

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  43f2e8b

  Copy full SHA for 43f2e8b

• [Fix]: Fix mae finetune algo type bug

  YuanLiuuuuuu

  committed
  f895c79

  Copy full SHA for f895c79

• [Feature]: Add unit test for algorithm

  YuanLiuuuuuu

  committed
  0d8a008

  Copy full SHA for 0d8a008

• [Feature]: Add unit test for remaining parts

  YuanLiuuuuuu

  committed
  145772a

  Copy full SHA for 145772a

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  2668560

  Copy full SHA for 2668560

• [Fix]: Fix typo

  YuanLiuuuuuu

  committed
  e1a18a5

  Copy full SHA for e1a18a5

• [Fix]: Delete some unnecessary modification in gitignore

  YuanLiuuuuuu

  committed
  967666b

  Copy full SHA for 967666b

• [Feature]: Change finetune setting in mae algo to mixup setting

  YuanLiuuuuuu

  committed
  9f48a8b

  Copy full SHA for 9f48a8b

• [Fix]: Change norm_pix_loss to norm_pix in pretrain head

  YuanLiuuuuuu

  committed
  8c70139

  Copy full SHA for 8c70139

• [Fix]: Delete modification in dist_train_linear.sh

  YuanLiuuuuuu

  committed
  f672434

  Copy full SHA for f672434

• [Refactor]: Delete global pool in mae_cls_vit.py

  YuanLiuuuuuu

  committed
  e817133

  Copy full SHA for e817133

• [Fix]: Change finetune param to mixup in test_mae_classification

  YuanLiuuuuuu

  committed
  cd72bed

  Copy full SHA for cd72bed

• [Fix]: Change norm_pix_loss to norm_pix of mae_pretrain_head in unit test

  YuanLiuuuuuu

  committed
  aa49693

  Copy full SHA for aa49693

• [Fix]: Change norm_pix_loss to norm_pix in unit test

  YuanLiuuuuuu

  committed
  6e7f4a4

  Copy full SHA for 6e7f4a4

• [Refactor]: Create init_weights for mae_finetune_head and mae_linprobe_head

  YuanLiuuuuuu

  committed
  bebf727

  Copy full SHA for bebf727

• [Refactor]: Construct 2d sin-cosine position embedding using torch

  YuanLiuuuuuu

  committed
  b3a2401

  Copy full SHA for b3a2401

• [Refactor]: Using classification and using mixup from mmcls

  YuanLiuuuuuu

  committed
  7b80971

  Copy full SHA for 7b80971

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  6d764cc

  Copy full SHA for 6d764cc

• [Fix]: Add False to finetune mae linprobe‘

  Show description for 7664df6
  YuanLiuuuuuu

  committed
  7664df6

  Copy full SHA for 7664df6

• [Fix]: Set drop_last to False

  YuanLiuuuuuu

  committed
  9ae1536

  Copy full SHA for 9ae1536

• [Fix]: Fix MAE finetune layerwise lr bug

  YuanLiuuuuuu

  committed
  7643e24

  Copy full SHA for 7643e24

• [Refactor]: Delete redundant MAE when registering MAE

  YuanLiuuuuuu

  committed
  89096de

  Copy full SHA for 89096de

• [Refactor]: Split initialize_weights in MAE to submodules

  YuanLiuuuuuu

  committed
  b76ce88

  Copy full SHA for b76ce88

• [Fix]: Change the min_lr of mae pretrain to 0.0

  YuanLiuuuuuu

  committed
  771682d

  Copy full SHA for 771682d

• [Refactor]: Delete unused _init_weights in mae_cls_vit

  YuanLiuuuuuu

  committed
  a1f1bc0

  Copy full SHA for a1f1bc0

• [Refactor]: Change MAE cls vit to a more general name

  YuanLiuuuuuu

  committed
  ec36e4b

  Copy full SHA for ec36e4b

• [Feature]: Add Epoch Fix cosine annealing lr updater

  YuanLiuuuuuu

  committed
  9dbd6ef

  Copy full SHA for 9dbd6ef

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  b686452

  Copy full SHA for b686452

• [Feature]: Add layer wise lr decay in optimizer constructor

  YuanLiuuuuuu

  committed
  95091a7

  Copy full SHA for 95091a7

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  4120bd2

  Copy full SHA for 4120bd2

• [Fix]: Fix set layer wise lr decay bug

  YuanLiuuuuuu

  committed
  5e00b4e

  Copy full SHA for 5e00b4e

• [Fix]: Fix UT for MAE

  YuanLiuuuuuu

  committed
  dca3e1f

  Copy full SHA for dca3e1f

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  74f33d9

  Copy full SHA for 74f33d9

• [Fix]: update algorithm readme format for MAE

  YuanLiuuuuuu

  committed
  f844b16

  Copy full SHA for f844b16

• [Fix]: Fix isort

  YuanLiuuuuuu

  committed
  cd36793

  Copy full SHA for cd36793

• [Fix]: Add Returns inmae_pretrain_vit

  YuanLiuuuuuu

  committed
  97d5410

  Copy full SHA for 97d5410

• [Fix]: Change bgr to rgb

  YuanLiuuuuuu

  committed
  722354a

  Copy full SHA for 722354a

• [Fix]: Change norm pix to True

  YuanLiuuuuuu

  committed
  f7796a9

  Copy full SHA for f7796a9

• [Fix]: Use cls_token to linear prob

  YuanLiuuuuuu

  committed
  102be55

  Copy full SHA for 102be55

• [Fix]: Delete mixup.py

  YuanLiuuuuuu

  committed
  6d06091

  Copy full SHA for 6d06091

• [Fix]: Fix MAE readme

  YuanLiuuuuuu

  committed
  e9c5ec3

  Copy full SHA for e9c5ec3

• [Feature]: Delete linprobe

  YuanLiuuuuuu

  committed
  35b6767

  Copy full SHA for 35b6767

• [Refactor]: Merge MAE head into one file

  YuanLiuuuuuu

  committed
  78c877b

  Copy full SHA for 78c877b

• [Fix]: Fix lint

  YuanLiuuuuuu

  committed
  7d80bdc

  Copy full SHA for 7d80bdc

• [Fix]: rename mae_pretrain_head to mae_head

  YuanLiuuuuuu

  committed
  1042cd8

  Copy full SHA for 1042cd8

• [Fix]: Fix import error in __init__.py

  YuanLiuuuuuu

  committed
  439834b

  Copy full SHA for 439834b

• [Feature]: skip MAE algo UT when running on windows

  YuanLiuuuuuu

  committed
  880b423

  Copy full SHA for 880b423

• [Fix]: Fix UT bug

  YuanLiuuuuuu

  committed
  03c6a73

  Copy full SHA for 03c6a73

• [Feature]: Update model_zoo

  YuanLiuuuuuu

  committed
  7a17ac0

  Copy full SHA for 7a17ac0

• [Fix]: Rename MAE pretrain model name

  YuanLiuuuuuu

  committed
  92d5b38

  Copy full SHA for 92d5b38

• [Fix]: Delete mae ft prefix

  YuanLiuuuuuu

  committed
  7e6c0e4

  Copy full SHA for 7e6c0e4

• [Feature]: Change b to base

  YuanLiuuuuuu

  committed
  9e285fc

  Copy full SHA for 9e285fc

• [Refactor]: Change b in MAE pt config to base

  YuanLiuuuuuu

  committed
  337de6e

  Copy full SHA for 337de6e

• [Fix]: Fix typo in docstring

  YuanLiuuuuuu

  committed
  e3323b5

  Copy full SHA for e3323b5

• [Fix]: Fix name bug

  YuanLiuuuuuu

  committed
  29521cd

  Copy full SHA for 29521cd

• [Feature]: Add new constructor for MAE finetune

  YuanLiuuuuuu

  committed
  e084a43

  Copy full SHA for e084a43

• [Fix]: Fix model_zoo link

  YuanLiuuuuuu

  committed
  3f1f8b8

  Copy full SHA for 3f1f8b8

• [Fix]: Skip UT for MAE

  YuanLiuuuuuu

  committed
  c8c8560

  Copy full SHA for c8c8560

• [Fix]: Change fixed channel order to param

  YuanLiuuuuuu

  committed
  cfb3bca

  Copy full SHA for cfb3bca