Two problems encountered when using AGPPruner

[ichejun]

Environment:

• NNI version:2.1
• NNI mode (local|remote|pai):local
• Client OS:
• Server OS (for remote mode only):
• Python version:3.7.5
• PyTorch/TensorFlow version:PyTorch 1.7.1
• Is conda/virtualenv/venv used?:no
• Is running in Docker?:no

Hi, I have encountered two problems when using AGPPruner, could you please help to solve?

Q1:
I encountered a problem when I tried AGPPruner.
The default configuration can run through, but when I try to add the exclude configuration. There is a problem with configuration parsing. AGP pruner did not support 'exclude' configration? Did any Pruning Schedule provided by nni support 'exclude' configration?
The code:

  config_list = [{
        'initial_sparsity': 0.,
        'final_sparsity': 0.05,
        'start_epoch': 0,
        'end_epoch': 10,
        'frequency': 1,
        'op_types': ['Conv2d']
    }, {
        'op_names':['model.24.m.0','model.24.m.1','model.24.m.2'],
        'exclude': True
    }]
    pruner = AGPPruner(model, config_list, optimizer, pruning_algorithm='fpgm')
    pruner.compress()

The error log:

Traceback (most recent call last):
  File "train_nni_up_agp.py", line 980, in <module>
    train(hyp, opt, device, tb_writer)
  File "train_nni_up_agp.py", line 537, in train
    pruner = AGPPruner(model, config_list, optimizer, pruning_algorithm='fpgm')
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/algorithms/compression/pytorch/pruning/agp.py", line 44, in __init__
    super().__init__(model, config_list, optimizer)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/compressor.py", line 322, in __init__
    super().__init__(model, config_list, optimizer)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/compressor.py", line 44, in __init__
    self.validate_config(model, config_list)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/algorithms/compression/pytorch/pruning/agp.py", line 70, in validate_config
    schema.validate(config_list)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/nni/compression/pytorch/utils/config_validation.py", line 53, in validate
    self.compressor_schema.validate(data)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/schema.py", line 357, in validate
    return type(data)(o.validate(d) for d in data)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/schema.py", line 357, in <genexpr>
    return type(data)(o.validate(d) for d in data)
  File "/home/admin/work_dir/.local/lib/python3.7/site-packages/schema.py", line 167, in validate
    [self._error.format(data) if self._error else None] + errors,
schema.SchemaError: Or(And({'initial_sparsity': And(<class 'float'>, <function AGPPruner.validate_config.<locals>.<lambda> at 0x7f987df74b90>), 'final_sparsity': And(<class 'float'>, <function AGPPruner.validate_config.<locals>.<lambda> at 0x7f987df74c20>), 'start_epoch': And(<class 'int'>, <function AGPPruner.validate_config.<locals>.<lambda> at 0x7f987df74cb0>), 'end_epoch': And(<class 'int'>, <function AGPPruner.validate_config.<locals>.<lambda> at 0x7f987df74d40>), 'frequency': And(<class 'int'>, <function AGPPruner.validate_config.<locals>.<lambda> at 0x7f987df74dd0>), Optional('op_types'): And([<class 'str'>], <function CompressorSchema._modify_schema.<locals>.<lambda> at 0x7f987df74e60>), Optional('op_names'): And([<class 'str'>], <function CompressorSchema._modify_schema.<locals>.<lambda> at 0x7f987df74ef0>)}, <function CompressorSchema._modify_schema.<locals>.<lambda> at 0x7f987df74f80>)) did not validate {'op_names': ['model.24.m.0', 'model.24.m.1', 'model.24.m.2'], 'exclude': True}
Missing keys: 'end_epoch', 'final_sparsity', 'frequency', 'initial_sparsity', 'start_epoch'

Q2:
Can AGPPruner with pruning_algorithm 'fpgm', work in dependency_aware mode? It seems that there is no corresponding api interface?
classnni.algorithms.compression.pytorch.pruning.AGPPruner(model, config_list, optimizer, pruning_algorithm='level')
Thanks a lot!

[J-shang]

hi @ichejun ,
Q1: We are sorry that none of our current schedule pruners supports exclude, and this is a feature that should be supported. For a workaround, you can modify the validate_config function to pass the validation, similar as _StructuredFilterPruner.validate_config().

Q2: AGPPruner can not work in dependency_aware mode, this is due to current implementation limitations. we will discuss if this needs refactor.

If you have any idea about these, welcome to discuss with us.