Add non_device_test pytest mark to filter out non-device tests

[fxmarty]

As per title.

Many tests in transformers do not use torch_device, .to(0), .to("cuda"), etc. and are always run on CPU. These tests are irrelevant for people who would like to run Transformers tests on their device.

With this proposed PR, one can now run pytest tests/ -m "not not_device_test" --collect-only to filter out non-device tests. We use pytest_collection_modifyitems in order to both capture the tests defined in test_modeling_common.py and in subclasses as well (no need to @pytest.mark.not_gpu_test everywhere) which does not add a significant overhead after tests collection.

We prefer a mark non_device_test rather than device_test, as people adding new tests may forget to add the device tests in the list. It is safer to be greedy (in a reasonable extent).

This now gives

================================================= 34914/54481 tests collected (19567 deselected) in 10.58s ==================================================

instead of the previous 54481 tests selected.

This is requested by AMD.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[fxmarty]

Actually now getting =========== 27353/54481 tests collected (27128 deselected) in 9.77s ============ skipping all the pipelinemixin tests that are run always on CPU.

[ArthurZucker]

Thanks this makes sense!
Do you think having a regex based marker would help? --regex ".*test_[a-bA-B]+$"