CUDA/HIP: fix tests/test-backend-ops #8896
Conversation
tests/test-backend-ops.cpp
Outdated
| @@ -539,7 +539,7 @@ struct test_case { | |||
| return false; | |||
| } | |||
|
|
|||
| bool eval_perf(ggml_backend_t backend, const char * op_name) { | |||
| bool eval_perf(ggml_backend_t backend1, ggml_backend_t backend2, const char * op_name) { | |||
There was a problem hiding this comment.
I don't understand the purpose of this change, the perf mode only uses one backend.
This should not be necessary. |
|
It seems that the issue is that the CUDA backend does not correctly report in the |
0231713 to
de7cf9d
Compare
|
In my opinion the reportedly supported ops being inaccurate and performance tests being executed for cases where the correctness cannot be asserted are both issues. But since the whole point of the tests is to reduce the time developers need to spend on quality assurance I don't want to discuss this at length. |
Fixes #8864 .
The problem is simply that I forgot to adapt the HIP test logic at some point.
Fixes #8863 .
The problem is that there are test cases that are (according to the corresponding functions) supported with the CUDA backend but not with the CPU backend. As a consequence, when running the tests normally these tests are not executed because there are no CPU results to compare the CUDA results to. In performance mode they are executed though and trigger asserts. Quite frankly though, without a way to assert that the results produced are actually correct such performance numbers would be useless anyways. So I edited the test code in such a way that performance is only evaluated for those ops where correctness can also be tested.