[feature] Add better CTest support to CMake helper

[fschoenm]

What is your suggestion?

Right now, the CMake helper has a test() method that uses the test/RUN_TESTS target for running tests. However, CTest has a lot of additional options that cannot be configured with just environment variables (see https://cmake.org/cmake/help/latest/manual/ctest.1.html).

For example, it does not seem possible to set the --output-junit parameter that way. But there are a lot more parameters that cannot be used, e.g. excluding and include tests, repeated runs, etc.

I think it would be nice to have an additional ctest() method that uses CTest directly instead of the limited method above. It should be possible to use presets anyway, so calling e.g. ctest --preset conan-debug on the command already starts CTest with the correct preset.

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[jcar87]

Hi @fschoenm - thank you for raising this issue.

Thank you for pointing out a use case where the recommended approach (environment variables) is not suitable.
Will discuss this with the team to try and improve the interface to cover this.

[PeteAudinate]

You can use CMAKE_CTEST_ARGUMENTS:

 def generate(self):
        deps = CMakeDeps(self)
        deps.generate()

        tc = CMakeToolchain(self)
        ctest_args = "--output-on-failure;--output-junit;test_results.xml"
        tc.cache_variables["CMAKE_CTEST_ARGUMENTS"] = ctest_args
        tc.generate()

    def build(self):
        cmake = CMake(self)
        cmake.configure()
        cmake.build()
        cmake.test()

But it would be nice to have a cmake.ctest() method.

[memsharded]

Implemented in #15282, for next 2.0.15, feedback welcome