From c63e66522682a8c04ce485c5f0a89d568bcd783e Mon Sep 17 00:00:00 2001 From: Caleb Leandre Jackson Date: Thu, 24 Aug 2023 13:48:16 -0600 Subject: [PATCH 1/7] Add SimpleTesting directory --- cmake/SimpleTesting/README.md | 139 +++++++++++++ .../cmake/ctest-CTestConfig.cmake | 38 ++++ .../cmake/ctest-cdash-setup.cmake | 142 +++++++++++++ cmake/SimpleTesting/cmake/ctest-common.cmake | 196 ++++++++++++++++++ cmake/SimpleTesting/cmake/ctest-driver.cmake | 24 +++ .../SimpleTesting/cmake/ctest-functions.cmake | 192 +++++++++++++++++ .../cmake/ctest-stage-build.cmake | 32 +++ .../cmake/ctest-stage-configure.cmake | 47 +++++ .../cmake/ctest-stage-coverage.cmake | 78 +++++++ .../cmake/ctest-stage-test.cmake | 66 ++++++ cmake/SimpleTesting/cmake/ctest-submit.cmake | 20 ++ 11 files changed, 974 insertions(+) create mode 100644 cmake/SimpleTesting/README.md create mode 100644 cmake/SimpleTesting/cmake/ctest-CTestConfig.cmake create mode 100644 cmake/SimpleTesting/cmake/ctest-cdash-setup.cmake create mode 100644 cmake/SimpleTesting/cmake/ctest-common.cmake create mode 100644 cmake/SimpleTesting/cmake/ctest-driver.cmake create mode 100644 cmake/SimpleTesting/cmake/ctest-functions.cmake create mode 100644 cmake/SimpleTesting/cmake/ctest-stage-build.cmake create mode 100644 cmake/SimpleTesting/cmake/ctest-stage-configure.cmake create mode 100644 cmake/SimpleTesting/cmake/ctest-stage-coverage.cmake create mode 100644 cmake/SimpleTesting/cmake/ctest-stage-test.cmake create mode 100644 cmake/SimpleTesting/cmake/ctest-submit.cmake diff --git a/cmake/SimpleTesting/README.md b/cmake/SimpleTesting/README.md new file mode 100644 index 000000000000..3149f43cba80 --- /dev/null +++ b/cmake/SimpleTesting/README.md @@ -0,0 +1,139 @@ +CMake File Structure +==================== +This CMake structure attempts to capture the structure of the original +`TFW_testing_single_configure_prototype` that is used by the current / old Trilinos +Pull-Request (PR) framework. + +**Note**: _Trilinos uses the **`pull_request_changes`** branch from this repository_. + +Specifically, this new configuration will replace the existing `simple_testing.cmake` +script. + +This diagram describes the current structure of the CMake files and how they +interact with each other. +
+ CMake Structure Diagram +
+ + +CMake Files +----------- + +| Filename | Guarded? | Purpose | +|-------------------------------|:--------:|--------------------------------------------------------------------------| +| `ctest-driver.cmake` | | Main driver script / entry-point | +| `ctest-common.cmake` | X | Common activities such as option handling, etc. | +| `ctest-cdash-setup.cmake` | X | Handle CDash configurations | +| `ctest-stage-configure.cmake` | | Handle the _configuration_ stage of the test | +| `ctest-stage-build.cmake` | | Handle the _build_ stage of the test | +| `ctest-stage-test.cmake` | | Handle the _test_ stage of the test. | +| `ctest-functions.cmake` | X | Functions and macros | +| `ctest-CTestConfig.cmake` | | Required file that is copied into the build directory at configure time. | + +The _guarded_ files use the CMake command [`include_guard()`][3] which should prevent that file +from being include more than once in an include chain. + +Options and Variables (`simple_testing.cmake`) +---------------------------------------------- +The `simple_testing.cmake` file has a number of optional parameters that can be sent into the +CMake system via `-D:BOOL=` parameters: + +| Option | Required? | Default | PR Override | Purpose | +|----------------------------|:---------:|---------------------------------------------|-------------|----------------------------------| +| `build_name` | YES | N/A | YES | Sets `CTEST_BUILD_NAME` | +| `ctest_submit_retry_count` | NO | 5 | | | +| `ctest_submit_retry_delay` | NO | 3 | | | +| `dashboard_model` | NO | `Experimental` | YES but NO | Set to the same value as default | +| `dashboard_track` | NO | `Experimental` | YES but NO | Set to the same value as default | +| `skip_by_parts_submit` | NO | `ON` | YES | | +| `skip_clean_build_dir` | NO | `ON` | | | +| `skip_single_submit` | NO | `ON` | | | +| `skip_update_step` | NO | `OFF` | YES | | +| `skip_upload_config_files` | NO | `OFF` | | | +| `PARALLEL_LEVEL` | NO | _num cores_ | YES | | +| `TEST_PARALLEL_LEVEL` | NO | `${PARALLEL_LEVEL}` | YES | | +| `build_root` | NO | `${CTEST_SOURCE_DIRECTORY}/nightly_testing` | | | +| `build_dir` | NO | `${build_root}/${CTEST_BUILD_NAME}` | YES | The CMake build dir | +| `configure_script` | YES | N/A | YES | See note below table | +| `package_enables` | YES | N/A | YES | `packageEnables.cmake` | +| `subprojects_file` | YES | N/A | YES | `package_subproject_list.cmake` | + +1. `configure_script` points to the `cmake/std/PullRequestLinuxTestingSettings.cmake` file. + - Example: `${WORKSPACE}/Trilinos/cmake/std/PullRequestLinuxGCC8.3.0TestingSettings.cmake` + +See `TrilinosPRConfigurationStandard.py` for information on what options are set to something +other than the default during normal Trilinos PR operations. + +Expected Operation +------------------ +The expected operation of this set of files to replace the old `simple_testing.cmake` is to load +the [`ctest-driver.cmake`](ctest-driver.cmake) file in its place. + + +`ctest-driver.cmake` Options +---------------------------- + +| Option | Type | Required? | Default | Description | +|----------------------------|--------|:---------:|-----------------------------------------------|--------------------------------------------------------------------| +| `build_name` | STRING | YES | N/A | The build name, see Jenkins' `${JOB_NAME}` envvar. | +| `subprojects_file` | STRING | YES | N/A | This is the `package_subproject_list.cmake` file. | +| `source_dir` | PATH | YES | N/A | Path to the source directory. | +| `configure_script` | STRING | YES | N/A | Test settings CMake script. | +| `package_enables` | STRING | YES | N/A | This is the `packageEnables.cmake` file. | +| `ctest_submit_retry_count` | STRING | NO | 5 | Number of times to retry a ctest submssion. | +| `ctest_submit_retry_delay` | STRING | NO | 3 | Delay (seconds) between attempts to submit to cdash. | +| `dashboard_model` | STRING | NO | Experimental | CDash model | +| `dashboard_track` | STRING | NO | Experimental | CDash track | +| `skip_clean_build_dir` | BOOL | NO | ON | Skip cleaning the build directory (`ctest_empty_binary_directory`) | +| `skip_update_step` | BOOL | NO | OFF | Skip the update step (`ctest_update()`) of the repository. | +| `skip_by_parts_submit` | BOOL | NO | ON | Skip submission to CDash after each phase. | +| `skip_single_submit` | BOOL | NO | ON | Skip single submission | +| `skip_upload_config_files` | BOOL | NO | OFF | Skip upload config files (???) | +| `build_root` | STRING | NO | `${source_dir}/nightly_testing` | Used to generate `build_dir` if `build_dir` is not defined. | +| `build_dir` | STRING | NO | `${build_root}/${CTEST_BUILD_NAME}` | Path to the build directory. | +| `PARALLEL_LEVEL` | STRING | NO | `` | | +| `TEST_PARALLEL_LEVEL` | STRING | NO | `${PARALLEL_LEVEL}` | | + +1. It might worthwhile to remove `build_root` since it's only used to create `build_dir` IF `build_dir` is not passed in + via a `-D` option. +2. Related to (1), we might also change `build_dir` to be `BUILD_DIR` and pass that in. + + + +Example CTest call from a Trilinos PR +------------------------------------- +This is an example, for reference, of how the `ctest` command is invoked in the current/old Trilinos +PR test driver. +```bash +ctest \ + -S simple_testing.cmake \ + -Dsource_dir=${WORKSPACE}/Trilinos \ + -Dbuild_name=PR-9495-test-Trilinos_pullrequest_gcc_8.3.0-5164 \ + -Dskip_by_parts_submit=OFF \ + -Dskip_update_step=ON \ + -Ddashboard_model=Experimental \ + -Ddashboard_track=Pull Request \ + -DPARALLEL_LEVEL=20 \ + -DTEST_PARALLEL_LEVEL=4 \ + -Dbuild_dir=${WORKSPACE}/pull_request_test \ + -Dconfigure_script=${WORKSPACE}/Trilinos/cmake/std/PullRequestLinuxGCC8.3.0TestingSettings.cmake \ + -Dpackage_enables=../packageEnables.cmake \ + -Dsubprojects_file=../package_subproject_list.cmake +``` + +Additional Notes and Pitfalls +============================= + +`ctest_test()` and `CAPTURE_CMAKE_ERROR` +---------------------------------------- +For Trilinos testing we should avoid checking the value returned by `ctest_test()` +for the `CAPTURE_CMAKE_ERROR` parameter. The reason for this is that CTest will +flag this as an error (i.e., -1 is returned) if there were _no tests run_. + +The _no tests were run_ though is a valid 'success' result for Trilinos PR tests since +this project enables packages dynamically based on what packages have modifications. +This allows some PR's to go through without building Trilinos, which is advantageous +when only documentation or perhaps the testing framework itself is modified and we do +not need to spend O(5 hours) for the test suite to run. + +[3]: https://cmake.org/cmake/help/latest/command/include_guard.html diff --git a/cmake/SimpleTesting/cmake/ctest-CTestConfig.cmake b/cmake/SimpleTesting/cmake/ctest-CTestConfig.cmake new file mode 100644 index 000000000000..c6b1dcf95fde --- /dev/null +++ b/cmake/SimpleTesting/cmake/ctest-CTestConfig.cmake @@ -0,0 +1,38 @@ +# This file will be copied to "CTestConfig.cmake" and should be placed +# in the ${CTEST_BINARY_DIRECTORY} directory. +# +# https://cmake.org/cmake/help/latest/module/CTest.html +message(">>> CTestConfig.cmake STARTED") + +# Must match what is in CDash project 'Trilinos' +set(CTEST_NIGHTLY_START_TIME "04:00:00 UTC") # 10 PM MDT or 9 PM MST + + +if (NOT DEFINED CTEST_DROP_METHOD) + set(CTEST_DROP_METHOD "http") +endif() + + +# Temporarily changing TEST_DROP_SITE from testing-vm.sandia.gov to +# testing.sandia.gov due to the broken CDash installation +if (CTEST_DROP_METHOD STREQUAL "http") + set(CTEST_PROJECT_NAME "Trilinos") + if (NOT DEFINED CTEST_DROP_SITE) + set(CTEST_DROP_SITE "testing.sandia.gov") + endif() + + if (${CTEST_DROP_SITE} STREQUAL "testing.sandia.gov") + set(CTEST_DROP_LOCATION "/cdash/submit.php?project=${CTEST_PROJECT_NAME}") + else() + set(CTEST_DROP_METHOD "https") + set(CTEST_DROP_LOCATION "/submit.php?project=${CTEST_PROJECT_NAME}") + endif() + + set(CTEST_TRIGGER_SITE "") + set(CTEST_DROP_SITE_CDASH TRUE) + message(">>> CTEST_DROP_SITE : ${CTEST_DROP_SITE}") + message(">>> CTEST_DROP_LOCATION : ${CTEST_DROP_LOCATION}") + message(">>> CTEST_PROJECT_NAME : ${CTEST_PROJECT_NAME}") +endif() + +message(">>> CTestConfig.cmake FINISHED") diff --git a/cmake/SimpleTesting/cmake/ctest-cdash-setup.cmake b/cmake/SimpleTesting/cmake/ctest-cdash-setup.cmake new file mode 100644 index 000000000000..44cdb85bd49e --- /dev/null +++ b/cmake/SimpleTesting/cmake/ctest-cdash-setup.cmake @@ -0,0 +1,142 @@ +include_guard() +message("+--------------------------------------+") +message("| ctest-cdash-setup.cmake START |") +message("+--------------------------------------+") +include(${CMAKE_CURRENT_LIST_DIR}/ctest-functions.cmake) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-common.cmake) + + + +# =============================================================================== +# -- +# -- Set up the Dashboard and Testing Infrastructure +# -- +# =============================================================================== +print_options_list() + +# ----------------------------------------------------------- +# -- Specify the Generator +# ----------------------------------------------------------- +set(CTEST_CMAKE_GENERATOR "Ninja") + + +set(CTEST_CONFIGURE_COMMAND_ARGS + "${CMAKE_COMMAND}" + "-C \"${configure_file}\"" + "-C \"${package_enables_file}\"" + "-G \"${CTEST_CMAKE_GENERATOR}\"" + "${CTEST_SOURCE_DIRECTORY}" +) + +list(JOIN CTEST_CONFIGURE_COMMAND_ARGS " " CTEST_CONFIGURE_COMMAND) + +banner("CMake Configuration Command") +message(">>> CTEST_CONFIGURE_COMMAND : ${CTEST_CONFIGURE_COMMAND}") +message(">>>") + + + +# ----------------------------------------------------------- +# -- Configure Preparation +# ----------------------------------------------------------- + +# Optionally skip cleaning the build directory +if(NOT skip_clean_build_dir) + ctest_empty_binary_directory(${CTEST_BINARY_DIRECTORY}) +endif() + +# Optionally upload the config files +# TODO: Note how this works / what it's doing in CMake-land. +message(">>> Write `configure_command_file`:") +if(skip_upload_config_files) + message(">>> - SKIPPED") +else() + message(">>> - WRITTEN") + #message(">>> - configure_command_file : ${configure_command_file}") + #message(">>> - CTEST_CONFIGURE_COMMAND: ${CTEST_CONFIGURE_COMMAND}") + file(WRITE ${configure_command_file} ${CTEST_CONFIGURE_COMMAND}) +endif() +message(">>>") + + +# ----------------------------------------------------------- +# -- COPY files +# -- ctest-CTestConfig.cmake -> ${CTEST_BINARY_DIRECTORY}/CTestConfig.cmake +# -- See: https://cmake.org/cmake/help/latest/command/configure_file.html +# ----------------------------------------------------------- +banner("Copy CTestConfig.cmake to BUILD dir") +configure_file(${CMAKE_CURRENT_LIST_DIR}/ctest-CTestConfig.cmake ${CTEST_BINARY_DIRECTORY}/CTestConfig.cmake COPYONLY) + + +# ----------------------------------------------------------- +# -- Start the testing engine +# -- CTEST_DROP_LOCATION defined in ctest-CTestConfig.cmake +# -- See: https://cmake.org/cmake/help/latest/command/ctest_start.html +# ----------------------------------------------------------- +banner("ctest_start() START") +ctest_start(${dashboard_model} GROUP ${dashboard_track}) +banner("ctest_start() FINISH") +message(">>> CTEST_DROP_LOCATION = ${CTEST_DROP_LOCATION}") +message(">>>") + +# ----------------------------------------------------------- +# -- Configure CDash Drop URLs +# -- CTEST_DROP_SITE is defined in ctest-CTestConfig.cmake +# -- CTEST_PROJECT_NAME is defined in ctest-CTestConfig.cmake +# -- CTEST_BUILD_NAME is defined in ctest-common.cmake +# -- URL_location is defined here +# -- build_stamp is defined here +# -- machine_name is defined here +# ----------------------------------------------------------- + +string(FIND ${CTEST_DROP_LOCATION} submit.php index) +string(SUBSTRING ${CTEST_DROP_LOCATION} 0 ${index} URL_location) + +file(STRINGS ${CTEST_BINARY_DIRECTORY}/Testing/TAG tag_strings LIMIT_COUNT 2) +foreach(item ${tag_strings}) + set(build_stamp_tmp "${build_stamp_tmp}-${item}") +endforeach() +# strip initial "-" using this method to avoid having to calculate string length +string(SUBSTRING ${build_stamp_tmp} 1 1024 build_stamp) + +generate_build_url1(build_url1 ${CTEST_DROP_SITE} ${URL_location} ${CTEST_PROJECT_NAME} ${CTEST_BUILD_NAME} ${build_stamp} ${machine_name}) +generate_build_url2(build_url2 ${CTEST_DROP_SITE} ${URL_location} ${CTEST_PROJECT_NAME} ${CTEST_BUILD_NAME} ${build_stamp}) +generate_build_url3(build_url3 ${CTEST_DROP_SITE} ${URL_location} ${CTEST_PROJECT_NAME} ${CTEST_BUILD_NAME} ${build_stamp}) +message(">>> CDash URL1 = ${build_url1}") +message(">>> CDash URL2 = ${build_url2}") +message(">>> CDash URL3 = ${build_url3}") + +# ----------------------------------------------------------- +# -- Optionally update the repository +# -- skip_update_step defined in ctest-common.cmake +# -- skip_by_parts_submit defined in ctest-common.cmake +# ----------------------------------------------------------- +# TODO: Do we really do this in the Trilinos CTest framework? +# Or is this all handled by the framework code around +# pulling in the source/target branches and merging them? +# If we don't use this, consider removing. +banner("Update repository START") +if(NOT skip_update_step) + message(">>> Updating repository") + ctest_update(RETURN_VALUE update_error) + if(${update_error} EQUAL -1) + message(WARNING ">>> Update failed. ") + else() + message(">>> Updated ${update_error} files.") + endif() + submit_by_parts("Update") +else() + set(CTEST_UPDATE_VERSION_ONLY ON) + ctest_update(RETURN_VALUE update_error) + if(${update_error} EQUAL -1) + message(WARNING ">>> Update failed. ") + else() + message(">>> Updated ${update_error} files.") + endif() + submit_by_parts("Update") +endif() +banner("Update repository FINISH") + +message("+--------------------------------------+") +message("| ctest-cdash-setup.cmake FINISH |") +message("+--------------------------------------+") diff --git a/cmake/SimpleTesting/cmake/ctest-common.cmake b/cmake/SimpleTesting/cmake/ctest-common.cmake new file mode 100644 index 000000000000..b000faca25d9 --- /dev/null +++ b/cmake/SimpleTesting/cmake/ctest-common.cmake @@ -0,0 +1,196 @@ +include_guard() +message("+--------------------------------------+") +message("| ctest-common.cmake START |") +message("+--------------------------------------+") + +cmake_minimum_required(VERSION 3.17.0 FATAL_ERROR) + +list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_LIST_DIR}) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-functions.cmake) + + +# ----------------------------------------------------------- +# -- Required Parameters +# ----------------------------------------------------------- +if(DEFINED build_name) + set(CTEST_BUILD_NAME ${build_name}) +else() + message(FATAL_ERROR "No build_name supplied. This is necessary to run and submit results.") +endif() + +# `subprojects_file` is a -D parameter that is passed in from Trilinos +# See: https://github.com/trilinos/Trilinos/blob/master/cmake/std/trilinosprhelpers/TrilinosPRConfigurationStandard.py +include( ${subprojects_file} OPTIONAL RESULT_VARIABLE subproject_include_success ) +if( NOT subproject_include_success ) + message(FATAL_ERROR "Subproject label file '${subprojects_file}' not found.") +else() + message("Subproject label file '${subprojects_file}' loaded.") +endif() +list(LENGTH CTEST_LABELS_FOR_SUBPROJECTS subproject_count) + +# configure_script is passed into CTest via -Dconfigure_script, points to +# Trilinos/cmake/std/ +get_filename_component(configure_file ${configure_script} ABSOLUTE) + +# package_enables is passed into CTest via -Dpackage_enables, points to the +# generated "PackageEnables.cmake file" +get_filename_component(package_enables_file ${package_enables} ABSOLUTE) + + +# ----------------------------------------------------------- +# -- Optional Parameters +# ----------------------------------------------------------- +if( NOT DEFINED ctest_submit_retry_count ) + set( ctest_submit_retry_count 5 ) +endif() + +if( NOT DEFINED ctest_submit_retry_delay ) + set( ctest_submit_retry_delay 3 ) +endif() + +if( NOT DEFINED dashboard_model ) + set( dashboard_model Experimental ) +endif() + +if( NOT DEFINED dashboard_track ) + set( dashboard_track Experimental ) +endif() + +if( NOT DEFINED skip_by_parts_submit ) + set( skip_by_parts_submit ON ) +endif() + +if( NOT DEFINED skip_clean_build_dir ) + set( skip_clean_build_dir ON ) +endif() + +if( NOT DEFINED skip_single_submit ) + set( skip_single_submit ON ) +endif() + +if( NOT DEFINED skip_update_step ) + set( skip_update_step OFF ) +endif() + +if( NOT DEFINED skip_upload_config_files ) + set( skip_upload_config_files OFF ) +endif() + + +# ----------------------------------------------------------- +# -- Miscellaneous Settings +# ----------------------------------------------------------- + +# Set the output to English +set($ENV{LC_MESSAGES} "en_EN") + +# Set the machine_name +cmake_host_system_information(RESULT machine_name QUERY HOSTNAME) +message(">>> machine_name : ${machine_name}") + + +# ----------------------------------------------------------- +# -- Set Git command +# ----------------------------------------------------------- +find_program(CTEST_GIT_COMMAND NAMES git) +set(CTEST_UPDATE_COMMAND "${CTEST_GIT_COMMAND}") + +# ----------------------------------------------------------- +# -- build specific +# ----------------------------------------------------------- + +# -- Set `CTEST_SOURCE_DIRECTORY` which would be the directory that contains the Trilinos repository. +# Note-1: CTEST_SOURCE_DIRECTORY is the path to the **this** cmake file. +# Note-2: use of `get_filename_component()` is just getting the path up 1 +# level. +if(DEFINED source_dir) + set(CTEST_SOURCE_DIRECTORY ${source_dir}) +else() + message(FATAL_ERROR "Missing required parameter: `source_dir`") +endif() +message(">>> CTEST_SOURCE_DIRECTORY = ${CTEST_SOURCE_DIRECTORY}") + + +## -- build_root +if( NOT DEFINED build_root ) + get_filename_component(bin_dir ${CTEST_SOURCE_DIRECTORY} DIRECTORY) + set(build_root "${bin_dir}/nightly_testing") +endif() + + +# -- build_dir +if(DEFINED build_dir) + message(">>> set CTEST_BINARY_DIRECTORY = ${build_dir} (A)") + set(CTEST_BINARY_DIRECTORY "${build_dir}") +else() + message(">>> set CTEST_BINARY_DIRECTORY = ${build_root}/${CTEST_BUILD_NAME} (B)") + set(CTEST_BINARY_DIRECTORY "${build_root}/${CTEST_BUILD_NAME}") +endif() +message(">>> CTEST_BINARY_DIRECTORY = ${CTEST_BINARY_DIRECTORY}") + + +# ----------------------------------------------------------- +# -- Configure parallelism +# ----------------------------------------------------------- +if( NOT DEFINED PARALLEL_LEVEL ) + cmake_host_system_information(RESULT PARALLEL_LEVEL QUERY NUMBER_OF_PHYSICAL_CORES) +endif() + +if( NOT DEFINED TEST_PARALLEL_LEVEL ) + set(TEST_PARALLEL_LEVEL "${PARALLEL_LEVEL}") +endif() + +# These are command line options to Ninja +# See: https://manpages.debian.org/testing/ninja-build/ninja.1.en.html +# -j${PARALLEL_LEVEL} - now many concurrent build jobs to allow +# -k NNN - keep going until NNN jobs fail. 0 == infinity +# -l N - might be an interesting option to reduce # of jobs based on LOAD AVERAGE +# but load average is dependent on the # of cores on a system. +set(CTEST_BUILD_FLAGS "-j${PARALLEL_LEVEL} -k 0") + + + + +# ----------------------------------------------------------- +# -- Dependent Variables +# ----------------------------------------------------------- + +# * REQUIRES `CTEST_BINARY_DIRECTORY` to be set. +if(NOT skip_upload_config_files) + set(configure_command_file ${CTEST_BINARY_DIRECTORY}/configure_command.txt) +endif() + + +# ----------------------------------------------------------- +# -- CTest Settings +# ----------------------------------------------------------- + +# Set CTEST_SITE to the name of the system. +cmake_host_system_information(RESULT HOSTNAME QUERY HOSTNAME) + +set(CTEST_SITE "${HOSTNAME}") + +# See: https://cmake.org/cmake/help/latest/command/site_name.html#command:site_name +site_name(${CTEST_SITE}) + +# This is required to make build information detailed enough to +# enable assignment to a specific subproject. +# This must be set for both cmake and ctest to work. +set(CTEST_USE_LAUNCHERS ON) +set(ENV{CTEST_USE_LAUNCHERS_DEFAULT} 1) + + +# ----------------------------------------------------------- +# -- Print out settings +# ----------------------------------------------------------- +print_options_list() + + +# ----------------------------------------------------------- +# -- Set up CDash +# ----------------------------------------------------------- +# include(${CMAKE_CURRENT_LIST_DIR}/ctest-cdash-setup.cmake) + +message("+--------------------------------------+") +message("| ctest-common.cmake FINISH |") +message("+--------------------------------------+") diff --git a/cmake/SimpleTesting/cmake/ctest-driver.cmake b/cmake/SimpleTesting/cmake/ctest-driver.cmake new file mode 100644 index 000000000000..124b3d70381d --- /dev/null +++ b/cmake/SimpleTesting/cmake/ctest-driver.cmake @@ -0,0 +1,24 @@ +message("+--------------------------------------+") +message("| ctest-driver.cmake START |") +message("+--------------------------------------+") +message("Snapshot: 2021-09-22 001") + +include(${CMAKE_CURRENT_LIST_DIR}/ctest-common.cmake) + +include(${CMAKE_CURRENT_LIST_DIR}/ctest-cdash-setup.cmake) + +include(${CMAKE_CURRENT_LIST_DIR}/ctest-stage-configure.cmake) + +include(${CMAKE_CURRENT_LIST_DIR}/ctest-stage-build.cmake) + +include(${CMAKE_CURRENT_LIST_DIR}/ctest-stage-test.cmake) + +include(${CMAKE_CURRENT_LIST_DIR}/ctest-stage-coverage.cmake) + +if(STAGE_CONFIGURE_ERROR OR STAGE_BUILD_ERROR OR STAGE_TEST_ERROR OR STAGE_COVERAGE_ERROR) + message(FATAL_ERROR "STAGE_CONFIGURE_ERROR: ${STAGE_CONFIGURE_ERROR}, STAGE_BUILD_ERROR: ${STAGE_BUILD_ERROR}, STAGE_TEST_ERROR: ${STAGE_TEST_ERROR}, STAGE_COVERAGE_ERROR: ${STAGE_COVERAGE_ERROR}") +endif() + +message("+--------------------------------------+") +message("| ctest-driver.cmake FINISH |") +message("+--------------------------------------+") \ No newline at end of file diff --git a/cmake/SimpleTesting/cmake/ctest-functions.cmake b/cmake/SimpleTesting/cmake/ctest-functions.cmake new file mode 100644 index 000000000000..df53579c3b3f --- /dev/null +++ b/cmake/SimpleTesting/cmake/ctest-functions.cmake @@ -0,0 +1,192 @@ +include_guard() +message("+--------------------------------------+") +message("| ctest-functions.cmake START |") +message("+--------------------------------------+") + + + +macro(banner arg_banner_message) + message("+----------------------------------------------------------+") + message("+ ${arg_banner_message}") + message("+----------------------------------------------------------+") +endmacro() + + + +macro(submit_by_parts arg_parts_value) + banner("submit_by_parts() START") + message(">>> arg_parts_value: ${arg_parts_value}") + message(">>> CTEST_DROP_METHOD : ${CTEST_DROP_METHOD}") + message(">>> CTEST_DROP_LOCATION : ${CTEST_DROP_LOCATION}") + message(">>> CDash URL1 = ${build_url1}") + message(">>> CDash URL2 = ${build_url2}") + message(">>> CDash URL3 = ${build_url3}") + if(NOT skip_by_parts_submit) + message(">>> ctest_submit(PARTS ${arg_parts_value}") + message(" RETRY_COUNT ${ctest_submit_retry_count}") + message(" RETRY_DELAY ${ctest_submit_retry_delay}") + message(" RETURN_VALUE ctest_submit_error)") + + # https://cmake.org/cmake/help/latest/command/ctest_submit.html + ctest_submit(PARTS ${arg_parts_value} + RETRY_COUNT ${ctest_submit_retry_count} + RETRY_DELAY ${ctest_submit_retry_delay} + BUILD_ID cdash_build_id + RETURN_VALUE ctest_submit_error) + + if(ctest_submit_error EQUAL 0) + message(">>> ${arg_parts_value} submit: OK") + message(">>> CDash Build-ID : ${cdash_build_id}") + else() + message(">>> ${arg_parts_value} submit: FAILED") + message(">>> - The ERROR code is ${ctest_submit_error}") + endif() + else() + message(">>> SKIPPED") + message(">>> skip_by_parts_submit : ${skip_by_parts_submit}") + endif() + banner("submit_by_parts() FINISH") +endmacro() + + + +macro(submit_single_submit) + banner("submit_single_submit() START") + if(NOT skip_single_submit) + message(">>> ctest_submit(RETRY_COUNT ${ctest_submit_retry_count}") + message(" RETRY_DELAY ${ctest_submit_retry_delay}") + message(" RETURN_VALUE error_code)") + + ctest_submit(RETRY_COUNT ${ctest_submit_retry_count} + RETRY_DELAY ${ctest_submit_retry_delay} + RETURN_VALUE error_code) + + if(error_code EQUAL 0) + message(">>> Single submit: OK") + else() + message(">>> Single submit: FAILED") + message(">>> - The ERROR code is ${error_code}") + endif() + else() + message(">>> SKIPPED") + message(">>> skip_single_submit : ${skip_single_submit}") + endif() + banner("submit_single_submit() FINISH") +endmacro() + + + +macro(submit_upload_config_files) + banner("submit_upload_config_files() START") + if( NOT skip_upload_config_files ) + if( NOT (skip_single_submit AND skip_by_parts_submit) ) + message(">>> ctest_upload(FILES ${configure_command_file}") + message(" ${configure_file}") + message(" ${package_enables_file} )") + ctest_upload(FILES ${configure_command_file} ${configure_file} ${package_enables_file}) + message(">>> ctest_submit(PARTS upload") + message(" RETRY_COUNT ${ctest_submit_retry_count}") + message(" RETRY_DELAY ${ctest_submit_retry_delay}") + message(" RETURN_VALUE file_upload_erorr)") + + ctest_submit(PARTS Upload + RETRY_COUNT ${ctest_submit_retry_count} + RETRY_DELAY ${ctest_submit_retry_delay} + RETURN_VALUE file_upload_error) + + if(file_upload_error EQUAL 0) + message(">>> Config Files Upload: OK") + else() + message(">>> Config Files Upload: FAILED") + message(">>> - The ERROR code is ${file_upload_error}") + endif() + else() + message(">>> SKIPPED") + message(">>> skip_single_submit : ${skip_single_submit}") + message(">>> skip_by_parts_submit : ${skip_by_parts_submit}") + endif() + else() + message(">>> SKIPPED") + message(">>> skip_upload_config_files : ${skip_upload_config_files}") + endif() + banner("submit_upload_config_files() FINISH") +endmacro() + + + +macro(print_options_list) + banner("OPTIONS") + message(">>> CTEST_BUILD_NAME = ${CTEST_BUILD_NAME}") + message(">>> PARALLEL_LEVEL = ${PARALLEL_LEVEL}") + message(">>> TEST_PARALLEL_LEVEL = ${TEST_PARALLEL_LEVEL}") + message(">>> skip_by_parts_submit = ${skip_by_parts_submit}") + message(">>> skip_single_submit = ${skip_single_submit}") + message(">>> skip_update_step = ${skip_update_step}") + message(">>> skip_upload_config_files = ${skip_upload_config_files}") + message(">>> skip_clean_build_dir = ${skip_clean_build_dir}") + message(">>> subproject_count = ${subproject_count}") + message(">>> dashboard_model = ${dashboard_model}") + message(">>> dashboard_track = ${dashboard_track}") + message(">>> configure_command_file = ${configure_command_file}") + message(">>> configure_file = ${configure_file}") + message(">>> build_root = ${build_root}") + message(">>> build_dir = ${build_dir}") + message(">>> SOURCE_DIR = ${SOURCE_DIR}") + message(">>> CMAKE_CURRENT_LIST_DIR = ${CMAKE_CURRENT_LIST_DIR}") + message(">>> CTEST_SOURCE_DIRECTORY = ${CTEST_SOURCE_DIRECTORY}") + message(">>> CTEST_BINARY_DIRECTORY = ${CTEST_BINARY_DIRECTORY}") +endmacro() + + + +function(generate_build_url1 url_output cdash_site cdash_location project_name build_name build_stamp machine_name) + banner("generate_build_url1() START") + message(">>> cdash_site : ${cdash_site}") + message(">>> cdash_location: ${cdash_location}") + message(">>> project_name : ${project_name}") + message(">>> build_name : ${build_name}") + message(">>> build_stamp : ${build_stamp}") + message(">>> machine_name : ${machine_name}") + string(REPLACE " " "%20" url_output_tmp + "https://${cdash_site}${cdash_location}index.php?project=${project_name}&display=project&filtercount=3&showfilters=1&filtercombine=and&field1=site&compare1=61&value1=${machine_name}&field2=buildname&compare2=61&value2=${build_name}&field3=buildstamp&compare3=61&value3=${build_stamp}" + ) + set(${url_output} ${url_output_tmp} PARENT_SCOPE) + banner("generate_build_url1() FINISH") +endfunction() + + + +function(generate_build_url2 url_output cdash_site cdash_location project_name build_name build_stamp) + banner("generate_build_url2() START") + message(">>> cdash_site : ${cdash_site}") + message(">>> cdash_location: ${cdash_location}") + message(">>> project_name : ${project_name}") + message(">>> build_name : ${build_name}") + message(">>> build_stamp : ${build_stamp}") + string(REPLACE " " "%20" url_output_tmp + "https://${cdash_site}${cdash_location}index.php?project=${project_name}&display=project&filtercount=2&showfilters=0&filtercombine=and&field1=buildname&compare1=61&value1=${build_name}&field2=buildstamp&compare2=61&value2=${build_stamp}" + ) + set(${url_output} ${url_output_tmp} PARENT_SCOPE) + banner("generate_build_url2() FINISH") +endfunction() + + + +function(generate_build_url3 url_output cdash_site cdash_location project_name build_name build_stamp) + banner("generate_build_url2() START") + message(">>> cdash_site : ${cdash_site}") + message(">>> cdash_location: ${cdash_location}") + message(">>> project_name : ${project_name}") + message(">>> build_name : ${build_name}") + message(">>> build_stamp : ${build_stamp}") + string(REPLACE " " "%20" url_output_tmp + "https://${cdash_site}${cdash_location}index.php?project=${project_name}&filtercount=2&showfilters=0&filtercombine=and&field1=buildname&compare1=61&value1=${build_name}&field2=buildstamp&compare2=61&value2=${build_stamp}" + ) + set(${url_output} ${url_output_tmp} PARENT_SCOPE) + banner("generate_build_url3() FINISH") +endfunction() + + +message("+--------------------------------------+") +message("| ctest-functions.cmake FINISH |") +message("+--------------------------------------+") diff --git a/cmake/SimpleTesting/cmake/ctest-stage-build.cmake b/cmake/SimpleTesting/cmake/ctest-stage-build.cmake new file mode 100644 index 000000000000..c96d9a53e59f --- /dev/null +++ b/cmake/SimpleTesting/cmake/ctest-stage-build.cmake @@ -0,0 +1,32 @@ +message("+--------------------------------------+") +message("| ctest-stage-build.cmake START |") +message("+--------------------------------------+") +set(STAGE_BUILD_ERROR OFF) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-functions.cmake) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-common.cmake) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-cdash-setup.cmake) + + +# ----------------------------------------------------------- +# -- Build +# ----------------------------------------------------------- +banner("START build step") + +# Execute the build step +ctest_build(RETURN_VALUE build_error) + +# Print out final stage banner +if(${build_error} EQUAL 0) + banner("END build step - SUCCESS") +else() + message(WARNING "Build failed with error ${build_error}") + banner("END build step - FAILURE") + set(STAGE_BUILD_ERROR ON) +endif() + +# Submit to CDash +submit_by_parts( "Build" ) + +message("+--------------------------------------+") +message("| ctest-stage-build.cmake FINISH |") +message("+--------------------------------------+") diff --git a/cmake/SimpleTesting/cmake/ctest-stage-configure.cmake b/cmake/SimpleTesting/cmake/ctest-stage-configure.cmake new file mode 100644 index 000000000000..82a8e650178d --- /dev/null +++ b/cmake/SimpleTesting/cmake/ctest-stage-configure.cmake @@ -0,0 +1,47 @@ +message("+--------------------------------------+") +message("| ctest-stage-configure.cmake START |") +message("+--------------------------------------+") +set(STAGE_CONFIGURE_ERROR OFF) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-functions.cmake) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-common.cmake) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-cdash-setup.cmake) + +# ----------------------------------------------------------- +# -- Configure +# ----------------------------------------------------------- +banner("START configure step") + +message(">>> CTEST_SOURCE_DIRECTORY: ${CTEST_SOURCE_DIRECTORY}") +message(">>> CTEST_BINARY_DIRECTORY: ${CTEST_BINARY_DIRECTORY}") + +ctest_configure(SOURCE ${CTEST_SOURCE_DIRECTORY} + BUILD ${CTEST_BINARY_DIRECTORY} + RETURN_VALUE configure_error + CAPTURE_CMAKE_ERROR tmp_cmake_error + ) + +ctest_read_custom_files(BUILD "${CTEST_BINARY_DIRECTORY}") +# NOTE: The CTestCustom.cmake file read in by the above command is +# automatically written in the binary dir by the configure of Trilinos in +# ctest_configure() above. + +message(">>> configure_error: ${configure_error}") +message(">>> tmp_cmake_error: ${tmp_cmake_error}") # NEW (testing this out) + +if( (${configure_error} EQUAL 0) AND (${tmp_cmake_error} EQUAL 0) ) + banner("END configure step - SUCCESS") +else() + banner("END configure step - FAILURE") + set(STAGE_CONFIGURE_ERROR ON) +endif() + +submit_by_parts("Configure") + +if((NOT ${configure_error} EQUAL 0) OR (NOT ${tmp_cmake_error} EQUAL 0)) + submit_single_submit() + message(WARNING "Configure failed with error ${configure_error}") +endif() + +message("+--------------------------------------+") +message("| ctest-stage-configure.cmake FINISH |") +message("+--------------------------------------+") diff --git a/cmake/SimpleTesting/cmake/ctest-stage-coverage.cmake b/cmake/SimpleTesting/cmake/ctest-stage-coverage.cmake new file mode 100644 index 000000000000..64826418b5be --- /dev/null +++ b/cmake/SimpleTesting/cmake/ctest-stage-coverage.cmake @@ -0,0 +1,78 @@ +message("+----------------------------------+") +message("| ctest-stage-coverage.cmake START |") +message("+----------------------------------+") +include(${CMAKE_CURRENT_LIST_DIR}/ctest-functions.cmake) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-common.cmake) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-cdash-setup.cmake) +include(CTestCoverageCollectGCOV) + + +# ----------------------------------------------------------- +# -- Coverage +# ----------------------------------------------------------- +set(STAGE_COVERAGE_ERROR OFF) + +# Rather than attempt to parse this out via GenConfig and +# pass it through the PR driver scripts, use a substring +# match here. +if(CTEST_BUILD_NAME MATCHES ".*coverage.*") + banner("START coverage step") + + find_program(GCOV_EXECUTABLE + NAMES gcov + DOC "Path to gcov executable") + + + # Execute the coverage parsing step + ctest_coverage_collect_gcov(TARBALL "${CTEST_BUILD_NAME}-gcov.tar.gz" + GCOV_COMMAND ${GCOV_EXECUTABLE} + GCOV_OPTIONS --branch-probabilities --hash-filenames --demangled-names --human-readable --display-progress + GLOB ON) + + + #find_program(GCOVR_EXECUTABLE + # NAMES gcovr + # DOC "Path to gcovr executable") + + # TODO: Try using gcovr + #execute_process(COMMAND ${GCOVR_EXECUTABLE} --json --output ${CTEST_BUILD_NAME}-gcovr-report.json --exclude-throw-branches --exlude-unreachable-branches -j 29 --gcov-executable ${GCOV_EXECUTABLE} --root ${CTEST_SOURCE_DIRECTORY} ${CTEST_BINARY_DIRECTORY} + # COMMAND rm -f ${CTEST_BUILD_NAME}-gcovr-report.tar.gz + # COMMAND tar -czf ${CTEST_BUILD_NAME}-gcovr-report.tar.gz ${CTEST_BUILD_NAME}-gcovr-report.json) + + # Print out final stage banner + banner("END coverage step") + + banner("ctest_upload() START") + + ctest_submit(CDASH_UPLOAD ${CTEST_BUILD_NAME}-gcov.tar.gz + CDASH_UPLOAD_TYPE GcovTar + RETRY_COUNT ${ctest_submit_retry_count} + RETRY_DELAY ${ctest_submit_retry_delay} + RETURN_VALUE coverage_upload_error) + + if(coverage_upload_error EQUAL 0) + message(">>> Coverage Upload: OK") + else() + message(">>> Coverage Upload: FAILED") + message(">>> - The ERROR code is ${coverage_upload_error}") + set(STAGE_COVERAGE_ERROR ON) + endif() + + banner("ctest_upload() FINISH") + + # TODO: Check for coverage percentage drop? + #if( NOT (coverage_error EQUAL 0) ) + # message(WARNING "Coverage has decreased from X to Y.") + # set(STAGE_COVERAGE_ERROR ON) + #else() + # message("Coverage passed.") + #endif() + + message("+-----------------------------------+") + message("| ctest-stage-coverage.cmake FINISH |") + message("+-----------------------------------+") +else() + message("+------------------------------------+") + message("| ctest-stage-coverage.cmake SKIPPED |") + message("+------------------------------------+") +endif() diff --git a/cmake/SimpleTesting/cmake/ctest-stage-test.cmake b/cmake/SimpleTesting/cmake/ctest-stage-test.cmake new file mode 100644 index 000000000000..76ec3a0656f3 --- /dev/null +++ b/cmake/SimpleTesting/cmake/ctest-stage-test.cmake @@ -0,0 +1,66 @@ +message("+--------------------------------------+") +message("| ctest-stage-test.cmake START |") +message("+--------------------------------------+") +include(${CMAKE_CURRENT_LIST_DIR}/ctest-functions.cmake) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-common.cmake) +include(${CMAKE_CURRENT_LIST_DIR}/ctest-cdash-setup.cmake) + + +# ----------------------------------------------------------- +# -- Test +# ----------------------------------------------------------- +banner("START test step") + +set(STAGE_TEST_ERROR OFF) + +if(CTEST_BUILD_NAME MATCHES .*_asan_.*) + set(CTEST_MEMORYCHECK_TYPE "AddressSanitizer") + set(ENV{LSAN_OPTIONS} "suppressions=${CTEST_SOURCE_DIRECTORY}/packages/framework/asan_assets/lsan.supp") + set(ENV{LD_PRELOAD} ${CTEST_SOURCE_DIRECTORY}/packages/framework/asan_assets/dummy_dlclose.so) + ctest_memcheck(PARALLEL_LEVEL ${TEST_PARALLEL_LEVEL} + CAPTURE_CMAKE_ERROR captured_cmake_error + RETURN_VALUE test_error) + unset(ENV{LD_PRELOAD}) + submit_by_parts( "MemCheck" ) +else() + ctest_test(PARALLEL_LEVEL ${TEST_PARALLEL_LEVEL} + CAPTURE_CMAKE_ERROR captured_cmake_error + RETURN_VALUE test_error) + submit_by_parts( "Test" ) +endif() + +# Print out final stage banner +if(${test_error} EQUAL 0) + banner("END test step - SUCCESS") +else() + banner("END test step - FAILURE") +endif() + +# Single submit +submit_single_submit() + +# Upload configure files +submit_upload_config_files() + +# Display what return values were captured. +message(">>> captured_cmake_error = ${captured_cmake_error} (unused)") +message(">>> test_error = ${test_error}") + +# We fail the test here if the `test_error` value from ctest_test is +# nonzero. +# We should NOT check `captured_cmake_error` (CAPTURE_CMAKE_ERROR) +# to determine if our test has failed since that value contain a +# nonzero value if there were _no tests to run_. For Trilinos, this +# can happen since we dynamically enable tests based on changes in +# a Pull-Request. +if( NOT (test_error EQUAL 0) ) + message(WARNING "There are errors detected in the test.") + set(STAGE_TEST_ERROR ON) +else() + message("Test(s) passed.") +endif() + +message("+--------------------------------------+") +message("| ctest-stage-test.cmake FINISH |") +message("+--------------------------------------+") + diff --git a/cmake/SimpleTesting/cmake/ctest-submit.cmake b/cmake/SimpleTesting/cmake/ctest-submit.cmake new file mode 100644 index 000000000000..a9cb232e8eb0 --- /dev/null +++ b/cmake/SimpleTesting/cmake/ctest-submit.cmake @@ -0,0 +1,20 @@ +# A utility driver for submitting partial results to CDash +# when an exception of failure occurs. +message("+--------------------------------------+") +message("| ctest-submit.cmake START |") +message("+--------------------------------------+") +message("Snapshot: 2021-09-22 001") + +include(${CMAKE_CURRENT_LIST_DIR}/ctest-common.cmake) + +include(${CMAKE_CURRENT_LIST_DIR}/ctest-cdash-setup.cmake) + +message(" -- Submit - ${CTEST_BUILD_NAME} --") + +ctest_start(APPEND) + +ctest_submit(RETURN_VALUE res) + +message("+--------------------------------------+") +message("| ctest-submit.cmake FINISH |") +message("+--------------------------------------+") From 445e909e2fbb97c536c049a144f53d1374feb9f9 Mon Sep 17 00:00:00 2001 From: Caleb Jackson Date: Sat, 26 Aug 2023 17:03:54 -0600 Subject: [PATCH 2/7] WIP:update pr-ctest-framework to SimpleTesting --- packages/framework/pr_tools/PullRequestLinuxDriver.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/framework/pr_tools/PullRequestLinuxDriver.sh b/packages/framework/pr_tools/PullRequestLinuxDriver.sh index 1eb949e73332..b1b2d2a43b9c 100755 --- a/packages/framework/pr_tools/PullRequestLinuxDriver.sh +++ b/packages/framework/pr_tools/PullRequestLinuxDriver.sh @@ -194,7 +194,7 @@ test_cmd_options=( --filename-subprojects=${WORKSPACE:?}/package_subproject_list.cmake --source-dir=${WORKSPACE}/Trilinos --build-dir=${TRILINOS_BUILD_DIR:?} - --ctest-driver=${WORKSPACE:?}/pr-ctest-framework/cmake/ctest-driver.cmake + --ctest-driver=${WORKSPACE:?}/SimpleTesting/cmake/ctest-driver.cmake --ctest-drop-site=${TRILINOS_CTEST_DROP_SITE:?} #--dry-run ) From 0c0025ac836d39f5e24e3734480d1902c57eb37e Mon Sep 17 00:00:00 2001 From: Caleb Jackson Date: Tue, 29 Aug 2023 10:24:20 -0600 Subject: [PATCH 3/7] Update to cmake/SimpleTesting --- packages/framework/pr_tools/PullRequestLinuxDriver.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/framework/pr_tools/PullRequestLinuxDriver.sh b/packages/framework/pr_tools/PullRequestLinuxDriver.sh index b1b2d2a43b9c..ad07fc74bd77 100755 --- a/packages/framework/pr_tools/PullRequestLinuxDriver.sh +++ b/packages/framework/pr_tools/PullRequestLinuxDriver.sh @@ -194,7 +194,7 @@ test_cmd_options=( --filename-subprojects=${WORKSPACE:?}/package_subproject_list.cmake --source-dir=${WORKSPACE}/Trilinos --build-dir=${TRILINOS_BUILD_DIR:?} - --ctest-driver=${WORKSPACE:?}/SimpleTesting/cmake/ctest-driver.cmake + --ctest-driver=${WORKSPACE:?}/cmake/SimpleTesting/cmake/ctest-driver.cmake --ctest-drop-site=${TRILINOS_CTEST_DROP_SITE:?} #--dry-run ) From b2c42120adc2a00e7ed85d04460747714d3723c0 Mon Sep 17 00:00:00 2001 From: Caleb Jackson Date: Tue, 29 Aug 2023 11:08:47 -0600 Subject: [PATCH 4/7] Update comments --- cmake/SimpleTesting/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/cmake/SimpleTesting/README.md b/cmake/SimpleTesting/README.md index 3149f43cba80..0212ac569332 100644 --- a/cmake/SimpleTesting/README.md +++ b/cmake/SimpleTesting/README.md @@ -15,6 +15,7 @@ interact with each other. CMake Structure Diagram + CMake Files ----------- From bf235f60d03da9731c202d6f808b471789e79200 Mon Sep 17 00:00:00 2001 From: Caleb Jackson Date: Tue, 29 Aug 2023 11:37:09 -0600 Subject: [PATCH 5/7] Update path name --- packages/framework/pr_tools/PullRequestLinuxDriver.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/framework/pr_tools/PullRequestLinuxDriver.sh b/packages/framework/pr_tools/PullRequestLinuxDriver.sh index ad07fc74bd77..477d3a9cc0b4 100755 --- a/packages/framework/pr_tools/PullRequestLinuxDriver.sh +++ b/packages/framework/pr_tools/PullRequestLinuxDriver.sh @@ -194,7 +194,7 @@ test_cmd_options=( --filename-subprojects=${WORKSPACE:?}/package_subproject_list.cmake --source-dir=${WORKSPACE}/Trilinos --build-dir=${TRILINOS_BUILD_DIR:?} - --ctest-driver=${WORKSPACE:?}/cmake/SimpleTesting/cmake/ctest-driver.cmake + --ctest-driver=${WORKSPACE:?}/Trilinos/cmake/SimpleTesting/cmake/ctest-driver.cmake --ctest-drop-site=${TRILINOS_CTEST_DROP_SITE:?} #--dry-run ) From 68b575e3675105353d9a3ab21e248bdcbe48e64b Mon Sep 17 00:00:00 2001 From: Caleb Jackson Date: Wed, 30 Aug 2023 15:41:02 -0600 Subject: [PATCH 6/7] Update documentation/remove unecessary comments --- cmake/SimpleTesting/README.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/cmake/SimpleTesting/README.md b/cmake/SimpleTesting/README.md index 0212ac569332..bb73a29d970c 100644 --- a/cmake/SimpleTesting/README.md +++ b/cmake/SimpleTesting/README.md @@ -15,8 +15,6 @@ interact with each other. CMake Structure Diagram - - CMake Files ----------- @@ -31,7 +29,7 @@ CMake Files | `ctest-functions.cmake` | X | Functions and macros | | `ctest-CTestConfig.cmake` | | Required file that is copied into the build directory at configure time. | -The _guarded_ files use the CMake command [`include_guard()`][3] which should prevent that file +The _guarded_ files use the CMake command [`include_guard()`][1] which should prevent that file from being include more than once in an include chain. Options and Variables (`simple_testing.cmake`) @@ -62,7 +60,7 @@ CMake system via `-D:BOOL=` parameters: 1. `configure_script` points to the `cmake/std/PullRequestLinuxTestingSettings.cmake` file. - Example: `${WORKSPACE}/Trilinos/cmake/std/PullRequestLinuxGCC8.3.0TestingSettings.cmake` -See `TrilinosPRConfigurationStandard.py` for information on what options are set to something +See `TrilinosPRConfigurationStandard.py`[4] for information on what options are set to something other than the default during normal Trilinos PR operations. Expected Operation @@ -137,4 +135,5 @@ This allows some PR's to go through without building Trilinos, which is advantag when only documentation or perhaps the testing framework itself is modified and we do not need to spend O(5 hours) for the test suite to run. -[3]: https://cmake.org/cmake/help/latest/command/include_guard.html +[1]: https://cmake.org/cmake/help/latest/command/include_guard.html +[2]: https://github.com/trilinos/Trilinos/blob/master/cmake/std/trilinosprhelpers/TrilinosPRConfigurationStandard.py From 7ade832dd79d9e852b9b521208460aa383a8ea3d Mon Sep 17 00:00:00 2001 From: Caleb Jackson Date: Wed, 30 Aug 2023 15:55:12 -0600 Subject: [PATCH 7/7] Fix typo --- cmake/SimpleTesting/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmake/SimpleTesting/README.md b/cmake/SimpleTesting/README.md index bb73a29d970c..948ce7ab403f 100644 --- a/cmake/SimpleTesting/README.md +++ b/cmake/SimpleTesting/README.md @@ -60,7 +60,7 @@ CMake system via `-D:BOOL=` parameters: 1. `configure_script` points to the `cmake/std/PullRequestLinuxTestingSettings.cmake` file. - Example: `${WORKSPACE}/Trilinos/cmake/std/PullRequestLinuxGCC8.3.0TestingSettings.cmake` -See `TrilinosPRConfigurationStandard.py`[4] for information on what options are set to something +See `TrilinosPRConfigurationStandard.py`[2] for information on what options are set to something other than the default during normal Trilinos PR operations. Expected Operation