cmake: Add ARMPL TPL support #880
Conversation
| @@ -660,10 +660,10 @@ elif [ "$MACHINE" = "blake" ]; then | |||
| SKIP_HWLOC=True | |||
| export SLURM_TASKS_PER_NODE=32 | |||
|
|
|||
| module load cmake/3.12.3 | |||
| module load cmake/3.19.3 | |||
There was a problem hiding this comment.
Thanks for updating this, just got the email this was updated on blake (I modified the nightlies to use the cm_generate_makefile and installed the cmake binaries, we can revert that revert that after this goes in)
scripts/cm_test_all_sandia
Outdated
| COMPILERS=("intel/18.1.163 $BASE_MODULE_LIST_INTEL $INTEL_BUILD_LIST icpc $INTEL_WARNING_FLAGS" | ||
| "gcc/7.2.0 $BASE_MODULE_LIST $GCC_BUILD_LIST g++ $GCC_WARNING_FLAGS" | ||
| COMPILERS=("intel/18.1.163 $BASE_MODULE_LIST_INTEL "OpenMP,Pthread" icpc $INTEL_WARNING_FLAGS" | ||
| "gcc/7.2.0 $BASE_MODULE_LIST "OpenMP_Serial" g++ $GCC_WARNING_FLAGS" |
There was a problem hiding this comment.
This test is failing with the autotester. When I worked on this part of the script initially the blas and lapack were available on the system path, but it's possible that has since changed with system updates. If so, I can help tomorrow to update the script to use a module and set the cmake options to point to the libs etc.
There was a problem hiding this comment.
@e10harvey one more change after adding the openblas module to the module list, change this line (around 1079, will vary with your changes) from
kokkos-kernels/scripts/cm_test_all_sandia
Line 1079 in 98e1c20
to
if ([[ "$MACHINE" = white* ]] || [[ "$MACHINE" = weaver* ]] || [[ "$MACHINE" = blake* ]]) && [[ "$mod" = openblas* ]]; then
This will make sure the openblas tpl is found, as well as provide extra flags that are typically needed (-lgfortran, -lm) to prevent linkage errors when using blas and lapack that are not on a system path
| @@ -0,0 +1,32 @@ | |||
| IF (ARMPL_LIBRARY_DIRS AND ARMPL_LIBRARIES) | |||
There was a problem hiding this comment.
These changes look reasonable, could you post any configure and test output from a system where you tested this? (I'm guessing maybe this was on mayer)? If this is being developed on mayer we can set up a nightly test with these tpls once this goes in
There was a problem hiding this comment.
Thanks for your review, @ndellingwood !
Expected configure fail
$ ../cm_generate_makefile.bash --with-tpls=armpl --with-scalars=float --arch=A64FX --with-serial --with-openmp --kokkos-path=$HOME/KOKKOS.base/kokkos --kokkos-prefix=$HOME/KOKKOS.base/kokkos/install -kokkoskernels-path=$HOME/KOKKOS.base/kokkos-kernels --prefix=$HOME/KOKKOS.base/kokkos-kernels/install
<snip>
CMake Error at cmake/Modules/FindTPLARMPL.cmake:21 (FIND_PACKAGE):
By not providing "FindARMPL.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "ARMPL", but
CMake did not find one.
Could not find a package configuration file provided by "ARMPL" with any of
the following names:
ARMPLConfig.cmake
armpl-config.cmake
Add the installation prefix of "ARMPL" to CMAKE_PREFIX_PATH or set
"ARMPL_DIR" to a directory containing one of the above files. If "ARMPL"
provides a separate development package or SDK, be sure it has been
installed.
Call Stack (most recent call first):
cmake/kokkoskernels_tpls.cmake:363 (FIND_PACKAGE)
cmake/kokkoskernels_tpls.cmake:485 (KOKKOSKERNELS_IMPORT_TPL)
CMakeLists.txt:175 (INCLUDE)
-- Configuring incomplete, errors occurred!Expected configure pass
$ rm -rf CMake*
$ module load armpl/20.3.0
$ ../cm_generate_makefile.bash --with-tpls=armpl --with-scalars=float --arch=A64FX --with-serial --with-openmp --kokkos-path=$HOME/KOKKOS.base/kokkos --kokkos-prefix=$HOME/KOKKOS.base/kokkos/install -kokkoskernels-path=$HOME/KOKKOS.base/kokkos-kernels --prefix=$HOME/KOKKOS.base/kokkos-kernels/install
<snip>
-- Found Kokkos at /path/to/kokkos/install/lib64/cmake/Kokkos
-- Found TPLARMPL: /path/to/include
=======================
KokkosKernels ETI Types
Devices: <OpenMP,HostSpace>;<Serial,HostSpace>
Scalars: double;float
Ordinals: int
Offsets: int;size_t
Layouts: LayoutLeft
KokkosKernels TPLs
ARMPL: /path/to/libamath.so;/path/to/libarmpl.so
=======================Test suite results
$ make -j8
$ # TODOLet's discuss setting up nightly tests with this tpl at the next meeting.
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL18
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA9
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720_GCC740
Jenkins Parameters
Using Repos:
Pull Request Author: e10harvey |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL18
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA9
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720_GCC740
Jenkins Parameters
Console Output (last 100 lines) : KokkosKernels_PullRequest_GCC720 # 22 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_Tpls_GCC720 # 15 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_Tpls_INTEL18 # 1 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_GCC720_Light # 43 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_Tpls_CUDA10 # 7 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_Tpls_CUDA9 # 1 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_Tpls_GCC720_GCC740 # 1 (click to expand) |
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL18
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA9
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720_GCC740
Jenkins Parameters
Using Repos:
Pull Request Author: e10harvey |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL18
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA9
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720_GCC740
Jenkins Parameters
|
|
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
There was a problem hiding this comment.
lgtm, thanks @e10harvey ! Let me know if this is ready for merge
|
Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ ndellingwood ]! |
|
Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - This Repo does not support Automerge |
Related to #874.