-
Notifications
You must be signed in to change notification settings - Fork 24
Sequoia BG Q
Those MPI enabled versions of CloverLeaf enable the MPI via a "use mpi". Doing this from XL Fortran on Sequoia and building with the default compiler wrappers, will encounter this error:
XL Fortran90 error: Module symbol file for module mpi is in a format not recognized by this compiler===
The cause of the error is that the mpi.mod referenced in the "default" software stack (/bgsys/drivers/ppcfloor/comm/default) is built with GNU Fortran.
MPI built using the XL compilers instead of the GNU compilers, thus containing a Fortran mpi module (mpi.mod) generated by XLF is avaiable from a non-default location. This software stack was introduced in V1R4M1. You may build with it using the wrappers in comm/xl:
/bgsys/drivers/ppcfloor/comm/xl/bin/mpixlf90
OpenMP Compilation Needs Thread Safe Versions of the XL Compilers
So to build CloverLeaf on Sequoia issue the following command:
make COMPILER=XL MPI_COMPILER=/bgsys/drivers/ppcfloor/comm/xl/bin/mpixlf90_r C_MPI_COMPILER=mpixlc_r
Current compiler versions: /bgsys/drivers/ppcfloor/comm/xl/bin/mpixlf90_r -qversion
IBM XL Fortran for Blue Gene, V14.1 Version: 14.01.0000.0001
mpixlc_r -qversion
IBM XL C/C++ for Blue Gene, V12.1 Version: 12.01.0000.0001
SLURM on here - different scripts needed to uDawn
Results
bm_short
1_64 (default on 1 node)
lover Version 1.000 MPI Version OpenMP Version Task Count 1 Thread Count: 64
Output file clover.out opened. All output will go there.
Clover Version 1.000 MPI Version OpenMP Version Task Count 1 Thread Count: 64
Clover will run from the following input:-
*clover Step 87 time 0.4971244 control sound timestep 5.85E-03 1, 1 x 5.21E-03 y 5.21E-03
Time 0.502972356941437826 Volume Mass Density Pressure Internal Energy Kinetic Energy Total Energy step: 87 0.1000E+03 0.2800E+02 0.2800E+00 0.1671E+00 0.4179E+02 0.1193E+01 0.4298E+02
Calculation complete Clover is finishing Wall clock 6.43371701240539551
Clover Version 1.000 MPI Version OpenMP Version Task Count 4 Thread Count: 16
Output file clover.out opened. All output will go there.
Clover Version 1.000 MPI Version OpenMP Version Task Count 4 Thread Count: 16
Clover will run from the following input:-
*clover Step 87 time 0.4971244 control sound timestep 5.85E-03 1, 1 x 5.21E-03 y 5.21E-03
Time 0.502972356941437826 Volume Mass Density Pressure Internal Energy Kinetic Energy Total Energy step: 87 0.1000E+03 0.2800E+02 0.2800E+00 0.1671E+00 0.4179E+02 0.1193E+01 0.4298E+02
Calculation complete Clover is finishing Wall clock 6.43540716171264648 Done Fri Feb 1 07:34:20 PST 2013
Clover Version 1.000 MPI Version OpenMP Version Task Count 64 Thread Count: 1
Output file clover.out opened. All output will go there.
Clover Version 1.000 MPI Version OpenMP Version Task Count 64 Thread Count: 1
Clover will run from the following input:-
*clover Step 87 time 0.4971244 control sound timestep 5.85E-03 1, 1 x 5.21E-03 y 5.21E-03
Time 0.502972356941437826 Volume Mass Density Pressure Internal Energy Kinetic Energy Total Energy step: 87 0.1000E+03 0.2800E+02 0.2800E+00 0.1671E+00 0.4179E+02 0.1193E+01 0.4298E+02
Calculation complete Clover is finishing Wall clock 6.72289299964904785 Done
Clover Version 1.000 MPI Version OpenMP Version Task Count 16 Thread Count: 4
Output file clover.out opened. All output will go there.
Clover Version 1.000 MPI Version OpenMP Version Task Count 16 Thread Count: 4
Clover will run from the following input:-
*clover
Step 87 time 0.4971244 control sound timestep 5.85E-03 1, 1 x 5.21E-03 y 5.21E-03
Time 0.502972356941437715 Volume Mass Density Pressure Internal Energy Kinetic Energy Total Energy step: 87 0.1000E+03 0.2800E+02 0.2800E+00 0.1671E+00 0.4179E+02 0.1193E+01 0.4298E+02
Calculation complete Clover is finishing Wall clock 6.45956182479858398 Done Fri Feb 1 07:45:24 PST 2013
Clover Version 1.000 MPI Version OpenMP Version Task Count 16 Thread Count: 1
Output file clover.out opened. All output will go there. 1
Clover Version 1.000 MPI Version OpenMP Version Task Count 16 Thread Count: 1
Clover will run from the following input:-
*clover
Step 87 time 0.4971244 control sound timestep 5.85E-03 1, 1 x 5.21E-03 y 5.21E-03
Time 0.502972356941437826 Volume Mass Density Pressure Internal Energy Kinetic Energy Total Energy step: 87 0.1000E+03 0.2800E+02 0.2800E+00 0.1671E+00 0.4179E+02 0.1193E+01 0.4298E+02
Calculation complete Clover is finishing Wall clock 10.2737510204315186 Done Fri Feb 1 08:40:01 PST 2013
bm
script: 8_64_1.cmd
Clover Version 1.000 MPI Version OpenMP Version Task Count 512 Thread Count: 1
Output file clover.out opened. All output will go there.
Clover Version 1.000 MPI Version OpenMP Version Task Count 512 Thread Count: 1
Clover will run from the following input:-
*clover
Step 2952 time 15.4956432 control sound timestep 5.11E-03 1, 1 x 5.21E-03 y 5.21E-03
Time 15.5007538630877200 Volume Mass Density Pressure Internal Energy Kinetic Energy Total Energy step: 2952 0.1000E+03 0.2800E+02 0.2800E+00 0.1604E+00 0.4009E+02 0.2590E+01 0.4268E+02
Calculation complete Clover is finishing Wall clock 37.7645969390869141 Done Fri Feb 1 08:55:03 PST 2013
8_16_4.cmd Clover Version 1.000 MPI Version OpenMP Version Task Count 128 Thread Count: 4
Output file clover.out opened. All output will go there.
Clover Version 1.000 MPI Version OpenMP Version Task Count 128 Thread Count: 4
Clover will run from the following input:-
*clover
Step 2952 time 15.4963148 control sound timestep 4.98E-03 1, 1 x 5.21E-03 y 5.21E-03
Time 15.5012992170708994 Volume Mass Density Pressure Internal Energy Kinetic Energy Total Energy step: 2952 0.1000E+03 0.2800E+02 0.2800E+00 0.1604E+00 0.4009E+02 0.2590E+01 0.4268E+02
Calculation complete Clover is finishing Wall clock 36.2393031120300293 Done Fri Feb 1 08:57:09 PST 2013
8_1_64.cmd
Clover Version 1.000 MPI Version OpenMP Version Task Count 8 Thread Count: 64
Output file clover.out opened. All output will go there.
Clover Version 1.000 MPI Version OpenMP Version Task Count 8 Thread Count: 64
Clover will run from the following input:-
*clover
Step 2953 time 15.4956906 control sound timestep 5.20E-03 1, 1 x 5.21E-03 y 5.21E-03
Time 15.5008953166272097 Volume Mass Density Pressure Internal Energy Kinetic Energy Total Energy step: 2953 0.1000E+03 0.2800E+02 0.2800E+00 0.1604E+00 0.4009E+02 0.2590E+01 0.4268E+02
Calculation complete Clover is finishing Wall clock 47.6886451244354248 Done Fri Feb 1 09:00:41 PST 2013
32_16_1.cmd
Clover Version 1.000 MPI Version OpenMP Version Task Count 512 Thread Count: 1
Output file clover.out opened. All output will go there.
Clover Version 1.000 MPI Version OpenMP Version Task Count 512 Thread Count: 1
Clover will run from the following input:-
*clover Step 2952 time 15.4956432 control sound timestep 5.11E-03 1, 1 x 5.21E-03 y 5.21E-03
Time 15.5007538630877200 Volume Mass Density Pressure Internal Energy Kinetic Energy Total Energy step: 2952 0.1000E+03 0.2800E+02 0.2800E+00 0.1604E+00 0.4009E+02 0.2590E+01 0.4268E+02
Calculation complete Clover is finishing Wall clock 25.7877268791198730 Done Sat Feb 2 03:52:11 PST 2013