Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closes #94. Update mpi variables in regrid #223

Merged
merged 1 commit into from
Nov 3, 2021
Merged

Closes #94. Update mpi variables in regrid #223

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- Changed the Intel MPI and MVAPICH2 flags in `regrid.pl` to be modern

### Removed

## [1.4.10] - 2021-10-08
Expand Down
36 changes: 15 additions & 21 deletions GEOS_Util/post/regrid.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2478,31 +2478,25 @@ sub regrid_upperair_rsts_CS {

if (\$?I_MPI_ROOT) then

# intel scaling suggestions
#--------------------------
# Based on GEOSgcm NCCS Intel MPI settings
#-----------------------------------------

setenv I_MPI_DAPL_UD on

setenv DAPL_UCM_CQ_SIZE 4096
setenv DAPL_UCM_QP_SIZE 4096

setenv I_MPI_DAPL_UD_SEND_BUFFER_NUM 4096
setenv I_MPI_DAPL_UD_RECV_BUFFER_NUM 4096
setenv I_MPI_DAPL_UD_ACK_SEND_POOL_SIZE 4096
setenv I_MPI_DAPL_UD_ACK_RECV_POOL_SIZE 4096
setenv I_MPI_DAPL_UD_RNDV_EP_NUM 2
setenv I_MPI_DAPL_UD_REQ_EVD_SIZE 2000

setenv DAPL_UCM_REP_TIME 2000
setenv DAPL_UCM_RTU_TIME 2000
setenv DAPL_UCM_RETRY 7
setenv DAPL_ACK_RETRY 7
setenv DAPL_ACK_TIMER 20
setenv DAPL_UCM_RETRY 10
setenv DAPL_ACK_RETRY 10
setenv I_MPI_ADJUST_ALLREDUCE 12
setenv I_MPI_ADJUST_GATHERV 3

setenv I_MPI_SHM_HEAP_VSIZE 512
setenv PSM2_MEMORY large
setenv I_MPI_EXTRA_FILESYSTEM 1
setenv I_MPI_EXTRA_FILESYSTEM_FORCE gpfs
setenv ROMIO_FSTYPE_FORCE "gpfs:"

else if (\$?MVAPICH2) then

setenv MV2_ENABLE_AFFINITY 0
setenv MV2_ENABLE_AFFINITY 0
setenv SLURM_DISTRIBUTION block
setenv MV2_MPIRUN_TIMEOUT 100
setenv MV2_GATHERV_SSEND_THRESHOLD 256

endif

Expand Down