Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use hpc-stack libraries in ufs-s2s-model #191

Merged
merged 38 commits into from
Sep 22, 2020
Merged

Use hpc-stack libraries in ufs-s2s-model #191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
77b5efa
Update ufs-s2s-model
binli2337 Jan 28, 2020
5e1bd90
update FV3, MOM6 and NEMS
binli2337 Jan 28, 2020
a05d12a
Merge remote-tracking branch 'upstream/develop' into develop
binli2337 Feb 3, 2020
b5deed6
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Feb 29, 2020
21c5620
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Mar 21, 2020
288874f
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Mar 26, 2020
8e0df44
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Apr 9, 2020
fdcabca
Updated FV3.
binli2337 Apr 9, 2020
a3a964c
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Apr 16, 2020
a47da44
Updated NEMS and .gitmodules.
binli2337 Apr 27, 2020
6071c58
Update .gitmodules and NEMS
binli2337 Apr 28, 2020
cc27ef2
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Apr 28, 2020
68e413d
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 May 1, 2020
65d8c6f
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 May 3, 2020
cbf19ad
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 May 13, 2020
5cd4f16
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 May 29, 2020
3ef06db
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Jun 4, 2020
ef771a5
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Jun 7, 2020
252769f
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Jun 13, 2020
9ef1e9d
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Jun 16, 2020
61dfe0a
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Jun 25, 2020
e727312
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Jun 26, 2020
0b14a96
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Jul 10, 2020
d300990
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Jul 16, 2020
fcaff06
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Jul 17, 2020
b252ae6
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Jul 20, 2020
0ee0cfd
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Jul 30, 2020
3435e86
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Aug 8, 2020
6d35107
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Aug 17, 2020
c4dc860
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Aug 21, 2020
7dbfbd0
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Sep 1, 2020
3bfb667
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Sep 16, 2020
da5c3b0
Merge branch 'develop' of https://github.com/ufs-community/ufs-s2s-mo…
binli2337 Sep 21, 2020
6145ea6
Updated module files to use hpc-stack libraries.
binli2337 Sep 21, 2020
f0214dc
Added log files from wcoss.
binli2337 Sep 21, 2020
0fb0dae
Added log files from Hera.
binli2337 Sep 22, 2020
30e4e2e
Added log files from Orion.
binli2337 Sep 22, 2020
29d8375
Revised configure.fv3.hera.intel, configure.fv3.orion.intel, and conf…
binli2337 Sep 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion conf/configure.fv3.hera.intel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,5 +181,5 @@ LDFLAGS += $(LIBS)
ifdef InNemsMakefile
FFLAGS += $(ESMF_INC)
CPPFLAGS += -traditional
EXTLIBS = $(NCEPLIBS) $(ESMF_LIB) $(LDFLAGS) $(NETCDF_LIB)
EXTLIBS = $(NCEPLIBS) $(ESMF_F90LINKPATHS) $(ESMF_LIB) $(LDFLAGS) $(NETCDF_LIB)
endif
4 changes: 2 additions & 2 deletions conf/configure.fv3.orion.intel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ include $(ESMFMKFILE)
ESMF_INC = $(ESMF_F90COMPILEPATHS)

NEMSIOINC = -I$(NEMSIO_INC)
NCEPLIBS = $(POST_LIB) $(NEMSIO_LIB) $(G2_LIB4) $(G2TMPL_LIB) $(BACIO_LIB4) $(SP_LIBd) $(W3EMC_LIBd) $(W3NCO_LIBd) $(CRTM_LIB) $(JASPER_LIB) -lpng -lz
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is JASPER_LIB defined in hpc-stack? what is the value of it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JASPER_LIB is not defined in hpc-stack. However, JASPER_ROOT is defined.

NCEPLIBS = $(POST_LIB) $(NEMSIO_LIB) $(G2_LIB4) $(G2TMPL_LIB) $(BACIO_LIB4) $(SP_LIBd) $(W3EMC_LIBd) $(W3NCO_LIBd) $(CRTM_LIB) $(JASPER_ROOT)/lib64/libjasper.a -lpng -lz

##############################################
# Need to use at least GNU Make version 3.81 #
Expand Down Expand Up @@ -184,5 +184,5 @@ LDFLAGS += $(LIBS)
ifdef InNemsMakefile
FFLAGS += $(ESMF_INC)
CPPFLAGS += -traditional
EXTLIBS = $(NCEPLIBS) $(ESMF_LIB) $(LDFLAGS) $(NETCDF_LIB)
EXTLIBS = $(NCEPLIBS) $(ESMF_F90LINKPATHS) $(ESMF_LIB) $(LDFLAGS) $(NETCDF_LIB)
endif
4 changes: 2 additions & 2 deletions conf/configure.fv3.wcoss_dell_p3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ include $(ESMFMKFILE)
ESMF_INC = $(ESMF_F90COMPILEPATHS)

NEMSIOINC = -I$(NEMSIO_INC)
NCEPLIBS = $(POST_LIB) $(NEMSIO_LIB) $(G2_LIB4) $(G2TMPL_LIB) $(BACIO_LIB4) $(SP_LIBd) $(W3EMC_LIBd) $(W3NCO_LIBd) $(CRTM_LIB) $(JASPER_LIB) -lpng -lz
NCEPLIBS = $(POST_LIB) $(NEMSIO_LIB) $(G2_LIB4) $(G2TMPL_LIB) $(BACIO_LIB4) $(SP_LIBd) $(W3EMC_LIBd) $(W3NCO_LIBd) $(CRTM_LIB) $(JASPER_ROOT)/lib64/libjasper.a -lpng -lz

##############################################
# Need to use at least GNU Make version 3.81 #
Expand Down Expand Up @@ -168,5 +168,5 @@ LDFLAGS += $(LIBS)
ifdef InNemsMakefile
FFLAGS += $(ESMF_INC)
CPPFLAGS += -traditional
EXTLIBS = $(NCEPLIBS) $(ESMF_LIB) $(LDFLAGS)
EXTLIBS = $(NCEPLIBS) $(ESMF_F90LINKPATHS) $(ESMF_LIB) $(LDFLAGS)
endif
77 changes: 29 additions & 48 deletions modulefiles/hera.intel/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,60 +1,41 @@
#%Module######################################################################
##
## NEMS FV3 Prerequisites: Hera/Intel
#%Module

proc ModulesHelp {} {
puts stderr "\tcit - loads modules required for building and running FV3 under NEMS on Hera/Intel"
puts stderr "\tcit - loads modules required for building and running UFS Model on Hera/Intel"
}

module-whatis "loads NEMS FV3 prerequisites for Hera/Intel"
module-whatis "loads UFS Model prerequisites for Hera/Intel"

##
## load contrib environment
## load slurm utils (arbitrary.pl layout.pl)
##
module use -a /contrib/sutils/modulefiles
module use /contrib/sutils/modulefiles
module load sutils

##
## load programming environment
## this typically includes compiler, MPI and job scheduler
##
module load intel/18.0.5.274
module load impi/2018.0.4

##
## NCEP libraries
##
module use -a /scratch2/NCEPDEV/nwprod/NCEPLIBS/cmake/install/NCEPLIBS/modules
module load bacio/2.4.0
module load crtm_dev/2.3.0
module load g2/3.4.0
module load g2tmpl/1.9.0
module load ip/3.3.0
module load nceppost/dceca26
module load nemsio/2.5.1
module load sp/2.3.0
module load w3emc/2.7.0
module load w3nco/2.4.0

module load gfsio/1.4.0
module load sfcio/1.4.0
module load sigio/2.3.0

##
## load ESMF library for above compiler / MPI combination
## use pre-compiled EMSF library for above compiler / MPI combination
##
module use -a /scratch1/NCEPDEV/nems/emc.nemspara/soft/modulefiles
module load netcdf_parallel/4.7.4.release
module load esmf/8.1.0bs27
module load hdf5_parallel/1.10.6.release

##
## load cmake
##
module load cmake/3.16.1
setenv CMAKE_C_COMPILER mpiicc
setenv CMAKE_CXX_COMPILER mpiicpc
setenv CMAKE_Fortran_COMPILER mpiifort
setenv CMAKE_Platform hera.intel

module use /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/v1.0.0-beta1/modulefiles/stack

module load hpc/1.0.0-beta1
module load hpc-intel/18.0.5.274
module load hpc-impi/2018.0.4

module load jasper/2.0.15
module load zlib/1.2.11
module load png/1.6.35

module load hdf5/1.10.6
module load netcdf/4.7.4
module load esmf/8_1_0_beta_snapshot_27

module load bacio/2.4.1
module load crtm/2.3.0
module load g2/3.4.1
module load g2tmpl/1.9.1
module load ip/3.3.3
module load nceppost/dceca26
module load nemsio/2.5.2
module load sp/2.3.3
module load w3emc/2.7.3
module load w3nco/2.4.1
77 changes: 29 additions & 48 deletions modulefiles/hera.intel/fv3_debug
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,60 +1,41 @@
#%Module######################################################################
##
## NEMS FV3 Prerequisites: Hera/Intel
#%Module

proc ModulesHelp {} {
puts stderr "\tcit - loads modules required for building and running FV3 under NEMS on Hera/Intel"
puts stderr "\tcit - loads modules required for building and running UFS Model on Hera/Intel"
}

module-whatis "loads NEMS FV3 prerequisites for Hera/Intel"
module-whatis "loads UFS Model prerequisites for Hera/Intel"

##
## load contrib environment
## load slurm utils (arbitrary.pl layout.pl)
##
module use -a /contrib/sutils/modulefiles
module use /contrib/sutils/modulefiles
module load sutils

##
## load programming environment
## this typically includes compiler, MPI and job scheduler
##
module load intel/18.0.5.274
module load impi/2018.0.4

##
## NCEP libraries
##
module use -a /scratch2/NCEPDEV/nwprod/NCEPLIBS/cmake/install/NCEPLIBS/modules
module load bacio/2.4.0
module load crtm_dev/2.3.0
module load g2/3.4.0
module load g2tmpl/1.9.0
module load ip/3.3.0
module load nceppost/dceca26
module load nemsio/2.5.1
module load sp/2.3.0
module load w3emc/2.7.0
module load w3nco/2.4.0

module load gfsio/1.4.0
module load sfcio/1.4.0
module load sigio/2.3.0

##
## load ESMF library for above compiler / MPI combination
## use pre-compiled EMSF library for above compiler / MPI combination
##
module use -a /scratch1/NCEPDEV/nems/emc.nemspara/soft/modulefiles
module load netcdf_parallel/4.7.4.release
module load esmf/8.1.0bs27g
module load hdf5_parallel/1.10.6.release

##
## load cmake
##
module load cmake/3.16.1
setenv CMAKE_C_COMPILER mpiicc
setenv CMAKE_CXX_COMPILER mpiicpc
setenv CMAKE_Fortran_COMPILER mpiifort
setenv CMAKE_Platform hera.intel

module use /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/v1.0.0-beta1/modulefiles/stack

module load hpc/1.0.0-beta1
module load hpc-intel/18.0.5.274
module load hpc-impi/2018.0.4

module load jasper/2.0.15
module load zlib/1.2.11
module load png/1.6.35

module load hdf5/1.10.6
module load netcdf/4.7.4
module load esmf/8_1_0_beta_snapshot_27-debug

module load bacio/2.4.1
module load crtm/2.3.0
module load g2/3.4.1
module load g2tmpl/1.9.1
module load ip/3.3.3
module load nceppost/dceca26
module load nemsio/2.5.2
module load sp/2.3.3
module load w3emc/2.7.3
module load w3nco/2.4.1
83 changes: 28 additions & 55 deletions modulefiles/orion.intel/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,67 +1,40 @@
#%Module######################################################################
##
## NEMS FV3 Prerequisites: Orion/Intel
#%Module

proc ModulesHelp {} {
puts stderr "\tcit - loads modules required for building and running FV3 under NEMS on Orion/Intel"
puts stderr "\tcit - loads modules required for building and running UFS Model on Orion/Intel"
}

module-whatis "loads NEMS FV3 prerequisites for Orion/Intel"
module-whatis "loads UFS Model prerequisites for Orion/Intel"

##
## load contrib environment
## load noaatools (slurm utils (arbitrary.pl layout.pl))
##
module load contrib noaatools

##
## load programming environment
## this typically includes compiler, MPI and job scheduler
##
module load intel/2018
module load impi/2018

##
## NCEP libraries
##
module use -a /apps/contrib/NCEPLIBS/orion/cmake/install/NCEPLIBS/modules
module load bacio/2.4.0
module load crtm_dev/2.3.0
module load g2/3.4.0
module load g2tmpl/1.9.0
module load ip/3.3.0
module load nceppost/dceca26
module load nemsio/2.5.1
module load sp/2.3.0
module load w3emc/2.7.0
module load w3nco/2.4.0
module load cmake/3.17.3
setenv CMAKE_C_COMPILER mpiicc
setenv CMAKE_CXX_COMPILER mpiicpc
setenv CMAKE_Fortran_COMPILER mpiifort
setenv CMAKE_Platform orion.intel

module load gfsio/1.4.0
module load sfcio/1.4.0
module load sigio/2.3.0
module use /apps/contrib/NCEP/libs/hpc-stack/v1.0.0-beta1/modulefiles/stack

module use /apps/contrib/NCEPLIBS/orion/modulefiles
module load jasper/1.900.2
module load png/1.2.44
module load z/1.2.6
module load hpc/1.0.0-beta1
module load hpc-intel/2018.4
module load hpc-impi/2018.4

##
## load ESMF library for above compiler / MPI combination
## use pre-compiled EMSF library for above compiler / MPI combination
##
module use /apps/contrib/NCEPLIBS/lib/modulefiles
module load netcdfp/4.7.4.release
module load esmflocal/8.1.0.27bs.release
module load jasper/2.0.15
module load zlib/1.2.11
module load png/1.6.35

###
### load Python 3
#module load python/3.7.5
module load hdf5/1.10.6
module load netcdf/4.7.4
module load esmf/8_1_0_beta_snapshot_27

##
## load cmake
##
module load cmake/3.15.4
setenv CMAKE_C_COMPILER mpiicc
setenv CMAKE_CXX_COMPILER mpiicpc
setenv CMAKE_Fortran_COMPILER mpiifort
setenv CMAKE_Platform orion.intel
module load bacio/2.4.1
module load crtm/2.3.0
module load g2/3.4.1
module load g2tmpl/1.9.1
module load ip/3.3.3
module load nceppost/dceca26
module load nemsio/2.5.2
module load sp/2.3.3
module load w3emc/2.7.3
module load w3nco/2.4.1
Loading