Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3 bugfix in ccpp-framework, ESMF 8.1.0bs27, cleanup rt_utils.sh, fix RRTMGP regression tests #201

Merged
merged 15 commits into from
Sep 15, 2020
Merged

Python 3 bugfix in ccpp-framework, ESMF 8.1.0bs27, cleanup rt_utils.sh, fix RRTMGP regression tests #201

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
f051537
Enable inline post for gnumake builds on hera.gnu, cheyenne.intel, ch…
climbfuji Sep 11, 2020
78206f9
Update pull request template, use correct Python interpreter when cal…
climbfuji Sep 11, 2020
7aaf241
tests/rt_utils.sh: remove machine-dependent configuration; skip-ci
climbfuji Sep 11, 2020
dc8604f
Update ESMF from 8.1.0bs21 to 8.1.0bs27 on almost all machines; skip-ci
climbfuji Sep 11, 2020
e4e8b08
Update .gitmodules and submodule pointer for fv3atm for code review a…
climbfuji Sep 11, 2020
a60714b
Fix RRTMGP regression test setups, temporarily disable RRTMGP tests i…
climbfuji Sep 11, 2020
b30a900
Updates for docker/ci
climbfuji Sep 11, 2020
0e63eff
Add note about unit test availability to pull request template
climbfuji Sep 11, 2020
e559272
Update rt.sh configuration for jet, remove tests/rt_rrtmgp.conf, add …
climbfuji Sep 14, 2020
34e596a
Regression test logs for wcoss_cray and wcoss_dell_p3; skip-ci
climbfuji Sep 14, 2020
cdeabf6
Regression test logs for cheyenne.intel and cheyenne.gnu; skip-ci
climbfuji Sep 14, 2020
9a087e7
Regression test logs for hera.intel and hera.gnu; skip-ci
climbfuji Sep 14, 2020
e4b0e0e
Regression test logs for orion.intel
climbfuji Sep 14, 2020
1af29d1
Update submodule pointer for fv3atm; skip-ci
climbfuji Sep 14, 2020
ce1ea89
Revert change to .gitmodules and update submodule pointer for fv3atm
climbfuji Sep 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 2 additions & 6 deletions .github/pull_request_template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@
Provide a detailed description of what this PR does.
What bug does it fix, or what feature does it add?
Is a change of answers expected from this PR?


Are any library updates included in this PR (modulefiles etc.)?

### Issue(s) addressed

Expand All @@ -14,15 +13,12 @@ Link the issues to be closed with this PR, whether in this repository, or in ano
- fixes #<issue_number>
- fixes noaa-emc/fv3atm/issues/<issue_number>



## Testing

How were these changes tested?
What compilers / HPCs was it tested with?
Are the changes covered by regression tests? (If not, why? Do new tests need to be added?)


Have regression tests and unit tests (utests) been run? On which platforms and with which compilers?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Utests can be run on Hera, Orion, Dell with Intel compilers (I had some trouble with ecFlow on Cray)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I add this information to the PR template?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since regression tests are supported on all platforms (tier 1, 2, etc.), maybe it would be helpful to add that utests are only supported on tier 1 (Cray will have to come later)? e.g. unit tests (utests on tier 1 platform)


## Dependencies

Expand Down
6 changes: 4 additions & 2 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
[submodule "FV3"]
path = FV3
url = https://github.com/NOAA-EMC/fv3atm
branch = develop
#url = https://github.com/NOAA-EMC/fv3atm
#branch = develop
url = https://github.com/climbfuji/fv3atm
branch = update_submodule_ccpp_framework_20200910
[submodule "NEMS"]
path = NEMS
url = https://github.com/NOAA-EMC/NEMS
Expand Down
6 changes: 4 additions & 2 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,8 @@ if(CCPP)

if(DEFINED CCPP_SUITES)
message("Calling CCPP code generator (ccpp_prebuild.py) for SUITES = ${CCPP_SUITES}")
execute_process(COMMAND FV3/ccpp/framework/scripts/ccpp_prebuild.py
execute_process(COMMAND ${Python_EXECUTABLE}
"FV3/ccpp/framework/scripts/ccpp_prebuild.py"
"--config=FV3/ccpp/config/ccpp_prebuild_config.py"
"--suites=${CCPP_SUITES}"
"--builddir=${PROJECT_BINARY_DIR}/FV3"
Expand All @@ -119,7 +120,8 @@ if(CCPP)
)
else()
message("Calling CCPP code generator (ccpp_prebuild.py) ...")
execute_process(COMMAND FV3/ccpp/framework/scripts/ccpp_prebuild.py
execute_process(COMMAND ${Python_EXECUTABLE}
"FV3/ccpp/framework/scripts/ccpp_prebuild.py"
"--config=FV3/ccpp/config/ccpp_prebuild_config.py"
"--builddir=${PROJECT_BINARY_DIR}/FV3"
WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
Expand Down
2 changes: 1 addition & 1 deletion FV3
Open in desktop
Submodule FV3 updated 2 files
+4 −2 .gitmodules
+1 −1 ccpp/framework
2 changes: 1 addition & 1 deletion cmake/configure_cheyenne.gnu.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
set(INLINE_POST ON CACHE BOOL "Enable inline post" FORCE)
set(INLINE_POST ON CACHE BOOL "Enable inline post" FORCE)
2 changes: 1 addition & 1 deletion conf/configure.fv3.cheyenne.gnu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ CFLAGS := $(INCLUDE)
FFLAGS := $(INCLUDE) -fcray-pointer -ffree-line-length-none -fno-range-check -fbacktrace

CPPDEFS += -Duse_libMPI -Duse_netCDF -DSPMD -DUSE_LOG_DIAG_FIELD_INFO -DUSE_GFSL63 -DGFS_PHYS -Duse_WRTCOMP
CPPDEFS += -DNEW_TAUCTMAX -DINTERNAL_FILE_NML -DNO_INLINE_POST
CPPDEFS += -DNEW_TAUCTMAX -DINTERNAL_FILE_NML

ifeq ($(HYDRO),Y)
CPPDEFS +=
Expand Down
2 changes: 1 addition & 1 deletion conf/configure.fv3.cheyenne.intel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ CFLAGS := $(INCLUDE)
FFLAGS := $(INCLUDE) -fno-alias -auto -safe-cray-ptr -save-temps -ftz -assume byterecl -nowarn -sox -align array64byte

CPPDEFS += -Duse_libMPI -Duse_netCDF -DSPMD -DUSE_LOG_DIAG_FIELD_INFO -DUSE_GFSL63 -DGFS_PHYS -Duse_WRTCOMP
CPPDEFS += -DNEW_TAUCTMAX -DINTERNAL_FILE_NML -DNO_INLINE_POST
CPPDEFS += -DNEW_TAUCTMAX -DINTERNAL_FILE_NML

ifeq ($(HYDRO),Y)
CPPDEFS +=
Expand Down
2 changes: 1 addition & 1 deletion conf/configure.fv3.hera.gnu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ CFLAGS := $(INCLUDE)
FFLAGS := $(INCLUDE) -fcray-pointer -ffree-line-length-none -fno-range-check -fbacktrace

CPPDEFS += -Duse_libMPI -Duse_netCDF -DSPMD -DUSE_LOG_DIAG_FIELD_INFO -DUSE_GFSL63 -DGFS_PHYS -Duse_WRTCOMP
CPPDEFS += -DNEW_TAUCTMAX -DINTERNAL_FILE_NML -DNO_INLINE_POST
CPPDEFS += -DNEW_TAUCTMAX -DINTERNAL_FILE_NML

ifeq ($(HYDRO),Y)
CPPDEFS +=
Expand Down
4 changes: 2 additions & 2 deletions modulefiles/cheyenne.gnu/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ module load ncarcompilers/0.5.0
##
## use pre-compiled PNG, NetCDF, ESMF and NCEP libraries for above compiler / MPI combination
##
module use -a /glade/p/ral/jntp/GMTB/tools/ufs-stack-20200825/gnu-9.1.0/mpt-2.19/modules
module use -a /glade/p/ral/jntp/GMTB/tools/ufs-stack-20200909/gnu-9.1.0/mpt-2.19/modules

module load libpng/1.6.35
module load netcdf/4.7.4
module load esmf/8.1.0bs21
module load esmf/8.1.0bs27

module load bacio/2.4.0
module load crtm/2.3.0
Expand Down
4 changes: 2 additions & 2 deletions modulefiles/cheyenne.intel/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ module load ncarcompilers/0.5.0
##
## use pre-compiled PNG, NetCDF, ESMF and NCEP libraries for above compiler / MPI combination
##
module use -a /glade/p/ral/jntp/GMTB/tools/ufs-stack-20200825/intel-19.1.1/mpt-2.19/modules
module use -a /glade/p/ral/jntp/GMTB/tools/ufs-stack-20200909/intel-19.1.1/mpt-2.19/modules

module load libpng/1.6.35
module load netcdf/4.7.4
module load esmf/8.1.0bs21
module load esmf/8.1.0bs27

module load bacio/2.4.0
module load crtm/2.3.0
Expand Down
4 changes: 2 additions & 2 deletions modulefiles/gaea.intel/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,11 @@ module load cray-mpich/7.7.11
##
## use pre-compiled PNG, NetCDF, ESMF and NCEP libraries for above compiler / MPI combination
##
module use -a /lustre/f2/pdata/esrl/gsd/ufs/ufs-stack-20200825/intel-18.0.6.288/cray-mpich-7.7.11/modules
module use -a /lustre/f2/pdata/esrl/gsd/ufs/ufs-stack-20200909/intel-18.0.6.288/cray-mpich-7.7.11/modules

module load libpng/1.6.35
module load netcdf/4.7.4
module load esmf/8.1.0bs21
module load esmf/8.1.0bs27

module load bacio/2.4.0
module load crtm/2.3.0
Expand Down
4 changes: 2 additions & 2 deletions modulefiles/hera.gnu/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ module load mpich/3.3.2
##
## use pre-compiled PNG, NetCDF, ESMF and NCEP libraries for above compiler / MPI combination
##
module use -a /scratch1/BMC/gmtb/software/ufs-stack-20200825/gnu-9.2.0/mpich-3.3.2/modules
module use -a /scratch1/BMC/gmtb/software/ufs-stack-20200909/gnu-9.2.0/mpich-3.3.2/modules

module load libpng/1.6.35
module load netcdf/4.7.4
module load esmf/8.1.0bs21
module load esmf/8.1.0bs27

module load bacio/2.4.0
module load crtm/2.3.0
Expand Down
2 changes: 1 addition & 1 deletion modulefiles/hera.intel/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ module load sigio/2.3.0
##
module use -a /scratch1/NCEPDEV/nems/emc.nemspara/soft/modulefiles
module load netcdf_parallel/4.7.4.release
module load esmf/8.1.0bs21_ParallelNetCDF.release
module load esmf/8.1.0bs27
module load hdf5_parallel/1.10.6.release

##
Expand Down
4 changes: 2 additions & 2 deletions modulefiles/jet.intel/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ module load impi/2018.4.274
##
## use pre-compiled PNG, NetCDF, ESMF and NCEP libraries for above compiler / MPI combination
##
module use -a /lfs4/HFIP/hfv3gfs/software/ufs-stack-20200825/intel-18.0.5.274/impi-2018.4.274/modules
module use -a /lfs4/HFIP/hfv3gfs/software/ufs-stack-20200909/intel-18.0.5.274/impi-2018.4.274/modules

module load libpng/1.6.35
module load netcdf/4.7.4
module load esmf/8.1.0bs21
module load esmf/8.1.0bs27

module load bacio/2.4.0
module load crtm/2.3.0
Expand Down
2 changes: 1 addition & 1 deletion modulefiles/orion.intel/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ module load z/1.2.6
##
module use /apps/contrib/NCEPLIBS/lib/modulefiles
module load netcdfp/4.7.4.release
module load esmflocal/8.1.0.21bs.release
module load esmflocal/8.1.0.27bs.release

###
### load Python 3
Expand Down
8 changes: 4 additions & 4 deletions modulefiles/stampede.intel/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,17 @@ module-whatis "loads NEMS FV3 prerequisites for Stampede/Intel"
module load intel/18.0.2
module load impi/18.0.2
module load python3/3.7.0
module load netcdf/4.6.2
setenv NETCDF /opt/apps/intel18/netcdf/4.6.2/x86_64

##
## use pre-compiled ESMF and NCEP libraries for above compiler / MPI combination
##
# DH* todo - shared directory
module use -a /work/06146/tg854455/stampede2/ufs-stack-20200728/intel-18.0.2/impi-18.0.2/modules
module use -a /work/06146/tg854455/stampede2/ufs-stack-20200909/intel-18.0.2/impi-18.0.2/modules
# *DH

module load esmf/8.0.0
module load libpng/1.6.35
module load netcdf/4.7.4
module load esmf/8.1.0bs27

module load bacio/2.4.0
module load crtm/2.3.0
Expand Down
2 changes: 1 addition & 1 deletion modulefiles/wcoss_cray/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ module load udreg
module load ugni

module use /gpfs/hps3/emc/nems/noscrub/emc.nemspara/soft/modulefiles
module load esmf/8.1.0bs21
module load esmf/8.1.0bs27

module swap pmi pmi/5.0.11

Expand Down
2 changes: 1 addition & 1 deletion modulefiles/wcoss_dell_p3/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ module load NetCDF-parallel/4.7.4
module load HDF5-parallel/1.10.6

module use /gpfs/dell2/emc/modeling/noscrub/emc.nemspara/soft/modulefiles
module load esmf/8.1.0bs21
module load esmf/8.1.0bs27

##
## load cmake
Expand Down
2 changes: 1 addition & 1 deletion parm/ccpp_v15p2_c96_rrtmgp.nml.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
fhout = 3
fhmaxhf = 120
fhouthf = 1
ccpp_suite = 'FV3_GFS_v15p2_RRTMGP'
ccpp_suite = '@[CCPP_SUITE]'
/

&diag_manager_nml
Expand Down
2 changes: 1 addition & 1 deletion parm/ccpp_v16beta_c96_rrtmgp.nml.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
fhout = 3
fhmaxhf = 120
fhouthf = 1
ccpp_suite = 'FV3_GFS_v16beta_RRTMGP'
ccpp_suite = '@[CCPP_SUITE]'
/

&diag_manager_nml
Expand Down
24 changes: 12 additions & 12 deletions tests/rt.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,12 +119,12 @@ RUN | fv3_ccpp_gfs_v15p2
RUN | fv3_ccpp_gfs_v16beta | standard | hera.intel | fv3 |
RUN | fv3_ccpp_gfs_v16beta | standard | orion.intel | fv3 |
RUN | fv3_ccpp_gfs_v16beta | standard | cheyenne.intel | fv3 |
RUN | fv3_ccpp_gfs_v15p2_RRTMGP | standard | hera.intel | fv3 |
RUN | fv3_ccpp_gfs_v15p2_RRTMGP | standard | orion.intel | fv3 |
RUN | fv3_ccpp_gfs_v15p2_RRTMGP | standard | cheyenne.intel | fv3 |
RUN | fv3_ccpp_gfs_v16beta_RRTMGP | standard | hera.intel | fv3 |
RUN | fv3_ccpp_gfs_v16beta_RRTMGP | standard | orion.intel | fv3 |
RUN | fv3_ccpp_gfs_v16beta_RRTMGP | standard | cheyenne.intel | fv3 |
#RUN | fv3_ccpp_gfs_v15p2_RRTMGP | standard | hera.intel | fv3 |
#RUN | fv3_ccpp_gfs_v15p2_RRTMGP | standard | orion.intel | fv3 |
#RUN | fv3_ccpp_gfs_v15p2_RRTMGP | standard | cheyenne.intel | fv3 |
#RUN | fv3_ccpp_gfs_v16beta_RRTMGP | standard | hera.intel | fv3 |
#RUN | fv3_ccpp_gfs_v16beta_RRTMGP | standard | orion.intel | fv3 |
#RUN | fv3_ccpp_gfs_v16beta_RRTMGP | standard | cheyenne.intel | fv3 |

COMPILE | CCPP=Y SUITES=FV3_GFS_v16_csawmg | standard | | fv3 |
# fv3_ccpp_gfsv16_csawmg and fv3_ccpp_gfsv16_csawmgt crash with a "bus error" on cheyenne.intel, turn off
Expand Down Expand Up @@ -155,12 +155,12 @@ RUN | fv3_ccpp_gfs_v15p2_debug
RUN | fv3_ccpp_gfs_v16beta_debug | standard | hera.intel | fv3 |
RUN | fv3_ccpp_gfs_v16beta_debug | standard | orion.intel | fv3 |
RUN | fv3_ccpp_gfs_v16beta_debug | standard | cheyenne.intel | fv3 |
RUN | fv3_ccpp_gfs_v15p2_RRTMGP_debug | standard | hera.intel | fv3 |
RUN | fv3_ccpp_gfs_v15p2_RRTMGP_debug | standard | orion.intel | fv3 |
RUN | fv3_ccpp_gfs_v15p2_RRTMGP_debug | standard | cheyenne.intel | fv3 |
RUN | fv3_ccpp_gfs_v16beta_RRTMGP_debug | standard | hera.intel | fv3 |
RUN | fv3_ccpp_gfs_v16beta_RRTMGP_debug | standard | orion.intel | fv3 |
RUN | fv3_ccpp_gfs_v16beta_RRTMGP_debug | standard | cheyenne.intel | fv3 |
#RUN | fv3_ccpp_gfs_v15p2_RRTMGP_debug | standard | hera.intel | fv3 |
#RUN | fv3_ccpp_gfs_v15p2_RRTMGP_debug | standard | orion.intel | fv3 |
#RUN | fv3_ccpp_gfs_v15p2_RRTMGP_debug | standard | cheyenne.intel | fv3 |
#RUN | fv3_ccpp_gfs_v16beta_RRTMGP_debug | standard | hera.intel | fv3 |
#RUN | fv3_ccpp_gfs_v16beta_RRTMGP_debug | standard | orion.intel | fv3 |
#RUN | fv3_ccpp_gfs_v16beta_RRTMGP_debug | standard | cheyenne.intel | fv3 |

COMPILE | CCPP=Y SUITES=FV3_GSD_v0,FV3_GFS_v15_thompson,FV3_RRFS_v1beta 32BIT=Y DEBUG=Y | standard | | fv3 |
RUN | fv3_ccpp_gsd_debug | standard | | fv3 |
Expand Down
8 changes: 4 additions & 4 deletions tests/rt_gnu.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ COMPILE | CCPP=Y SUITES=FV3_GFS_v15p2,FV3_GFS_v16beta,FV3_GFS_v16beta_flake,FV3_
RUN | fv3_ccpp_gfs_v15p2 | standard | | fv3 |
RUN | fv3_ccpp_gfs_v16beta | standard | | fv3 |
RUN | fv3_ccpp_gfs_v16beta_flake | standard | | fv3 |
RUN | fv3_ccpp_gfs_v15p2_RRTMGP | standard | | fv3 |
RUN | fv3_ccpp_gfs_v16beta_RRTMGP | standard | | fv3 |
#RUN | fv3_ccpp_gfs_v15p2_RRTMGP | standard | | fv3 |
#RUN | fv3_ccpp_gfs_v16beta_RRTMGP | standard | | fv3 |

COMPILE | CCPP=Y SUITES=FV3_GSD_v0,FV3_GFS_v15_thompson,FV3_RRFS_v1beta 32BIT=Y | standard | | fv3 |

Expand All @@ -31,8 +31,8 @@ RUN | fv3_ccpp_control_debug
COMPILE | CCPP=Y SUITES=FV3_GFS_v15p2,FV3_GFS_v16beta,FV3_GFS_v15p2_RRTMGP,FV3_GFS_v16beta_RRTMGP DEBUG=Y | standard | | fv3 |
RUN | fv3_ccpp_gfs_v15p2_debug | standard | | fv3 |
RUN | fv3_ccpp_gfs_v16beta_debug | standard | | fv3 |
RUN | fv3_ccpp_gfs_v15p2_RRTMGP_debug | standard | | fv3 |
RUN | fv3_ccpp_gfs_v16beta_RRTMGP_debug | standard | | fv3 |
#RUN | fv3_ccpp_gfs_v15p2_RRTMGP_debug | standard | | fv3 |
#RUN | fv3_ccpp_gfs_v16beta_RRTMGP_debug | standard | | fv3 |

COMPILE | CCPP=Y SUITES=FV3_GFS_2017_fv3wam 32BIT=Y MULTI_GASES=Y | standard | | fv3 |
RUN | fv3_ccpp_multigases | standard | | fv3 |
Expand Down
Loading