Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster compile and HAFS app to be compiled with 64bit #1578

Merged
merged 27 commits into from
Mar 7, 2023
Merged

Faster compile and HAFS app to be compiled with 64bit #1578

Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
7a2c58e
Updating BL Date to March 02, 2023
sadeghitabas Mar 3, 2023
1b547c7
Faster Compile option turned on for a compile and test per project
sadeghitabas Mar 3, 2023
d6d31e6
added sample tests from each project to be tested with faster compile
sadeghitabas Mar 3, 2023
afed098
log file for full RT on acorn.intel (passed!)
sadeghitabas Mar 3, 2023
95cabd3
log file for full RT on jet.intel (passed!)
sadeghitabas Mar 3, 2023
d1475db
log file for full RT on hera.intel (passed!)
sadeghitabas Mar 3, 2023
45ffbd6
log file for full RT on orion.intel (passed!)
sadeghitabas Mar 3, 2023
ed4f2af
log file for full RT on gaea.intel (passed!)
sadeghitabas Mar 3, 2023
61002ca
mom6/fv3 restart file format updated based on control_p8 test
sadeghitabas Mar 3, 2023
82e036c
update test CNTL_DIR name
sadeghitabas Mar 3, 2023
a24449d
cpld_control_p8_faster test updated for new restart file format
sadeghitabas Mar 3, 2023
c66ebbd
datm_cdeps_control_cfsr_faster test updated for new restart file format
sadeghitabas Mar 3, 2023
4ea21e7
regional_atmaq_faster test updated for new restart file format
sadeghitabas Mar 3, 2023
1880360
revert BL_DATE change
jkbk2004 Mar 3, 2023
10d9730
combining with PR1610
Mar 3, 2023
8e091a9
[AutoRT] cheyenne.gnu Job Completed.
epic-cicd-jenkins Mar 4, 2023
2c20211
add cheyenne.intel RT log: passed
jkbk2004 Mar 4, 2023
15745b4
[AutoRT] hera.intel Job Completed.
jkbk2004 Mar 4, 2023
2aecac8
[AutoRT] hera.gnu Job Completed.
jkbk2004 Mar 4, 2023
9cdced1
[AutoRT] orion.intel Job Completed.
jkbk2004 Mar 4, 2023
ca2bb72
[AutoRT] jet.intel Job Completed.
epic-cicd-jenkins Mar 5, 2023
2f2a561
add gaea.intel RT log: passed
jkbk2004 Mar 6, 2023
b2b63a0
Acorn Intel RT Log
BrianCurtis-NOAA Mar 7, 2023
52c76fa
WCOSS2 Intel RT Log
BrianCurtis-NOAA Mar 7, 2023
a245bab
revert gitmodule changes and pointer
Mar 7, 2023
dca0c43
cosmetic update on gitmodules
jkbk2004 Mar 7, 2023
a4b056b
updating poiter to the correct hash
Mar 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[submodule "FV3"]
path = FV3
url = https://github.com/NOAA-EMC/fv3atm
branch = develop
url = https://github.com/DusanJovic-NOAA/fv3atm
branch = hafs_64bit_fms
[submodule "WW3"]
path = WW3
url = https://github.com/NOAA-EMC/WW3
Expand Down
2 changes: 1 addition & 1 deletion FV3
Open in desktop
Submodule FV3 updated 5 files
+4 −0 moving_nest/fv_moving_nest.F90
+4 −0 moving_nest/fv_moving_nest_main.F90
+4 −0 moving_nest/fv_moving_nest_physics.F90
+4 −0 moving_nest/fv_moving_nest_utils.F90
+4 −0 moving_nest/fv_tracker.F90
5,244 changes: 2,717 additions & 2,527 deletions tests/RegressionTests_acorn.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

426 changes: 213 additions & 213 deletions tests/RegressionTests_cheyenne.gnu.log
View file
Open in desktop

Large diffs are not rendered by default.

2,153 changes: 1,145 additions & 1,008 deletions tests/RegressionTests_cheyenne.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

1,865 changes: 1,058 additions & 807 deletions tests/RegressionTests_gaea.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

324 changes: 162 additions & 162 deletions tests/RegressionTests_hera.gnu.log
View file
Open in desktop

Large diffs are not rendered by default.

1,680 changes: 932 additions & 748 deletions tests/RegressionTests_hera.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

1,548 changes: 866 additions & 682 deletions tests/RegressionTests_jet.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

1,686 changes: 935 additions & 751 deletions tests/RegressionTests_orion.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

13 changes: 13 additions & 0 deletions tests/rt.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ RUN | cpld_control_c48
RUN | cpld_warmstart_c48 | | fv3 |
RUN | cpld_restart_c48 | | | cpld_warmstart_c48

COMPILE | -DAPP=S2SWA -DCCPP_SUITES=FV3_GFS_v17_coupled_p8,FV3_GFS_cpld_rasmgshocnsstnoahmp_ugwp -DFASTER=ON | | fv3 |
RUN | cpld_control_p8_faster | | fv3 |

###################################################################################################################################################################################
# ATM tests #
###################################################################################################################################################################################
Expand Down Expand Up @@ -119,6 +122,10 @@ RUN | control_ras
COMPILE | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v16_fv3wam -D32BIT=ON -DMULTI_GASES=ON | | fv3 |
RUN | control_wam | | fv3 |

COMPILE | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v16,FV3_GFS_v15_thompson_mynn,FV3_GFS_v17_p8,FV3_GFS_v17_p8_rrtmgp,FV3_GFS_v15_thompson_mynn_lam3km,FV3_WoFS_v0 -D32BIT=ON -DFASTER=ON | | fv3 |
RUN | control_p8_faster | | fv3 |
RUN | regional_control_faster | | fv3 |

###################################################################################################################################################################################
# DEBUG ATM tests #
###################################################################################################################################################################################
Expand Down Expand Up @@ -237,6 +244,9 @@ RUN | datm_cdeps_gfs
COMPILE | -DAPP=NG-GODAS -DDEBUG=ON | - wcoss2.intel acorn.intel | fv3 |
RUN | datm_cdeps_debug_cfsr | - wcoss2.intel acorn.intel | fv3 |

COMPILE | -DAPP=NG-GODAS -DFASTER=ON | - wcoss2.intel | fv3 |
RUN | datm_cdeps_control_cfsr_faster | - wcoss2.intel | fv3 |

###################################################################################################################################################################################
# CDEPS Data Atmosphere tests with LND #
###################################################################################################################################################################################
Expand Down Expand Up @@ -276,3 +286,6 @@ COMPILE | -DAPP=ATMAQ -DCCPP_SUITES=FV3_GFS_v15p2
RUN | regional_atmaq | | fv3 |
COMPILE | -DAPP=ATMAQ -DCCPP_SUITES=FV3_GFS_v15p2 -DDEBUG=ON | | fv3 |
RUN | regional_atmaq_debug | - jet.intel gaea.intel cheyenne.intel | fv3 |

COMPILE | -DAPP=ATMAQ -DCCPP_SUITES=FV3_GFS_v15p2 -DFASTER=ON | | fv3 |
RUN | regional_atmaq_faster | | fv3 |
173 changes: 173 additions & 0 deletions tests/tests/control_p8_faster
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
###############################################################################
#
# Global control test GFSv16 atmosphere only at C96L127, P8 configuration
#
###############################################################################

export TEST_DESCR="Compare global control results with previous trunk version"

export CNTL_DIR=control_p8_faster

export LIST_FILES="sfcf000.nc \
sfcf021.nc \
sfcf024.nc \
atmf000.nc \
atmf021.nc \
atmf024.nc \
GFSFLX.GrbF00 \
GFSFLX.GrbF21 \
GFSFLX.GrbF24 \
GFSPRS.GrbF00 \
GFSPRS.GrbF21 \
GFSPRS.GrbF24 \
RESTART/20210323.060000.coupler.res \
RESTART/20210323.060000.fv_core.res.nc \
RESTART/20210323.060000.fv_core.res.tile1.nc \
RESTART/20210323.060000.fv_core.res.tile2.nc \
RESTART/20210323.060000.fv_core.res.tile3.nc \
RESTART/20210323.060000.fv_core.res.tile4.nc \
RESTART/20210323.060000.fv_core.res.tile5.nc \
RESTART/20210323.060000.fv_core.res.tile6.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile1.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile2.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile3.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile4.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile5.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile6.nc \
RESTART/20210323.060000.fv_tracer.res.tile1.nc \
RESTART/20210323.060000.fv_tracer.res.tile2.nc \
RESTART/20210323.060000.fv_tracer.res.tile3.nc \
RESTART/20210323.060000.fv_tracer.res.tile4.nc \
RESTART/20210323.060000.fv_tracer.res.tile5.nc \
RESTART/20210323.060000.fv_tracer.res.tile6.nc \
RESTART/20210323.060000.phy_data.tile1.nc \
RESTART/20210323.060000.phy_data.tile2.nc \
RESTART/20210323.060000.phy_data.tile3.nc \
RESTART/20210323.060000.phy_data.tile4.nc \
RESTART/20210323.060000.phy_data.tile5.nc \
RESTART/20210323.060000.phy_data.tile6.nc \
RESTART/20210323.060000.sfc_data.tile1.nc \
RESTART/20210323.060000.sfc_data.tile2.nc \
RESTART/20210323.060000.sfc_data.tile3.nc \
RESTART/20210323.060000.sfc_data.tile4.nc \
RESTART/20210323.060000.sfc_data.tile5.nc \
RESTART/20210323.060000.sfc_data.tile6.nc"

export_fv3
export NPZ=127
export NPZP=128
export DT_ATMOS=720
export DT_INNER=${DT_ATMOS}
export SYEAR=2021
export SMONTH=03
export SDAY=22
export SHOUR=06
export RESTART_INTERVAL="12 -1"
export OUTPUT_GRID='gaussian_grid'
export WRITE_DOPOST=.true.
export OUTPUT_FH='0 21 24'

# P7 default
export IALB=2
export IEMS=2
export LSM=2
export IOPT_DVEG=4
export IOPT_CRS=2
export IOPT_RAD=3
export IOPT_ALB=1
export IOPT_STC=3
# P8
export IOPT_SFC=3
export IOPT_TRS=2

# FV3 P7 settings
export D2_BG_K1=0.20
export D2_BG_K2=0.04
export PSM_BC=1
# P8
export DDDMP=0.1

# P7 Merra2 Aerosols & NSST
export USE_MERRA2=.true.
export IAER=1011
export NSTF_NAME=2,1,0,0,0

export LHEATSTRG=.false.
export LSEASPRAY=.true.

# P7 UGWP1
export GWD_OPT=2
export DO_UGWP_V1=.false.
export KNOB_UGWP_VERSION=0
export KNOB_UGWP_NSLOPE=1
export DO_UGWP_V0=.true.
export DO_GSL_DRAG_LS_BL=.false.
export DO_GSL_DRAG_SS=.true.
export DO_GSL_DRAG_TOFD=.false.
export DO_UGWP_V1_OROG_ONLY=.false.
export DO_UGWP_V0_NST_ONLY=.false.
export LDIAG_UGWP=.false.

# P7 CA
export DO_CA=.true.
export CA_SGS=.true.
export CA_GLOBAL=.false.
export NCA=1
export NCELLS=5
export NLIVES=12
export NTHRESH=18
export NSEED=1
export NFRACSEED=0.5
export CA_TRIGGER=.true.
export NSPINUP=1
export ISEED_CA=12345

# P7 settings
export TILEDFIX=.true.
export FNALBC="'C96.snowfree_albedo.tileX.nc'"
export FNALBC2="'C96.facsf.tileX.nc'"
export FNTG3C="'C96.substrate_temperature.tileX.nc'"
export FNVEGC="'C96.vegetation_greenness.tileX.nc'"
export FNVETC="'C96.vegetation_type.tileX.nc'"
export FNSOTC="'C96.soil_type.tileX.nc'"
export FNSMCC=${FNSMCC_control}
export FNMSKH=${FNMSKH_control}
export FNVMNC="'C96.vegetation_greenness.tileX.nc'"
export FNVMXC="'C96.vegetation_greenness.tileX.nc'"
export FNSLPC="'C96.slope_type.tileX.nc'"
export FNABSC="'C96.maximum_snow_albedo.tileX.nc'"
export LANDICE=".false."
export FSICL=0
export FSICS=0

export IMP_PHYSICS=8
export LGFDLMPRAD=.false.
export DO_SAT_ADJ=.false.
export DNATS=0
export DZ_MIN=6

#required for NML.IN sharing
export MIN_SEAICE=0.15
export FRAC_GRID=.true.
export MOM6_RESTART_SETTING=n
# P8 (not used for standalone)
export USE_CICE_ALB=.false.

export WRITE_NSFLIP=.true.

export FV3_RUN=control_run.IN
export CCPP_SUITE=FV3_GFS_v17_p8
export FIELD_TABLE=field_table_thompson_noaero_tke
export DIAG_TABLE=diag_table_p8_template
# use same namelist for standalone,coupled P7
export INPUT_NML=cpld_control.nml.IN

# RRTMGP
export DO_RRTMGP=.false.
export DOGP_CLDOPTICS_LUT=.true.
export DOGP_LWSCAT=.true.
export DOGP_SGS_CNV=.true.

if [[ $MACHINE_ID = cheyenne.* ]]; then
TPN=18
fi
87 changes: 87 additions & 0 deletions tests/tests/cpld_control_p8_faster
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
#
# cpld_control P8 test
#

export TEST_DESCR="Fully coupled FV3-CCPP-GOCART-MOM6-CICE-CMEPS-WW3 system - C96MX100"

export CNTL_DIR="cpld_control_p8_faster"

export LIST_FILES="sfcf021.tile1.nc \
sfcf021.tile2.nc \
sfcf021.tile3.nc \
sfcf021.tile4.nc \
sfcf021.tile5.nc \
sfcf021.tile6.nc \
atmf021.tile1.nc \
atmf021.tile2.nc \
atmf021.tile3.nc \
atmf021.tile4.nc \
atmf021.tile5.nc \
atmf021.tile6.nc \
sfcf024.tile1.nc \
sfcf024.tile2.nc \
sfcf024.tile3.nc \
sfcf024.tile4.nc \
sfcf024.tile5.nc \
sfcf024.tile6.nc \
atmf024.tile1.nc \
atmf024.tile2.nc \
atmf024.tile3.nc \
atmf024.tile4.nc \
atmf024.tile5.nc \
atmf024.tile6.nc \
gocart.inst_aod.20210323_0600z.nc4 \
RESTART/20210323.060000.coupler.res \
RESTART/20210323.060000.fv_core.res.nc \
RESTART/20210323.060000.fv_core.res.tile1.nc \
RESTART/20210323.060000.fv_core.res.tile2.nc \
RESTART/20210323.060000.fv_core.res.tile3.nc \
RESTART/20210323.060000.fv_core.res.tile4.nc \
RESTART/20210323.060000.fv_core.res.tile5.nc \
RESTART/20210323.060000.fv_core.res.tile6.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile1.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile2.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile3.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile4.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile5.nc \
RESTART/20210323.060000.fv_srf_wnd.res.tile6.nc \
RESTART/20210323.060000.fv_tracer.res.tile1.nc \
RESTART/20210323.060000.fv_tracer.res.tile2.nc \
RESTART/20210323.060000.fv_tracer.res.tile3.nc \
RESTART/20210323.060000.fv_tracer.res.tile4.nc \
RESTART/20210323.060000.fv_tracer.res.tile5.nc \
RESTART/20210323.060000.fv_tracer.res.tile6.nc \
RESTART/20210323.060000.phy_data.tile1.nc \
RESTART/20210323.060000.phy_data.tile2.nc \
RESTART/20210323.060000.phy_data.tile3.nc \
RESTART/20210323.060000.phy_data.tile4.nc \
RESTART/20210323.060000.phy_data.tile5.nc \
RESTART/20210323.060000.phy_data.tile6.nc \
RESTART/20210323.060000.sfc_data.tile1.nc \
RESTART/20210323.060000.sfc_data.tile2.nc \
RESTART/20210323.060000.sfc_data.tile3.nc \
RESTART/20210323.060000.sfc_data.tile4.nc \
RESTART/20210323.060000.sfc_data.tile5.nc \
RESTART/20210323.060000.sfc_data.tile6.nc \
RESTART/20210323.060000.MOM.res.nc \
RESTART/iced.2021-03-23-21600.nc \
RESTART/ufs.cpld.cpl.r.2021-03-23-21600.nc \
20210323.060000.out_pnt.ww3 \
20210323.060000.out_grd.ww3 "

export_fv3
export_cpl

export RESTART_N=12
export RESTART_INTERVAL="${RESTART_N} -1"
export OUTPUT_FH='0 21 24'

export TOPOEDITS=ufs.topo_edits_011818.nc
export MOM6_ALLOW_LANDMASK_CHANGES=True
export eps_imesh=2.5e-1

export FV3_RUN=cpld_control_run.IN

if [[ $MACHINE_ID = cheyenne.* ]]; then
TPN=18
fi
23 changes: 23 additions & 0 deletions tests/tests/datm_cdeps_control_cfsr_faster
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#
# DATM_CDEPS_MOM6_CICE_CMEPS_CFSR test
#

export TEST_DESCR="DATM_CDEPS-MOM6-CICE-CMEPS_CFSR - control "

export CNTL_DIR="datm_cdeps_control_cfsr_faster"

export LIST_FILES="RESTART/20111002.000000.MOM.res.nc \
RESTART/iced.2011-10-02-00000.nc \
RESTART/DATM_CFSR.cpl.r.2011-10-02-00000.nc"

export_datm_cdeps
export DATM_SRC="CFSR"
export FILENAME_BASE='cfsr.'
export IATM=1760
export JATM=880
export RESTART_N=12
export eps_imesh='2.5e-1'
export TOPOEDITS=ufs.topo_edits_011818.nc
export MOM6_ALLOW_LANDMASK_CHANGES=True

export FV3_RUN=cpld_datm_cdeps.IN
Loading