Sync merge dev_v16 updates into GFSv16.3.0 release branch #807
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- update versions/orion.ver to change python version to 3.7.5 - using python/3.9.2 results in DA job errors Refs: #665
- retire WCOSS_C references from config.fv3.emc.dyn and config.resources.emc.dyn - remove WCOSS_DELL_P3 references from config.fv3.emc.dyn and config.resources.emc.dyn; WCOSS_DELL will disappear after WCOSS2 go-live Refs: #665
- add cdate10 setting to config.prep to support development - use $PDY$cyc to build $cdate10 Refs: #665
- change C96 DELTIM to 450 based on testing on Orion - DELTIM=450 also matches current setting in develop branch Refs: #665
…les - (modify the current dev_v16 with developer testing COM assignment)
- update gldas block in ORION.env to set USE_CFP=NO Refs: #665
- update C384 WRITE_GROUP_GFS to be 2 to match develop and use more nodes - update WRTTASK_PER_GROUP and WRTTASK_PER_GROUP_GFS to set WCOSS2 values but then if values are greater than npe_node_max change variables to be npe_node_max (retains what works on Hera/Orion with 40 pes per node) Refs: #665
- add memory setting for prep job (40GB) - add missing _gfs variables for anal job - add npe_node_$step greater than npe_node_max checks for the analdiag, gldas, post, and esfc jobs (set npe_node_$step to npe_node_max if so) - increase wtime_fcst_gfs to 6hrs - update echgres resources to be prior values and wall off WCOSS2 values - update eobs resources to be consistent with appropriate values from develop - wall off C768 WCOSS2 resources for eobs - update eupd resources to be consistent with develop values Updated resources performed well in two cycled tests on Orion: 1) C192C96L127 2) C384C192L127 Need to test these values on WCOSS2 and Hera for further checks. Refs: #665
- The HOMEobsproc setting in config.base.nco.static is not used in operations and thus not needed in this version of config.base. Refs: #399
… into the envir-p1.h
Comment out the if-block that sets DUMP_SUFFIX="p" when between 2019092100 and 2019110700. The p-dumps aren't available everywhere and should only be used optionally. May remove in future. Refs: #665
- add launcher_PREP="srun" to Hera and Orion env files - add launcher_PREP="mpiexec" to WCOSS2 env file - cleanup prep section of HERA.env to remove unneeded module unload and load of netcdf Refs: #665
Adds a preliminary version of an ecFlow suite generator in a new workflow_generator directory. The suite generator takes in a configuration as a YAML file and generates the appropriate ecFlow suite definition file, as well as copies the appropriate scripts into the correct structure as dictated by the suite. For full details on use, see the README in the workflow_generator directory. Refs #501
Will remove WCOSS_DELL_P3 references after official retirement Refs: #665
Low resolution Orion testing updates for dev_v16
- Update obsproc and prepobs version to rd ones - Add tracker_ver and fit_ver Refs: #665
Based on changes made for WCOSS2, change the enkf forecast jobs to run serially everywhere for stable completions. Refs: #665
- Turn off VSDB - Consolidate tracker section while retaining WCOSS_DELL_P3 support - Consolidate Fit2Obs sections for new version - Remove WCOSS_C references Refs: #665
External package and serial enkf forecast updates
- Update on Dogwood implemented cgroups, which means memory limits are now enforced. - Exclusive jobs must now use "place=exclhost" insted of "place=excl". - Associated exclusive ecf script PBS statements are updated to exclhost. Refs: #399
Need to set exclhost for exclusive jobs on WCOSS2 now after cgroups was implemented. Matches updates to exclusive job ecf script PBS statements. Refs: #399
Will consider removing at later date. Refs: #399
Update "excl" to "exclhost" for exclusive jobs on WCOSS2
- Update config.efcs to run EnKF forecast job with serial netcdf instead of parallel netcdf. Based on joint decision between NCO and EMC. - Update C384 config.fv3.nco.static block to set DELTIM=200 (NCO request). - Update C384 config.fv3.nco.static block to set WRITE_GROUP=2 to speed up serial EnKF forecast jobs to fit inside needed window in ops. Refs: #399
Sync feature/ops wcoss2 into dev_v16 (excl -> exclhost updates)
- remove hyper=true in jgdas_atmos_analysis_calc.ecf - add export nth_echgres=$nth_echgres_gfs when CDUMP=gfs in config.analcalc; for correct thread setting at runtime - add export nth_echgres=4 to analcalc block in config.resources - add export nth_echgres_gfs=12 to analcalc block in config.resources Refs: #399
Hand-off tag to NCO is now EMC-v16.2.0.7 Refs: #399
Final pre-production freeze updates for GFSv16.2.0 package on WCOSS2
wcoss2 cycled ecflow script with developer assigned path
Sync merge feature/ops-wcoss2 into dev_v16
- NCO updated the default path for HOMENHC and tested it in prod on WCOSS2 during NHC test Refs: #399
- Based on testing on Dogwood after some WCOSS2 updates some memory and resource adjustments were made by NCO. - Memory updates to the gempak, awips, and fbwnd job ecf scripts. - Resource adjustments to remedy oversubscription errors in the post and postsnd jobs. Refs: #399
The gfspostsnd job was oversubscribing CPUs on WCOSS2 after updates on Dogwood. Updating resources settings to get them matching and working. Refs: #399
- Add updated memory values for awips and gempak jobs into resource configs to match similar updates in ecf scripts Refs: #399
WCOSS2 GFSv16.2.0 resource updates and NHC change
Sync feature/ops-wcoss2 into dev_v16
WalterKolczynski-NOAA
approved these changes
May 23, 2022
kayeekayee
pushed a commit
to kayeekayee/global-workflow
that referenced
this pull request
May 30, 2024
…AA-EMC#447) * CCPP physics updates from PR NOAA-EMC#806(only diagnostic changes for RRTMGP - small diag-only RT impact for tests involving RRTMGP),NOAA-EMC#807 (only aborting model if sfc emis file not present when needed) ,NOAA-EMC#813(only SCM-specific physics changes)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR is a sync merge to bring recent GFSv16.2.0 (
feature/ops-wcoss2anddev_v16) updates into therelease/gfs.v16.3.0branch via a PR from thedev_v16branch. Updates include:dev_v16on Orion and WCOSS2 for supporting GFSv16.2.0 dev users: PRs Low resolution Orion testing updates for dev_v16 #779, External package and serial enkf forecast updates #782dev_v16: PR ecFlow Suite Generator Application #777There will be additional PRs in the coming weeks as testing of the
dev_v16branch on WCOSS2, Orion, and Hera continues. Priorities are to preparedev_v16(andrelease/gfs.v.16.3.0by extension) to support dev users on WCOSS2 first and then the R&D platforms.Type of change
Sync merge with parent branches.
How Has This Been Tested?
Refs: #744
@emilyhcliu FYI