From 683692db13f0df1e2b7f1b1577eaa6beb3226e8e Mon Sep 17 00:00:00 2001 From: YaliMao-NOAA <53870326+YaliMao-NOAA@users.noreply.github.com> Date: Tue, 8 Oct 2024 19:43:05 +0000 Subject: [PATCH] Merge changes made for Release/wafs.v7 to develop branch (#73) * remove rdhpcs options (#42) * remove hera/orion modulefiles. rename drivers without wcoss2 and remove detect_machine.sh ush scripts * cleanup versions and make fix files not exec * Update README.md * EE2 review updates (#44) * update wafs_upp to EE2 * update upp job per EE2 standards * fix scripting errors * ignore the dirty upp.fd directory as it creates build artifacts that are not captured in its .gitignore * itag is not a namlist in this version of UPP. Go Figure! * remove copying of analysis master file, and move setting of some variables to exscript * EE2 mods for grib2 1p25 and 0p25 (no blending) * update blending scripts for EE2 * fix grib1 jobs * apply EE2 fixes to gcip * some more updates on gcip * Bugfixes on previous PR that was merged prior to testing (#45) * remove unnecessary hours for grib, the offline UPP executable should match EE2 convention, setting up ecflow for development use with multiple expids * revert changes .gitmodules * move upp.fd to wafs_upp.fd per EE2 * ensure git submodule update is performed in the right directory * fix a couple of COMIN bugs * update experiment paths * update JWAFS_GFS_MANAGER so it is similar to all other jjobs * bugfixes discovered while testing * Copy the folder of upp parm to WAFS/parm after copying gtg.config.gfs from GTG repository to upp parm * UPP didn't generate WAFS master file correctly. To fix it, add a line (even blank) between 'flxfile' and '&nampgb' to UPP itag. * Made the non-ecflow version back to work and added HOMEwafs flexibility * only copy relevant UPP parm files to WAFS vertical structure * update doco * add draft of release_notes --------- Co-authored-by: yali mao * Update script document blocks, bug fixes of previous PRs (#48) * Change all command with "``" to "$()" * GCIP doesn't need SENDDBN. * Don't need wmo folder since GRIB2_0P25 products are not added WMO headers. * Change back to {EXECwafs}/${pgm} from {DATA}/${pgm} * For UPP, move environment variables from scripts to jobs * Add SENDDBN_NTC to jobs and correct dbn_alert for SENDDBN_NTC and SENDDBN * Update document blocks of the scripts * For WAFS GRIB1 scripts, move defination of jobsuffix from ush/mkwfsgbl.sh to script/exwafs_grib.sh since fhr doesn't have the same value. * Add descriptions of JWAFS_GFS_MANAGER --------- Co-authored-by: yali mao Co-authored-by: yali mao * Extend waiting time window of UK data to 25 minutes (#49) Co-authored-by: yali mao * Adjust forecast hours up to 36 for the additional levels per AWC request (#50) * AWC needs extra levels up to F036 * Update branch of UPP in .gitmodules --------- Co-authored-by: yali mao Co-authored-by: yali mao * Update UPP tag to upp_wafs_v7.0.0 (#52) * update UPP code revision to upp_wafs_v7.0.0 * Update UPP tag in .gitmodules --------- Co-authored-by: yali mao * AWC request adjusted, blending wall time extended (#53) * Extend the wall time of the job card for the additional 5 minutes of waiting UK data * Modified scripts for the additional levels on the second request from AWC --------- Co-authored-by: yali mao * NRT with ecflow (#54) * add possibilty of doing in NRT * depend on all GFS data, not just log files * fix extensions to atm and sfc files * GCIP jobs in addition to JWAFS_GFS_MANAGER, have a time trigger in NRT * gcip time trigger can be anytime after the time specified * remove GFS forecast job triggers for NRT and rely on time * remove GFS job triggers in experimental runs. * add time triggers for blending jobs based on PR review comments * First version of Release Note for WAFS.v7.0.0 (#55) * First version of Release Note for WAFS.v7.0.0 * Update docs/Release_Notes.md Co-authored-by: Kate Friedman * Update docs/Release_Notes.md Co-authored-by: Rahul Mahajan * Update docs/Release_Notes.md Co-authored-by: Rahul Mahajan * Modified Release Notes from feedback from Rahual and Huiya * Adjust a table in Release Notes * Update Release Notes according to the WAFS separation kickout slides --------- Co-authored-by: yali mao Co-authored-by: Kate Friedman Co-authored-by: Rahul Mahajan Co-authored-by: yali mao * Add ecflow manual text to .ecf files (#58) Co-authored-by: yali mao * Remove processing for fhrs = 1,2,3,4,5 for UPP in WAFS (#59) * remove processing of forecast hours 1-5 for UPP in WAFS * unindent the task * update exwafs_gfs_manager.sh for hrs 1-5 in upp * Update release note and .ecf manuals (#60) * Add more details to .ecf manuals of upp and grib2_0p25 * Update Release Notes of stopping WAFS master files when FFF is between [001-005] * Update UPP com size after removing WAFS master files for forecast hours between [001-005] --------- Co-authored-by: yali mao * Update blending script to send email when UK data is missing (#61) * Update blending script to send email when UK data is missing 1. usonly.emailbody is differentiated for each forecast hour with missing UK data 2. Remove the condition of sending UK unblended data if US unblended data is missing. It won't happen because the job itself won't get triggered if US unblended data is missing * Added an ecflow client test script * Update dev/ecf/README.md * Update ecf README.md --------- Co-authored-by: yali mao * make the NRT suite repeat daily (#62) * To fix bugzilla 1370 and 1371 for WAFS blending job, (#68) 1. change variable name 'maillist' to 'MAILTO' 2. assign the value in job cards instead of in scripts Co-authored-by: yali mao * Change blending job to MPMD to fix bugzilla 1593. Fix bugzilla 1226 (#69) * Change blending job to MPMD to fix bugzilla 1593, meanwhile fix bugzilla 1226 The MPMD change for bugzilla 1593 is for NCO who wants to receive one single email combining all forecast hours with missing UK data For bugzilla 1226, AWC is fine with dbn_alert of US unblended data earlier in JWAFS_GRIB2_0P25 job Bugzilla 1593 - Improve email notification for missing UK WAFS data Bugzilla 1226 - Eliminate the duplicated dbn_alert for unblended gfs wafs data * Update release note and modify the driver * 1. If US unblended data is missing, don't quit silently, instead send out email and dbn_alert. 2. Add not-blended email and dbn_alert if both UK and US unblended files are missing 3. Change fhours from a string to an array 4. Bug fix and code improvement * Update blending scripts * Bug fix * Update the way of handling err and removing np variable for MPIRUN * Update scripts/exwafs_grib2_0p25_blending.sh Co-authored-by: Rahul Mahajan --------- Co-authored-by: yali mao Co-authored-by: yali mao Co-authored-by: yali mao Co-authored-by: yali mao Co-authored-by: Rahul Mahajan * Update ecflow after switching blending to MPMD parallel run. (#72) * Update ecflow after switching blending to MPMD parallel run. 1. Don't need to setup ecflow links for blending 2. In ecflow definations, change event triggers of each forecast hour to f048 of the upstream completion Change COMROOT from 'com' to '%ENVIR%/com' * Remove ecf/scripts/grib2/0p25/blending/jwafs_grib2_0p25_blending_f*.ecf from .gitignore --------- Co-authored-by: yali mao --------- Co-authored-by: Rahul Mahajan Co-authored-by: yali mao Co-authored-by: yali mao Co-authored-by: yali mao Co-authored-by: yali mao Co-authored-by: yali mao Co-authored-by: yali mao Co-authored-by: Kate Friedman Co-authored-by: yali mao Co-authored-by: yali mao Co-authored-by: yali mao Co-authored-by: yali mao --- .gitignore | 3 +- .gitmodules | 6 +- dev/driver/driver_WAFS.README | 33 +- .../{run_JWAFS_GCIP.wcoss2 => run_JWAFS_GCIP} | 5 +- .../{run_JWAFS_GRIB.wcoss2 => run_JWAFS_GRIB} | 8 +- ...GRIB2_0P25.wcoss2 => run_JWAFS_GRIB2_0P25} | 8 +- ...G.wcoss2 => run_JWAFS_GRIB2_0P25_BLENDING} | 27 +- ...GRIB2_1P25.wcoss2 => run_JWAFS_GRIB2_1P25} | 5 +- .../{run_JWAFS_UPP.wcoss2 => run_JWAFS_UPP} | 17 +- dev/driver/submit.run_JWAFS.sh | 64 + dev/driver/submit.run_JWAFS.wcoss2.sh | 47 - dev/ecf/README.md | 49 +- dev/ecf/run.ecflow_pdy.sh | 49 + dev/ecf/setup_ecf.sh | 95 +- dev/modulefiles/wafs.sh | 3 + docs/Release_Notes.md | 223 +++ ecf/def/wafs.def.tmpl | 240 +-- ecf/def/wafs_nrt.def.tmpl | 1640 +++++++++++++++++ ecf/scripts/gcip/jwafs_gcip_master.ecf | 15 + ecf/scripts/grib/jwafs_grib_master.ecf | 10 + .../blending/jwafs_grib2_0p25_blending.ecf | 71 + .../jwafs_grib2_0p25_blending_master.ecf | 52 - .../grib2/0p25/jwafs_grib2_0p25_master.ecf | 12 + .../grib2/1p25/jwafs_grib2_1p25_master.ecf | 10 + ecf/scripts/jwafs_gfs_manager.ecf | 15 +- ecf/scripts/upp/jwafs_upp_master.ecf | 13 + ecf/setup_ecf_links.sh | 29 +- fix/wafs/grib_wafs.namelist | 0 fix/wafs/grib_wfsgfs00 | 0 fix/wafs/grib_wfsgfs06 | 0 fix/wafs/grib_wfsgfs12a | 0 fix/wafs/grib_wfsgfs12b | 0 fix/wafs/grib_wfsgfs18a | 0 fix/wafs/grib_wfsgfs18b | 0 fix/wafs/grib_wfsgfs24a | 0 fix/wafs/grib_wfsgfs24b | 0 fix/wafs/grib_wfsgfs30a | 0 fix/wafs/grib_wfsgfs30b | 0 fix/wafs/grib_wfsgfs36 | 0 fix/wafs/grib_wfsgfs42 | 0 fix/wafs/grib_wfsgfs48 | 0 fix/wafs/grib_wfsgfs60 | 0 fix/wafs/grib_wfsgfs72 | 0 fix/wafs/wafs_blending_0p25_admin_msg | 0 jobs/JWAFS_GCIP | 149 +- jobs/JWAFS_GFS_MANAGER | 101 +- jobs/JWAFS_GRIB | 119 +- jobs/JWAFS_GRIB2_0P25 | 112 +- jobs/JWAFS_GRIB2_0P25_BLENDING | 131 +- jobs/JWAFS_GRIB2_1P25 | 118 +- jobs/JWAFS_UPP | 93 +- modulefiles/wafs_common.lua | 17 - modulefiles/wafs_hera.intel.lua | 17 - modulefiles/wafs_orion.intel.lua | 17 - scripts/exwafs_gcip.sh | 261 ++- scripts/exwafs_gfs_manager.sh | 30 +- scripts/exwafs_grib.sh | 101 +- scripts/exwafs_grib2_0p25.sh | 189 +- scripts/exwafs_grib2_0p25_blending.sh | 350 +--- scripts/exwafs_grib2_1p25.sh | 267 ++- scripts/exwafs_upp.sh | 228 ++- sorc/CMakeLists.txt | 2 +- sorc/build_all.sh | 11 +- sorc/build_upp.sh | 33 +- sorc/build_wafs.sh | 10 +- sorc/checkout_upp.sh | 4 +- sorc/upp.fd | 1 - sorc/wafs_upp.fd | 1 + ush/detect_machine.sh | 92 - ush/mkwfsgbl.sh | 130 +- ush/module-reset.sh | 115 -- ush/wafs_grib2_0p25_blending.sh | 150 ++ ush/wafs_upp.sh | 43 - versions/build.ver | 0 versions/run.ver | 25 +- 75 files changed, 3481 insertions(+), 2185 deletions(-) rename dev/driver/{run_JWAFS_GCIP.wcoss2 => run_JWAFS_GCIP} (97%) mode change 100644 => 100755 rename dev/driver/{run_JWAFS_GRIB.wcoss2 => run_JWAFS_GRIB} (95%) rename dev/driver/{run_JWAFS_GRIB2_0P25.wcoss2 => run_JWAFS_GRIB2_0P25} (95%) mode change 100644 => 100755 rename dev/driver/{run_JWAFS_GRIB2_0P25_BLENDING.wcoss2 => run_JWAFS_GRIB2_0P25_BLENDING} (82%) mode change 100644 => 100755 rename dev/driver/{run_JWAFS_GRIB2_1P25.wcoss2 => run_JWAFS_GRIB2_1P25} (97%) mode change 100644 => 100755 rename dev/driver/{run_JWAFS_UPP.wcoss2 => run_JWAFS_UPP} (91%) create mode 100755 dev/driver/submit.run_JWAFS.sh delete mode 100644 dev/driver/submit.run_JWAFS.wcoss2.sh create mode 100755 dev/ecf/run.ecflow_pdy.sh create mode 100644 docs/Release_Notes.md create mode 100644 ecf/def/wafs_nrt.def.tmpl create mode 100755 ecf/scripts/grib2/0p25/blending/jwafs_grib2_0p25_blending.ecf delete mode 100755 ecf/scripts/grib2/0p25/blending/jwafs_grib2_0p25_blending_master.ecf mode change 100755 => 100644 fix/wafs/grib_wafs.namelist mode change 100755 => 100644 fix/wafs/grib_wfsgfs00 mode change 100755 => 100644 fix/wafs/grib_wfsgfs06 mode change 100755 => 100644 fix/wafs/grib_wfsgfs12a mode change 100755 => 100644 fix/wafs/grib_wfsgfs12b mode change 100755 => 100644 fix/wafs/grib_wfsgfs18a mode change 100755 => 100644 fix/wafs/grib_wfsgfs18b mode change 100755 => 100644 fix/wafs/grib_wfsgfs24a mode change 100755 => 100644 fix/wafs/grib_wfsgfs24b mode change 100755 => 100644 fix/wafs/grib_wfsgfs30a mode change 100755 => 100644 fix/wafs/grib_wfsgfs30b mode change 100755 => 100644 fix/wafs/grib_wfsgfs36 mode change 100755 => 100644 fix/wafs/grib_wfsgfs42 mode change 100755 => 100644 fix/wafs/grib_wfsgfs48 mode change 100755 => 100644 fix/wafs/grib_wfsgfs60 mode change 100755 => 100644 fix/wafs/grib_wfsgfs72 mode change 100755 => 100644 fix/wafs/wafs_blending_0p25_admin_msg delete mode 100644 modulefiles/wafs_common.lua delete mode 100644 modulefiles/wafs_hera.intel.lua delete mode 100644 modulefiles/wafs_orion.intel.lua delete mode 160000 sorc/upp.fd create mode 160000 sorc/wafs_upp.fd delete mode 100755 ush/detect_machine.sh delete mode 100755 ush/module-reset.sh create mode 100755 ush/wafs_grib2_0p25_blending.sh delete mode 100755 ush/wafs_upp.sh mode change 100755 => 100644 versions/build.ver diff --git a/.gitignore b/.gitignore index 9d24c3c..8474b27 100644 --- a/.gitignore +++ b/.gitignore @@ -17,9 +17,8 @@ ecf/scripts/upp/jwafs_upp_f*.ecf ecf/scripts/upp/jwafs_upp_anl.ecf ecf/scripts/grib2/1p25/jwafs_grib2_1p25_f*.ecf ecf/scripts/grib2/0p25/jwafs_grib2_0p25_f*.ecf -ecf/scripts/grib2/0p25/blending/jwafs_grib2_0p25_blending_f*.ecf ecf/scripts/gcip/jwafs_gcip_f*.ecf ecf/scripts/grib/jwafs_grib_f*.ecf # Ignore wafs.def -ecf/def/wafs.def +ecf/def/wafs*.def diff --git a/.gitmodules b/.gitmodules index 3f884a8..65ea029 100644 --- a/.gitmodules +++ b/.gitmodules @@ -1,3 +1,5 @@ -[submodule "sorc/upp.fd"] - path = sorc/upp.fd +[submodule "sorc/wafs_upp.fd"] + path = sorc/wafs_upp.fd url = https://github.com/NOAA-EMC/UPP + branch = upp_wafs_v7.0.0 + ignore = dirty diff --git a/dev/driver/driver_WAFS.README b/dev/driver/driver_WAFS.README index ea4b398..1e7db7b 100644 --- a/dev/driver/driver_WAFS.README +++ b/dev/driver/driver_WAFS.README @@ -3,12 +3,12 @@ How to use the drivers for WAFS jobs ********************************************************************* There are 1 standalone UPP job and 5 WAFS jobs -run_JWAFS_UPP.wcoss2 -run_JWAFS_GRIB.wcoss2 -run_JWAFS_GCIP.wcoss2 -run_JWAFS_GRIB2.wcoss2 -run_JWAFS_GRIB2_0P25.wcoss2 -run_JWAFS_BLENDING_0P25.wcoss2 +run_JWAFS_UPP +run_JWAFS_GRIB +run_JWAFS_GCIP +run_JWAFS_GRIB2 +run_JWAFS_GRIB2_0P25 +run_JWAFS_BLENDING_0P25 JWAFS_UPP is a downstream of GFS model outputs in netcdf @@ -30,7 +30,7 @@ Project location, defined by HOMEwafs export NWROOT=(the location holds your WAFS local inventory) export HOMEwafs=${HOMEwafs:-${NWROOT}/{the folder of your WAFS local inventory)} -A user needs to modify NWROOT or HOMEwafs to the project location +A user needs to modify NWROOT or HOMEwafs to the project location Step 2: output and working folder ===================== @@ -47,22 +47,23 @@ Change file names of error messages and script printout: Step 3: input data ===================== Specify date and cycle: -export PDY=(date as YYYYMMDD) -export cyc=(cycle as CC) +export CDATE=(date as YYYYMMDDHH) +PDY is inferred from CDATE as YYYYMMDD +cyc is inferred from CDATE as HH Specify COMIN by either setting COMIN or by setting COMPATH used by compath.py export COMPATH=(location of canned data ending with $envir/com/$RUN structure) export COMIN=(folder with a full path) -1) run_JWAFS_GRIB2.WCOSS2 and run_JWAFS_GRIB2_0P25.WCOSS2 +1) run_JWAFS_GRIB2 and run_JWAFS_GRIB2_0P25 Ensure COMPATH includes both GFS and WAFS -2) run_JWAFS_BLENDING_0P25.wcoss2 has two inputs, UK and US. +2) run_JWAFS_BLENDING_0P25 has two inputs, UK and US. US data is a downstream product of JWAFS_GRIB2_0P25, so either: -export COMPATH=($COMROOT/wafs where COMROOT is of run_JWAFS_GRIB2_0P25.wcoss2) +export COMPATH=($COMROOT/wafs where COMROOT is of run_JWAFS_GRIB2_0P25) or: -export COMINus=($COMOUT of run_JWAFS_GRIB2_0P25.wcoss2) +export COMINus=($COMOUT of run_JWAFS_GRIB2_0P25) UK data is from DCOM, either: @@ -70,7 +71,7 @@ export DCOMROOT=(a folder canned data with structure of $PDY/wgrbbul/ukmet_wafs) or: export COMINuk=${COMINuk:-$DCOMROOT/$PDY/wgrbbul/ukmet_wafs} -3) run_JWAFS_GCIP.wcoss2 has 4 inputs: gfs master files of f000 and f003, bufr, satellite and radar +3) run_JWAFS_GCIP has 4 inputs: gfs master files of f000 and f003, bufr, satellite and radar Ensure COMPATH includes GFS BUFR data is dumped from DCOM. @@ -89,7 +90,7 @@ either ensure COMPATH includes radar, or: export COMINradar=${COMINradar:-$COMROOT/radarl2/$radarl2_ver)/radar.$PDY} -4) run_JWAFS_UPP.wcoss2 and run_JWAFS_GRIB.wcoss2 +4) run_JWAFS_UPP and run_JWAFS_GRIB Ensure COMPATH includes GFS @@ -101,5 +102,5 @@ Test by comparing outputs to operational products in different ways. 2) Use wgrib2 to check date, cycle and forecast hour are correct. 3) Check the number of fields and records are correct by comparing the control files generated by g2ctl -4) Plot and compare by using GrADS to have a sanity check whether +4) Plot and compare by using GrADS to have a sanity check whether the differences are reasonable. diff --git a/dev/driver/run_JWAFS_GCIP.wcoss2 b/dev/driver/run_JWAFS_GCIP old mode 100644 new mode 100755 similarity index 97% rename from dev/driver/run_JWAFS_GCIP.wcoss2 rename to dev/driver/run_JWAFS_GCIP index 25cc552..5164afb --- a/dev/driver/run_JWAFS_GCIP.wcoss2 +++ b/dev/driver/run_JWAFS_GCIP @@ -55,10 +55,9 @@ export RUN=wafs ############################################ # user defined ############################################ -export PDY=20240703 - export cyc=${cyc:-00} export fhr=000 +export PDY=$(cut -c 7-14 $COMROOT/date/t${cyc}z) # wafs_gcip for generating global icing analysis for every 3 hours export job=wafs_gcip_${cyc} @@ -66,7 +65,7 @@ export job=wafs_gcip_${cyc} export pid=${pid:-$$} export jobid=${job}.${pid} -USER=`whoami` +USER=$(whoami) ############################################ # SENDCOM=YES--Copy output file to /com diff --git a/dev/driver/run_JWAFS_GRIB.wcoss2 b/dev/driver/run_JWAFS_GRIB similarity index 95% rename from dev/driver/run_JWAFS_GRIB.wcoss2 rename to dev/driver/run_JWAFS_GRIB index 277cca4..58ca373 100755 --- a/dev/driver/run_JWAFS_GRIB.wcoss2 +++ b/dev/driver/run_JWAFS_GRIB @@ -51,13 +51,9 @@ export RUN=wafs ############################################ # user defined ############################################ -#export PDY=`$NDATE -24 | cut -c 1-8` -export PDY=`cut -c 7-14 $COMROOT/date/t00z` -export PDY=20240703 - export cyc=${cyc:-00} - export fhr=006 +export PDY=$(cut -c 7-14 $COMROOT/date/t${cyc}z) # wafs for octants (8 WAFS files) export job=wafs_grib_${cyc} @@ -65,7 +61,7 @@ export job=wafs_grib_${cyc} export pid=${pid:-$$} export jobid=${job}.${pid} -USER=`whoami` +USER=$(whoami) ############################################ # SENDCOM=YES--Copy output file to /com diff --git a/dev/driver/run_JWAFS_GRIB2_0P25.wcoss2 b/dev/driver/run_JWAFS_GRIB2_0P25 old mode 100644 new mode 100755 similarity index 95% rename from dev/driver/run_JWAFS_GRIB2_0P25.wcoss2 rename to dev/driver/run_JWAFS_GRIB2_0P25 index 6a9c607..d5e4432 --- a/dev/driver/run_JWAFS_GRIB2_0P25.wcoss2 +++ b/dev/driver/run_JWAFS_GRIB2_0P25 @@ -52,13 +52,9 @@ export RUN=wafs ############################################ # user defined ############################################ -#export PDY=`$NDATE -24 | cut -c 1-8` -export PDY=`cut -c 7-14 $COMROOT/date/t00z` -export PDY=20240703 - export cyc=${cyc:-00} - export fhr=006 +export PDY=$(cut -c 7-14 $COMROOT/date/t${cyc}z) # wafs_grib2 for generating global WAFS on grid 45 export job=wafs_grib2_0p25_${cyc} @@ -66,7 +62,7 @@ export job=wafs_grib2_0p25_${cyc} export pid=${pid:-$$} export jobid=${job}.${pid} -USER=`whoami` +USER=$(whoami) ############################################ # SENDCOM=YES--Copy output file to /com diff --git a/dev/driver/run_JWAFS_GRIB2_0P25_BLENDING.wcoss2 b/dev/driver/run_JWAFS_GRIB2_0P25_BLENDING old mode 100644 new mode 100755 similarity index 82% rename from dev/driver/run_JWAFS_GRIB2_0P25_BLENDING.wcoss2 rename to dev/driver/run_JWAFS_GRIB2_0P25_BLENDING index a52e129..e3b3ca5 --- a/dev/driver/run_JWAFS_GRIB2_0P25_BLENDING.wcoss2 +++ b/dev/driver/run_JWAFS_GRIB2_0P25_BLENDING @@ -3,8 +3,8 @@ #PBS -j oe #PBS -o /lfs/h2/emc/ptmp/yali.mao/working_wafs/log.wafs_grib2_0p25_blending #PBS -N wafs_blending_0p25 -#PBS -l walltime=00:25:00 -#PBS -l select=1:ncpus=1 +#PBS -l walltime=00:30:00 +#PBS -l select=1:ncpus=27:mem=50GB #PBS -q debug #PBS -l debug=true #PBS -A GFS-DEV @@ -57,13 +57,8 @@ export RUN=wafs ############################################ # user defined ############################################ -#export PDY=`$NDATE -24 | cut -c 1-8` -export PDY=`cut -c 7-14 $COMROOT/date/t00z` -export PDY=20240703 - export cyc=${cyc:-00} - -export fhr=006 +export PDY=$(cut -c 7-14 $COMROOT/date/t${cyc}z) # wafs_blending for blending icing turbulence of US and UK export job=wafs_blending_0p25_${cyc} @@ -71,7 +66,7 @@ export job=wafs_blending_0p25_${cyc} export pid=${pid:-$$} export jobid=${job}.${pid} -USER=`whoami` +USER=$(whoami) ############################################ # SENDCOM=YES--Copy output file to /com @@ -87,16 +82,24 @@ export KEEPDATA=YES ############################################ # Define DATA PCOM, COMOUT and COMIN ############################################ - +# For COMOUT export COMROOT=/lfs/h2/emc/ptmp/$USER/wafs_dwn/$envir/com -export COMPATH=$COMROOT/wafs +# For COMIN +# export COMPATH=$COMROOT/wafs +export COMPATH=/lfs/h2/emc/ptmp/yali.mao/wafsx001/prod/com/wafs -export DATA=/lfs/h2/emc/ptmp/$USER/working_wafs/blending.$jobid +export DATA=/lfs/h2/emc/ptmp/$USER/working_wafs ############################################ # run the job ############################################# +# Set up mailing list +if [[ "${envir}" != "prod" ]]; then + MAILTO="nco.spa@noaa.gov" +fi +export MAILTO=${MAILTO:-"nco.spa@noaa.gov,ncep.sos@noaa.gov,nco.sos@noaa.gov,nco.hpc.dataflow@noaa.gov"} + sh $HOMEwafs/jobs/JWAFS_GRIB2_0P25_BLENDING exit diff --git a/dev/driver/run_JWAFS_GRIB2_1P25.wcoss2 b/dev/driver/run_JWAFS_GRIB2_1P25 old mode 100644 new mode 100755 similarity index 97% rename from dev/driver/run_JWAFS_GRIB2_1P25.wcoss2 rename to dev/driver/run_JWAFS_GRIB2_1P25 index 28bfeec..0c4437c --- a/dev/driver/run_JWAFS_GRIB2_1P25.wcoss2 +++ b/dev/driver/run_JWAFS_GRIB2_1P25 @@ -52,10 +52,9 @@ export RUN=wafs ############################################ # user defined ############################################ -export PDY=20240703 - export cyc=${cyc:-00} export fhr=006 +export PDY=$(cut -c 7-14 $COMROOT/date/t${cyc}z) export job=wafs_grib2_1p25_${cyc} @@ -64,7 +63,7 @@ export jobid=${job}.${pid} # wafs_grib2 for generating global WAFS on grid 45 -USER=`whoami` +USER=$(whoami) ############################################ # SENDCOM=YES--Copy output file to /com diff --git a/dev/driver/run_JWAFS_UPP.wcoss2 b/dev/driver/run_JWAFS_UPP similarity index 91% rename from dev/driver/run_JWAFS_UPP.wcoss2 rename to dev/driver/run_JWAFS_UPP index 8fdc5e5..4812e79 100755 --- a/dev/driver/run_JWAFS_UPP.wcoss2 +++ b/dev/driver/run_JWAFS_UPP @@ -15,9 +15,7 @@ set -x # specify computation resource -export threads=1 export MP_LABELIO=yes -export OMP_NUM_THREADS=$threads export MPIRUN='mpiexec -l -n 126 -ppn 126 --cpu-bind depth --depth 1' echo "starting time $PBS_JOBNAME" @@ -63,19 +61,13 @@ export envir=prod export NET=wafs export RUN=wafs -export RUNupp=gfs - ############################################ # user defined ############################################ -#export PDY=`$NDATE -24 | cut -c 1-8` -export PDY=`cut -c 7-14 $COMROOT/date/t00z` -export PDY=20240617 - export cyc=${cyc:-00} export cycle=t${cyc}z - -export fhr="018" +export fhr=006 +export PDY=$(cut -c 7-14 $COMROOT/date/t${cyc}z) # wafs_grib2 for generating global WAFS on grid 45 export job=wafs_upp_${cyc} @@ -83,7 +75,7 @@ export job=wafs_upp_${cyc} export pid=${pid:-$$} export jobid=${job}.${pid} -USER=`whoami` +USER=$(whoami) ############################################ # SENDCOM=YES--Copy output file to /com @@ -101,7 +93,7 @@ export KEEPDATA=YES export COMROOT=/lfs/h2/emc/ptmp/$USER/wafs_dwn/$envir/com -export COMPATH=${COMPATHgfs:-/lfs/h1/ops/prod/com/$RUNupp} +export COMPATH=${COMPATHgfs:-/lfs/h1/ops/prod/com/gfs} DATAroot=/lfs/h2/emc/ptmp/$USER/working_wafs @@ -112,4 +104,3 @@ export DATA=$DATAroot/upp.f$fhr.${jobid} $HOMEwafs/jobs/JWAFS_UPP echo $? - diff --git a/dev/driver/submit.run_JWAFS.sh b/dev/driver/submit.run_JWAFS.sh new file mode 100755 index 0000000..e3e8e41 --- /dev/null +++ b/dev/driver/submit.run_JWAFS.sh @@ -0,0 +1,64 @@ +#!/bin/bash + +set -eu + +# Get the root of the cloned WAFS directory +readonly DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}")")/../.." && pwd -P) + +job=${1?"Must specify a job to submit"} +PDYcyc=${2:-"2024081918"} + +tmpdir=/lfs/h2/emc/ptmp/${USER}/working_wafs.${job}_${PDYcyc:0:8} +mkdir -p $tmpdir +cd $tmpdir + +jobcard=run_JWAFS_${job^^} +cp "${DIR_ROOT}/dev/driver/${jobcard}" . + +if [ $job = 'upp' ]; then + FHOURS="anl 000 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 \ + 027 030 033 036 039 042 045 048 054 060 066 072 078 084 090 096 102 108 114 120" +elif [ $job = 'gcip' ]; then + FHOURS="000 003" +elif [ $job = 'grib2_0p25' ]; then + export FHOUT_GFS=${FHOUT_GFS:-1} + if [ $FHOUT_GFS -eq 3 ]; then #27 + export FHOURS=${FHOURS:-"6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 54 60 66 72 78 84 90 96 102 108 114 120"} + else #39 + export FHOURS=${FHOURS:-"6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 27 30 33 36 39 42 45 48 54 60 66 72 78 84 90 96 102 108 114 120"} + fi +elif [ $job = 'grib2_1p25' ]; then + export FHOURS=${FHOURS:-"00 06 09 12 15 18 21 24 27 30 33 36 42 48 54 60 66 72"} +elif [ $job = 'grib' ]; then + export FHOURS=${FHOURS:-"06 12 18 24 30 36 42 48 54 60 66 72"} +elif [ $job = 'grib2_0p25_blending' ]; then + sed -e "s|log.wafs_$job|log.wafs_$job|g" \ + -e "s|HOMEwafs=.*|HOMEwafs=$DIR_ROOT|g" \ + -e "s|PDY=.*|PDY=${PDYcyc:0:8}|g" \ + -e "s|cyc=.*|cyc=${PDYcyc:8:2}|g" \ + -e "s|working_wafs|working_wafs.${job}_${PDYcyc:0:8}|g" \ + -i $jobcard + qsub $jobcard + exit +fi + +for fhr in $FHOURS; do + if [ $job = 'grib' ]; then + fhr="$(printf "%02d" $(( 10#$fhr )) )" + else + if [ ! $fhr = "anl" ] ; then + fhr="$(printf "%03d" $(( 10#$fhr )) )" + fi + fi + + sed -e "s|log.wafs_$job|log.wafs_$job.$fhr|g" \ + -e "s|HOMEwafs=.*|HOMEwafs=$DIR_ROOT|g" \ + -e "s|PDY=.*|PDY=${PDYcyc:0:8}|g" \ + -e "s|cyc=.*|cyc=${PDYcyc:8:2}|g" \ + -e "s|fhr=.*|fhr=$fhr|g" \ + -e "s|working_wafs|working_wafs.${job}_${PDYcyc:0:8}|g" \ + $jobcard >$jobcard.$fhr + + qsub $jobcard.$fhr +done + diff --git a/dev/driver/submit.run_JWAFS.wcoss2.sh b/dev/driver/submit.run_JWAFS.wcoss2.sh deleted file mode 100644 index 481ba10..0000000 --- a/dev/driver/submit.run_JWAFS.wcoss2.sh +++ /dev/null @@ -1,47 +0,0 @@ -job=$1 -JOB=`echo $job | tr 'a-z' 'A-Z'` - -PDY=20240819 -cyc=18 - -tmpdir=/lfs/h2/emc/ptmp/yali.mao/working_wafs.$job.$PDY -mkdir -p $tmpdir -cd $tmpdir - -jobcard=run_JWAFS_$JOB.wcoss2 -cp /lfs/h2/emc/vpppg/noscrub/yali.mao/git/WAFS.fork/dev/driver/$jobcard . - -if [ $job = 'upp' ] ; then - FHOURS="anl 000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 \ - 027 030 033 036 039 042 045 048 054 060 066 072 078 084 090 096 102 108 114 120" -elif [ $job = 'gcip' ] ; then - FHOURS="000 003" -elif [ $job = 'grib2_0p25' ] ; then - export FHOUT_GFS=${FHOUT_GFS:-1} - if [ $FHOUT_GFS -eq 3 ] ; then #27 - export FHOURS=${FHOURS:-"6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 54 60 66 72 78 84 90 96 102 108 114 120"} - else #39 - export FHOURS=${FHOURS:-"6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 27 30 33 36 39 42 45 48 54 60 66 72 78 84 90 96 102 108 114 120"} - fi -elif [ $job = 'grib2_1p25' ] ; then - export FHOURS=${FHOURS:-"00 06 09 12 15 18 21 24 27 30 33 36 42 48 54 60 66 72"} -elif [ $job = 'grib' ] ; then - export FHOURS=${FHOURS:-"06 12 18 24 30 36 42 48 54 60 66 72"} -elif [ $job = 'grib2_0p25_blending' ] ; then - export FHOUT_GFS=${FHOUT_GFS:-1} - if [ $FHOUT_GFS -eq 3 ] ; then - export FHOURS=${FHOURS:-"6 9 12 15 18 21 24 27 30 33 36 39 42 45 48"} - else - export FHOURS=${FHOURS:-"6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 27 30 33 36 39 42 45 48"} - fi -fi - -for fhr in $FHOURS ; do - sed -e "s/log.wafs_$job/log.wafs_$job.$fhr/g" \ - -e "s/PDY=.*/PDY=$PDY/g" \ - -e "s/cyc=.*/cyc=$cyc/g" \ - -e "s/fhr=.*/fhr=$fhr/g" \ - -e "s/working_wafs/working_wafs.$job.$PDY/g" \ - $jobcard > $jobcard.$fhr - qsub $jobcard.$fhr -done diff --git a/dev/ecf/README.md b/dev/ecf/README.md index ecff506..507c4ce 100644 --- a/dev/ecf/README.md +++ b/dev/ecf/README.md @@ -10,17 +10,15 @@ cd dev/ecf ``` This will create a suite definition file called `wafs.def` in the `ecf/def` directory as well as links to the individual forecast hour ecf scripts. -## Loading `ecflow` -`ecflow` can be loaded using the following command: -```bash -module load ecflow -``` -This will load `ecflow` in your environment and setup the necessary value for `ECF_PORT`. It will also add `ecflow` calls to your `PATH`. - ## Starting `ecflow_server` `ecflow_server` can only be started on dedicated ecflow server nodes. On WCOSS2, the ecflow server nodes for development are: - `cdecflow01`, `cdecflow02` (cactus) -- `ddcflow01`, `ddcflow02` (dogwood) +- `ddecflow01`, `ddecflow02` (dogwood) + +`ssh` to one of the above ecflow server nodes (e.g. `cdecflow01`). +```bash +ssh cdecflow01 +``` Before starting the `ecflow_server`, one has to set the following variables. This only needs to be set once before starting the `ecflow_server`. ```bash @@ -33,17 +31,29 @@ mkdir -p ${ECF_ROOT} You are now ready to start the `ecflow_server`. ```bash +module load ecflow server_check.sh ${ECF_ROOT} ``` This will start the `ecflow_server` and print out the port number that the server is running on. -You can now exit the host where you started the `ecflow_server` and return to the usual WCOSS2 login nodes. +`ecflow_server` needs to be started **ONLY** once. Once the server is running, this window can be closed. +You can now exit the ecflow host where you started the `ecflow_server` and return to the usual WCOSS2 login nodes. + +## Starting `ecflow` and loading a Suite Definition File +Load the `ecflow` module on any WCOSS2 login nodes where you want to load the suite definition file. +```bash +module load ecflow +``` +This will load `ecflow` in your environment and setup the necessary value for `ECF_PORT`. It will also add `ecflow` calls to your `PATH`. + +Declare `ECF_HOST` on the WCOSS2 login node. `ECF_HOST` should be the `hostname` on which `ecflow_server` is running. +```bash +export ECF_HOST="cdecflow01" # This is the hostname on which the `ecflow_server` process is active. +``` -## Loading a Suite Definition File -Load the `ecflow` module on the WCOSS2 login node where you want to load the suite definition file. Check to ensure the `ecflow_client` can ping the `ecflow_server`: ```bash -`ecflow_client --ping` +ecflow_client --ping ``` If this is successful, one can launch the `ecflow_ui` and place it in the background. @@ -51,6 +61,8 @@ If this is successful, one can launch the `ecflow_ui` and place it in the backgr ecflow_ui & ``` +You can use the `ecflow_ui` GUI to start/halt the `ecflow_server` + Navigate to the directory where the suite definition file is located (typically `ecf/def`). ```bash ecflow_client --load $PWD/wafs.def @@ -62,5 +74,18 @@ To begin running the suite, use the following command: ecflow_client --begin wafs ``` +## Some useful commands +`ecflow_client --stats` To check the status of ecflow_server: HALTED/RUNNING + +`ecflow_client --restart` To start ecflow_server if it is HALTED + +`ecflow_client --delete=/yoursuite` To delete a suite which was previously loaded + +`ecflow_stop.sh -p NNNNN` To stop ecflow_server, where NNNNN is a port number assigned to the ecflow_server + +`qstat -u $USER` To check any job is running, including triggered by a loaded suite + +`ecflow_client --replace=/yoursuite1 $PWD/yoursuite2.def` To load a new suite if not loaded, otherwise replace the suite + ## Additional Resources For more information on using ecflow, refer to the official [documentation](https://ecflow.readthedocs.io/en/latest/overview.html) diff --git a/dev/ecf/run.ecflow_pdy.sh b/dev/ecf/run.ecflow_pdy.sh new file mode 100755 index 0000000..38fbf31 --- /dev/null +++ b/dev/ecf/run.ecflow_pdy.sh @@ -0,0 +1,49 @@ +#!/bin/bash + +# Instructions: https://github.com/NOAA-EMC/WAFS/tree/release/wafs.v7/dev/ecf +# +# Two requirements to run this script +# 1. ecflow_server has been started +# 2. ecflow_server is started on either cdecflow01 or ddecflow01 +# +# What the script will do for a PDYcyc: +# 1. Follow the instructions from dev/ecf/setup_ecf.sh and creates a suite definition file for a PDYcyc +# 2. Load the suite def file to the ecflow_server which was started already +# 3. Begin ecflow for a PDYcyc + +set -eu + +echo "Usage: ./run.ecflow_pdy.sh [PDYcyc]" + +PDYcyc=${1:-$(date --utc "+%Y%m%d")00} + +# Get the root of the script +readonly DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}")")/../.." && pwd -P) +cd $DIR_ROOT/dev/ecf +# create a suite def file in real time +./setup_ecf.sh -d $PDYcyc -x xPDY + +suitename="wafsxPDY" + +# set ECF_HOST according to which WCOSS2 machine +if [[ $(hostname) =~ ^[d][login|dxfer] ]] ; then + export ECF_HOST="ddecflow01" +elif [[ $(hostname) =~ ^[c][login|dxfer] ]] ; then + export ECF_HOST="cdecflow01" +fi +echo $ECF_HOST +module load ecflow + +# Make sure ecflow_server is not halted +ecflow_client --restart + +cd $DIR_ROOT/ecf/def + +# echo "yes" | ecflow_client --delete=/$suitename +# ecflow_client --load $PWD/$suitename.def + +# Replace: it can either load a new suite def or +# replace with a new suite def UNLESS a job is active for the current suite +ecflow_client --replace=/$suitename $PWD/$suitename.def + +ecflow_client --begin $suitename diff --git a/dev/ecf/setup_ecf.sh b/dev/ecf/setup_ecf.sh index 4b5b288..19fbb02 100755 --- a/dev/ecf/setup_ecf.sh +++ b/dev/ecf/setup_ecf.sh @@ -1,28 +1,109 @@ #!/bin/bash +# The script sets up the ecflow suite definition file for the WAFS workflow +# Usage: ./setup_ecf.sh [-d PDYcyc] [-x EXPID] +# PDYcyc: Test date in YYYYMMDDHH format. If blank, the current date and NRT mode is used (default: blank) +# EXPID: Experiment ID to distinguish different test runs (default: None) +# Example: 1. ./setup_ecf.sh -d 2020011012 -x x001 # Use the specified date and experiment ID +# 2. ./setup_ecf.sh -x x001 # Use the current date and NRT mode +# +# Setting up ecflow suite requires the package to be cloned in a directory matching 'wafs.vX.Y.Z' +# where X, Y, Z are numbers +# The script replaces @VARIABLE@ names in suite definition files with values +# and links ecflow scripts in the ecf/scripts directory +# +# The script is expected to be run after the package is cloned and executables are built + set -eu +# Function to print usage +_usage() { + echo "Usage: ./setup_ecf.sh [-d PDYcyc] [-x EXPID]" + echo " PDYcyc: Test date in YYYYMMDDHH format. If blank, the current date and NRT mode is used (default: blank)" + echo " EXPID: Experiment ID to distinguish different test runs (default: None)" + echo "Example: 1. ./setup_ecf.sh -d 2020011012 -x x001 # Use the specified date and experiment ID" + echo " 2. ./setup_ecf.sh -x x001 # Use the current date and NRT mode" +} -# Get the root of the cloned WAFS directory -readonly DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}")")/../.." && pwd -P) +# Set defaults for key-value arguments +PDYcyc="" +EXPID="" -ECF_DIR="${DIR_ROOT}/ecf" +# Parse key-value arguments using getopts +while getopts ":d:x:h" opt; do + case ${opt} in + d) + PDYcyc=${OPTARG} + ;; + x) + EXPID=${OPTARG} + ;; + h) + _usage + exit 0 + ;; + \?) + echo "Invalid option: -${OPTARG}" >&2 + _usage + exit 1 + ;; + :) + echo "Option -${OPTARG} requires an argument." >&2 + _usage + exit 1 + ;; + esac +done +shift $((OPTIND - 1)) + +# Get the root of the cloned WAFS directory +declare -r DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}")")/../.." && pwd -P) model="wafs" modelver=$(echo ${DIR_ROOT} | perl -pe "s:.*?/${model}\.(v[\d\.a-z]+).*:\1:") packageroot=$(dirname ${DIR_ROOT}) +# Check if the directory ends with "wafs.vX.Y.Z" +packagename=$(basename ${DIR_ROOT}) +pattern="^wafs\.v([0-9\.a-z]+).$" +if [[ ! "${packagename}" =~ ${pattern} ]]; then + echo "FATAL ERROR: The package '${packagename}' should be cloned in a directory matching 'wafs.vX.Y.Z'" + echo " X, Y, Z are numbers" + exit 1 +fi + +# Check if PDYcyc is provided and in the right format to set template used to either wafs.def.tmpl or wafs_nrt.def.tmpl +if [[ -n "${PDYcyc}" ]]; then + template="wafs.def.tmpl" +else + PDYcyc=$(date --utc "+%Y%m%d%H") + template="wafs_nrt.def.tmpl" +fi + +# Echo out the settings for the user +echo "Settings:" +echo " Model: ${model}.${modelver}" +echo " Package Root: ${packageroot}" +if [[ -n "${EXPID}" ]]; then + echo " Experiment ID: ${EXPID}" +fi +echo " Test date: ${PDYcyc}" +echo " ecflow suite def template: ${template}" + # Replace @VARIABLE@ names in suite definition files with values -echo "Create ecflow suite definition file in ... ecf/def/wafs.def" -sed -e "s|@MACHINE_SITE@|${MACHINE_SITE:-development}|g" \ +echo "Create ecflow suite definition file 'wafs${EXPID}.def' in ... ecf/def" +sed -e "s|@EXPID@|${EXPID}|g" \ + -e "s|@MACHINE_SITE@|${MACHINE_SITE:-development}|g" \ -e "s|@USER@|${USER}|g" \ -e "s|@MODELVER@|${modelver}|g" \ -e "s|@PACKAGEROOT@|${packageroot}|g" \ - "${ECF_DIR}/def/wafs.def.tmpl" > "${ECF_DIR}/def/wafs.def" + -e "s|@PDY@|${PDYcyc:0:8}|g" \ + -e "s|@CYC@|${PDYcyc:8:2}|g" \ + "${DIR_ROOT}/ecf/def/${template}" >"${DIR_ROOT}/ecf/def/wafs${EXPID}.def" # Link ecflow scripts echo "Link ecflow scripts in ... ecf/scripts" -cd "${ECF_DIR}" || exit 1 +cd "${DIR_ROOT}/ecf" || exit 1 ./setup_ecf_links.sh echo "... done" diff --git a/dev/modulefiles/wafs.sh b/dev/modulefiles/wafs.sh index 2479fb7..c0fd0e3 100644 --- a/dev/modulefiles/wafs.sh +++ b/dev/modulefiles/wafs.sh @@ -6,6 +6,9 @@ module load intel/${intel_ver} module load PrgEnv-intel/${PrgEnvintel_ver} module load craype/$craype_ver +module load cray-pals/$craypals_ver +module load cfp/$cfp_ver + module load libjpeg/$libjpeg_ver module load prod_util/$prod_util_ver module load prod_envir/$prod_envir_ver diff --git a/docs/Release_Notes.md b/docs/Release_Notes.md new file mode 100644 index 0000000..ba55cd8 --- /dev/null +++ b/docs/Release_Notes.md @@ -0,0 +1,223 @@ +WAFS v7.0.0 RELEASE NOTES + +------- +Prelude +------- + +This implementation is a separation of the WAFS component from the GFSv16 application, no science change. + +Implementation Instructions +--------------------------- + +The NOAA-EMC and NCAR organization spaces on GitHub are used to manage the WAFS code. The SPA(s) handling the WAFS implementation need to have permissions to clone the private NCAR UPP_GTG repository. All NOAA-EMC organization repositories are publicly readable and do not require access permissions. Please proceed with the following steps to checkout, build, and install the package on WCOSS2: + +Checkout the package from GitHub and `cd` into the directory: +```bash +cd ${PACKAGEROOT} +git clone --recursive -b wafs.v7.0.0 https://github.com/noaa-emc/wafs wafs.v7.0.0 +cd wafs.v7.0.0 +``` + +The checkout procedure extracts the following WAFS components, while GTG is a subcomponent of UPP.: +| Component | Tag | POC | +| --------- | -------------------- | ----------------- | +| UPP | upp_wafs_v7.0.0 | Wen.Meng@noaa.gov | +| GTG | ncep_post_gtg.v2.1.0 | Yali.Mao@noaa.gov | + +To build all the WAFS components, execute: +```bash +./sorc/build_all.sh +``` +The `build_all.sh` script compiles all WAFS components including UPP. Runtime output from the build is written to log files in `sorc/logs` directory. To build an individual program, for instance, `wafs_upp.x`, use `sorc/build_upp.sh`. + +Lastly, link the `ecflow` scripts by executing: +```bash +./ecf/setup_ecf_links.sh +``` + +Version File Changes +-------------------- +New files: build.ver and run.ver + + +Sorc Changes +------------ +No change from GFSv16 + +Job Changes +----------- +Filename changes according to EE2 standards: + | wafs.v7 | GFSv16 + | ------------------------- | ----------------------------------------------------| + | JWAFS_GCIP |<- JGFS_ATMOS_WAFS_GCIP | + | JWAFS_GRIB |<- JGFS_ATMOS_WAFS | + | JWAFS_GRIB2_0P25 |<- JGFS_ATMOS_WAFS_GRIB2_0P25 | + | JWAFS_GRIB2_0P25_BLENDING |<- JGFS_ATMOS_WAFS_BLENDING_0P25 | + | JWAFS_GRIB2_1P25 |<- JGFS_ATMOS_WAFS_GRIB2 | + | | JGFS_ATMOS_WAFS_BLENDING (removed after cleaning up) | + | JWAFS_GFS_MANAGER (new) | | + | JWAFS_UPP (new) | | + + +Parm Changes +------------ +1. Remove parm/wafs/legend +2. parm/upp is created after system building. Per AWC request, WAFS UPP control files add 4 low levels for icing and 1 upper lower for turbulence: + - postxconfig-NT-GFS-WAFS.txt + - postcntrl_gfs_wafs.xml + +Script Changes +-------------- +First of all, filename changes according to EE2 standards: + | wafs.v7 | GFSv16 + | ----------------------------- | ---------------------------------------------------------| + | exwafs_gcip.sh |<- exgfs_atmos_wafs_gcip.sh | + | exwafs_grib2_0p25_blending.sh |<- exgfs_atmos_wafs_blending_0p25.sh | + | exwafs_grib2_0p25.sh |<- exgfs_atmos_wafs_grib2_0p25.sh | + | exwafs_grib2_1p25.sh |<- exgfs_atmos_wafs_grib2.sh | + | exwafs_grib.sh |<- exgfs_atmos_wafs_grib.sh | + | | exgfs_atmos_wafs_blending.sh (removed after cleaning up) | + | exwafs_gfs_manager.sh (new) | | + | exwafs_upp.sh (new) | | + | ush/wafs_grib2_0p25_blending.sh (new) | | + +Additionally there are other changes: +1. In exwafs_grib2_0p25_blending.sh, extend the waiting time window for UK data from 15 to 25 minutes +2. In exwafs_grib2_0p25.sh, only include fields at the extra levels when forecast hour is between 06 and 36 per AWC request +3. In ush/mkwfsgbl.sh: change input dependency from GFS pgrb2.1p00 to master file. +4. Remove files under ush/ folder: wafs_blending.sh wafs_grib2.regrid.sh wafs_intdsk.sh +5. In exwafs_grib2_0p25_blending.sh, use MPMD for each forecast hour to call ush/wafs_grib2_0p25_blending.sh. + - Collect missing files from ush/wafs_grib2_0p25_blending.sh, send out warning email and dbn_alert of missing data once per cycle + - Add not-blended email and dbn_alert if both UK and US unblended files are missing + +Fix Changes +----------- +1. Remove fix/wafs/legend folder +2. Remove files under fix/wafs: + - grib2_blended_wafs_wifs_fFF.0p25 + - grib2_gfs_wafs_wifs_fFF.0p25 + - grib_wafsgfs_intdsk + - grib_wafsgfs_intdskf00 +3. Under fix/wafs, filenames are changed. + - faa_gfsmaster.grb2.list → grib2_gfs_awf_master.list + - gfs_master.grb2_0p25.list → grib2_0p25_gfs_master2d.list + - gfs_wafs.grb2_0p25.list → grib2_0p25_wafs_hazard.list + - grib2_gfs_awffFF.45 → grib2_gfs_awffFFF.45 (FF FFF - forecast hours) + - grib2_gfs_wafsfFF.45 → grib2_wafsfFFF.45 (FF FFF - forecast hours) + - wafs_0p25_admin_msg → wafs_blending_0p25_admin_msg + - wafs_gfsmaster.grb2.list → grib2_wafs.gfs_master.list + - wafs.namelist → grib_wafs.namelist + +Module Changes +-------------- +No change + +Changes to File Sizes +--------------------- +* UPP: increased by 35G (new, moved from GFS to WAFS) +* GRIB2_0P25: increased by 0.1G + +Environment and Resource Changes +-------------------------------- +1. Add ecFlow to WAFS package +2. Add UPP as a WAFS component +3. Add MPMD to WAFS_GRIB2_0P25_BLENDING; For other jobs, get rid of MPMD, each forecast hour will be run in its own job card. +4. WAFS_GRIB job dependency is changed from GFS pgrb2.1p00 to GFS master file. +5. WAFS_GRIB2_0P25_BLENDING runtime decreases from 4 minutes to 0.5 minutes after switching from sequential to parallel run for each forecast hour +6. Package increases from 33M to 288M (increase due to offline UPP source) +7. According to EE2 standards, input data are copied to DATA work folder instead of being soft linked. For this reason, the overall DATA folder size rockets up to 709.6G from 39G + +Pre-implementation Testing Requirements +--------------------------------------- +* Which production jobs should be tested as part of this implementation? + * The entire WAFS v7.0.0 package needs to be installed and tested on WCOSS-2 +* Does this change require a 30-day evaluation? + * No + + +Product Changes +--------------- +* Directory changes + * From com/gfs/v16.3/gfs.YYYYMMDD/CC/atmos to com/wafs/v7.0/wafs.YYYYMMDD/CC + * Inside WAFS, there are subfolders categoried by job names + * |-- upp + * |-- gcip + * |-- grib + * |-- grib2 + * |----- 1p25 + * |----- 0p25 + * |-------- blending +* Files to be retired + * `gfs.tCCz.wafs_icao.grb2fFFF` + * wafs.tCCz.master.fFFF.grib2 where FFF is from 001 to 005 +* Filename changes + * Renamed according to EE2 implementation standards + * Exceptions: files sent to UK keep the original names except forecast hour is changed to 3 digits + * Details: + | GFSv16 | wafs.v7 | + | -------------------------------------- | ---------------------------------------- | + | gfs.tCCz.wafs.0p25.anl | wafs.tCCz.0p25.anl.grib2 | + | gfs.tCCz.wafs.grb2fFFF | wafs.tCCz.master.fFFF.grib2 | + | gfs.tCCz.wafs_0p25_unblended.fFF.grib2 | WAFS_0p25_unblended_YYYYMMDDHHfFFF.grib2 | + | gfs.tCCz.awf_0p25.fFFF.grib2 | wafs.tCCz.awf.0p25.fFFF.grib2 | + | gfs.tCCz.awf_grb45fFF.grib2 | wafs.tCCz.awf_grid45.fFFF.grib2 | + | wmo/grib2.tCCz.awf_grbfFF.45 | wmo/grib2.wafs.tCCz.awf_grid45.fFFF | + | gfs.tCCz.wafs_grb45fFF.grib2 | gfs.tCCz.wafs_grb45fFFF.grib2 | + | wmo/grib2.tCCz.wafs_grbfFF.45 | wmo/grib2.wafs.tCCz.grid45.fFFF | + | gfs.tCCz.gcip.fFF.grib2 | wafs.tCCz.gcip.fFFF.grib2 | + | WAFS_0p25_blended_YYYYMMDDHHfFF.grib2 | WAFS_0p25_blended_ YYYYMMDDHHfFFF.grib2 | + | gfs.tCCz.wafs_blend_0p25_usonly.emailbody | wafs.tCCz.wafs_blend_0p25_ukmissing.emailbody | + | gfs.tCCz.wafs_blend_0p25_ukonly.emailbody | wafs.tCCz.wafs_blend_0p25_usmissing.emailbody | + | | wafs.tCCz.wafs_blend_0p25_noblending.emailbody (new) | + + +* File content changes + * Add EDPARM CATEDR MWTURB on 127.7 mb, ICESEV on 875.1 908.1 942.1 977.2 mb to: + * wafs.tCCz.master.fFFF.grib2 when FFF<=048 + * grib2/0p25/wafs.tCCz.awf.0p25.fFFF.grib2 when FFF<=036 + + +Dissemination Information +------------------------- +* dbn_alert subtype changes + | | GFSv16 | wafs.v7 | + | ---------------------------------------- | --------------------- |------------------ | + | gfs.tCCz.wafs_0p25.fFFF.grib2 | GFS_WAFS_0P25_GB2 | WAFS_0P25_GB2 | + | WAFS_0p25_unblended_YYYYMMDDHHfFFF.grib2 | GFS_WAFS_0P25_UBL_GB2 | WAFS_0P25_UBL_GB2 | + | wafs.tCCz.awf.0p25.fFFF.grib2 | GFS_AWF_0P25_GB2 | WAFS_AWF_0P25_GB2 | + | wmo/grib2.wafs.tCCz.awf_grid45.fFFF | gfs | wafs | + | gfs.tCCz.wafs_grb45fFFF.grib2 | GFS_WAFS_1P25_GB2 | WAFS_1P25_GB2 | + | wmo/grib2.wafs.tCCz.grid45.fFFF | gfs | wafs | + | WAFS_0p25_blended_ YYYYMMDDHHfFFF.grib2 | GFS_WAFS_0P25_BL_GB2 | WAFS_0P25_BL_GB2 | + +* Where should this output be sent? + * Same as current operations in GFS WAFS +* Who are the users? + * AWC, UK Met Office, SPC, and ICAO subscribed users +* Which output files should be transferred from PROD WCOSS to DEV WCOSS? + * All WAFS files should be transferred + + +HPSS Archive +------------ +* Directory changes and filename changes, refer back to 'Product Changes' +* Add wafs.tCCz.0p25.anl.grib2 + + +Job Dependencies and flow diagram +--------------------------------- +* Job dependencies refers to this document: https://docs.google.com/spreadsheets/d/1Nt343Z9x9UycweFik3HRFpXkqIjs7m20s15yGOhsgUY/edit?gid=1172497604#gid=1172497604 +* Flow diagram refer to page 6 in this document: https://docs.google.com/presentation/d/1yhdTfTHoBvV7K6jR2nfvkNAWn_eDJ2lTvDueRp9C89w/edit#slide=id.g2eeab8aa817_0_0 + + +Documentation +------------- +* WAFS.V7 Implementation Kick-off Meeting Slides https://docs.google.com/presentation/d/1yhdTfTHoBvV7K6jR2nfvkNAWn_eDJ2lTvDueRp9C89w +* WAFSv7 products and dbn_alert: https://docs.google.com/spreadsheets/d/1Nt343Z9x9UycweFik3HRFpXkqIjs7m20s15yGOhsgUY + + +Prepared By +----------- +* yali.mao@noaa.gov +* rahul.mahajan@noaa.gov +* Hui-Ya.Chuang@noaa.gov diff --git a/ecf/def/wafs.def.tmpl b/ecf/def/wafs.def.tmpl index c633350..51bd223 100644 --- a/ecf/def/wafs.def.tmpl +++ b/ecf/def/wafs.def.tmpl @@ -1,4 +1,4 @@ -suite wafs +suite wafs@EXPID@ edit MACHINE_SITE '@MACHINE_SITE@' edit USER '@USER@' edit wafs_ver '@MODELVER@' @@ -10,87 +10,77 @@ suite wafs edit QUEUE_ARCH 'dev_transfer' edit PROJENVIR 'DEV' edit PROJ 'GFS' # TODO: Change to 'WAFS' when ready - edit PDY '20240824' + edit PDY '@PDY@' edit PARATEST 'NO' edit SENDDBN 'NO' edit SENDDBN_NTC 'NO' edit SENDCANNEDDBN 'NO' edit KEEPDATA 'NO' edit RUN_ENVIR 'dev' - edit COMROOT '/lfs/h2/emc/ptmp/%USER%/%ENVIR%/com' - edit DATAROOT '/lfs/h2/emc/stmp/%USER%/tmp' - edit DCOMROOT '/lfs/h1/ops/dev/dcom/test' - edit OUTPUTDIR '/lfs/h2/emc/stmp/%USER%/output' + edit COMROOT '/lfs/h2/emc/ptmp/%USER%/wafs@EXPID@/%ENVIR%/com' + edit DATAROOT '/lfs/h2/emc/stmp/%USER%/wafs@EXPID@/tmp' + edit OUTPUTDIR '/lfs/h2/emc/stmp/%USER%/wafs@EXPID@/output' edit COMPATH '%COMROOT%/wafs' family primary family wafs edit NET 'wafs' edit RUN 'wafs' - family v7.0 - family 00 - edit CYC '00' + family @MODELVER@ + family @CYC@ + edit CYC '@CYC@' task jwafs_gfs_manager - #trigger /00/gfs/jgfs_forecast == active or /00/gfs/jgfs_forecast == complete - event 1 release_wafs_upp_anl + event 1 release_wafs_upp_anl # upp event 2 release_wafs_upp_000 - event 3 release_wafs_upp_001 - event 4 release_wafs_upp_002 - event 5 release_wafs_upp_003 - event 6 release_wafs_upp_004 - event 7 release_wafs_upp_005 - event 8 release_wafs_upp_006 - event 9 release_wafs_upp_007 - event 10 release_wafs_upp_008 - event 11 release_wafs_upp_009 - event 12 release_wafs_upp_010 - event 13 release_wafs_upp_011 - event 14 release_wafs_upp_012 - event 15 release_wafs_upp_013 - event 16 release_wafs_upp_014 - event 17 release_wafs_upp_015 - event 18 release_wafs_upp_016 - event 19 release_wafs_upp_017 - event 20 release_wafs_upp_018 - event 21 release_wafs_upp_019 - event 22 release_wafs_upp_020 - event 23 release_wafs_upp_021 - event 24 release_wafs_upp_022 - event 25 release_wafs_upp_023 - event 26 release_wafs_upp_024 - event 27 release_wafs_upp_027 - event 28 release_wafs_upp_030 - event 29 release_wafs_upp_033 - event 30 release_wafs_upp_036 - event 31 release_wafs_upp_039 - event 32 release_wafs_upp_042 - event 33 release_wafs_upp_045 - event 34 release_wafs_upp_048 - event 35 release_wafs_upp_054 - event 36 release_wafs_upp_060 - event 37 release_wafs_upp_066 - event 38 release_wafs_upp_072 - event 39 release_wafs_upp_078 - event 40 release_wafs_upp_084 - event 41 release_wafs_upp_090 - event 42 release_wafs_upp_096 - event 43 release_wafs_upp_102 - event 44 release_wafs_upp_108 - event 45 release_wafs_upp_114 - event 46 release_wafs_upp_120 - event 47 release_wafs_gcip_000 - event 48 release_wafs_gcip_003 - event 49 release_wafs_grib_006 - event 50 release_wafs_grib_012 - event 51 release_wafs_grib_018 - event 52 release_wafs_grib_024 - event 53 release_wafs_grib_030 - event 54 release_wafs_grib_036 - event 55 release_wafs_grib_042 - event 56 release_wafs_grib_048 - event 57 release_wafs_grib_054 - event 58 release_wafs_grib_060 - event 59 release_wafs_grib_066 - event 60 release_wafs_grib_072 + event 3 release_wafs_upp_006 + event 4 release_wafs_upp_007 + event 5 release_wafs_upp_008 + event 6 release_wafs_upp_009 + event 7 release_wafs_upp_010 + event 8 release_wafs_upp_011 + event 9 release_wafs_upp_012 + event 10 release_wafs_upp_013 + event 11 release_wafs_upp_014 + event 12 release_wafs_upp_015 + event 13 release_wafs_upp_016 + event 14 release_wafs_upp_017 + event 15 release_wafs_upp_018 + event 16 release_wafs_upp_019 + event 17 release_wafs_upp_020 + event 18 release_wafs_upp_021 + event 19 release_wafs_upp_022 + event 20 release_wafs_upp_023 + event 21 release_wafs_upp_024 + event 22 release_wafs_upp_027 + event 23 release_wafs_upp_030 + event 24 release_wafs_upp_033 + event 25 release_wafs_upp_036 + event 26 release_wafs_upp_039 + event 27 release_wafs_upp_042 + event 28 release_wafs_upp_045 + event 29 release_wafs_upp_048 + event 30 release_wafs_upp_054 + event 31 release_wafs_upp_060 + event 32 release_wafs_upp_066 + event 33 release_wafs_upp_072 + event 34 release_wafs_upp_078 + event 35 release_wafs_upp_084 + event 36 release_wafs_upp_090 + event 37 release_wafs_upp_096 + event 38 release_wafs_upp_102 + event 39 release_wafs_upp_108 + event 40 release_wafs_upp_114 + event 41 release_wafs_upp_120 + event 42 release_wafs_gcip_000 # gcip + event 43 release_wafs_gcip_003 + event 44 release_wafs_grib_012 # grib + event 45 release_wafs_grib_018 + event 46 release_wafs_grib_024 + event 47 release_wafs_grib_030 + event 48 release_wafs_grib_036 + event 49 release_wafs_grib_042 + event 50 release_wafs_grib_048 + event 51 release_wafs_grib_060 + event 52 release_wafs_grib_072 family upp task jwafs_upp_anl trigger ../jwafs_gfs_manager:release_wafs_upp_anl @@ -98,21 +88,6 @@ suite wafs task jwafs_upp_f000 trigger ../jwafs_gfs_manager:release_wafs_upp_000 edit FHR 000 - task jwafs_upp_f001 - trigger ../jwafs_gfs_manager:release_wafs_upp_001 - edit FHR 001 - task jwafs_upp_f002 - trigger ../jwafs_gfs_manager:release_wafs_upp_002 - edit FHR 002 - task jwafs_upp_f003 - trigger ../jwafs_gfs_manager:release_wafs_upp_003 - edit FHR 003 - task jwafs_upp_f004 - trigger ../jwafs_gfs_manager:release_wafs_upp_004 - edit FHR 004 - task jwafs_upp_f005 - trigger ../jwafs_gfs_manager:release_wafs_upp_005 - edit FHR 005 task jwafs_upp_f006 trigger ../jwafs_gfs_manager:release_wafs_upp_006 edit FHR 006 @@ -240,9 +215,6 @@ suite wafs edit FHR 003 endfamily # endfamily gcip family grib - task jwafs_grib_f006 - trigger ../jwafs_gfs_manager:release_wafs_grib_006 - edit FHR 06 task jwafs_grib_f012 trigger ../jwafs_gfs_manager:release_wafs_grib_012 edit FHR 12 @@ -264,15 +236,9 @@ suite wafs task jwafs_grib_f048 trigger ../jwafs_gfs_manager:release_wafs_grib_048 edit FHR 48 - task jwafs_grib_f054 - trigger ../jwafs_gfs_manager:release_wafs_grib_054 - edit FHR 54 task jwafs_grib_f060 trigger ../jwafs_gfs_manager:release_wafs_grib_060 edit FHR 60 - task jwafs_grib_f066 - trigger ../jwafs_gfs_manager:release_wafs_grib_066 - edit FHR 66 task jwafs_grib_f072 trigger ../jwafs_gfs_manager:release_wafs_grib_072 edit FHR 72 @@ -453,92 +419,14 @@ suite wafs trigger ../../upp/jwafs_upp_f120 == complete edit FHR 120 family blending - task jwafs_grib2_0p25_blending_f006 - trigger ../jwafs_grib2_0p25_f006 == complete - edit FHR 006 - task jwafs_grib2_0p25_blending_f007 - trigger ../jwafs_grib2_0p25_f007 == complete - edit FHR 007 - task jwafs_grib2_0p25_blending_f008 - trigger ../jwafs_grib2_0p25_f008 == complete - edit FHR 008 - task jwafs_grib2_0p25_blending_f009 - trigger ../jwafs_grib2_0p25_f009 == complete - edit FHR 009 - task jwafs_grib2_0p25_blending_f010 - trigger ../jwafs_grib2_0p25_f010 == complete - edit FHR 010 - task jwafs_grib2_0p25_blending_f011 - trigger ../jwafs_grib2_0p25_f011 == complete - edit FHR 011 - task jwafs_grib2_0p25_blending_f012 - trigger ../jwafs_grib2_0p25_f012 == complete - edit FHR 012 - task jwafs_grib2_0p25_blending_f013 - trigger ../jwafs_grib2_0p25_f013 == complete - edit FHR 013 - task jwafs_grib2_0p25_blending_f014 - trigger ../jwafs_grib2_0p25_f014 == complete - edit FHR 014 - task jwafs_grib2_0p25_blending_f015 - trigger ../jwafs_grib2_0p25_f015 == complete - edit FHR 015 - task jwafs_grib2_0p25_blending_f016 - trigger ../jwafs_grib2_0p25_f016 == complete - edit FHR 016 - task jwafs_grib2_0p25_blending_f017 - trigger ../jwafs_grib2_0p25_f017 == complete - edit FHR 017 - task jwafs_grib2_0p25_blending_f018 - trigger ../jwafs_grib2_0p25_f018 == complete - edit FHR 018 - task jwafs_grib2_0p25_blending_f019 - trigger ../jwafs_grib2_0p25_f019 == complete - edit FHR 019 - task jwafs_grib2_0p25_blending_f020 - trigger ../jwafs_grib2_0p25_f020 == complete - edit FHR 020 - task jwafs_grib2_0p25_blending_f021 - trigger ../jwafs_grib2_0p25_f021 == complete - edit FHR 021 - task jwafs_grib2_0p25_blending_f022 - trigger ../jwafs_grib2_0p25_f022 == complete - edit FHR 022 - task jwafs_grib2_0p25_blending_f023 - trigger ../jwafs_grib2_0p25_f023 == complete - edit FHR 023 - task jwafs_grib2_0p25_blending_f024 - trigger ../jwafs_grib2_0p25_f024 == complete - edit FHR 024 - task jwafs_grib2_0p25_blending_f027 - trigger ../jwafs_grib2_0p25_f027 == complete - edit FHR 027 - task jwafs_grib2_0p25_blending_f030 - trigger ../jwafs_grib2_0p25_f030 == complete - edit FHR 030 - task jwafs_grib2_0p25_blending_f033 - trigger ../jwafs_grib2_0p25_f033 == complete - edit FHR 033 - task jwafs_grib2_0p25_blending_f036 - trigger ../jwafs_grib2_0p25_f036 == complete - edit FHR 036 - task jwafs_grib2_0p25_blending_f039 - trigger ../jwafs_grib2_0p25_f039 == complete - edit FHR 039 - task jwafs_grib2_0p25_blending_f042 - trigger ../jwafs_grib2_0p25_f042 == complete - edit FHR 042 - task jwafs_grib2_0p25_blending_f045 - trigger ../jwafs_grib2_0p25_f045 == complete - edit FHR 045 - task jwafs_grib2_0p25_blending_f048 + edit DCOMROOT '/lfs/h1/ops/dev/dcom/test' + task jwafs_grib2_0p25_blending trigger ../jwafs_grib2_0p25_f048 == complete - edit FHR 048 endfamily # endfamily blending endfamily # endfamily 0p25 endfamily # endfamily grib2 - endfamily # endfamily 00 - endfamily # endfamily v7.0 + endfamily # endfamily @CYC@ + endfamily # endfamily @MODELVER@ endfamily # endfamily wafs endfamily # endfamily primary -endsuite # endsuite wafs +endsuite # endsuite wafs@EXPID@ diff --git a/ecf/def/wafs_nrt.def.tmpl b/ecf/def/wafs_nrt.def.tmpl new file mode 100644 index 0000000..53fbf3d --- /dev/null +++ b/ecf/def/wafs_nrt.def.tmpl @@ -0,0 +1,1640 @@ +suite wafs@EXPID@ + edit MACHINE_SITE '@MACHINE_SITE@' + edit USER '@USER@' + edit wafs_ver '@MODELVER@' + edit PACKAGEHOME '@PACKAGEROOT@/wafs.%wafs_ver%' + edit ECF_INCLUDE '@PACKAGEROOT@/wafs.@MODELVER@/ecf/include' + edit ECF_FILES '%PACKAGEHOME%/ecf/scripts' + edit ENVIR 'prod' + edit QUEUE 'dev' + edit QUEUE_ARCH 'dev_transfer' + edit PROJENVIR 'DEV' + edit PROJ 'GFS' # TODO: Change to 'WAFS' when ready + edit PARATEST 'NO' + edit SENDDBN 'NO' + edit SENDDBN_NTC 'NO' + edit SENDCANNEDDBN 'NO' + edit KEEPDATA 'NO' + edit RUN_ENVIR 'dev' + edit COMROOT '/lfs/h2/emc/ptmp/%USER%/wafs@EXPID@/%ENVIR%/com' + edit DATAROOT '/lfs/h2/emc/stmp/%USER%/wafs@EXPID@/tmp' + edit OUTPUTDIR '/lfs/h2/emc/stmp/%USER%/wafs@EXPID@/output' + edit COMPATH '%COMROOT%/wafs' + repeat day 1 + family primary + family wafs + edit NET 'wafs' + edit RUN 'wafs' + family @MODELVER@ + family 00 + edit CYC '00' + task jwafs_gfs_manager + trigger :TIME >= 0335 and :TIME < 0935 + event 1 release_wafs_upp_anl # upp + event 2 release_wafs_upp_000 + event 3 release_wafs_upp_006 + event 4 release_wafs_upp_007 + event 5 release_wafs_upp_008 + event 6 release_wafs_upp_009 + event 7 release_wafs_upp_010 + event 8 release_wafs_upp_011 + event 9 release_wafs_upp_012 + event 10 release_wafs_upp_013 + event 11 release_wafs_upp_014 + event 12 release_wafs_upp_015 + event 13 release_wafs_upp_016 + event 14 release_wafs_upp_017 + event 15 release_wafs_upp_018 + event 16 release_wafs_upp_019 + event 17 release_wafs_upp_020 + event 18 release_wafs_upp_021 + event 19 release_wafs_upp_022 + event 20 release_wafs_upp_023 + event 21 release_wafs_upp_024 + event 22 release_wafs_upp_027 + event 23 release_wafs_upp_030 + event 24 release_wafs_upp_033 + event 25 release_wafs_upp_036 + event 26 release_wafs_upp_039 + event 27 release_wafs_upp_042 + event 28 release_wafs_upp_045 + event 29 release_wafs_upp_048 + event 30 release_wafs_upp_054 + event 31 release_wafs_upp_060 + event 32 release_wafs_upp_066 + event 33 release_wafs_upp_072 + event 34 release_wafs_upp_078 + event 35 release_wafs_upp_084 + event 36 release_wafs_upp_090 + event 37 release_wafs_upp_096 + event 38 release_wafs_upp_102 + event 39 release_wafs_upp_108 + event 40 release_wafs_upp_114 + event 41 release_wafs_upp_120 + event 42 release_wafs_gcip_000 # gcip + event 43 release_wafs_gcip_003 + event 44 release_wafs_grib_012 # grib + event 45 release_wafs_grib_018 + event 46 release_wafs_grib_024 + event 47 release_wafs_grib_030 + event 48 release_wafs_grib_036 + event 49 release_wafs_grib_042 + event 50 release_wafs_grib_048 + event 51 release_wafs_grib_060 + event 52 release_wafs_grib_072 + family upp + task jwafs_upp_anl + trigger ../jwafs_gfs_manager:release_wafs_upp_anl + edit FHR anl + task jwafs_upp_f000 + trigger ../jwafs_gfs_manager:release_wafs_upp_000 + edit FHR 000 + task jwafs_upp_f006 + trigger ../jwafs_gfs_manager:release_wafs_upp_006 + edit FHR 006 + task jwafs_upp_f007 + trigger ../jwafs_gfs_manager:release_wafs_upp_007 + edit FHR 007 + task jwafs_upp_f008 + trigger ../jwafs_gfs_manager:release_wafs_upp_008 + edit FHR 008 + task jwafs_upp_f009 + trigger ../jwafs_gfs_manager:release_wafs_upp_009 + edit FHR 009 + task jwafs_upp_f010 + trigger ../jwafs_gfs_manager:release_wafs_upp_010 + edit FHR 010 + task jwafs_upp_f011 + trigger ../jwafs_gfs_manager:release_wafs_upp_011 + edit FHR 011 + task jwafs_upp_f012 + trigger ../jwafs_gfs_manager:release_wafs_upp_012 + edit FHR 012 + task jwafs_upp_f013 + trigger ../jwafs_gfs_manager:release_wafs_upp_013 + edit FHR 013 + task jwafs_upp_f014 + trigger ../jwafs_gfs_manager:release_wafs_upp_014 + edit FHR 014 + task jwafs_upp_f015 + trigger ../jwafs_gfs_manager:release_wafs_upp_015 + edit FHR 015 + task jwafs_upp_f016 + trigger ../jwafs_gfs_manager:release_wafs_upp_016 + edit FHR 016 + task jwafs_upp_f017 + trigger ../jwafs_gfs_manager:release_wafs_upp_017 + edit FHR 017 + task jwafs_upp_f018 + trigger ../jwafs_gfs_manager:release_wafs_upp_018 + edit FHR 018 + task jwafs_upp_f019 + trigger ../jwafs_gfs_manager:release_wafs_upp_019 + edit FHR 019 + task jwafs_upp_f020 + trigger ../jwafs_gfs_manager:release_wafs_upp_020 + edit FHR 020 + task jwafs_upp_f021 + trigger ../jwafs_gfs_manager:release_wafs_upp_021 + edit FHR 021 + task jwafs_upp_f022 + trigger ../jwafs_gfs_manager:release_wafs_upp_022 + edit FHR 022 + task jwafs_upp_f023 + trigger ../jwafs_gfs_manager:release_wafs_upp_023 + edit FHR 023 + task jwafs_upp_f024 + trigger ../jwafs_gfs_manager:release_wafs_upp_024 + edit FHR 024 + task jwafs_upp_f027 + trigger ../jwafs_gfs_manager:release_wafs_upp_027 + edit FHR 027 + task jwafs_upp_f030 + trigger ../jwafs_gfs_manager:release_wafs_upp_030 + edit FHR 030 + task jwafs_upp_f033 + trigger ../jwafs_gfs_manager:release_wafs_upp_033 + edit FHR 033 + task jwafs_upp_f036 + trigger ../jwafs_gfs_manager:release_wafs_upp_036 + edit FHR 036 + task jwafs_upp_f039 + trigger ../jwafs_gfs_manager:release_wafs_upp_039 + edit FHR 039 + task jwafs_upp_f042 + trigger ../jwafs_gfs_manager:release_wafs_upp_042 + edit FHR 042 + task jwafs_upp_f045 + trigger ../jwafs_gfs_manager:release_wafs_upp_045 + edit FHR 045 + task jwafs_upp_f048 + trigger ../jwafs_gfs_manager:release_wafs_upp_048 + edit FHR 048 + task jwafs_upp_f054 + trigger ../jwafs_gfs_manager:release_wafs_upp_054 + edit FHR 054 + task jwafs_upp_f060 + trigger ../jwafs_gfs_manager:release_wafs_upp_060 + edit FHR 060 + task jwafs_upp_f066 + trigger ../jwafs_gfs_manager:release_wafs_upp_066 + edit FHR 066 + task jwafs_upp_f072 + trigger ../jwafs_gfs_manager:release_wafs_upp_072 + edit FHR 072 + task jwafs_upp_f078 + trigger ../jwafs_gfs_manager:release_wafs_upp_078 + edit FHR 078 + task jwafs_upp_f084 + trigger ../jwafs_gfs_manager:release_wafs_upp_084 + edit FHR 084 + task jwafs_upp_f090 + trigger ../jwafs_gfs_manager:release_wafs_upp_090 + edit FHR 090 + task jwafs_upp_f096 + trigger ../jwafs_gfs_manager:release_wafs_upp_096 + edit FHR 096 + task jwafs_upp_f102 + trigger ../jwafs_gfs_manager:release_wafs_upp_102 + edit FHR 102 + task jwafs_upp_f108 + trigger ../jwafs_gfs_manager:release_wafs_upp_108 + edit FHR 108 + task jwafs_upp_f114 + trigger ../jwafs_gfs_manager:release_wafs_upp_114 + edit FHR 114 + task jwafs_upp_f120 + trigger ../jwafs_gfs_manager:release_wafs_upp_120 + edit FHR 120 + endfamily # endfamily upp + family gcip + task jwafs_gcip_f000 + trigger :TIME >= 0440 and ../jwafs_gfs_manager:release_wafs_gcip_000 + edit FHR 000 + task jwafs_gcip_f003 + trigger :TIME >= 0440 and ../jwafs_gfs_manager:release_wafs_gcip_003 + edit FHR 003 + endfamily # endfamily gcip + family grib + task jwafs_grib_f012 + trigger ../jwafs_gfs_manager:release_wafs_grib_012 + edit FHR 12 + task jwafs_grib_f018 + trigger ../jwafs_gfs_manager:release_wafs_grib_018 + edit FHR 18 + task jwafs_grib_f024 + trigger ../jwafs_gfs_manager:release_wafs_grib_024 + edit FHR 24 + task jwafs_grib_f030 + trigger ../jwafs_gfs_manager:release_wafs_grib_030 + edit FHR 30 + task jwafs_grib_f036 + trigger ../jwafs_gfs_manager:release_wafs_grib_036 + edit FHR 36 + task jwafs_grib_f042 + trigger ../jwafs_gfs_manager:release_wafs_grib_042 + edit FHR 42 + task jwafs_grib_f048 + trigger ../jwafs_gfs_manager:release_wafs_grib_048 + edit FHR 48 + task jwafs_grib_f060 + trigger ../jwafs_gfs_manager:release_wafs_grib_060 + edit FHR 60 + task jwafs_grib_f072 + trigger ../jwafs_gfs_manager:release_wafs_grib_072 + edit FHR 72 + endfamily # endfamily grib + family grib2 + family 1p25 + task jwafs_grib2_1p25_f000 + trigger ../../upp/jwafs_upp_f000 == complete + edit FHR 000 + task jwafs_grib2_1p25_f006 + trigger ../../upp/jwafs_upp_f006 == complete + edit FHR 006 + task jwafs_grib2_1p25_f009 + trigger ../../upp/jwafs_upp_f009 == complete + edit FHR 009 + task jwafs_grib2_1p25_f012 + trigger ../../upp/jwafs_upp_f012 == complete + edit FHR 012 + task jwafs_grib2_1p25_f015 + trigger ../../upp/jwafs_upp_f015 == complete + edit FHR 015 + task jwafs_grib2_1p25_f018 + trigger ../../upp/jwafs_upp_f018 == complete + edit FHR 018 + task jwafs_grib2_1p25_f021 + trigger ../../upp/jwafs_upp_f021 == complete + edit FHR 021 + task jwafs_grib2_1p25_f024 + trigger ../../upp/jwafs_upp_f024 == complete + edit FHR 024 + task jwafs_grib2_1p25_f027 + trigger ../../upp/jwafs_upp_f027 == complete + edit FHR 027 + task jwafs_grib2_1p25_f030 + trigger ../../upp/jwafs_upp_f030 == complete + edit FHR 030 + task jwafs_grib2_1p25_f033 + trigger ../../upp/jwafs_upp_f033 == complete + edit FHR 033 + task jwafs_grib2_1p25_f036 + trigger ../../upp/jwafs_upp_f036 == complete + edit FHR 036 + task jwafs_grib2_1p25_f042 + trigger ../../upp/jwafs_upp_f042 == complete + edit FHR 042 + task jwafs_grib2_1p25_f048 + trigger ../../upp/jwafs_upp_f048 == complete + edit FHR 048 + task jwafs_grib2_1p25_f054 + trigger ../../upp/jwafs_upp_f054 == complete + edit FHR 054 + task jwafs_grib2_1p25_f060 + trigger ../../upp/jwafs_upp_f060 == complete + edit FHR 060 + task jwafs_grib2_1p25_f066 + trigger ../../upp/jwafs_upp_f066 == complete + edit FHR 066 + task jwafs_grib2_1p25_f072 + trigger ../../upp/jwafs_upp_f072 == complete + edit FHR 072 + endfamily # endfamily 1p25 + family 0p25 + task jwafs_grib2_0p25_f006 + trigger ../../upp/jwafs_upp_f006 == complete + edit FHR 006 + task jwafs_grib2_0p25_f007 + trigger ../../upp/jwafs_upp_f007 == complete + edit FHR 007 + task jwafs_grib2_0p25_f008 + trigger ../../upp/jwafs_upp_f008 == complete + edit FHR 008 + task jwafs_grib2_0p25_f009 + trigger ../../upp/jwafs_upp_f009 == complete + edit FHR 009 + task jwafs_grib2_0p25_f010 + trigger ../../upp/jwafs_upp_f010 == complete + edit FHR 010 + task jwafs_grib2_0p25_f011 + trigger ../../upp/jwafs_upp_f011 == complete + edit FHR 011 + task jwafs_grib2_0p25_f012 + trigger ../../upp/jwafs_upp_f012 == complete + edit FHR 012 + task jwafs_grib2_0p25_f013 + trigger ../../upp/jwafs_upp_f013 == complete + edit FHR 013 + task jwafs_grib2_0p25_f014 + trigger ../../upp/jwafs_upp_f014 == complete + edit FHR 014 + task jwafs_grib2_0p25_f015 + trigger ../../upp/jwafs_upp_f015 == complete + edit FHR 015 + task jwafs_grib2_0p25_f016 + trigger ../../upp/jwafs_upp_f016 == complete + edit FHR 016 + task jwafs_grib2_0p25_f017 + trigger ../../upp/jwafs_upp_f017 == complete + edit FHR 017 + task jwafs_grib2_0p25_f018 + trigger ../../upp/jwafs_upp_f018 == complete + edit FHR 018 + task jwafs_grib2_0p25_f019 + trigger ../../upp/jwafs_upp_f019 == complete + edit FHR 019 + task jwafs_grib2_0p25_f020 + trigger ../../upp/jwafs_upp_f020 == complete + edit FHR 020 + task jwafs_grib2_0p25_f021 + trigger ../../upp/jwafs_upp_f021 == complete + edit FHR 021 + task jwafs_grib2_0p25_f022 + trigger ../../upp/jwafs_upp_f022 == complete + edit FHR 022 + task jwafs_grib2_0p25_f023 + trigger ../../upp/jwafs_upp_f023 == complete + edit FHR 023 + task jwafs_grib2_0p25_f024 + trigger ../../upp/jwafs_upp_f024 == complete + edit FHR 024 + task jwafs_grib2_0p25_f027 + trigger ../../upp/jwafs_upp_f027 == complete + edit FHR 027 + task jwafs_grib2_0p25_f030 + trigger ../../upp/jwafs_upp_f030 == complete + edit FHR 030 + task jwafs_grib2_0p25_f033 + trigger ../../upp/jwafs_upp_f033 == complete + edit FHR 033 + task jwafs_grib2_0p25_f036 + trigger ../../upp/jwafs_upp_f036 == complete + edit FHR 036 + task jwafs_grib2_0p25_f039 + trigger ../../upp/jwafs_upp_f039 == complete + edit FHR 039 + task jwafs_grib2_0p25_f042 + trigger ../../upp/jwafs_upp_f042 == complete + edit FHR 042 + task jwafs_grib2_0p25_f045 + trigger ../../upp/jwafs_upp_f045 == complete + edit FHR 045 + task jwafs_grib2_0p25_f048 + trigger ../../upp/jwafs_upp_f048 == complete + edit FHR 048 + task jwafs_grib2_0p25_f054 + trigger ../../upp/jwafs_upp_f054 == complete + edit FHR 054 + task jwafs_grib2_0p25_f060 + trigger ../../upp/jwafs_upp_f060 == complete + edit FHR 060 + task jwafs_grib2_0p25_f066 + trigger ../../upp/jwafs_upp_f066 == complete + edit FHR 066 + task jwafs_grib2_0p25_f072 + trigger ../../upp/jwafs_upp_f072 == complete + edit FHR 072 + task jwafs_grib2_0p25_f078 + trigger ../../upp/jwafs_upp_f078 == complete + edit FHR 078 + task jwafs_grib2_0p25_f084 + trigger ../../upp/jwafs_upp_f084 == complete + edit FHR 084 + task jwafs_grib2_0p25_f090 + trigger ../../upp/jwafs_upp_f090 == complete + edit FHR 090 + task jwafs_grib2_0p25_f096 + trigger ../../upp/jwafs_upp_f096 == complete + edit FHR 096 + task jwafs_grib2_0p25_f102 + trigger ../../upp/jwafs_upp_f102 == complete + edit FHR 102 + task jwafs_grib2_0p25_f108 + trigger ../../upp/jwafs_upp_f108 == complete + edit FHR 108 + task jwafs_grib2_0p25_f114 + trigger ../../upp/jwafs_upp_f114 == complete + edit FHR 114 + task jwafs_grib2_0p25_f120 + trigger ../../upp/jwafs_upp_f120 == complete + edit FHR 120 + family blending + trigger :TIME >= 0430 and :TIME < 1030 + edit DCOMROOT '/lfs/h1/ops/dev/dcom/test' + task jwafs_grib2_0p25_blending + trigger ../jwafs_grib2_0p25_f048 == complete + endfamily # endfamily blending + endfamily # endfamily 0p25 + endfamily # endfamily grib2 + endfamily # endfamily 00 + family 06 + edit CYC '06' + task jwafs_gfs_manager + trigger :TIME >= 0935 and :TIME < 1535 + event 1 release_wafs_upp_anl # upp + event 2 release_wafs_upp_000 + event 3 release_wafs_upp_006 + event 4 release_wafs_upp_007 + event 5 release_wafs_upp_008 + event 6 release_wafs_upp_009 + event 7 release_wafs_upp_010 + event 8 release_wafs_upp_011 + event 9 release_wafs_upp_012 + event 10 release_wafs_upp_013 + event 11 release_wafs_upp_014 + event 12 release_wafs_upp_015 + event 13 release_wafs_upp_016 + event 14 release_wafs_upp_017 + event 15 release_wafs_upp_018 + event 16 release_wafs_upp_019 + event 17 release_wafs_upp_020 + event 18 release_wafs_upp_021 + event 19 release_wafs_upp_022 + event 20 release_wafs_upp_023 + event 21 release_wafs_upp_024 + event 22 release_wafs_upp_027 + event 23 release_wafs_upp_030 + event 24 release_wafs_upp_033 + event 25 release_wafs_upp_036 + event 26 release_wafs_upp_039 + event 27 release_wafs_upp_042 + event 28 release_wafs_upp_045 + event 29 release_wafs_upp_048 + event 30 release_wafs_upp_054 + event 31 release_wafs_upp_060 + event 32 release_wafs_upp_066 + event 33 release_wafs_upp_072 + event 34 release_wafs_upp_078 + event 35 release_wafs_upp_084 + event 36 release_wafs_upp_090 + event 37 release_wafs_upp_096 + event 38 release_wafs_upp_102 + event 39 release_wafs_upp_108 + event 40 release_wafs_upp_114 + event 41 release_wafs_upp_120 + event 42 release_wafs_gcip_000 # gcip + event 43 release_wafs_gcip_003 + event 44 release_wafs_grib_012 # grib + event 45 release_wafs_grib_018 + event 46 release_wafs_grib_024 + event 47 release_wafs_grib_030 + event 48 release_wafs_grib_036 + event 49 release_wafs_grib_042 + event 50 release_wafs_grib_048 + event 51 release_wafs_grib_060 + event 52 release_wafs_grib_072 + family upp + task jwafs_upp_anl + trigger ../jwafs_gfs_manager:release_wafs_upp_anl + edit FHR anl + task jwafs_upp_f000 + trigger ../jwafs_gfs_manager:release_wafs_upp_000 + edit FHR 000 + task jwafs_upp_f006 + trigger ../jwafs_gfs_manager:release_wafs_upp_006 + edit FHR 006 + task jwafs_upp_f007 + trigger ../jwafs_gfs_manager:release_wafs_upp_007 + edit FHR 007 + task jwafs_upp_f008 + trigger ../jwafs_gfs_manager:release_wafs_upp_008 + edit FHR 008 + task jwafs_upp_f009 + trigger ../jwafs_gfs_manager:release_wafs_upp_009 + edit FHR 009 + task jwafs_upp_f010 + trigger ../jwafs_gfs_manager:release_wafs_upp_010 + edit FHR 010 + task jwafs_upp_f011 + trigger ../jwafs_gfs_manager:release_wafs_upp_011 + edit FHR 011 + task jwafs_upp_f012 + trigger ../jwafs_gfs_manager:release_wafs_upp_012 + edit FHR 012 + task jwafs_upp_f013 + trigger ../jwafs_gfs_manager:release_wafs_upp_013 + edit FHR 013 + task jwafs_upp_f014 + trigger ../jwafs_gfs_manager:release_wafs_upp_014 + edit FHR 014 + task jwafs_upp_f015 + trigger ../jwafs_gfs_manager:release_wafs_upp_015 + edit FHR 015 + task jwafs_upp_f016 + trigger ../jwafs_gfs_manager:release_wafs_upp_016 + edit FHR 016 + task jwafs_upp_f017 + trigger ../jwafs_gfs_manager:release_wafs_upp_017 + edit FHR 017 + task jwafs_upp_f018 + trigger ../jwafs_gfs_manager:release_wafs_upp_018 + edit FHR 018 + task jwafs_upp_f019 + trigger ../jwafs_gfs_manager:release_wafs_upp_019 + edit FHR 019 + task jwafs_upp_f020 + trigger ../jwafs_gfs_manager:release_wafs_upp_020 + edit FHR 020 + task jwafs_upp_f021 + trigger ../jwafs_gfs_manager:release_wafs_upp_021 + edit FHR 021 + task jwafs_upp_f022 + trigger ../jwafs_gfs_manager:release_wafs_upp_022 + edit FHR 022 + task jwafs_upp_f023 + trigger ../jwafs_gfs_manager:release_wafs_upp_023 + edit FHR 023 + task jwafs_upp_f024 + trigger ../jwafs_gfs_manager:release_wafs_upp_024 + edit FHR 024 + task jwafs_upp_f027 + trigger ../jwafs_gfs_manager:release_wafs_upp_027 + edit FHR 027 + task jwafs_upp_f030 + trigger ../jwafs_gfs_manager:release_wafs_upp_030 + edit FHR 030 + task jwafs_upp_f033 + trigger ../jwafs_gfs_manager:release_wafs_upp_033 + edit FHR 033 + task jwafs_upp_f036 + trigger ../jwafs_gfs_manager:release_wafs_upp_036 + edit FHR 036 + task jwafs_upp_f039 + trigger ../jwafs_gfs_manager:release_wafs_upp_039 + edit FHR 039 + task jwafs_upp_f042 + trigger ../jwafs_gfs_manager:release_wafs_upp_042 + edit FHR 042 + task jwafs_upp_f045 + trigger ../jwafs_gfs_manager:release_wafs_upp_045 + edit FHR 045 + task jwafs_upp_f048 + trigger ../jwafs_gfs_manager:release_wafs_upp_048 + edit FHR 048 + task jwafs_upp_f054 + trigger ../jwafs_gfs_manager:release_wafs_upp_054 + edit FHR 054 + task jwafs_upp_f060 + trigger ../jwafs_gfs_manager:release_wafs_upp_060 + edit FHR 060 + task jwafs_upp_f066 + trigger ../jwafs_gfs_manager:release_wafs_upp_066 + edit FHR 066 + task jwafs_upp_f072 + trigger ../jwafs_gfs_manager:release_wafs_upp_072 + edit FHR 072 + task jwafs_upp_f078 + trigger ../jwafs_gfs_manager:release_wafs_upp_078 + edit FHR 078 + task jwafs_upp_f084 + trigger ../jwafs_gfs_manager:release_wafs_upp_084 + edit FHR 084 + task jwafs_upp_f090 + trigger ../jwafs_gfs_manager:release_wafs_upp_090 + edit FHR 090 + task jwafs_upp_f096 + trigger ../jwafs_gfs_manager:release_wafs_upp_096 + edit FHR 096 + task jwafs_upp_f102 + trigger ../jwafs_gfs_manager:release_wafs_upp_102 + edit FHR 102 + task jwafs_upp_f108 + trigger ../jwafs_gfs_manager:release_wafs_upp_108 + edit FHR 108 + task jwafs_upp_f114 + trigger ../jwafs_gfs_manager:release_wafs_upp_114 + edit FHR 114 + task jwafs_upp_f120 + trigger ../jwafs_gfs_manager:release_wafs_upp_120 + edit FHR 120 + endfamily # endfamily upp + family gcip + task jwafs_gcip_f000 + trigger :TIME >= 1040 and ../jwafs_gfs_manager:release_wafs_gcip_000 + edit FHR 000 + task jwafs_gcip_f003 + trigger :TIME >= 1040 and ../jwafs_gfs_manager:release_wafs_gcip_003 + edit FHR 003 + endfamily # endfamily gcip + family grib + task jwafs_grib_f012 + trigger ../jwafs_gfs_manager:release_wafs_grib_012 + edit FHR 12 + task jwafs_grib_f018 + trigger ../jwafs_gfs_manager:release_wafs_grib_018 + edit FHR 18 + task jwafs_grib_f024 + trigger ../jwafs_gfs_manager:release_wafs_grib_024 + edit FHR 24 + task jwafs_grib_f030 + trigger ../jwafs_gfs_manager:release_wafs_grib_030 + edit FHR 30 + task jwafs_grib_f036 + trigger ../jwafs_gfs_manager:release_wafs_grib_036 + edit FHR 36 + task jwafs_grib_f042 + trigger ../jwafs_gfs_manager:release_wafs_grib_042 + edit FHR 42 + task jwafs_grib_f048 + trigger ../jwafs_gfs_manager:release_wafs_grib_048 + edit FHR 48 + task jwafs_grib_f060 + trigger ../jwafs_gfs_manager:release_wafs_grib_060 + edit FHR 60 + task jwafs_grib_f072 + trigger ../jwafs_gfs_manager:release_wafs_grib_072 + edit FHR 72 + endfamily # endfamily grib + family grib2 + family 1p25 + task jwafs_grib2_1p25_f000 + trigger ../../upp/jwafs_upp_f000 == complete + edit FHR 000 + task jwafs_grib2_1p25_f006 + trigger ../../upp/jwafs_upp_f006 == complete + edit FHR 006 + task jwafs_grib2_1p25_f009 + trigger ../../upp/jwafs_upp_f009 == complete + edit FHR 009 + task jwafs_grib2_1p25_f012 + trigger ../../upp/jwafs_upp_f012 == complete + edit FHR 012 + task jwafs_grib2_1p25_f015 + trigger ../../upp/jwafs_upp_f015 == complete + edit FHR 015 + task jwafs_grib2_1p25_f018 + trigger ../../upp/jwafs_upp_f018 == complete + edit FHR 018 + task jwafs_grib2_1p25_f021 + trigger ../../upp/jwafs_upp_f021 == complete + edit FHR 021 + task jwafs_grib2_1p25_f024 + trigger ../../upp/jwafs_upp_f024 == complete + edit FHR 024 + task jwafs_grib2_1p25_f027 + trigger ../../upp/jwafs_upp_f027 == complete + edit FHR 027 + task jwafs_grib2_1p25_f030 + trigger ../../upp/jwafs_upp_f030 == complete + edit FHR 030 + task jwafs_grib2_1p25_f033 + trigger ../../upp/jwafs_upp_f033 == complete + edit FHR 033 + task jwafs_grib2_1p25_f036 + trigger ../../upp/jwafs_upp_f036 == complete + edit FHR 036 + task jwafs_grib2_1p25_f042 + trigger ../../upp/jwafs_upp_f042 == complete + edit FHR 042 + task jwafs_grib2_1p25_f048 + trigger ../../upp/jwafs_upp_f048 == complete + edit FHR 048 + task jwafs_grib2_1p25_f054 + trigger ../../upp/jwafs_upp_f054 == complete + edit FHR 054 + task jwafs_grib2_1p25_f060 + trigger ../../upp/jwafs_upp_f060 == complete + edit FHR 060 + task jwafs_grib2_1p25_f066 + trigger ../../upp/jwafs_upp_f066 == complete + edit FHR 066 + task jwafs_grib2_1p25_f072 + trigger ../../upp/jwafs_upp_f072 == complete + edit FHR 072 + endfamily # endfamily 1p25 + family 0p25 + task jwafs_grib2_0p25_f006 + trigger ../../upp/jwafs_upp_f006 == complete + edit FHR 006 + task jwafs_grib2_0p25_f007 + trigger ../../upp/jwafs_upp_f007 == complete + edit FHR 007 + task jwafs_grib2_0p25_f008 + trigger ../../upp/jwafs_upp_f008 == complete + edit FHR 008 + task jwafs_grib2_0p25_f009 + trigger ../../upp/jwafs_upp_f009 == complete + edit FHR 009 + task jwafs_grib2_0p25_f010 + trigger ../../upp/jwafs_upp_f010 == complete + edit FHR 010 + task jwafs_grib2_0p25_f011 + trigger ../../upp/jwafs_upp_f011 == complete + edit FHR 011 + task jwafs_grib2_0p25_f012 + trigger ../../upp/jwafs_upp_f012 == complete + edit FHR 012 + task jwafs_grib2_0p25_f013 + trigger ../../upp/jwafs_upp_f013 == complete + edit FHR 013 + task jwafs_grib2_0p25_f014 + trigger ../../upp/jwafs_upp_f014 == complete + edit FHR 014 + task jwafs_grib2_0p25_f015 + trigger ../../upp/jwafs_upp_f015 == complete + edit FHR 015 + task jwafs_grib2_0p25_f016 + trigger ../../upp/jwafs_upp_f016 == complete + edit FHR 016 + task jwafs_grib2_0p25_f017 + trigger ../../upp/jwafs_upp_f017 == complete + edit FHR 017 + task jwafs_grib2_0p25_f018 + trigger ../../upp/jwafs_upp_f018 == complete + edit FHR 018 + task jwafs_grib2_0p25_f019 + trigger ../../upp/jwafs_upp_f019 == complete + edit FHR 019 + task jwafs_grib2_0p25_f020 + trigger ../../upp/jwafs_upp_f020 == complete + edit FHR 020 + task jwafs_grib2_0p25_f021 + trigger ../../upp/jwafs_upp_f021 == complete + edit FHR 021 + task jwafs_grib2_0p25_f022 + trigger ../../upp/jwafs_upp_f022 == complete + edit FHR 022 + task jwafs_grib2_0p25_f023 + trigger ../../upp/jwafs_upp_f023 == complete + edit FHR 023 + task jwafs_grib2_0p25_f024 + trigger ../../upp/jwafs_upp_f024 == complete + edit FHR 024 + task jwafs_grib2_0p25_f027 + trigger ../../upp/jwafs_upp_f027 == complete + edit FHR 027 + task jwafs_grib2_0p25_f030 + trigger ../../upp/jwafs_upp_f030 == complete + edit FHR 030 + task jwafs_grib2_0p25_f033 + trigger ../../upp/jwafs_upp_f033 == complete + edit FHR 033 + task jwafs_grib2_0p25_f036 + trigger ../../upp/jwafs_upp_f036 == complete + edit FHR 036 + task jwafs_grib2_0p25_f039 + trigger ../../upp/jwafs_upp_f039 == complete + edit FHR 039 + task jwafs_grib2_0p25_f042 + trigger ../../upp/jwafs_upp_f042 == complete + edit FHR 042 + task jwafs_grib2_0p25_f045 + trigger ../../upp/jwafs_upp_f045 == complete + edit FHR 045 + task jwafs_grib2_0p25_f048 + trigger ../../upp/jwafs_upp_f048 == complete + edit FHR 048 + task jwafs_grib2_0p25_f054 + trigger ../../upp/jwafs_upp_f054 == complete + edit FHR 054 + task jwafs_grib2_0p25_f060 + trigger ../../upp/jwafs_upp_f060 == complete + edit FHR 060 + task jwafs_grib2_0p25_f066 + trigger ../../upp/jwafs_upp_f066 == complete + edit FHR 066 + task jwafs_grib2_0p25_f072 + trigger ../../upp/jwafs_upp_f072 == complete + edit FHR 072 + task jwafs_grib2_0p25_f078 + trigger ../../upp/jwafs_upp_f078 == complete + edit FHR 078 + task jwafs_grib2_0p25_f084 + trigger ../../upp/jwafs_upp_f084 == complete + edit FHR 084 + task jwafs_grib2_0p25_f090 + trigger ../../upp/jwafs_upp_f090 == complete + edit FHR 090 + task jwafs_grib2_0p25_f096 + trigger ../../upp/jwafs_upp_f096 == complete + edit FHR 096 + task jwafs_grib2_0p25_f102 + trigger ../../upp/jwafs_upp_f102 == complete + edit FHR 102 + task jwafs_grib2_0p25_f108 + trigger ../../upp/jwafs_upp_f108 == complete + edit FHR 108 + task jwafs_grib2_0p25_f114 + trigger ../../upp/jwafs_upp_f114 == complete + edit FHR 114 + task jwafs_grib2_0p25_f120 + trigger ../../upp/jwafs_upp_f120 == complete + edit FHR 120 + family blending + trigger :TIME >= 1030 and :TIME < 1630 + edit DCOMROOT '/lfs/h1/ops/dev/dcom/test' + task jwafs_grib2_0p25_blending + trigger ../jwafs_grib2_0p25_f048 == complete + endfamily # endfamily blending + endfamily # endfamily 0p25 + endfamily # endfamily grib2 + endfamily # endfamily 06 + family 12 + edit CYC '12' + task jwafs_gfs_manager + trigger :TIME >= 1535 and :TIME < 2135 + event 1 release_wafs_upp_anl # upp + event 2 release_wafs_upp_000 + event 3 release_wafs_upp_006 + event 4 release_wafs_upp_007 + event 5 release_wafs_upp_008 + event 6 release_wafs_upp_009 + event 7 release_wafs_upp_010 + event 8 release_wafs_upp_011 + event 9 release_wafs_upp_012 + event 10 release_wafs_upp_013 + event 11 release_wafs_upp_014 + event 12 release_wafs_upp_015 + event 13 release_wafs_upp_016 + event 14 release_wafs_upp_017 + event 15 release_wafs_upp_018 + event 16 release_wafs_upp_019 + event 17 release_wafs_upp_020 + event 18 release_wafs_upp_021 + event 19 release_wafs_upp_022 + event 20 release_wafs_upp_023 + event 21 release_wafs_upp_024 + event 22 release_wafs_upp_027 + event 23 release_wafs_upp_030 + event 24 release_wafs_upp_033 + event 25 release_wafs_upp_036 + event 26 release_wafs_upp_039 + event 27 release_wafs_upp_042 + event 28 release_wafs_upp_045 + event 29 release_wafs_upp_048 + event 30 release_wafs_upp_054 + event 31 release_wafs_upp_060 + event 32 release_wafs_upp_066 + event 33 release_wafs_upp_072 + event 34 release_wafs_upp_078 + event 35 release_wafs_upp_084 + event 36 release_wafs_upp_090 + event 37 release_wafs_upp_096 + event 38 release_wafs_upp_102 + event 39 release_wafs_upp_108 + event 40 release_wafs_upp_114 + event 41 release_wafs_upp_120 + event 42 release_wafs_gcip_000 # gcip + event 43 release_wafs_gcip_003 + event 44 release_wafs_grib_012 # grib + event 45 release_wafs_grib_018 + event 46 release_wafs_grib_024 + event 47 release_wafs_grib_030 + event 48 release_wafs_grib_036 + event 49 release_wafs_grib_042 + event 50 release_wafs_grib_048 + event 51 release_wafs_grib_060 + event 52 release_wafs_grib_072 + family upp + task jwafs_upp_anl + trigger ../jwafs_gfs_manager:release_wafs_upp_anl + edit FHR anl + task jwafs_upp_f000 + trigger ../jwafs_gfs_manager:release_wafs_upp_000 + edit FHR 000 + task jwafs_upp_f006 + trigger ../jwafs_gfs_manager:release_wafs_upp_006 + edit FHR 006 + task jwafs_upp_f007 + trigger ../jwafs_gfs_manager:release_wafs_upp_007 + edit FHR 007 + task jwafs_upp_f008 + trigger ../jwafs_gfs_manager:release_wafs_upp_008 + edit FHR 008 + task jwafs_upp_f009 + trigger ../jwafs_gfs_manager:release_wafs_upp_009 + edit FHR 009 + task jwafs_upp_f010 + trigger ../jwafs_gfs_manager:release_wafs_upp_010 + edit FHR 010 + task jwafs_upp_f011 + trigger ../jwafs_gfs_manager:release_wafs_upp_011 + edit FHR 011 + task jwafs_upp_f012 + trigger ../jwafs_gfs_manager:release_wafs_upp_012 + edit FHR 012 + task jwafs_upp_f013 + trigger ../jwafs_gfs_manager:release_wafs_upp_013 + edit FHR 013 + task jwafs_upp_f014 + trigger ../jwafs_gfs_manager:release_wafs_upp_014 + edit FHR 014 + task jwafs_upp_f015 + trigger ../jwafs_gfs_manager:release_wafs_upp_015 + edit FHR 015 + task jwafs_upp_f016 + trigger ../jwafs_gfs_manager:release_wafs_upp_016 + edit FHR 016 + task jwafs_upp_f017 + trigger ../jwafs_gfs_manager:release_wafs_upp_017 + edit FHR 017 + task jwafs_upp_f018 + trigger ../jwafs_gfs_manager:release_wafs_upp_018 + edit FHR 018 + task jwafs_upp_f019 + trigger ../jwafs_gfs_manager:release_wafs_upp_019 + edit FHR 019 + task jwafs_upp_f020 + trigger ../jwafs_gfs_manager:release_wafs_upp_020 + edit FHR 020 + task jwafs_upp_f021 + trigger ../jwafs_gfs_manager:release_wafs_upp_021 + edit FHR 021 + task jwafs_upp_f022 + trigger ../jwafs_gfs_manager:release_wafs_upp_022 + edit FHR 022 + task jwafs_upp_f023 + trigger ../jwafs_gfs_manager:release_wafs_upp_023 + edit FHR 023 + task jwafs_upp_f024 + trigger ../jwafs_gfs_manager:release_wafs_upp_024 + edit FHR 024 + task jwafs_upp_f027 + trigger ../jwafs_gfs_manager:release_wafs_upp_027 + edit FHR 027 + task jwafs_upp_f030 + trigger ../jwafs_gfs_manager:release_wafs_upp_030 + edit FHR 030 + task jwafs_upp_f033 + trigger ../jwafs_gfs_manager:release_wafs_upp_033 + edit FHR 033 + task jwafs_upp_f036 + trigger ../jwafs_gfs_manager:release_wafs_upp_036 + edit FHR 036 + task jwafs_upp_f039 + trigger ../jwafs_gfs_manager:release_wafs_upp_039 + edit FHR 039 + task jwafs_upp_f042 + trigger ../jwafs_gfs_manager:release_wafs_upp_042 + edit FHR 042 + task jwafs_upp_f045 + trigger ../jwafs_gfs_manager:release_wafs_upp_045 + edit FHR 045 + task jwafs_upp_f048 + trigger ../jwafs_gfs_manager:release_wafs_upp_048 + edit FHR 048 + task jwafs_upp_f054 + trigger ../jwafs_gfs_manager:release_wafs_upp_054 + edit FHR 054 + task jwafs_upp_f060 + trigger ../jwafs_gfs_manager:release_wafs_upp_060 + edit FHR 060 + task jwafs_upp_f066 + trigger ../jwafs_gfs_manager:release_wafs_upp_066 + edit FHR 066 + task jwafs_upp_f072 + trigger ../jwafs_gfs_manager:release_wafs_upp_072 + edit FHR 072 + task jwafs_upp_f078 + trigger ../jwafs_gfs_manager:release_wafs_upp_078 + edit FHR 078 + task jwafs_upp_f084 + trigger ../jwafs_gfs_manager:release_wafs_upp_084 + edit FHR 084 + task jwafs_upp_f090 + trigger ../jwafs_gfs_manager:release_wafs_upp_090 + edit FHR 090 + task jwafs_upp_f096 + trigger ../jwafs_gfs_manager:release_wafs_upp_096 + edit FHR 096 + task jwafs_upp_f102 + trigger ../jwafs_gfs_manager:release_wafs_upp_102 + edit FHR 102 + task jwafs_upp_f108 + trigger ../jwafs_gfs_manager:release_wafs_upp_108 + edit FHR 108 + task jwafs_upp_f114 + trigger ../jwafs_gfs_manager:release_wafs_upp_114 + edit FHR 114 + task jwafs_upp_f120 + trigger ../jwafs_gfs_manager:release_wafs_upp_120 + edit FHR 120 + endfamily # endfamily upp + family gcip + task jwafs_gcip_f000 + trigger :TIME >= 1640 and ../jwafs_gfs_manager:release_wafs_gcip_000 + edit FHR 000 + task jwafs_gcip_f003 + trigger :TIME >= 1640 and ../jwafs_gfs_manager:release_wafs_gcip_003 + edit FHR 003 + endfamily # endfamily gcip + family grib + task jwafs_grib_f012 + trigger ../jwafs_gfs_manager:release_wafs_grib_012 + edit FHR 12 + task jwafs_grib_f018 + trigger ../jwafs_gfs_manager:release_wafs_grib_018 + edit FHR 18 + task jwafs_grib_f024 + trigger ../jwafs_gfs_manager:release_wafs_grib_024 + edit FHR 24 + task jwafs_grib_f030 + trigger ../jwafs_gfs_manager:release_wafs_grib_030 + edit FHR 30 + task jwafs_grib_f036 + trigger ../jwafs_gfs_manager:release_wafs_grib_036 + edit FHR 36 + task jwafs_grib_f042 + trigger ../jwafs_gfs_manager:release_wafs_grib_042 + edit FHR 42 + task jwafs_grib_f048 + trigger ../jwafs_gfs_manager:release_wafs_grib_048 + edit FHR 48 + task jwafs_grib_f060 + trigger ../jwafs_gfs_manager:release_wafs_grib_060 + edit FHR 60 + task jwafs_grib_f072 + trigger ../jwafs_gfs_manager:release_wafs_grib_072 + edit FHR 72 + endfamily # endfamily grib + family grib2 + family 1p25 + task jwafs_grib2_1p25_f000 + trigger ../../upp/jwafs_upp_f000 == complete + edit FHR 000 + task jwafs_grib2_1p25_f006 + trigger ../../upp/jwafs_upp_f006 == complete + edit FHR 006 + task jwafs_grib2_1p25_f009 + trigger ../../upp/jwafs_upp_f009 == complete + edit FHR 009 + task jwafs_grib2_1p25_f012 + trigger ../../upp/jwafs_upp_f012 == complete + edit FHR 012 + task jwafs_grib2_1p25_f015 + trigger ../../upp/jwafs_upp_f015 == complete + edit FHR 015 + task jwafs_grib2_1p25_f018 + trigger ../../upp/jwafs_upp_f018 == complete + edit FHR 018 + task jwafs_grib2_1p25_f021 + trigger ../../upp/jwafs_upp_f021 == complete + edit FHR 021 + task jwafs_grib2_1p25_f024 + trigger ../../upp/jwafs_upp_f024 == complete + edit FHR 024 + task jwafs_grib2_1p25_f027 + trigger ../../upp/jwafs_upp_f027 == complete + edit FHR 027 + task jwafs_grib2_1p25_f030 + trigger ../../upp/jwafs_upp_f030 == complete + edit FHR 030 + task jwafs_grib2_1p25_f033 + trigger ../../upp/jwafs_upp_f033 == complete + edit FHR 033 + task jwafs_grib2_1p25_f036 + trigger ../../upp/jwafs_upp_f036 == complete + edit FHR 036 + task jwafs_grib2_1p25_f042 + trigger ../../upp/jwafs_upp_f042 == complete + edit FHR 042 + task jwafs_grib2_1p25_f048 + trigger ../../upp/jwafs_upp_f048 == complete + edit FHR 048 + task jwafs_grib2_1p25_f054 + trigger ../../upp/jwafs_upp_f054 == complete + edit FHR 054 + task jwafs_grib2_1p25_f060 + trigger ../../upp/jwafs_upp_f060 == complete + edit FHR 060 + task jwafs_grib2_1p25_f066 + trigger ../../upp/jwafs_upp_f066 == complete + edit FHR 066 + task jwafs_grib2_1p25_f072 + trigger ../../upp/jwafs_upp_f072 == complete + edit FHR 072 + endfamily # endfamily 1p25 + family 0p25 + task jwafs_grib2_0p25_f006 + trigger ../../upp/jwafs_upp_f006 == complete + edit FHR 006 + task jwafs_grib2_0p25_f007 + trigger ../../upp/jwafs_upp_f007 == complete + edit FHR 007 + task jwafs_grib2_0p25_f008 + trigger ../../upp/jwafs_upp_f008 == complete + edit FHR 008 + task jwafs_grib2_0p25_f009 + trigger ../../upp/jwafs_upp_f009 == complete + edit FHR 009 + task jwafs_grib2_0p25_f010 + trigger ../../upp/jwafs_upp_f010 == complete + edit FHR 010 + task jwafs_grib2_0p25_f011 + trigger ../../upp/jwafs_upp_f011 == complete + edit FHR 011 + task jwafs_grib2_0p25_f012 + trigger ../../upp/jwafs_upp_f012 == complete + edit FHR 012 + task jwafs_grib2_0p25_f013 + trigger ../../upp/jwafs_upp_f013 == complete + edit FHR 013 + task jwafs_grib2_0p25_f014 + trigger ../../upp/jwafs_upp_f014 == complete + edit FHR 014 + task jwafs_grib2_0p25_f015 + trigger ../../upp/jwafs_upp_f015 == complete + edit FHR 015 + task jwafs_grib2_0p25_f016 + trigger ../../upp/jwafs_upp_f016 == complete + edit FHR 016 + task jwafs_grib2_0p25_f017 + trigger ../../upp/jwafs_upp_f017 == complete + edit FHR 017 + task jwafs_grib2_0p25_f018 + trigger ../../upp/jwafs_upp_f018 == complete + edit FHR 018 + task jwafs_grib2_0p25_f019 + trigger ../../upp/jwafs_upp_f019 == complete + edit FHR 019 + task jwafs_grib2_0p25_f020 + trigger ../../upp/jwafs_upp_f020 == complete + edit FHR 020 + task jwafs_grib2_0p25_f021 + trigger ../../upp/jwafs_upp_f021 == complete + edit FHR 021 + task jwafs_grib2_0p25_f022 + trigger ../../upp/jwafs_upp_f022 == complete + edit FHR 022 + task jwafs_grib2_0p25_f023 + trigger ../../upp/jwafs_upp_f023 == complete + edit FHR 023 + task jwafs_grib2_0p25_f024 + trigger ../../upp/jwafs_upp_f024 == complete + edit FHR 024 + task jwafs_grib2_0p25_f027 + trigger ../../upp/jwafs_upp_f027 == complete + edit FHR 027 + task jwafs_grib2_0p25_f030 + trigger ../../upp/jwafs_upp_f030 == complete + edit FHR 030 + task jwafs_grib2_0p25_f033 + trigger ../../upp/jwafs_upp_f033 == complete + edit FHR 033 + task jwafs_grib2_0p25_f036 + trigger ../../upp/jwafs_upp_f036 == complete + edit FHR 036 + task jwafs_grib2_0p25_f039 + trigger ../../upp/jwafs_upp_f039 == complete + edit FHR 039 + task jwafs_grib2_0p25_f042 + trigger ../../upp/jwafs_upp_f042 == complete + edit FHR 042 + task jwafs_grib2_0p25_f045 + trigger ../../upp/jwafs_upp_f045 == complete + edit FHR 045 + task jwafs_grib2_0p25_f048 + trigger ../../upp/jwafs_upp_f048 == complete + edit FHR 048 + task jwafs_grib2_0p25_f054 + trigger ../../upp/jwafs_upp_f054 == complete + edit FHR 054 + task jwafs_grib2_0p25_f060 + trigger ../../upp/jwafs_upp_f060 == complete + edit FHR 060 + task jwafs_grib2_0p25_f066 + trigger ../../upp/jwafs_upp_f066 == complete + edit FHR 066 + task jwafs_grib2_0p25_f072 + trigger ../../upp/jwafs_upp_f072 == complete + edit FHR 072 + task jwafs_grib2_0p25_f078 + trigger ../../upp/jwafs_upp_f078 == complete + edit FHR 078 + task jwafs_grib2_0p25_f084 + trigger ../../upp/jwafs_upp_f084 == complete + edit FHR 084 + task jwafs_grib2_0p25_f090 + trigger ../../upp/jwafs_upp_f090 == complete + edit FHR 090 + task jwafs_grib2_0p25_f096 + trigger ../../upp/jwafs_upp_f096 == complete + edit FHR 096 + task jwafs_grib2_0p25_f102 + trigger ../../upp/jwafs_upp_f102 == complete + edit FHR 102 + task jwafs_grib2_0p25_f108 + trigger ../../upp/jwafs_upp_f108 == complete + edit FHR 108 + task jwafs_grib2_0p25_f114 + trigger ../../upp/jwafs_upp_f114 == complete + edit FHR 114 + task jwafs_grib2_0p25_f120 + trigger ../../upp/jwafs_upp_f120 == complete + edit FHR 120 + family blending + trigger :TIME >= 1630 and :TIME < 2230 + edit DCOMROOT '/lfs/h1/ops/dev/dcom/test' + task jwafs_grib2_0p25_blending + trigger ../jwafs_grib2_0p25_f048 == complete + endfamily # endfamily blending + endfamily # endfamily 0p25 + endfamily # endfamily grib2 + endfamily # endfamily 12 + family 18 + edit CYC '18' + task jwafs_gfs_manager + trigger :TIME >= 2135 + event 1 release_wafs_upp_anl # upp + event 2 release_wafs_upp_000 + event 3 release_wafs_upp_006 + event 4 release_wafs_upp_007 + event 5 release_wafs_upp_008 + event 6 release_wafs_upp_009 + event 7 release_wafs_upp_010 + event 8 release_wafs_upp_011 + event 9 release_wafs_upp_012 + event 10 release_wafs_upp_013 + event 11 release_wafs_upp_014 + event 12 release_wafs_upp_015 + event 13 release_wafs_upp_016 + event 14 release_wafs_upp_017 + event 15 release_wafs_upp_018 + event 16 release_wafs_upp_019 + event 17 release_wafs_upp_020 + event 18 release_wafs_upp_021 + event 19 release_wafs_upp_022 + event 20 release_wafs_upp_023 + event 21 release_wafs_upp_024 + event 22 release_wafs_upp_027 + event 23 release_wafs_upp_030 + event 24 release_wafs_upp_033 + event 25 release_wafs_upp_036 + event 26 release_wafs_upp_039 + event 27 release_wafs_upp_042 + event 28 release_wafs_upp_045 + event 29 release_wafs_upp_048 + event 30 release_wafs_upp_054 + event 31 release_wafs_upp_060 + event 32 release_wafs_upp_066 + event 33 release_wafs_upp_072 + event 34 release_wafs_upp_078 + event 35 release_wafs_upp_084 + event 36 release_wafs_upp_090 + event 37 release_wafs_upp_096 + event 38 release_wafs_upp_102 + event 39 release_wafs_upp_108 + event 40 release_wafs_upp_114 + event 41 release_wafs_upp_120 + event 42 release_wafs_gcip_000 # gcip + event 43 release_wafs_gcip_003 + event 44 release_wafs_grib_012 # grib + event 45 release_wafs_grib_018 + event 46 release_wafs_grib_024 + event 47 release_wafs_grib_030 + event 48 release_wafs_grib_036 + event 49 release_wafs_grib_042 + event 50 release_wafs_grib_048 + event 51 release_wafs_grib_060 + event 52 release_wafs_grib_072 + family upp + task jwafs_upp_anl + trigger ../jwafs_gfs_manager:release_wafs_upp_anl + edit FHR anl + task jwafs_upp_f000 + trigger ../jwafs_gfs_manager:release_wafs_upp_000 + edit FHR 000 + task jwafs_upp_f006 + trigger ../jwafs_gfs_manager:release_wafs_upp_006 + edit FHR 006 + task jwafs_upp_f007 + trigger ../jwafs_gfs_manager:release_wafs_upp_007 + edit FHR 007 + task jwafs_upp_f008 + trigger ../jwafs_gfs_manager:release_wafs_upp_008 + edit FHR 008 + task jwafs_upp_f009 + trigger ../jwafs_gfs_manager:release_wafs_upp_009 + edit FHR 009 + task jwafs_upp_f010 + trigger ../jwafs_gfs_manager:release_wafs_upp_010 + edit FHR 010 + task jwafs_upp_f011 + trigger ../jwafs_gfs_manager:release_wafs_upp_011 + edit FHR 011 + task jwafs_upp_f012 + trigger ../jwafs_gfs_manager:release_wafs_upp_012 + edit FHR 012 + task jwafs_upp_f013 + trigger ../jwafs_gfs_manager:release_wafs_upp_013 + edit FHR 013 + task jwafs_upp_f014 + trigger ../jwafs_gfs_manager:release_wafs_upp_014 + edit FHR 014 + task jwafs_upp_f015 + trigger ../jwafs_gfs_manager:release_wafs_upp_015 + edit FHR 015 + task jwafs_upp_f016 + trigger ../jwafs_gfs_manager:release_wafs_upp_016 + edit FHR 016 + task jwafs_upp_f017 + trigger ../jwafs_gfs_manager:release_wafs_upp_017 + edit FHR 017 + task jwafs_upp_f018 + trigger ../jwafs_gfs_manager:release_wafs_upp_018 + edit FHR 018 + task jwafs_upp_f019 + trigger ../jwafs_gfs_manager:release_wafs_upp_019 + edit FHR 019 + task jwafs_upp_f020 + trigger ../jwafs_gfs_manager:release_wafs_upp_020 + edit FHR 020 + task jwafs_upp_f021 + trigger ../jwafs_gfs_manager:release_wafs_upp_021 + edit FHR 021 + task jwafs_upp_f022 + trigger ../jwafs_gfs_manager:release_wafs_upp_022 + edit FHR 022 + task jwafs_upp_f023 + trigger ../jwafs_gfs_manager:release_wafs_upp_023 + edit FHR 023 + task jwafs_upp_f024 + trigger ../jwafs_gfs_manager:release_wafs_upp_024 + edit FHR 024 + task jwafs_upp_f027 + trigger ../jwafs_gfs_manager:release_wafs_upp_027 + edit FHR 027 + task jwafs_upp_f030 + trigger ../jwafs_gfs_manager:release_wafs_upp_030 + edit FHR 030 + task jwafs_upp_f033 + trigger ../jwafs_gfs_manager:release_wafs_upp_033 + edit FHR 033 + task jwafs_upp_f036 + trigger ../jwafs_gfs_manager:release_wafs_upp_036 + edit FHR 036 + task jwafs_upp_f039 + trigger ../jwafs_gfs_manager:release_wafs_upp_039 + edit FHR 039 + task jwafs_upp_f042 + trigger ../jwafs_gfs_manager:release_wafs_upp_042 + edit FHR 042 + task jwafs_upp_f045 + trigger ../jwafs_gfs_manager:release_wafs_upp_045 + edit FHR 045 + task jwafs_upp_f048 + trigger ../jwafs_gfs_manager:release_wafs_upp_048 + edit FHR 048 + task jwafs_upp_f054 + trigger ../jwafs_gfs_manager:release_wafs_upp_054 + edit FHR 054 + task jwafs_upp_f060 + trigger ../jwafs_gfs_manager:release_wafs_upp_060 + edit FHR 060 + task jwafs_upp_f066 + trigger ../jwafs_gfs_manager:release_wafs_upp_066 + edit FHR 066 + task jwafs_upp_f072 + trigger ../jwafs_gfs_manager:release_wafs_upp_072 + edit FHR 072 + task jwafs_upp_f078 + trigger ../jwafs_gfs_manager:release_wafs_upp_078 + edit FHR 078 + task jwafs_upp_f084 + trigger ../jwafs_gfs_manager:release_wafs_upp_084 + edit FHR 084 + task jwafs_upp_f090 + trigger ../jwafs_gfs_manager:release_wafs_upp_090 + edit FHR 090 + task jwafs_upp_f096 + trigger ../jwafs_gfs_manager:release_wafs_upp_096 + edit FHR 096 + task jwafs_upp_f102 + trigger ../jwafs_gfs_manager:release_wafs_upp_102 + edit FHR 102 + task jwafs_upp_f108 + trigger ../jwafs_gfs_manager:release_wafs_upp_108 + edit FHR 108 + task jwafs_upp_f114 + trigger ../jwafs_gfs_manager:release_wafs_upp_114 + edit FHR 114 + task jwafs_upp_f120 + trigger ../jwafs_gfs_manager:release_wafs_upp_120 + edit FHR 120 + endfamily # endfamily upp + family gcip + task jwafs_gcip_f000 + trigger :TIME >= 2240 and ../jwafs_gfs_manager:release_wafs_gcip_000 + edit FHR 000 + task jwafs_gcip_f003 + trigger :TIME >= 2240 and ../jwafs_gfs_manager:release_wafs_gcip_003 + edit FHR 003 + endfamily # endfamily gcip + family grib + task jwafs_grib_f012 + trigger ../jwafs_gfs_manager:release_wafs_grib_012 + edit FHR 12 + task jwafs_grib_f018 + trigger ../jwafs_gfs_manager:release_wafs_grib_018 + edit FHR 18 + task jwafs_grib_f024 + trigger ../jwafs_gfs_manager:release_wafs_grib_024 + edit FHR 24 + task jwafs_grib_f030 + trigger ../jwafs_gfs_manager:release_wafs_grib_030 + edit FHR 30 + task jwafs_grib_f036 + trigger ../jwafs_gfs_manager:release_wafs_grib_036 + edit FHR 36 + task jwafs_grib_f042 + trigger ../jwafs_gfs_manager:release_wafs_grib_042 + edit FHR 42 + task jwafs_grib_f048 + trigger ../jwafs_gfs_manager:release_wafs_grib_048 + edit FHR 48 + task jwafs_grib_f060 + trigger ../jwafs_gfs_manager:release_wafs_grib_060 + edit FHR 60 + task jwafs_grib_f072 + trigger ../jwafs_gfs_manager:release_wafs_grib_072 + edit FHR 72 + endfamily # endfamily grib + family grib2 + family 1p25 + task jwafs_grib2_1p25_f000 + trigger ../../upp/jwafs_upp_f000 == complete + edit FHR 000 + task jwafs_grib2_1p25_f006 + trigger ../../upp/jwafs_upp_f006 == complete + edit FHR 006 + task jwafs_grib2_1p25_f009 + trigger ../../upp/jwafs_upp_f009 == complete + edit FHR 009 + task jwafs_grib2_1p25_f012 + trigger ../../upp/jwafs_upp_f012 == complete + edit FHR 012 + task jwafs_grib2_1p25_f015 + trigger ../../upp/jwafs_upp_f015 == complete + edit FHR 015 + task jwafs_grib2_1p25_f018 + trigger ../../upp/jwafs_upp_f018 == complete + edit FHR 018 + task jwafs_grib2_1p25_f021 + trigger ../../upp/jwafs_upp_f021 == complete + edit FHR 021 + task jwafs_grib2_1p25_f024 + trigger ../../upp/jwafs_upp_f024 == complete + edit FHR 024 + task jwafs_grib2_1p25_f027 + trigger ../../upp/jwafs_upp_f027 == complete + edit FHR 027 + task jwafs_grib2_1p25_f030 + trigger ../../upp/jwafs_upp_f030 == complete + edit FHR 030 + task jwafs_grib2_1p25_f033 + trigger ../../upp/jwafs_upp_f033 == complete + edit FHR 033 + task jwafs_grib2_1p25_f036 + trigger ../../upp/jwafs_upp_f036 == complete + edit FHR 036 + task jwafs_grib2_1p25_f042 + trigger ../../upp/jwafs_upp_f042 == complete + edit FHR 042 + task jwafs_grib2_1p25_f048 + trigger ../../upp/jwafs_upp_f048 == complete + edit FHR 048 + task jwafs_grib2_1p25_f054 + trigger ../../upp/jwafs_upp_f054 == complete + edit FHR 054 + task jwafs_grib2_1p25_f060 + trigger ../../upp/jwafs_upp_f060 == complete + edit FHR 060 + task jwafs_grib2_1p25_f066 + trigger ../../upp/jwafs_upp_f066 == complete + edit FHR 066 + task jwafs_grib2_1p25_f072 + trigger ../../upp/jwafs_upp_f072 == complete + edit FHR 072 + endfamily # endfamily 1p25 + family 0p25 + task jwafs_grib2_0p25_f006 + trigger ../../upp/jwafs_upp_f006 == complete + edit FHR 006 + task jwafs_grib2_0p25_f007 + trigger ../../upp/jwafs_upp_f007 == complete + edit FHR 007 + task jwafs_grib2_0p25_f008 + trigger ../../upp/jwafs_upp_f008 == complete + edit FHR 008 + task jwafs_grib2_0p25_f009 + trigger ../../upp/jwafs_upp_f009 == complete + edit FHR 009 + task jwafs_grib2_0p25_f010 + trigger ../../upp/jwafs_upp_f010 == complete + edit FHR 010 + task jwafs_grib2_0p25_f011 + trigger ../../upp/jwafs_upp_f011 == complete + edit FHR 011 + task jwafs_grib2_0p25_f012 + trigger ../../upp/jwafs_upp_f012 == complete + edit FHR 012 + task jwafs_grib2_0p25_f013 + trigger ../../upp/jwafs_upp_f013 == complete + edit FHR 013 + task jwafs_grib2_0p25_f014 + trigger ../../upp/jwafs_upp_f014 == complete + edit FHR 014 + task jwafs_grib2_0p25_f015 + trigger ../../upp/jwafs_upp_f015 == complete + edit FHR 015 + task jwafs_grib2_0p25_f016 + trigger ../../upp/jwafs_upp_f016 == complete + edit FHR 016 + task jwafs_grib2_0p25_f017 + trigger ../../upp/jwafs_upp_f017 == complete + edit FHR 017 + task jwafs_grib2_0p25_f018 + trigger ../../upp/jwafs_upp_f018 == complete + edit FHR 018 + task jwafs_grib2_0p25_f019 + trigger ../../upp/jwafs_upp_f019 == complete + edit FHR 019 + task jwafs_grib2_0p25_f020 + trigger ../../upp/jwafs_upp_f020 == complete + edit FHR 020 + task jwafs_grib2_0p25_f021 + trigger ../../upp/jwafs_upp_f021 == complete + edit FHR 021 + task jwafs_grib2_0p25_f022 + trigger ../../upp/jwafs_upp_f022 == complete + edit FHR 022 + task jwafs_grib2_0p25_f023 + trigger ../../upp/jwafs_upp_f023 == complete + edit FHR 023 + task jwafs_grib2_0p25_f024 + trigger ../../upp/jwafs_upp_f024 == complete + edit FHR 024 + task jwafs_grib2_0p25_f027 + trigger ../../upp/jwafs_upp_f027 == complete + edit FHR 027 + task jwafs_grib2_0p25_f030 + trigger ../../upp/jwafs_upp_f030 == complete + edit FHR 030 + task jwafs_grib2_0p25_f033 + trigger ../../upp/jwafs_upp_f033 == complete + edit FHR 033 + task jwafs_grib2_0p25_f036 + trigger ../../upp/jwafs_upp_f036 == complete + edit FHR 036 + task jwafs_grib2_0p25_f039 + trigger ../../upp/jwafs_upp_f039 == complete + edit FHR 039 + task jwafs_grib2_0p25_f042 + trigger ../../upp/jwafs_upp_f042 == complete + edit FHR 042 + task jwafs_grib2_0p25_f045 + trigger ../../upp/jwafs_upp_f045 == complete + edit FHR 045 + task jwafs_grib2_0p25_f048 + trigger ../../upp/jwafs_upp_f048 == complete + edit FHR 048 + task jwafs_grib2_0p25_f054 + trigger ../../upp/jwafs_upp_f054 == complete + edit FHR 054 + task jwafs_grib2_0p25_f060 + trigger ../../upp/jwafs_upp_f060 == complete + edit FHR 060 + task jwafs_grib2_0p25_f066 + trigger ../../upp/jwafs_upp_f066 == complete + edit FHR 066 + task jwafs_grib2_0p25_f072 + trigger ../../upp/jwafs_upp_f072 == complete + edit FHR 072 + task jwafs_grib2_0p25_f078 + trigger ../../upp/jwafs_upp_f078 == complete + edit FHR 078 + task jwafs_grib2_0p25_f084 + trigger ../../upp/jwafs_upp_f084 == complete + edit FHR 084 + task jwafs_grib2_0p25_f090 + trigger ../../upp/jwafs_upp_f090 == complete + edit FHR 090 + task jwafs_grib2_0p25_f096 + trigger ../../upp/jwafs_upp_f096 == complete + edit FHR 096 + task jwafs_grib2_0p25_f102 + trigger ../../upp/jwafs_upp_f102 == complete + edit FHR 102 + task jwafs_grib2_0p25_f108 + trigger ../../upp/jwafs_upp_f108 == complete + edit FHR 108 + task jwafs_grib2_0p25_f114 + trigger ../../upp/jwafs_upp_f114 == complete + edit FHR 114 + task jwafs_grib2_0p25_f120 + trigger ../../upp/jwafs_upp_f120 == complete + edit FHR 120 + family blending + trigger :TIME >= 2230 + edit DCOMROOT '/lfs/h1/ops/dev/dcom/test' + task jwafs_grib2_0p25_blending + trigger ../jwafs_grib2_0p25_f048 == complete + endfamily # endfamily blending + endfamily # endfamily 0p25 + endfamily # endfamily grib2 + endfamily # endfamily 18 + endfamily # endfamily @MODELVER@ + endfamily # endfamily wafs + endfamily # endfamily primary +endsuite # endsuite wafs@EXPID@ diff --git a/ecf/scripts/gcip/jwafs_gcip_master.ecf b/ecf/scripts/gcip/jwafs_gcip_master.ecf index e674ba6..e53e257 100755 --- a/ecf/scripts/gcip/jwafs_gcip_master.ecf +++ b/ecf/scripts/gcip/jwafs_gcip_master.ecf @@ -52,4 +52,19 @@ fi %include %manual +TASK: JWAFS_GCIP + +PURPOSE: Triggered by JWAFS_GFS_MANAGER to produce icing analysis. It needs observation data including: + Satellite data from $DCOMROOT/YYYYMMDD/mcidas, manditory + METAR dumped from BUFR by 'dumpjb', manditory + PIREP/ships/lightning data dumped from BUFR by 'dumpjb', optional + Radar data, optional + +This job will be triggered by time trigger at T+4:40 +This job will be triggered by JWAFS_GFS_MANAGER once GFS forecast model output, at F000 and F003 respectively, is available. +This job will quit if satellite data is missing and/or no METAR data is dumped. +This job will continue if radar data or PIREP/ships/lightning data is missing + +TROUBLESHOOTING +This job may fail if the mandiory observation data (satellite and METAR) is missing. If the observation data becomes available, rebooting the job will work. It is ok for occasional failures. %end diff --git a/ecf/scripts/grib/jwafs_grib_master.ecf b/ecf/scripts/grib/jwafs_grib_master.ecf index 3e25a1b..d368ff8 100755 --- a/ecf/scripts/grib/jwafs_grib_master.ecf +++ b/ecf/scripts/grib/jwafs_grib_master.ecf @@ -49,4 +49,14 @@ fi %include %manual +TASK: JWAFS_GRIB + +PURPOSE: Triggered by JWAFS_GFS_MANAGER to produce WAFS grib1 data. + +This job will be triggered for each forecast hour by JWAFS_GFS_MANAGER once GFS forecast model output is available. +This job will produce data which is saved to wmo/ folder only. +This job will skip any intermediate forecast hour if the forecast model output is not available for that forecast hour + +TROUBLESHOOTING +If this job fails, make sure GFS forecast model output is available and re-run this job for the failed forecast hour. %end diff --git a/ecf/scripts/grib2/0p25/blending/jwafs_grib2_0p25_blending.ecf b/ecf/scripts/grib2/0p25/blending/jwafs_grib2_0p25_blending.ecf new file mode 100755 index 0000000..f59bce6 --- /dev/null +++ b/ecf/scripts/grib2/0p25/blending/jwafs_grib2_0p25_blending.ecf @@ -0,0 +1,71 @@ +#PBS -S /bin/bash +#PBS -N %RUN%_grib2_0p25_blending_%CYC% +#PBS -j oe +#PBS -q %QUEUE% +#PBS -A %PROJ%-%PROJENVIR% +#PBS -l walltime=00:30:00 +#PBS -l select=1:ncpus=27:mem=50GB +#PBS -l debug=true + +export model=wafs + +%include +%include + +set -x + +export NET=%NET% +export RUN=%RUN% +export cyc=%CYC% + +############################################################ +# Load modules +############################################################ +set +x +module load PrgEnv-intel/${PrgEnv_intel_ver} +module load craype/${craype_ver} +module load intel/${intel_ver} +module load libjpeg/${libjpeg_ver} +module load grib_util/${grib_util_ver} +module load util_shared/${util_shared_ver} + +module load cray-pals/${craypals_ver} +module load cfp/${cfp_ver} + +module list +set -x + +############################################################# +# WCOSS environment settings +############################################################# + +############################################################ +# CALL executable job script here +############################################################ +# Set up mailing list +if [[ "${envir}" != "prod" ]]; then + MAILTO="nco.spa@noaa.gov" +fi +export MAILTO=${MAILTO:-"nco.spa@noaa.gov,ncep.sos@noaa.gov,nco.sos@noaa.gov,nco.hpc.dataflow@noaa.gov"} + +${HOMEwafs}/jobs/JWAFS_GRIB2_0P25_BLENDING +if [ $? -ne 0 ]; then + ecflow_client --msg="***JOB ${ECF_NAME} ERROR RUNNING J-SCRIPT ***" + ecflow_client --abort + exit +fi + +%include +%manual +TASK: JWAFS_GRIB2_0P25_BLENDING + +PURPOSE: Triggered by JWAFS_GRIB2_0P25 to blend US and UK unblended hazard data at 0.25 degree + +This job will be triggered by time trigger at T+4:30. +This job will be triggered after JWAFS_GRIB2_0P25 f048 is completed. In non real time, it will sleep/wait 60 seconds if JWAFS_GRIB2_0P25 at another forecast hour is not completed yet when f048 is completed. +This single MPMD job will dispatch subtasks to each forecast hour, collect the subtask results and send out warning email and dbn_alert of missing data once per cycle. +This job will start blending if UK unblended data from DCOM is available; or wait up to 25 minutes for UK data. + +TROUBLESHOOTING +If this job fails, make sure both its upstream JWAFS_GRIB2_0P25 output and UK data are available then re-run this job for all the forecast hours. +%end diff --git a/ecf/scripts/grib2/0p25/blending/jwafs_grib2_0p25_blending_master.ecf b/ecf/scripts/grib2/0p25/blending/jwafs_grib2_0p25_blending_master.ecf deleted file mode 100755 index a01780a..0000000 --- a/ecf/scripts/grib2/0p25/blending/jwafs_grib2_0p25_blending_master.ecf +++ /dev/null @@ -1,52 +0,0 @@ -#PBS -S /bin/bash -#PBS -N %RUN%_grib2_0p25_blending_f%FHR%_%CYC% -#PBS -j oe -#PBS -q %QUEUE% -#PBS -A %PROJ%-%PROJENVIR% -#PBS -l walltime=00:25:00 -#PBS -l select=1:ncpus=1 -#PBS -l debug=true - -export model=wafs - -%include -%include - -set -x - -export NET=%NET% -export RUN=%RUN% -export cyc=%CYC% -export fhr=%FHR% - -############################################################ -# Load modules -############################################################ -set +x -module load PrgEnv-intel/${PrgEnv_intel_ver} -module load craype/${craype_ver} -module load intel/${intel_ver} -module load libjpeg/${libjpeg_ver} -module load grib_util/${grib_util_ver} -module load util_shared/${util_shared_ver} - -module list -set -x - -############################################################# -# WCOSS environment settings -############################################################# - -############################################################ -# CALL executable job script here -############################################################ -${HOMEwafs}/jobs/JWAFS_GRIB2_0P25_BLENDING -if [ $? -ne 0 ]; then - ecflow_client --msg="***JOB ${ECF_NAME} ERROR RUNNING J-SCRIPT ***" - ecflow_client --abort - exit -fi - -%include -%manual -%end diff --git a/ecf/scripts/grib2/0p25/jwafs_grib2_0p25_master.ecf b/ecf/scripts/grib2/0p25/jwafs_grib2_0p25_master.ecf index 8dc3587..cfb6a6e 100755 --- a/ecf/scripts/grib2/0p25/jwafs_grib2_0p25_master.ecf +++ b/ecf/scripts/grib2/0p25/jwafs_grib2_0p25_master.ecf @@ -50,4 +50,16 @@ fi %include %manual +TASK: JWAFS_GRIB2_0P25 + +PURPOSE: Triggered by JWAFS_UPP to produce WAFS grid files at 0.25 degree, and trigger the blending job: + JWAFS_GRIB2_0P25_BLENDING + +This job will be triggered for each forecast hour once JWAFS_UPP is finished. +This job will produce WAFS grid files at 0.25 degree resolution. +This job will produce wafs.tCCz.awf.0p25.fFFF.grib2, while FFF [006-036] has extra levels of icing and turbulence than FFF [039-048] +This job will skip any intermediate forecast hour if GFS model output is not available or the upsteam JWAFS_UPP fails for that forecast hour. + +TROUBLESHOOTING +If this job fails, make sure its upstream JWAFS_UPP output is available and re-run this job for the failed forecast hour. %end diff --git a/ecf/scripts/grib2/1p25/jwafs_grib2_1p25_master.ecf b/ecf/scripts/grib2/1p25/jwafs_grib2_1p25_master.ecf index 54b90c0..c036b26 100755 --- a/ecf/scripts/grib2/1p25/jwafs_grib2_1p25_master.ecf +++ b/ecf/scripts/grib2/1p25/jwafs_grib2_1p25_master.ecf @@ -50,4 +50,14 @@ fi %include %manual +TASK: JWAFS_GRIB2_1P25 + +PURPOSE: Triggered by JWAFS_UPP to produce WAFS files on grid 45 + +This job will be triggered for each forecast hour once JWAFS_UPP is finished. +This job will produce WAFS files on grid 45 , which is at 1.25 degree. +This job will skip any intermediate forecast hour if GFS model output is not available or the upsteam JWAFS_UPP fails for that forecast hour. + +TROUBLESHOOTING +If this job fails, make sure its upstream JWAFS_UPP output is available and re-run this job for the failed forecast hour. %end diff --git a/ecf/scripts/jwafs_gfs_manager.ecf b/ecf/scripts/jwafs_gfs_manager.ecf index 2939ac0..64376e7 100755 --- a/ecf/scripts/jwafs_gfs_manager.ecf +++ b/ecf/scripts/jwafs_gfs_manager.ecf @@ -38,4 +38,17 @@ fi %include %manual -%end +TASK: JWAFS_GFS_MANAGER + +PURPOSE: Look for GFS forecast model output and trigger the following jobs: + JWAFS_UPP: Offline UPP with GTG + JWAFS_GCIP: Icing analysis as a downstream of GFS master file + JWAFS_GRIB: GFS WAFS GRIB1 product as a downstream of GFS master file + +This job will look for GFS forecast model output, and trigger the above jobs for each of the forecast hour that the products are desired. +The job will timeout in 10800 seconds (3 hours) if the forecast model output is not available. +The job will skip any intermediate forecast hour if the forecast model output is not available for that forecast hour + +TROUBLESHOOTING +If this job fails, re-queuing the job is probably not the best idea as it may release the triggers for already launched/completed downstream jobs. In that case, it is perhaps better to boot the downstream jobs mentioned in the purpose for the failed forecast hours. +%end \ No newline at end of file diff --git a/ecf/scripts/upp/jwafs_upp_master.ecf b/ecf/scripts/upp/jwafs_upp_master.ecf index b249224..91f0932 100755 --- a/ecf/scripts/upp/jwafs_upp_master.ecf +++ b/ecf/scripts/upp/jwafs_upp_master.ecf @@ -56,4 +56,17 @@ fi %include %manual +TASK: JWAFS_UPP + +PURPOSE: Triggered by JWAFS_GFS_MANAGER to produce WAFS master files, and trigger the following two jobs: + JWAFS_GRIB2_0P25 + JWAFS_GRIB2_1P25 + +This job will be triggered for each forecast hour by JWAFS_GFS_MANAGER once GFS forecast model output is available. +Different from genearl UPP, this job will be with GTG to produce WAFS master files for its downstream jobs. +This job will use different control files for forecast hour [000-048] and [054-120], so forecast products are different. forecast hour [000-048] contains icing and turublence, while [054-120] doesn't. +This job will skip any intermediate forecast hour if the forecast model output is not available for that forecast hour + +TROUBLESHOOTING +If this job fails, make sure GFS forecast model output is available and re-run this job for the failed forecast hour. To debug, make sure GTG related source code, scripts and parm files work correctly. %end diff --git a/ecf/setup_ecf_links.sh b/ecf/setup_ecf_links.sh index 0990752..1290f72 100755 --- a/ecf/setup_ecf_links.sh +++ b/ecf/setup_ecf_links.sh @@ -15,22 +15,22 @@ function link_master_to_fhr() { local clean_only=${3:-"NO"} # Clean only flag to remove existing links local fhr3 master target for fhr in ${fhrs[@]}; do - fhr3=$(printf %03d $fhr) - master=${tmpl}_master.ecf - target=${tmpl}_f${fhr3}.ecf - rm -f ${target} - case ${clean_only} in + fhr3=$(printf %03d ${fhr}) + master="${tmpl}_master.ecf" + target="${tmpl}_f${fhr3}.ecf" + rm -f "${target}" + case "${clean_only}" in "YES") continue ;; *) - ln -sf ${master} ${target} + ln -sf "${master}" "${target}" ;; esac done } -CLEAN=${1:-${CLEAN:-"NO"}} # Remove links only; do not create links (YES) +CLEAN=${1:-${CLEAN:-"NO"}} # Remove links only; do not create links (YES) # JWAFS_UPP cd "${ECF_DIR}/scripts/upp" @@ -40,10 +40,11 @@ rm -f jwafs_upp_anl.ecf if [[ "${CLEAN}" != "YES" ]]; then ln -sf jwafs_upp_master.ecf jwafs_upp_anl.ecf fi -seq1=$(seq -s ' ' 0 1 24) # 000 -> 024; 1-hourly +seq0="0" # 000 +seq1=$(seq -s ' ' 6 1 24) # 006 -> 024; 1-hourly seq2=$(seq -s ' ' 27 3 48) # 027 -> 048; 3-hourly seq3=$(seq -s ' ' 54 6 120) # 054 -> 120; 6-hourly -fhrs="${seq1} ${seq2} ${seq3}" +fhrs="${seq0} ${seq1} ${seq2} ${seq3}" link_master_to_fhr "jwafs_upp" "${fhrs}" "${CLEAN}" # JWAFS_GRIB2 @@ -65,12 +66,6 @@ fhrs="${seq1} ${seq2} ${seq3}" link_master_to_fhr "jwafs_grib2_0p25" "${fhrs}" "${CLEAN}" # JWAFS_BLENDING_0P25 -cd "${ECF_DIR}/scripts/grib2/0p25/blending" -echo "Linking grib2/0p25/blending ..." -seq1=$(seq -s ' ' 6 1 24) # 006 -> 024; 1-hourly -seq2=$(seq -s ' ' 27 3 48) # 027 -> 048; 3-hourly -fhrs="${seq1} ${seq2}" -link_master_to_fhr "jwafs_grib2_0p25_blending" "${fhrs}" "${CLEAN}" # JWAFS_GCIP cd "${ECF_DIR}/scripts/gcip" @@ -81,5 +76,7 @@ link_master_to_fhr "jwafs_gcip" "${fhrs}" "${CLEAN}" # JWAFS_GRIB cd "${ECF_DIR}/scripts/grib" echo "Linking grib ..." -fhrs=$(seq -s ' ' 6 6 72) # 006 -> 072; 6-hourly +seq1=$(seq -s ' ' 12 6 48) # 012 -> 048; 6-hourly +seq2="60 72" # 060, 072 +fhrs="${seq1} ${seq2}" link_master_to_fhr "jwafs_grib" "${fhrs}" "${CLEAN}" diff --git a/fix/wafs/grib_wafs.namelist b/fix/wafs/grib_wafs.namelist old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs00 b/fix/wafs/grib_wfsgfs00 old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs06 b/fix/wafs/grib_wfsgfs06 old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs12a b/fix/wafs/grib_wfsgfs12a old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs12b b/fix/wafs/grib_wfsgfs12b old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs18a b/fix/wafs/grib_wfsgfs18a old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs18b b/fix/wafs/grib_wfsgfs18b old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs24a b/fix/wafs/grib_wfsgfs24a old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs24b b/fix/wafs/grib_wfsgfs24b old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs30a b/fix/wafs/grib_wfsgfs30a old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs30b b/fix/wafs/grib_wfsgfs30b old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs36 b/fix/wafs/grib_wfsgfs36 old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs42 b/fix/wafs/grib_wfsgfs42 old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs48 b/fix/wafs/grib_wfsgfs48 old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs60 b/fix/wafs/grib_wfsgfs60 old mode 100755 new mode 100644 diff --git a/fix/wafs/grib_wfsgfs72 b/fix/wafs/grib_wfsgfs72 old mode 100755 new mode 100644 diff --git a/fix/wafs/wafs_blending_0p25_admin_msg b/fix/wafs/wafs_blending_0p25_admin_msg old mode 100755 new mode 100644 diff --git a/jobs/JWAFS_GCIP b/jobs/JWAFS_GCIP index c881ffe..e02fbf1 100755 --- a/jobs/JWAFS_GCIP +++ b/jobs/JWAFS_GCIP @@ -1,121 +1,78 @@ #!/bin/bash - -############################################ +######################################## # WAFS GCIP PRODUCT GENERATION -############################################ +######################################## date -export PS4='$SECONDS + ' +export PS4='$SECONDS + ' set -x -# keep the working directory or not -export KEEPDATA=${KEEPDATA:-NO} - -############################################ -# Specify it is production or developmen -############################################ -export envir=${envir:-prod} -# print current environment -env - -############################################ -# Working Directory -############################################ +#################################### +# make temp directory +#################################### export DATA=${DATA:-${DATAROOT}/${jobid:?}} -mkdir -p $DATA -cd $DATA - -############################################ -# Output for executables -############################################ -export pgmout=OUTPUT.$$ - -############################################ -# Load the UTILITIES module -############################################ -#### module load prod_util -#### module load grib_util +mkdir -p "${DATA}" && cd "${DATA}" -############################################ +########################################### # Run setpdy and initialize PDY variables -############################################ -export cycle=t${cyc}z +########################################### +export cycle=${cycle:-"t${cyc}z"} setpdy.sh . ./PDY -############################################ -# Set up the NET and RUN -############################################ +#################################### +# SEND to COM, DBN, etc +#################################### +export SENDCOM=${SENDCOM:-"YES"} + +#################################### +# Specify NET and RUN Name and model +#################################### export NET=${NET:-wafs} export RUN=${RUN:-wafs} -############################################ -# Specify HOME Directory -############################################ -export HOMEwafs=${HOMEwafs:-${NWROOT}/wafs.${wafs_ver}} -export EXECwafs=$HOMEwafs/exec -export FIXwafs=$HOMEwafs/fix/wafs -export PARMwafs=$HOMEwafs/parm/wafs -export USHwafs=$HOMEwafs/ush -export SCRIPTSwafs=$HOMEwafs/scripts - -# For BUFR dump, TMPDIR must be defined -export TMPDIR=$DATA # will be overwritten in exwafs script for parallel runs on fhr -# For BUFR dump, these two environment variables are defined by module load -# HOMEobsproc_shared_bufr_dumplist <= module load bufr_dumplist/1.5.0 -# HOMEobsproc_dump <= module load dumpjb/4.0.0 - - -################################################ -# Set up the input/output directory -################################################ -# model data -export COMINgfs=${COMINgfs:-$(compath.py ${envir}/gfs/${gfs_ver})/gfs.${PDY}/${cyc}/atmos} - -# satellite data -#ftp://satepsanone.nesdis.noaa.gov/2day/gmosaic/ -# Have to change IP address to digital ones, which BSUB can identify -#export COMINsat=${COMINsat:-ftp://140.90.213.161/2day/gmosaic} -export COMINsat=${COMINsat:-$DCOMROOT/$PDY/mcidas} - -# radar data -export COMINradar=${COMINradar:-$(compath.py ${envir}/radarl2/$radarl2_ver)/radar.$PDY} - -# metar/ships/lightning/pireps -# data are dumped by $USHobsproc_dump/dumpjb -# - -# COMOUT -export COMOUT=${COMOUT:-$(compath.py -o $NET/$wafs_ver)/$RUN.$PDY/$cyc/gcip} - -mkdir -p $COMOUT +############################################## +# Define COM directories +############################################## +export COMINgfs=${COMINgfs:-$(compath.py "${envir}/gfs/${gfs_ver}")"/gfs.${PDY}/${cyc}/atmos"} +export COMINsat=${COMINsat:-"${DCOMROOT}/${PDY}/mcidas"} +export COMINradar=${COMINradar:-$(compath.py "${envir}/radarl2/${radarl2_ver}")"/radar.${PDY}"} +export COMOUT=${COMOUT:-$(compath.py -o "${NET}/${wafs_ver}")"/${RUN}.${PDY}/${cyc}/gcip"} +mkdir -p "${COMOUT}" + +#################################### +# Determine Job Output Name on System +#################################### +export pgmout="OUTPUT.$$" + +#################################### +# Specify Execution Areas +#################################### +export EXECwafs="${HOMEwafs}/exec" +export PARMwafs="${HOMEwafs}/parm" +export FIXwafs="${HOMEwafs}/fix" + +######################################################### +# print current environment +######################################################### +env ############################################ -# Execute the script, parallel run for 000 003 +# Execute the script. ############################################ -${SCRIPTSwafs}/exwafs_gcip.sh $fhr +"${HOMEwafs}/scripts/exwafs_gcip.sh" export err=$? +err_chk -if [ $err -eq 0 ] ; then - echo "JOB $job HAS COMPLETED NORMALLY!" -elif [ $err -eq 1 ] ; then - echo "WARNING!!! JOB $job incomplete. Missing satellite data." -else - echo "JOB $job FAILED!!!!" -fi - -############################################ -# print exec output -############################################ -if [ -e "$pgmout" ] ; then - cat $pgmout +if [[ -e "${pgmout}" ]]; then + cat "${pgmout}" fi -############################################ -# remove temporary working directory -############################################ -if [ $KEEPDATA != YES ] ; then - rm -rf $DATA +############################## +# Remove the Temporary working directory +############################## +if [[ "${KEEPDATA^^}" != "YES" ]]; then + rm -rf "${DATA}" fi date diff --git a/jobs/JWAFS_GFS_MANAGER b/jobs/JWAFS_GFS_MANAGER index fa6657f..385ff5c 100755 --- a/jobs/JWAFS_GFS_MANAGER +++ b/jobs/JWAFS_GFS_MANAGER @@ -1,89 +1,70 @@ #!/bin/bash - ######################################## -# GFS manager +# WAFS GFS MANAGER ######################################## -export PS4='$SECONDS + ' date +export PS4='$SECONDS + ' set -x -# keep the working directory or not -export KEEPDATA=${KEEPDATA:-NO} - -############################################ -# Specify it is production or developmen -############################################ -export envir=${envir:-prod} -# print current environment -env - -############################################ -# Working Directory -############################################ +#################################### +# make temp directory +#################################### export DATA=${DATA:-${DATAROOT}/${jobid:?}} -mkdir -p $DATA -cd $DATA - -############################################ -# Output for executables -############################################ -export pgmout=OUTPUT.$$ - -############################################ -# Load the UTILITIES module -############################################ -#### module load prod_util -#### module load grib_util +mkdir -p "${DATA}" && cd "${DATA}" ########################################### # Run setpdy and initialize PDY variables ########################################### -export cycle=t${cyc}z +export cycle=${cycle:-"t${cyc}z"} setpdy.sh . ./PDY -############################################ -# Set up the NET and RUN -############################################ +#################################### +# SEND to COM, DBN, etc +#################################### + +#################################### +# Specify NET and RUN Name and model +#################################### export NET=${NET:-wafs} export RUN=${RUN:-wafs} -############################################ -# Specify HOME Directory -############################################ -export HOMEwafs=${HOMEwafs:-${NWROOT}/wafs.${wafs_ver}} -export EXECwafs=$HOMEwafs/exec -export FIXwafs=$HOMEwafs/fix/wafs -export PARMwafs=$HOMEwafs/parm/wafs -export USHwafs=$HOMEwafs/ush -export SCRIPTSwafs=$HOMEwafs/scripts +############################################## +# Define COM directories +############################################## +export COMINgfs=${COMINgfs:-$(compath.py "${envir}/gfs/${gfs_ver}")"/gfs.${PDY}/${cyc}/atmos"} -################################################ -# Set up the input/output directory -################################################ +#################################### +# Determine Job Output Name on System +#################################### +export pgmout="OUTPUT.$$" -export COMIN=${COMIN:-$(compath.py $envir/$NET/$wafs_ver)/$RUN.$PDY/$cyc} -export COMOUT=${COMOUT:-$(compath.py -o $NET/$wafs_ver)/$RUN.$PDY/$cyc} -export COMINgfs=${COMINgfs:-$(compath.py $envir/gfs/$gfs_ver)/gfs.$PDY/$cyc/atmos} +#################################### +# Specify Execution Areas +#################################### -######################################################## -# Execute the script. -$HOMEwafs/scripts/exwafs_gfs_manager.sh -######################################################## +######################################################### +# print current environment +######################################################### +env ############################################ -# print exec output +# Execute the script. ############################################ -if [ -e "$pgmout" ]; then - cat $pgmout +"${HOMEwafs}/scripts/exwafs_gfs_manager.sh" +export err=$? +err_chk + +if [[ -e "${pgmout}" ]]; then + cat "${pgmout}" fi -############################################ -# remove temporary working directory -############################################ -if [ $KEEPDATA != YES ]; then - rm -rf $DATA +############################## +# Remove the Temporary working directory +############################## +if [[ "${KEEPDATA^^}" != "YES" ]]; then + rm -rf "${DATA}" fi date diff --git a/jobs/JWAFS_GRIB b/jobs/JWAFS_GRIB index 0548779..e8140f3 100755 --- a/jobs/JWAFS_GRIB +++ b/jobs/JWAFS_GRIB @@ -1,97 +1,78 @@ #!/bin/bash - ######################################## -# WAFS GRIB AWIPS PRODUCT GENERATION +# WAFS GRIB AWIPS PRODUCT GENERATION ######################################## + date -export PS4='$SECONDS + ' +export PS4='$SECONDS + ' set -x -export KEEPDATA=${KEEPDATA:-NO} - -############################################ -# Specify it is production or developmen -############################################ -export envir=${envir:-prod} -# print current environment -env - -############################################ -# Working Directory -############################################ +#################################### +# make temp directory +#################################### export DATA=${DATA:-${DATAROOT}/${jobid:?}} -mkdir -p $DATA -cd $DATA - -############################################ -# Output for executables -############################################ -export pgmout=OUTPUT.$$ - -############################################ -# Load the UTILITIES module -############################################ -#### module load prod_util -#### module load grib_util +mkdir -p "${DATA}" && cd "${DATA}" ########################################### # Run setpdy and initialize PDY variables ########################################### -export cycle=t${cyc}z +export cycle=${cycle:-"t${cyc}z"} setpdy.sh . ./PDY -export RERUN=${RERUN:-NO} +#################################### +# SEND to COM, DBN, etc +#################################### +export SENDCOM=${SENDCOM:-"YES"} +export SENDDBN_NTC=${SENDDBN_NTC:-"YES"} -############################################ -# Set up the NET and RUN -############################################ +#################################### +# Specify NET and RUN Name and model +#################################### export NET=${NET:-wafs} export RUN=${RUN:-wafs} -############################################ -# Specify HOME Directory -############################################ -export HOMEwafs=${HOMEwafs:-${NWROOT}/wafs.${wafs_ver}} -export EXECwafs=$HOMEwafs/exec -export FIXwafs=$HOMEwafs/fix/wafs -export PARMwafs=$HOMEwafs/parm/wafs -export USHwafs=$HOMEwafs/ush -export SCRIPTSwafs=$HOMEwafs/scripts - -################################################ -# Set up the input/output directory -################################################ -export COMOUT=${COMOUT:-$(compath.py -o $NET/$wafs_ver)/$RUN.$PDY/$cyc/grib} -export PCOM=${PCOM:-$COMOUT/wmo} - -export COMINgfs=${COMINgfs:-$(compath.py ${envir}/gfs/${gfs_ver})/gfs.${PDY}/${cyc}/atmos} - -if [ $SENDCOM = YES ] ; then - mkdir -p $COMOUT $PCOM -fi +############################################## +# Define COM directories +############################################## +export COMINgfs=${COMINgfs:-$(compath.py "${envir}/gfs/${gfs_ver}")"/gfs.${PDY}/${cyc}/atmos"} +export COMOUT=${COMOUT:-$(compath.py -o "${NET}/${wafs_ver}")"/${RUN}.${PDY}/${cyc}/grib"} +export COMOUTwmo=${COMOUTwmo:-"${COMOUT}/wmo"} +mkdir -p "${COMOUT}" "${COMOUTwmo}" + +#################################### +# Determine Job Output Name on System +#################################### +export pgmout="OUTPUT.$$" + +#################################### +# Specify Execution Areas +#################################### +export EXECwafs="${HOMEwafs}/exec" +export FIXwafs="${HOMEwafs}/fix" +export USHwafs="${HOMEwafs}/ush" + +######################################################### +# print current environment +######################################################### +env ############################################ # Execute the script. ############################################ -${SCRIPTSwafs}/exwafs_grib.sh $fhr - -export err=$?; err_chk - -echo "JOB $job HAS COMPLETED NORMALLY!" +"${HOMEwafs}/scripts/exwafs_grib.sh" +export err=$? +err_chk -############################################ -# print exec output -############################################ -if [ -e "$pgmout" ] ; then - cat $pgmout +if [[ -e "${pgmout}" ]]; then + cat "${pgmout}" fi -############################################ -# remove temporary working directory -############################################ -if [ $KEEPDATA != YES ] ; then - rm -rf $DATA +############################## +# Remove the Temporary working directory +############################## +if [[ "${KEEPDATA^^}" != "YES" ]]; then + rm -rf "${DATA}" fi date diff --git a/jobs/JWAFS_GRIB2_0P25 b/jobs/JWAFS_GRIB2_0P25 index fd75a24..bd63ff6 100755 --- a/jobs/JWAFS_GRIB2_0P25 +++ b/jobs/JWAFS_GRIB2_0P25 @@ -1,93 +1,77 @@ #!/bin/bash ######################################## -# WAFS GRIB 0P25 PRODUCT GENERATION +# WAFS GRIB2 0P25 PRODUCT GENERATION ######################################## date -export PS4='$SECONDS + ' +export PS4='$SECONDS + ' set -x -# keep the working directory or not -export KEEPDATA=${KEEPDATA:-NO} - -############################################ -# Specify it is production or developmen -############################################ -export envir=${envir:-prod} -# print current environment -env - -############################################ -# Working Directory -############################################ +#################################### +# make temp directory +#################################### export DATA=${DATA:-${DATAROOT}/${jobid:?}} -mkdir -p $DATA -cd $DATA - -############################################ -# Output for executables -############################################ -export pgmout=OUTPUT.$$ +mkdir -p "${DATA}" && cd "${DATA}" ########################################### # Run setpdy and initialize PDY variables ########################################### -export cycle=t${cyc}z +export cycle=${cycle:-"t${cyc}z"} setpdy.sh . ./PDY -############################################ -# Set up the NET and RUN -############################################ +#################################### +# SEND to COM, DBN, etc +#################################### +export SENDCOM=${SENDCOM:-"YES"} +export SENDDBN=${SENDDBN:-"YES"} + +#################################### +# Specify NET and RUN Name and model +#################################### export NET=${NET:-wafs} export RUN=${RUN:-wafs} -############################################ -# Specify HOME Directory -############################################ -export HOMEwafs=${HOMEwafs:-${NWROOT}/wafs.${wafs_ver}} -export EXECwafs=$HOMEwafs/exec -export FIXwafs=$HOMEwafs/fix/wafs -export PARMwafs=$HOMEwafs/parm/wafs -export USHwafs=$HOMEwafs/ush -export SCRIPTSwafs=$HOMEwafs/scripts - -################################################ -# Set up the input/output directory -################################################ -export COMIN=${COMIN:-$(compath.py $envir/$NET/$wafs_ver)/$RUN.$PDY/$cyc/upp} -export COMOUT=${COMOUT:-$(compath.py -o $NET/$wafs_ver)/$RUN.$PDY/$cyc/grib2/0p25} -export PCOM=${PCOM:-$COMOUT/wmo} - -export COMINgfs=${COMINgfs:-$(compath.py $envir/gfs/$gfs_ver)/gfs.$PDY/$cyc/atmos} - -if [ $SENDCOM = YES ] ; then - mkdir -p $COMOUT # $PCOM -fi +############################################## +# Define COM directories +############################################## +export COMINgfs=${COMINgfs:-$(compath.py "${envir}/gfs/${gfs_ver}")"/gfs.${PDY}/${cyc}/atmos"} +export COMIN=${COMIN:-$(compath.py "${envir}/${NET}/${wafs_ver}")"/${RUN}.${PDY}/${cyc}/upp"} +export COMOUT=${COMOUT:-$(compath.py -o "${NET}/${wafs_ver}")"/${RUN}.${PDY}/${cyc}/grib2/0p25"} +mkdir -p "${COMOUT}" + +#################################### +# Determine Job Output Name on System +#################################### +export pgmout="OUTPUT.$$" + +#################################### +# Specify Execution Areas +#################################### +export FIXwafs="${HOMEwafs}/fix" + +######################################################### +# print current environment +######################################################### +env ############################################ # Execute the script. ############################################ -${SCRIPTSwafs}/exwafs_grib2_0p25.sh $fhr +"${HOMEwafs}/scripts/exwafs_grib2_0p25.sh" +export err=$? +err_chk -export err=$?; err_chk - -echo "JOB $job HAS COMPLETED NORMALLY!" - -############################################ -# print exec output -############################################ -if [ -e "$pgmout" ] ; then - cat $pgmout +if [[ -e "${pgmout}" ]]; then + cat "${pgmout}" fi -############################################ -# remove temporary working directory -############################################ -if [ $KEEPDATA != YES ] ; then - rm -rf $DATA +############################## +# Remove the Temporary working directory +############################## +if [[ "${KEEPDATA^^}" != "YES" ]]; then + rm -rf "${DATA}" fi date - diff --git a/jobs/JWAFS_GRIB2_0P25_BLENDING b/jobs/JWAFS_GRIB2_0P25_BLENDING index 67569a5..164fa30 100755 --- a/jobs/JWAFS_GRIB2_0P25_BLENDING +++ b/jobs/JWAFS_GRIB2_0P25_BLENDING @@ -1,107 +1,82 @@ #!/bin/bash ######################################################## -# This job runs the code to blend US's and UK's WAFS products at 0.25 deg +# This job runs the code to blend US's and UK's WAFS products at 0.25 deg ######################################################## date -export PS4='$SECONDS + ' +export PS4='$SECONDS + ' set -x -# keep the working directory or not -export KEEPDATA=${KEEPDATA:-NO} - -############################################ -# Specify it is production or developmen -############################################ -export envir=${envir:-prod} -# print current environment -env - -############################################ -# Working Directory -############################################ +#################################### +# make temp directory +#################################### export DATA=${DATA:-${DATAROOT}/${jobid:?}} -mkdir -p $DATA -cd $DATA - -############################################ -# Output for executables -############################################ -export pgmout=OUTPUT.$$ +mkdir -p "${DATA}" && cd "${DATA}" ########################################### # Run setpdy and initialize PDY variables ########################################### -export cycle=t${cyc}z +export cycle=${cycle:-"t${cyc}z"} setpdy.sh . ./PDY -export RERUN=${RERUN:-NO} +#################################### +# SEND to COM, DBN, etc +#################################### +export SENDCOM=${SENDCOM:-"YES"} +export SENDDBN=${SENDDBN:-"YES"} +export SENDDBN_NTC=${SENDDBN_NTC:-"YES"} -############################################ -# Set up the NET and RUN -############################################ +#################################### +# Specify NET and RUN Name and model +#################################### export NET=${NET:-wafs} export RUN=${RUN:-wafs} -############################################ -# Specify HOME Directory -############################################ -export HOMEwafs=${HOMEwafs:-${NWROOT}/wafs.${wafs_ver}} -export EXECwafs=$HOMEwafs/exec -export FIXwafs=$HOMEwafs/fix/wafs -export PARMwafs=$HOMEwafs/parm/wafs -export USHwafs=$HOMEwafs/ush -export SCRIPTSwafs=$HOMEwafs/scripts - -################################################ -# Set up the INPUT and OUTPUT directories -################################################ -export COMIN=${COMIN:-$(compath.py $envir/$NET/$wafs_ver)/$RUN.$PDY/$cyc/grib2/0p25} -export COMOUT=${COMOUT:-$(compath.py -o $NET/$wafs_ver)/$RUN.$PDY/$cyc/grib2/0p25/blending} -export PCOM=${PCOM:-$COMOUT/wmo} - -if [ $SENDCOM = YES ] ; then - mkdir -p $COMOUT $PCOM -fi - -export COMINus=${COMINus:-$COMIN} -export COMINuk=${COMINuk:-$DCOMROOT/$PDY/wgrbbul/ukmet_wafs} - -############################################### -# Specify Timeout Behavior of WAFS blending -# -# SLEEP_TIME - Amount of time to wait for -# a input file before exiting -# SLEEP_INT - Amount of time to wait between -# checking for input files -############################################### -# export SLEEP_TIME=300 # changed to 60 to avoid hitting wall_clock when miss umket wafs files ... -# JY -0129: export SLEEP_TIME=600 -# Aug2024: export SLEEP_TIME=1200 (20 minutes, T+4:30 - T+4:50) -export SLEEP_TIME=1200 -export SLEEP_INT=10 +############################################## +# Define COM directories +############################################## +export COMIN=${COMIN:-$(compath.py "${envir}/${NET}/${wafs_ver}")"/${RUN}.${PDY}/${cyc}/grib2/0p25"} +export COMOUT=${COMOUT:-$(compath.py -o "${NET}/${wafs_ver}")"/${RUN}.${PDY}/${cyc}/grib2/0p25/blending"} +export COMOUTwmo=${COMOUTwmo:-"${COMOUT}/wmo"} +mkdir -p "${COMOUT}" "${COMOUTwmo}" + +export COMINus=${COMINus:-"${COMIN}"} +export COMINuk=${COMINuk:-"${DCOMROOT}/${PDY}/wgrbbul/ukmet_wafs"} + +#################################### +# Determine Job Output Name on System +#################################### +export pgmout="OUTPUT.$$" + +#################################### +# Specify Execution Areas +#################################### +export EXECwafs="${HOMEwafs}/exec" +export FIXwafs="${HOMEwafs}/fix" +export USHwafs="${HOMEwafs}/ush" + +######################################################### +# print current environment +######################################################### +env ############################################ # Execute the script. ############################################ -${SCRIPTSwafs}/exwafs_grib2_0p25_blending.sh -export err=$?; err_chk - -echo "JOB $job HAS COMPLETED NORMALLY." +"${HOMEwafs}/scripts/exwafs_grib2_0p25_blending.sh" +export err=$? +err_chk -############################################ -# print exec output -############################################ -if [ -e "$pgmout" ] ; then - cat $pgmout +if [[ -e "${pgmout}" ]]; then + cat "${pgmout}" fi -############################################ -# remove temporary working directory -############################################ -if [ $KEEPDATA != YES ] ; then - rm -rf $DATA +############################## +# Remove the Temporary working directory +############################## +if [[ "${KEEPDATA^^}" != "YES" ]]; then + rm -rf "${DATA}" fi date diff --git a/jobs/JWAFS_GRIB2_1P25 b/jobs/JWAFS_GRIB2_1P25 index 6d1af56..d02152d 100755 --- a/jobs/JWAFS_GRIB2_1P25 +++ b/jobs/JWAFS_GRIB2_1P25 @@ -5,95 +5,75 @@ ######################################## date -export PS4='$SECONDS + ' +export PS4='$SECONDS + ' set -x -# keep the working directory or not -export KEEPDATA=${KEEPDATA:-NO} - -############################################ -# Specify it is production or developmen -############################################ -export envir=${envir:-prod} -# print current environment -env - -############################################ -# Working Directory -############################################ +#################################### +# make temp directory +#################################### export DATA=${DATA:-${DATAROOT}/${jobid:?}} -mkdir -p $DATA -cd $DATA - -############################################ -# Output for executables -############################################ -export pgmout=OUTPUT.$$ - -############################################ -# Load the UTILITIES module -############################################ -#### module load prod_util -#### module load grib_util +mkdir -p "${DATA}" && cd "${DATA}" ########################################### # Run setpdy and initialize PDY variables ########################################### -export cycle=t${cyc}z +export cycle=${cycle:-"t${cyc}z"} setpdy.sh . ./PDY -############################################ -# Set up the NET and RUN -############################################ +#################################### +# SEND to COM, DBN, etc +#################################### +export SENDCOM=${SENDCOM:-"YES"} +export SENDDBN=${SENDDBN:-"YES"} +export SENDDBN_NTC=${SENDDBN_NTC:-"YES"} + +#################################### +# Specify NET and RUN Name and model +#################################### export NET=${NET:-wafs} export RUN=${RUN:-wafs} -############################################ -# Specify HOME Directory -############################################ -export HOMEwafs=${HOMEwafs:-${NWROOT}/wafs.${wafs_ver}} -export EXECwafs=$HOMEwafs/exec -export FIXwafs=$HOMEwafs/fix/wafs -export PARMwafs=$HOMEwafs/parm/wafs -export USHwafs=$HOMEwafs/ush -export SCRIPTSwafs=$HOMEwafs/scripts - -################################################ -# Set up the input/output directory -################################################ - -export COMIN=${COMIN:-$(compath.py $envir/$NET/$wafs_ver)/$RUN.$PDY/$cyc/upp} -export COMOUT=${COMOUT:-$(compath.py -o $NET/$wafs_ver)/$RUN.$PDY/$cyc/grib2/1p25} -export PCOM=${PCOM:-$COMOUT/wmo} - -export COMINgfs=${COMINgfs:-$(compath.py $envir/gfs/$gfs_ver)/gfs.$PDY/$cyc/atmos} - -if [ $SENDCOM = YES ] ; then - mkdir -p $COMOUT $PCOM -fi +############################################## +# Define COM directories +############################################## +export COMINgfs=${COMINgfs:-$(compath.py "${envir}/gfs/${gfs_ver}")"/gfs.${PDY}/${cyc}/atmos"} +export COMIN=${COMIN:-$(compath.py "${envir}/${NET}/${wafs_ver}")"/${RUN}.${PDY}/${cyc}/upp"} +export COMOUT=${COMOUT:-$(compath.py -o "${NET}/${wafs_ver}")"/${RUN}.${PDY}/${cyc}/grib2/1p25"} +export COMOUTwmo=${COMOUTwmo:-"${COMOUT}/wmo"} +mkdir -p "${COMOUT}" "${COMOUTwmo}" + +#################################### +# Determine Job Output Name on System +#################################### +export pgmout="OUTPUT.$$" + +#################################### +# Specify Execution Areas +#################################### +export FIXwafs="${HOMEwafs}/fix" + +######################################################### +# print current environment +######################################################### +env ############################################ # Execute the script. ############################################ -${SCRIPTSwafs}/exwafs_grib2_1p25.sh $fhr -export err=$?; err_chk +"${HOMEwafs}/scripts/exwafs_grib2_1p25.sh" +export err=$? +err_chk -echo "JOB $job HAS COMPLETED NORMALLY!" - -############################################ -# print exec output -############################################ -if [ -e "$pgmout" ] ; then - cat $pgmout +if [[ -e "${pgmout}" ]]; then + cat "${pgmout}" fi -############################################ -# remove temporary working directory -############################################ -if [ $KEEPDATA != YES ] ; then - rm -rf $DATA +############################## +# Remove the Temporary working directory +############################## +if [[ "${KEEPDATA^^}" != "YES" ]]; then + rm -rf "${DATA}" fi date - diff --git a/jobs/JWAFS_UPP b/jobs/JWAFS_UPP index 85f4296..da020ba 100755 --- a/jobs/JWAFS_UPP +++ b/jobs/JWAFS_UPP @@ -4,93 +4,76 @@ # WAFS UPP PRODUCT GENERATION ######################################## -export PS4='$SECONDS + ' date +export PS4='$SECONDS + ' set -x -# keep the working directory or not -export KEEPDATA=${KEEPDATA:-NO} - -#################################### -# Specify whether the run is production or development -#################################### -export envir=${envir:-prod} -# print current environment -env - -#################################### -# Specify NET and RUN Name and model -#################################### -export NET=${NET:-wafs} -export RUN=${RUN:-wafs} - -export RUNupp=${RUNupp:-gfs} - #################################### -# obtain unique process id (pid) and make temp directory +# make temp directory #################################### -export pid=${pid:-$$} -export outid=${outid:-"LL$job"} export DATA=${DATA:-${DATAROOT}/${jobid:?}} -mkdir -p $DATA -cd $DATA - -#################################### -# Determine Job Output Name on System -#################################### -export pgmout="OUTPUT.${pid}" +mkdir -p "${DATA}" && cd "${DATA}" ########################################### # Run setpdy and initialize PDY variables ########################################### -export cycle=t${cyc}z +export cycle=${cycle:-"t${cyc}z"} setpdy.sh . ./PDY - #################################### # SENDCOM - Copy Files From TMPDIR to $COMOUT -# SENDDBN - Issue DBNet Client Calls -# RERUN - Rerun posts from beginning (default no) -# VERBOSE - Specify Verbose Output in global_postgp.sh #################################### -export SENDCOM=${SENDCOM:-YES} -export SENDDBN=${SENDDBN:-YES} -export RERUN=${RERUN:-NO} -export VERBOSE=${VERBOSE:-YES} +export SENDCOM=${SENDCOM:-"YES"} #################################### -# Specify Execution Areas +# Specify NET and RUN Name and model #################################### -export HOMEwafs=${HOMEwafs:-${NWROOT}/wafs.${wafs_ver}} -export EXECwafs=$HOMEwafs/exec -export FIXwafs=$HOMEwafs/fix/upp -export PARMwafs=$HOMEwafs/parm/upp -export USHwafs=$HOMEwafs/ush -export SCRIPTSwafs=$HOMEwafs/scripts - -export POSTGPEXEC=${POSTGPEXEC:-$EXECwafs/ncep_post} -export POSTGRB2TBL=${POSTGRB2TBL:-${g2tmpl_ROOT}/share/params_grib2_tbl_new} +export NET=${NET:-wafs} +export RUN=${RUN:-wafs} ############################################## # Define COM directories ############################################## -export COMINgfs=${COMINgfs:-$(compath.py $envir/$RUNupp/$gfs_ver)/gfs.$PDY/$cyc/atmos} +export COMINgfs=${COMINgfs:-$(compath.py "${envir}/gfs/${gfs_ver}")"/gfs.${PDY}/${cyc}/atmos"} +export COMOUT=${COMOUT:-$(compath.py -o "${NET}/${wafs_ver}")"/${RUN}.${PDY}/${cyc}/upp"} +mkdir -p "${COMOUT}" + +#################################### +# Determine Job Output Name on System +#################################### +export pgmout="OUTPUT.$$" + +#################################### +# Specify Execution Areas +#################################### +export EXECwafs="${HOMEwafs}/exec" +export PARMwafs="${HOMEwafs}/parm" -export COMOUT=${COMOUT:-$(compath.py -o $NET/$wafs_ver)/$RUN.$PDY/$cyc/upp} -mkdir -p $COMOUT +######################################################### +# print current environment +######################################################### +env ######################################################### # Execute the Script ######################################################### -export MPIRUN=${MPIRUN:-'mpiexec -l -n 126 -ppn 126 --cpu-bind depth --depth 1'} -$SCRIPTSwafs/exwafs_upp.sh +# Run UPP with 1 thread +export OMP_NUM_THREADS=1 + +"${HOMEwafs}/scripts/exwafs_upp.sh" +export err=$? +err_chk + +if [[ -e "${pgmout}" ]]; then + cat "${pgmout}" +fi ############################## # Remove the Temporary working directory ############################## -if [ $KEEPDATA = NO ]; then - rm -rf $DATA +if [[ "${KEEPDATA^^}" != "YES" ]]; then + rm -rf "${DATA}" fi date diff --git a/modulefiles/wafs_common.lua b/modulefiles/wafs_common.lua deleted file mode 100644 index 5288d9b..0000000 --- a/modulefiles/wafs_common.lua +++ /dev/null @@ -1,17 +0,0 @@ -help([[ -Load common modules to build WAFS on all machines -]]) - -local bufr_ver=os.getenv("bufr_ver") or "11.7.0" -local bacio_ver=os.getenv("bacio_ver") or "2.4.1" -local w3emc_ver=os.getenv("w3emc_ver") or "2.10.0" -local sp_ver=os.getenv("sp_ver") or "2.5.0" -local ip_ver=os.getenv("ip_ver") or "4.3.0" -local g2_ver=os.getenv("g2_ver") or "3.4.5" - -load(pathJoin("bufr", bufr_ver)) -load(pathJoin("bacio", bacio_ver)) -load(pathJoin("w3emc", w3emc_ver)) -load(pathJoin("sp", sp_ver)) -load(pathJoin("ip", ip_ver)) -load(pathJoin("g2", g2_ver)) diff --git a/modulefiles/wafs_hera.intel.lua b/modulefiles/wafs_hera.intel.lua deleted file mode 100644 index 6c29f70..0000000 --- a/modulefiles/wafs_hera.intel.lua +++ /dev/null @@ -1,17 +0,0 @@ -help([[ -Build environment for WAFS on Hera -]]) - -prepend_path("MODULEPATH", "/scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.6.0/envs/gsi-addon-dev/install/modulefiles/Core") - -local stack_intel_ver=os.getenv("stack_intel_ver") or "2021.5.0" -local stack_impi_ver=os.getenv("stack_impi_ver") or "2021.5.1" -local cmake_ver=os.getenv("cmake_ver") or "3.23.1" - -load(pathJoin("stack-intel", stack_intel_ver)) -load(pathJoin("stack-intel-oneapi-mpi", stack_impi_ver)) -load(pathJoin("cmake", cmake_ver)) - -load("wafs_common") - -whatis("Description: WAFS environment on Hera with Intel Compilers") diff --git a/modulefiles/wafs_orion.intel.lua b/modulefiles/wafs_orion.intel.lua deleted file mode 100644 index 1815a58..0000000 --- a/modulefiles/wafs_orion.intel.lua +++ /dev/null @@ -1,17 +0,0 @@ -help([[ -Build environment for WAFS on Orion -]]) - -prepend_path("MODULEPATH", "/work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.6.0/envs/gsi-addon-env/install/modulefiles/Core") - -local stack_intel_ver=os.getenv("stack_intel_ver") or "2022.0.2" -local stack_impi_ver=os.getenv("stack_impi_ver") or "2021.5.1" -local cmake_ver=os.getenv("cmake_ver") or "3.23.1" - -load(pathJoin("stack-intel", stack_intel_ver)) -load(pathJoin("stack-intel-oneapi-mpi", stack_impi_ver)) -load(pathJoin("cmake", cmake_ver)) - -load("wafs_common") - -whatis("Description: WAFS environment on Orion with Intel Compilers") diff --git a/scripts/exwafs_gcip.sh b/scripts/exwafs_gcip.sh index 79053cb..47350a4 100755 --- a/scripts/exwafs_gcip.sh +++ b/scripts/exwafs_gcip.sh @@ -4,193 +4,152 @@ # UTILITY SCRIPT NAME : exwafs_gcip.sh # DATE WRITTEN : 01/28/2015 # -# Abstract: This utility script produces the WAFS GCIP. +# Abstract: This utility script produces the WAFS GCIP. # -# GCIP runs f000 f003 for each cycle, 4 times/day, -# to make the output valid every 3 hours +# GCIP runs f000 f003 for each cycle, 4 times/day, +# to make the output valid every 3 hours # -# History: 01/28/2015 -# - GFS master file as first guess -# /com/prod/gfs.YYYYMMDD -# - Nesdis composite global satellite data -# /dcom (ftp?) -# - Metar/ships/lightning/pireps -# dumpjb YYYYMMDDHH hours output >/dev/null -# - Radar data over CONUS -# /com/hourly/prod/radar.YYYYMMDD/refd3d.tHHz.grbf00 -# - output of current icing potential -##################################################################### -echo "-----------------------------------------------------" -echo "JWAFS_GCIP at 00Z/06Z/12Z/18Z GFS postprocessing" -echo "-----------------------------------------------------" -echo "History: 2015 - First implementation of this new script." -echo "Oct 2021 - Remove jlogfile" -echo "May 2024 - WAFS separation" -echo " " +# History: 01/28/2015 +# - First implementation of this new script. +# - GFS master file as first guess +# - Nesdis composite global satellite data +# - Metar/ships/lightning/pireps +# - Radar data over CONUS +# - output of current icing potential +# Oct 2021 - Remove jlogfile +# 09/08/2024 - WAFS separation +# - Filename changes according to EE2 standard +# - Fix bugzilla 1213: Filename should use fHHH instead of FHH ##################################################################### set -x -# Set up working dir for parallel runs based on fhr -fhr=$1 -DATA=$DATA/$fhr -mkdir -p $DATA -cd $DATA -# Overwrite TMPDIR for dumpjb -export TMPDIR=$DATA - -configFile=gcip.config - -echo 'before preparing data' `date` +cd "${DATA}" || err_exit "FATAL ERROR: Could not 'cd ${DATA}'; ABORT!" # valid time. no worry, it won't be across to another date -vhour=$(( $fhr + $cyc )) -vhour="$(printf "%02d" $(( 10#$vhour )) )" - -######################################################## -# Preparing data - -# model data -masterFile=$COMINgfs/gfs.t${cyc}z.master.grb2f$fhr -cp $PARMwafs/wafs_gcip_gfs.cfg $configFile - -modelFile=modelfile.grb -# ln -sf $masterFile $modelFile -$WGRIB2 $masterFile | egrep ":HGT:|:VVEL:|:CLWMR:|:TMP:|:SPFH:|:RWMR:|:SNMR:|:GRLE:|:ICMR:|:RH:" | egrep "00 mb:|25 mb:|50 mb:|75 mb:|:HGT:surface" | $WGRIB2 -i $masterFile -grib $modelFile +vhour=$((fhr + cyc)) +vhour="$(printf "%02d" $((10#$vhour)))" # metar / ships / lightning / pireps -# dumped data files' suffix is ".ibm" +export TMPDIR="${DATA}" # dumpjb uses TMPDIR obsfiles="metar ships ltngsr pirep" -for obsfile in $obsfiles ; do - $DUMPJB ${PDY}${vhour} 1.5 $obsfile +for obsfile in ${obsfiles}; do + ${DUMPJB} "${PDY}${vhour}" 1.5 "${obsfile}" done -metarFile=metar.ibm -shipFile=ships.ibm -lightningFile=ltngsr.ibm -pirepFile=pirep.ibm + +# dumped data files suffix is ".ibm" +metarFile="metar.ibm" +shipFile="ships.ibm" +lightningFile="ltngsr.ibm" +pirepFile="pirep.ibm" + +# Setup mailing list once +if [[ "${envir}" != "prod" ]]; then + maillist="nco.spa@noaa.gov" +fi +maillist=${maillist:-"nco.spa@noaa.gov,ncep.sos@noaa.gov"} satFiles="" channels="VIS SIR LIR SSR" # If one channel is missing, satFiles will be empty -for channel in $channels ; do - satFile=GLOBCOMP$channel.${PDY}${vhour} - if [[ $COMINsat == *ftp:* ]] ; then - curl -O $COMINsat/$satFile - else - # check the availability of satellite data file - if [ -s $COMINsat/$satFile ] ; then - cp $COMINsat/$satFile . +for channel in ${channels}; do + satFile="GLOBCOMP${channel}.${PDY}${vhour}" + if [[ "${COMINsat}" == *ftp:* ]]; then + curl -O "${COMINsat}/${satFile}" + else + # check the availability of satellite data file + if [[ -s "${COMINsat}/${satFile}" ]]; then + cpreq "${COMINsat}/${satFile}" . + else + msg="GCIP at ${vhour}z ABORTING, no satellite ${channel} file!" + echo "${msg}" + echo "${msg}" >>"${COMOUT}/${RUN}.gcip.log" + + subject="Missing GLOBCOMPVIS Satellite Data for ${PDY} t${cyc}z ${job}" + echo "*************************************************************" >mailmsg + echo "*** WARNING !! COULD NOT FIND GLOBCOMPVIS Satellite Data *** " >>mailmsg + echo "*************************************************************" >>mailmsg + echo >>mailmsg + echo "One or more GLOBCOMPVIS Satellite Data files are missing, including " >>mailmsg + echo " ${COMINsat}/${satFile} " >>mailmsg + echo >>mailmsg + echo "${job} will gracfully exit" >>mailmsg + cat mailmsg >"${COMOUT}/${RUN}.t${cyc}z.gcip.emailbody" + cat "${COMOUT}/${RUN}.t${cyc}z.gcip.emailbody" | mail.py -s "${subject}" "${maillist}" -v + + exit 1 + fi + fi + if [[ -s "${satFile}" ]]; then + satFiles="${satFiles} ${satFile}" else - msg="GCIP at ${vhour}z ABORTING, no satellite $channel file!" - echo "$msg" - echo $msg >> $COMOUT/${RUN}.gcip.log - - if [ $envir != prod ]; then - export maillist='nco.spa@noaa.gov' - fi - export maillist=${maillist:-'nco.spa@noaa.gov,ncep.sos@noaa.gov'} - - export subject="Missing GLOBCOMPVIS Satellite Data for $PDY t${cyc}z $job" - echo "*************************************************************" > mailmsg - echo "*** WARNING !! COULD NOT FIND GLOBCOMPVIS Satellite Data *** " >> mailmsg - echo "*************************************************************" >> mailmsg - echo >> mailmsg - echo "One or more GLOBCOMPVIS Satellite Data files are missing, including " >> mailmsg - echo " $COMINsat/$satFile " >> mailmsg - echo >> mailmsg - echo "$job will gracfully exited" >> mailmsg - cat mailmsg > $COMOUT/${RUN}.t${cyc}z.gcip.emailbody - cat $COMOUT/${RUN}.t${cyc}z.gcip.emailbody | mail.py -s "$subject" $maillist -v - - exit 1 + satFiles="" + break fi - fi - if [[ -s $satFile ]] ; then - satFiles="$satFiles $satFile" - else - satFiles="" - break - fi done -# radar data -sourceRadar=$COMINradar/refd3d.t${vhour}z.grb2f00 -radarFile=radarFile.grb -if [ -s $sourceRadar ] ; then - cp $sourceRadar $radarFile -fi +# Copy GFS master file and prepare modelFile +cpreq "${COMINgfs}/gfs.t${cyc}z.master.grb2f${fhr}" ./gfs_master.grib2 +modelFile="modelfile.grb" +${WGRIB2} "gfs_master.grib2" | grep -E ":HGT:|:VVEL:|:CLWMR:|:TMP:|:SPFH:|:RWMR:|:SNMR:|:GRLE:|:ICMR:|:RH:" | grep -E "00 mb:|25 mb:|50 mb:|75 mb:|:HGT:surface" | ${WGRIB2} -i "gfs_master.grib2" -grib "${modelFile}" -######################################################## # Composite gcip command options - -outputfile=wafs.t${vhour}z.gcip.f000.grib2 - -cmdoptions="-t ${PDY}${vhour} -c $configFile -model $modelFile" -if [[ -s $metarFile ]] ; then - cmdoptions="$cmdoptions -metar $metarFile" +configFile="gcip.config" +cmdoptions="-t ${PDY}${vhour} -c ${configFile} -model ${modelFile}" +if [[ -s "${metarFile}" ]]; then + cmdoptions="${cmdoptions} -metar ${metarFile}" else - err_exit "There are no METAR observations." + err_exit "FATAL ERROR: There are no METAR observations." fi -if [[ -s $shipFile ]] ; then - cmdoptions="$cmdoptions -ship $shipFile" +if [[ -s "${shipFile}" ]]; then + cmdoptions="${cmdoptions} -ship ${shipFile}" +else + echo "WARNING: There are no SHIP observations" fi # empty if a channel data is missing -if [[ -n $satFiles ]] ; then - cmdoptions="$cmdoptions -sat $satFiles" +if [[ -n "${satFiles}" ]]; then + cmdoptions="${cmdoptions} -sat ${satFiles}" else - err_exit "Satellite data are not available or completed." + err_exit "FATAL ERROR: Satellite data are not available or completed." fi -if [[ -s $lightningFile ]] ; then - cmdoptions="$cmdoptions -lightning $lightningFile" +if [[ -s "${lightningFile}" ]]; then + cmdoptions="${cmdoptions} -lightning ${lightningFile}" fi -if [[ -s $pirepFile ]] ; then - cmdoptions="$cmdoptions -pirep $pirepFile" +if [[ -s "${pirepFile}" ]]; then + cmdoptions="${cmdoptions} -pirep ${pirepFile}" +else + echo "WARNING: There are no PIREP observations" fi -if [[ -s $radarFile ]] ; then - cmdoptions="$cmdoptions -radar $radarFile" +# radar data +sourceRadar="${COMINradar}/refd3d.t${vhour}z.grb2f00" +radarFile="radarFile.grb" +if [[ -s "${sourceRadar}" ]]; then + cpreq "${sourceRadar}" "${radarFile}" + cmdoptions="${cmdoptions} -radar ${radarFile}" +else + echo "WARNING: There are no RADAR observations" fi -cmdoptions="$cmdoptions -o $outputfile" - -####################################################### -# Run GCIP -echo 'after preparing data' `date` +outputfile="wafs.t${vhour}z.gcip.f000.grib2" +cmdoptions="${cmdoptions} -o ${outputfile}" -export pgm=wafs_gcip.x +# Copy the configuration files and the executable +cpreq "${PARMwafs}/wafs/wafs_gcip_gfs.cfg" "${configFile}" +cpreq "${FIXwafs}/wafs/gcip_near_ir_refl.table" ./near_ir_refl.table -cp $FIXwafs/gcip_near_ir_refl.table near_ir_refl.table +export pgm="wafs_gcip.x" -startmsg -$EXECwafs/$pgm >> $pgmout $cmdoptions 2> errfile & -wait -export err=$?; err_chk +. prep_step +${EXECwafs}/${pgm} ${cmdoptions} >>"${pgmout}" 2>errfile +export err=$? +err_chk -if [[ -s $outputfile ]] ; then - ############################## - # Post Files to COM - ############################## - if [ $SENDCOM = "YES" ] ; then - cpfs $outputfile $COMOUT/$outputfile - if [ $SENDDBN = "YES" ] ; then - : - fi - fi -else - err_exit "Output $outputfile was not generated" +if [[ ! -f "${outputfile}" ]]; then + err_exit "FATAL ERROR: '${pgm}' failed to produce output '${outputfile}', ABORT!" fi - -################################################################################ -# GOOD RUN -set +x -echo "**************JOB EXWAFS_GCIP.SH COMPLETED NORMALLY ON THE IBM" -echo "**************JOB EXWAFS_GCIP.SH COMPLETED NORMALLY ON THE IBM" -echo "**************JOB EXWAFS_GCIP.SH COMPLETED NORMALLY ON THE IBM" -set -x -################################################################################ - -exit 0 - -############## END OF SCRIPT ####################### - +# Send output to COM +if [[ "${SENDCOM}" == "YES" ]]; then + cpfs "${outputfile}" "${COMOUT}/${outputfile}" +fi diff --git a/scripts/exwafs_gfs_manager.sh b/scripts/exwafs_gfs_manager.sh index c0918d1..c6c67a9 100755 --- a/scripts/exwafs_gfs_manager.sh +++ b/scripts/exwafs_gfs_manager.sh @@ -3,9 +3,10 @@ # UTILITY SCRIPT NAME : exwafs_gfs_manager.sh # DATE WRITTEN : 07/22/2024 # -# Abstract: This script manages upstream GFS data +# Abstract: This script checks for upstream GFS data availability +# and triggers downstream JWAFS jobs # -# History: 07/22/2024 +# History: 07/22/2024 # - initial version ##################################################################### @@ -15,13 +16,16 @@ set -x gfs_fhrs=$(seq -s ' ' 0 1 120) # Forecast hours for JWAFS_UPP -seq1=$(seq -s ' ' 0 1 24) # 000 -> 024; 1-hourly +seq0="0" # 000 +seq1=$(seq -s ' ' 6 1 24) # 006 -> 024; 1-hourly seq2=$(seq -s ' ' 27 3 48) # 027 -> 048; 3-hourly seq3=$(seq -s ' ' 54 6 120) # 054 -> 120; 6-hourly -jwafs_upp_fhrs="${seq1} ${seq2} ${seq3}" +jwafs_upp_fhrs="${seq0} ${seq1} ${seq2} ${seq3}" -# Forecast hours for JWAFS_GRIB; 006 - 072; 6-hourly -jwafs_grib_fhrs=$(seq -s ' ' 6 6 72) +# Forecast hours for JWAFS_GRIB +seq1=$(seq -s ' ' 12 6 48) # 012 -> 048; 6-hourly +seq2="60 72" # 060, 072 +jwafs_grib_fhrs="${seq1} ${seq2}" # Forecast hours for JWAFS_GCIP; 000, 003 jwafs_gcip_fhrs="0 3" @@ -36,7 +40,11 @@ for ((iter = 1; iter <= MAX_ITER; iter++)); do fhr3=$(printf "%03d" "${fhr}") # Trigger jobs based on GFS forecast output availability - if [[ -s "${COMINgfs}/gfs.${cycle}.logf${fhr3}.txt" ]]; then + gfs_log="${COMINgfs}/gfs.t${cyc}z.logf${fhr3}.txt" + gfs_atm="${COMINgfs}/gfs.t${cyc}z.atmf${fhr3}.nc" + gfs_sfc="${COMINgfs}/gfs.t${cyc}z.sfcf${fhr3}.nc" + gfs_grb="${COMINgfs}/gfs.t${cyc}z.master.grb2f${fhr3}" + if [[ -s "${gfs_log}" ]] && [[ -s "${gfs_atm}" ]] && [[ -s "${gfs_sfc}" ]] && [[ -s "${gfs_grb}" ]]; then # Release the JWAFS_UPP analysis job if this is f000 if ((fhr == 0)); then @@ -62,6 +70,7 @@ for ((iter = 1; iter <= MAX_ITER; iter++)); do ecflow_client --event "release_wafs_gcip_${fhr3}" fi + # Release JWAFS_GRIB job if [[ " ${jwafs_grib_fhrs} " == *" ${fhr} "* ]]; then set +x echo "Releasing JWAFS_GRIB job for fhr=${fhr3}" @@ -87,10 +96,5 @@ for ((iter = 1; iter <= MAX_ITER; iter++)); do done # end of loop over all iterations if ((iter > MAX_ITER)); then - msg="FATAL ERROR: ABORTING after 3 hours of waiting for GFS forecast output at hours ${gfs_fhrs}." - err_exit "${msg}" + err_exit "FATAL ERROR: ABORTING after 3 hours of waiting for GFS forecast output at hours ${gfs_fhrs}." fi - -echo "HAS COMPLETED NORMALLY!" - -exit 0 diff --git a/scripts/exwafs_grib.sh b/scripts/exwafs_grib.sh index 3245249..56e0155 100755 --- a/scripts/exwafs_grib.sh +++ b/scripts/exwafs_grib.sh @@ -13,48 +13,16 @@ # added variable of the a or b in the process accordingly. # The other fhrs, the a or b is dropped. # +# History: Oct 2004 - First implementation of this new script." +# Aug 2015 - Modified for Phase II" +# Dec 2015 - Modified for input model data in Grib2" +# Oct 2021 - Remove jlogfile" +# May 2024 - WAFS separation" ##################################################################### -echo "------------------------------------------------" -echo "JWAFS_00/06/12/18 GFS postprocessing" -echo "------------------------------------------------" -echo "History: OCT 2004 - First implementation of this new script." -echo " Aug 2015 - Modified for Phase II" -echo " Dec 2015 - Modified for input model data in Grib2" -echo " Oct 2021 - Remove jlogfile" -echo " May 2024 - WAFS separation" -echo " " -##################################################################### -set +x -fhr="$1" -num=$# - -if test "$num" -ge 1 -then - echo " Appropriate number of arguments were passed" - set -x - export DBNALERT_TYPE=${DBNALERT_TYPE:-GRIB} -# export job=${job:-interactive} -else - echo "" - echo "Usage: exwafs_grib.sh \$fhr " - echo "" - exit 16 -fi - -cd $DATA set -x -# To fix bugzilla 628 ( removing 'j' ahead of $job ) -export jobsuffix=gfs_atmos_wafs_f${fhr}_$cyc - -######################################## -echo "HAS BEGUN!" -######################################## - -echo " ------------------------------------------" -echo " BEGIN MAKING GFS WAFS PRODUCTS" -echo " ------------------------------------------" +ifhr=$((10#$fhr)) #################################################### # @@ -74,52 +42,31 @@ echo "#####################################" echo " " set -x -if test $fhr -eq 0 -then - echo " " -fi +cd "${DATA}" || err_exit "FATAL ERROR: Could not 'cd ${DATA}'; ABORT!" + +export jobsuffix="gfs_atmos_wafs_f${fhr}_$cyc" -# If we are processing fhrs 12-30, we have the -# added variable of the a or b in the process. -# The other fhrs, the a or b is dropped. +# If we are processing fhrs 12-30, we have the +# added variable of the a or b in the process. +# The other fhrs, the a or b is dropped. -if test $fhr -ge 12 -a $fhr -le 24 -then - $USHwafs/mkwfsgbl.sh $fhr a +if ((ifhr >= 12 && ifhr <= 24)); then + "${USHwafs}/mkwfsgbl.sh" "${fhr}" a fi -if test $fhr -eq 30 -then - $USHwafs/mkwfsgbl.sh $fhr a - for fhr in 12 18 24 30 - do - $USHwafs/mkwfsgbl.sh $fhr b +if ((ifhr == 30)); then + "${USHwafs}/mkwfsgbl.sh" "${fhr}" a + for hr in 12 18 24 30; do + "${USHwafs}/mkwfsgbl.sh" "${hr}" b done - $USHwafs/mkwfsgbl.sh 00 x - $USHwafs/mkwfsgbl.sh 06 x + "${USHwafs}/mkwfsgbl.sh" 00 x + "${USHwafs}/mkwfsgbl.sh" 06 x fi -if test $fhr -gt 30 -a $fhr -le 48 -then - $USHwafs/mkwfsgbl.sh $fhr x +if ((ifhr > 30 && ifhr <= 48)); then + "${USHwafs}/mkwfsgbl.sh" "${fhr}" x fi -if test $fhr -eq 60 -o $fhr -eq 72 -then - $USHwafs/mkwfsgbl.sh $fhr x +if ((ifhr == 60 || ifhr == 72)); then + "${USHwafs}/mkwfsgbl.sh" "${fhr}" x fi - -################################################################################ -# GOOD RUN -set +x -echo "**************JOB EXWAFS_GRIB.SH COMPLETED NORMALLY ON THE IBM" -echo "**************JOB EXWAFS_GRIB.SH COMPLETED NORMALLY ON THE IBM" -echo "**************JOB EXWAFS_GRIB.SH COMPLETED NORMALLY ON THE IBM" -set -x -################################################################################ - -echo "HAS COMPLETED NORMALLY!" - -exit 0 - -############## END OF SCRIPT ####################### diff --git a/scripts/exwafs_grib2_0p25.sh b/scripts/exwafs_grib2_0p25.sh index 323c475..92d62c6 100755 --- a/scripts/exwafs_grib2_0p25.sh +++ b/scripts/exwafs_grib2_0p25.sh @@ -6,7 +6,7 @@ # # Abstract: This utility script produces the WAFS GRIB2 at 0.25 degree. # The output GRIB files are posted on NCEP ftp server and the -# grib2 files are pushed via dbnet to TOC to WAFS (ICSC). +# grib2 files are pushed via dbnet to TOC to WAFS (ICSC). # This is a joint project of WAFC London and WAFC Washington. # # We are processing WAFS grib2 for fhr: @@ -14,163 +14,116 @@ # 3 hour: 027 - 048 # 6 hour: 054 - 120 (for U/V/T/RH, not for turbulence/icing/CB) # -# History: +# History: Mar 2020 - First implementation of this new script. +# Oct 2021 - Remove jlogfile +# Aug 2022 - fhr expanded from 36 to 120 +# 09/08/2024 - WAFS separation +# - Filename changes according to EE2 standard except for files sent to UK +# - dbn_alert subtype is changed from gfs to WAFS +# - Fix bugzilla 1213: Filename should use fHHH instead of FHH +# - Add additional levels of icing and turbulence to AWF files fhr<=36 ##################################################################### -echo "-----------------------------------------------------" -echo "JWAFS_GRIB2_0P25 at 00Z/06Z/12Z/18Z GFS&WAFS postprocessing" -echo "-----------------------------------------------------" -echo "History: MARCH 2020 - First implementation of this new script." -echo "Oct 2021 - Remove jlogfile" -echo "Aug 2022 - fhr expanded from 36 to 120" -echo "May 2024 - WAFS separation" -echo " " -##################################################################### - -cd $DATA set -x +GFS_MASTER="${COMINgfs}/gfs.t${cyc}z.master.grb2f${fhr}" +WAFS_MASTER="${COMIN}/${RUN}.t${cyc}z.master.f${fhr}.grib2" -fhr=$1 -export fhr="$(printf "%03d" $(( 10#$fhr )) )" - -DATA=$DATA/$fhr -mkdir -p $DATA -cd $DATA - - -if [ $fhr -le 48 ] ; then - hazard_timewindow=yes +ifhr="$((10#${fhr}))" +if ((ifhr <= 48)); then + hazard_timewindow="YES" else - hazard_timewindow=no + hazard_timewindow="NO" fi +cd "${DATA}" || err_exit "FATAL ERROR: Could not 'cd ${DATA}'; ABORT!" -########################################################## -# Wait for the availability of the gfs WAFS file -########################################################## - -# 3D data (on new ICAO model pressure levels) and 2D data (CB) -wafs2=$COMIN/${RUN}.${cycle}.master.f$fhr.grib2 -wafs2i=$COMIN/${RUN}.${cycle}.master.f$fhr.grib2.idx - -# 2D data from master file (U/V/H on max wind level, T/H at tropopause) -master2=$COMINgfs/gfs.${cycle}.master.grb2f${fhr} -master2i=$COMINgfs/gfs.${cycle}.master.grb2if${fhr} - -######################################## -echo "HAS BEGUN!" -######################################## - -echo " ------------------------------------------" -echo " BEGIN MAKING WAFS GRIB2 0.25 DEG PRODUCTS" -echo " ------------------------------------------" - -set +x -echo " " -echo "#####################################" -echo " Process GRIB2 WAFS 0.25 DEG PRODUCTS " -echo "#####################################" -echo " " -set -x - +# WGRIB2 options opt1=' -set_grib_type same -new_grid_winds earth ' opt21=' -new_grid_interpolation bilinear -if ' opt22="(:ICESEV|parm=37):" opt23=' -new_grid_interpolation neighbor -fi ' opt24=' -set_bitmap 1 -set_grib_max_bits 16 ' -opt25=":(UGRD|VGRD):max wind" newgrid="latlon 0:1440:0.25 90:721:-0.25" # WAFS 3D data -$WGRIB2 $wafs2 $opt1 $opt21 $opt22 $opt23 $opt24 -new_grid $newgrid tmp_wafs_0p25.grb2 -# Master 2D data -$WGRIB2 $master2 | grep -F -f $FIXwafs/grib2_0p25_gfs_master2d.list \ - | $WGRIB2 -i $master2 -set master_table 25 -grib tmp_master.grb2 -$WGRIB2 tmp_master.grb2 $opt1 $opt21 ":(UGRD|VGRD):max wind" $opt23 $opt24 -new_grid $newgrid tmp_master_0p25.grb2 +cpreq "${WAFS_MASTER}" ./wafs_master.grib2 +${WGRIB2} "./wafs_master.grib2" ${opt1} ${opt21} ${opt22} ${opt23} ${opt24} -new_grid ${newgrid} tmp_wafs_0p25.grb2 +# GFS 2D data +cpreq "${GFS_MASTER}" ./gfs_master.grib2 +${WGRIB2} "./gfs_master.grib2" | grep -F -f "${FIXwafs}/wafs/grib2_0p25_gfs_master2d.list" | + ${WGRIB2} -i "./gfs_master.grib2" -set master_table 25 -grib tmp_master.grb2 +${WGRIB2} tmp_master.grb2 ${opt1} ${opt21} ":(UGRD|VGRD):max wind" ${opt23} ${opt24} -new_grid ${newgrid} tmp_master_0p25.grb2 #--------------------------- # Product 1: WAFS u/v/t/rh wafs.tHHz.0p25.fFFF.grib2 #--------------------------- -$WGRIB2 tmp_wafs_0p25.grb2 | egrep "UGRD|VGRD|TMP|HGT|RH" \ - | $WGRIB2 -i tmp_wafs_0p25.grb2 -set master_table 25 -grib tmp.gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2 -cat tmp_master_0p25.grb2 >> tmp.gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2 +${WGRIB2} tmp_wafs_0p25.grb2 | grep -E "UGRD|VGRD|TMP|HGT|RH" | + ${WGRIB2} -i tmp_wafs_0p25.grb2 -set master_table 25 -grib "tmp.gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2" +cat tmp_master_0p25.grb2 >>"tmp.gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2" # Convert template 5 to 5.40 -#$WGRIB2 tmp.gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2 -set_grib_type jpeg -grib_out gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2 -mv tmp.gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2 gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2 -$WGRIB2 -s gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2 > gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2.idx - -if [ $hazard_timewindow = 'yes' ] ; then -#--------------------------- -# Product 2: For AWC and Delta airline: EDPARM CAT MWT ICESEV CB wafs.tHHz.awf.0p25.fFFF.grib2 -#--------------------------- +#${WGRIB2} "tmp.gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2" -set_grib_type jpeg -grib_out "gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2" +mv "tmp.gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2" "gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2" +${WGRIB2} -s "gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2" >"gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2.idx" + +if [[ "${hazard_timewindow}" == "YES" ]]; then + #--------------------------- + # Product 2: For AWC and Delta airline: EDPARM CAT MWT ICESEV CB wafs.tHHz.awf.0p25.fFFF.grib2 + #--------------------------- criteria1=":EDPARM:|:ICESEV:|parm=37:" criteria2=":CATEDR:|:MWTURB:" criteria3=":CBHE:|:ICAHT:" - $WGRIB2 tmp_wafs_0p25.grb2 | egrep "${criteria1}|$criteria2|$criteria3" \ - | $WGRIB2 -i tmp_wafs_0p25.grb2 -grib ${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2 - $WGRIB2 -s ${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2 > ${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2.idx + # AWC requests additional low levels of icing and high level of turbulence if fhr<=36 + if ((ifhr > 36 )); then + extra_levels=":977.2 mb:|:942.1 mb:|:908.1 mb:|:875.1 mb:|:127.7 mb:" + ${WGRIB2} tmp_wafs_0p25.grb2 | grep -E "$criteria1|$criteria2|$criteria3" | grep -Ev "$extra_levels" | + ${WGRIB2} -i tmp_wafs_0p25.grb2 -grib "${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2" + else + ${WGRIB2} tmp_wafs_0p25.grb2 | grep -E "${criteria1}|$criteria2|$criteria3" | + ${WGRIB2} -i tmp_wafs_0p25.grb2 -grib "${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2" + fi + ${WGRIB2} -s "${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2" >"${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2.idx" -#--------------------------- -# Product 3: WAFS unblended EDPARM, ICESEV, CB (No CAT MWT) wafs.tHHz.unblended.0p25.fFFF.grib2 -#--------------------------- - $WGRIB2 tmp_wafs_0p25.grb2 | grep -F -f $FIXwafs/grib2_0p25_wafs_hazard.list \ - | $WGRIB2 -i tmp_wafs_0p25.grb2 -set master_table 25 -grib tmp_wafs_0p25.grb2.forblend + #--------------------------- + # Product 3: WAFS unblended EDPARM, ICESEV, CB (No CAT MWT) wafs.tHHz.unblended.0p25.fFFF.grib2 + #--------------------------- + ${WGRIB2} tmp_wafs_0p25.grb2 | grep -F -f "${FIXwafs}/wafs/grib2_0p25_wafs_hazard.list" | + ${WGRIB2} -i tmp_wafs_0p25.grb2 -set master_table 25 -grib tmp_wafs_0p25.grb2.forblend # Convert template 5 to 5.40 - #$WGRIB2 tmp_wafs_0p25.grb2.forblend -set_grib_type jpeg -grib_out WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 - mv tmp_wafs_0p25.grb2.forblend WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 - $WGRIB2 -s WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 > WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2.idx + #${WGRIB2} tmp_wafs_0p25.grb2.forblend -set_grib_type jpeg -grib_out "WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2" + mv tmp_wafs_0p25.grb2.forblend "WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2" + ${WGRIB2} -s "WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2" >"WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2.idx" fi -if [ $SENDCOM = "YES" ] ; then +# Send data to COM +if [[ "${SENDCOM}" == "YES" ]]; then - ############################## - # Post Files to COM - ############################## + cpfs "gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2" "${COMOUT}/gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2" + cpfs "gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2.idx" "${COMOUT}/gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2.idx" - cpfs gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2 $COMOUT/gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2 - cpfs gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2.idx $COMOUT/gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2.idx + if [[ "${hazard_timewindow}" == "YES" ]]; then + cpfs "${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2" "${COMOUT}/${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2" + cpfs "${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2.idx" "${COMOUT}/${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2.idx" - if [ $hazard_timewindow = 'yes' ] ; then - cpfs ${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2 $COMOUT/${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2 - cpfs ${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2.idx $COMOUT/${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2.idx - - cpfs WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 $COMOUT/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 - cpfs WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2.idx $COMOUT/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2.idx - fi + cpfs "WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2" "${COMOUT}/WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2" + cpfs "WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2.idx" "${COMOUT}/WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2.idx" + fi fi -if [ $SENDDBN = "YES" ] ; then - ###################### - # Distribute Data - ###################### +# Alert via DBN +if [[ "${SENDDBN}" == "YES" ]]; then - if [ $hazard_timewindow = 'yes' ] ; then - # Hazard WAFS data (ICESEV EDR CAT MWT on 100mb to 1000mb or on new ICAO levels) sent to AWC and to NOMADS for US stakeholders - $DBNROOT/bin/dbn_alert MODEL WAFS_AWF.0P25_GB2 $job $COMOUT/${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2 + if [[ "${hazard_timewindow}" == "YES" ]]; then + # Hazard WAFS data (ICESEV EDR CAT MWT on 100mb to 1000mb or on new ICAO levels) sent to AWC and to NOMADS for US stakeholders + "${DBNROOT}/bin/dbn_alert" MODEL WAFS_AWF_0P25_GB2 "${job}" "${COMOUT}/${RUN}.t${cyc}z.awf.0p25.f${fhr}.grib2" - # Unblended US WAFS data sent to UK for blending, to the same server as 1.25 deg unblended data: wmo/grib2.tCCz.wafs_grb_wifsfFF.45 - $DBNROOT/bin/dbn_alert MODEL WAFS_0P25_UBL_GB2 $job $COMOUT/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 + # Unblended US WAFS data sent to UK for blending, also sent to AWC + "${DBNROOT}/bin/dbn_alert" MODEL WAFS_0P25_UBL_GB2 "${job}" "${COMOUT}/WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2" fi # WAFS U/V/T/RH data sent to the same server as the unblended data as above - $DBNROOT/bin/dbn_alert MODEL WAFS_0P25_GB2 $job $COMOUT/gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2 + "${DBNROOT}/bin/dbn_alert" MODEL WAFS_0P25_GB2 "${job}" "${COMOUT}/gfs.t${cyc}z.wafs_0p25.f${fhr}.grib2" fi - -################################################################################ -# GOOD RUN -set +x -echo "**************JOB EXWAFS_GRIB2_0P25.SH COMPLETED NORMALLY ON THE IBM" -echo "**************JOB EXWAFS_GRIB2_0P25.SH COMPLETED NORMALLY ON THE IBM" -echo "**************JOB EXWAFS_GRIB2_0P25.SH COMPLETED NORMALLY ON THE IBM" -set -x -################################################################################ - -echo "HAS COMPLETED NORMALLY!" - -exit 0 - -############## END OF SCRIPT ####################### diff --git a/scripts/exwafs_grib2_0p25_blending.sh b/scripts/exwafs_grib2_0p25_blending.sh index 0ec6dbf..314e803 100755 --- a/scripts/exwafs_grib2_0p25_blending.sh +++ b/scripts/exwafs_grib2_0p25_blending.sh @@ -1,289 +1,105 @@ #!/bin/bash ################################################################################ -#### UNIX Script Documentation Block -# . . -# Script name: exwafs_grib2_0p25_blending.sh -# Script description: This scripts looks for US and UK WAFS Grib2 products at 1/4 deg, -# wait for specified period of time. If both WAFS data are available. -# Otherwise, the job aborts with error massage +# UTILITY SCRIPT NAME : exwafs_grib2_0p25_blending.sh +# DATE WRITTEN : 10/02/2024 # -# Author: Y Mao Org: EMC Date: 2020-04-02 +# Abstract: This script runs blending script, ush/wafs_grib2_0p25_blending.sh, +# using MPMD to parallel run for each forcast hour. +# It handles the situation of UK missing data and sends out email per cycle # -# -# Script history log: -# 2020-04-02 Y Mao -# Oct 2021 - Remove jlogfile -# 2022-05-25 | Y Mao | Add ICAO new milestone Nov 2023 -# May 2024 - WAFS separation +# History: 10/02/2024 - WAFS separation +# - MPMD parallel run for each forecast hour, changed from sequential run. +# - Fix bugzilla 1593: Improve email notification for missing UK WAFS data. +# - Fix bugzilla 1226: Eliminate the duplicated dbn_alert for unblended wafs data +################################################################################ set -x -echo "JOB $job HAS BEGUN" -export SEND_AWC_US_ALERT=NO -export SEND_AWC_UK_ALERT=NO -export SEND_US_WAFS=NO -export SEND_UK_WAFS=NO - -YYYY=`echo $PDY | cut -c1-4` -MM=`echo $PDY | cut -c5-6` -DD=`echo $PDY | cut -c7-8` - -cd $DATA -export SLEEP_LOOP_MAX=`expr $SLEEP_TIME / $SLEEP_INT` - -echo "start blending US and UK WAFS products at 1/4 degree for " $cyc " z cycle" - -export ic_uk=1 - -fhr="$(printf "%03d" $(( 10#$fhr )) )" -########################## -# look for US WAFS data -########################## - -export ic=1 -while [ $ic -le $SLEEP_LOOP_MAX ] -do - if [ -s $COMINus/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 ] ; then - break - fi - if [ $ic -eq $SLEEP_LOOP_MAX ] ; then - echo "US WAFS GRIB2 file $COMINus/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 not found after waiting over $SLEEP_TIME seconds" - echo "US WAFS GRIB2 file " $COMINus/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 "not found after waiting ",$SLEEP_TIME, "exitting" - SEND_UK_WAFS=YES - break - else - ic=`expr $ic + 1` - sleep $SLEEP_INT - fi -done - -########################## -# look for UK WAFS data. -########################## -SLEEP_LOOP_MAX_UK=$SLEEP_LOOP_MAX +cd "${DATA}" || err_exit "FATAL ERROR: Could not 'cd ${DATA}'; ABORT!" -# export ic=1 -while [ $ic_uk -le $SLEEP_LOOP_MAX_UK ] -do - # Three(3) unblended UK files for each cycle+fhour: icing, turb, cb - #ukfiles=`ls $COMINuk/EGRR_WAFS_0p25_*_unblended_${PDY}_${cyc}z_t${fhr2}.grib2 | wc -l` - ukfiles=`ls $COMINuk/egrr_wafshzds_unblended_*_0p25_${YYYY}-${MM}-${DD}T${cyc}:00Z_t$fhr.grib2 | wc -l` - if [ $ukfiles -ge 3 ] ; then - break - fi +fhours=($(seq -s ' ' -f "%03g" 6 1 24; seq -s ' ' -f "%03g" 27 3 48)) - if [ $ic_uk -eq $SLEEP_LOOP_MAX_UK ] ; then - echo "UK WAFS GRIB2 unblended data is not completed" - products="cb ice turb" - for prod in $products ; do - ukfile=egrr_wafshzds_unblended_${prod}_0p25_${YYYY}-${MM}-${DD}T${cyc}:00Z_t$fhr.grib2 - if [ ! -f $COMINuk/$ukfile ] ; then - echo $ukfile >> missing_uk_files - fi - done - export SEND_US_WAFS=YES - break - else - ic_uk=`expr $ic_uk + 1` - sleep $SLEEP_INT - fi +rm -f wafsgrib2_0p25.cmdfile +for fhr in ${fhours[@]}; do + echo "${USHwafs}/wafs_grib2_0p25_blending.sh $fhr > $DATA/${fhr}.log 2>&1">> wafsgrib2_0p25_blending.cmdfile done +export MP_PGMMODEL=mpmd +MPIRUN="mpiexec -np ${#fhours[@]} -cpu-bind verbose,core cfp" +$MPIRUN wafsgrib2_0p25_blending.cmdfile -########################## -# If both UK and US data are missing. -########################## - -if [ $SEND_UK_WAFS = 'YES' -a $SEND_US_WAFS = 'YES' ] ; then - SEND_US_WAFS=NO - SEND_UK_WAFS=NO - echo "BOTH UK and US data are missing, no blended for $PDY$cyc$fhr" - export err=1; err_chk - exit 1 +export err=$? +if (( err != 0 )); then + echo "FATAL ERROR: An error occured processing blending" fi - -########################## -# Blending or unblended -########################## - -if [ $SEND_US_WAFS = 'YES' ] ; then - echo "turning back on dbn alert for unblended US WAFS product" -elif [ $SEND_UK_WAFS = 'YES' ] ; then - echo "turning back on dbn alert for unblended UK WAFS product" - # retrieve UK products - # Three(3) unblended UK files for each cycle+fhour: icing, turb, cb - cat $COMINuk/egrr_wafshzds_unblended_*_0p25_${YYYY}-${MM}-${DD}T${cyc}:00Z_t$fhr.grib2 > EGRR_WAFS_0p25_unblended_${PDY}_${cyc}z_t${fhr}.grib2 -else # elif [ $SEND_US_WAFS = "NO" -a $SEND_UK_WAFS = "NO" ] ; then - # retrieve UK products - # Three(3) unblended UK files for each cycle+fhour: icing, turb, cb - cat $COMINuk/egrr_wafshzds_unblended_*_0p25_${YYYY}-${MM}-${DD}T${cyc}:00Z_t$fhr.grib2 > EGRR_WAFS_0p25_unblended_${PDY}_${cyc}z_t${fhr}.grib2 - - # pick up US data - cp $COMINus/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 . - - # run blending code - export pgm=wafs_blending_0p25.x - . prep_step - - startmsg - $EXECwafs/$pgm WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 \ - EGRR_WAFS_0p25_unblended_${PDY}_${cyc}z_t${fhr}.grib2 \ - 0p25_blended_${PDY}${cyc}f${fhr}.grib2 > f${fhr}.out - err1=$? - if test "$err1" -ne 0 - then - echo "WAFS blending 0p25 program failed at " ${PDY}${cyc}F${fhr} " turning back on dbn alert for unblended US WAFS product" - SEND_US_WAFS=YES +for fhr in ${fhours[@]}; do + echo "=================== log file of fhr=$fhr ===================" + cat "${DATA}/${fhr}.log" +done +echo "===================== end of log files =====================" + +missing_uk_files="$(find $DATA -name 'missing_uk_files*')" +missing_us_files="$(find $DATA -name 'missing_us_files*')" +no_blending_files="$(find $DATA -name 'no_blending_files*')" + +if [[ ! -z "$missing_uk_files" ]] || [[ ! -z "$missing_us_files" ]] || [[ ! -z "$no_blending_files" ]] ; then + echo "WARNING: No WAFS GRIB2 0P25 blending. Send alert message to AWC ......" + make_NTC_file.pl NOXX10 KKCI "${PDY}${cyc}" NONE "${FIXwafs}/wafs/wafs_blending_0p25_admin_msg" "${COMOUTwmo}/wifs_0p25_admin_msg" + make_NTC_file.pl NOXX10 KWBC "${PDY}${cyc}" NONE "${FIXwafs}/wafs/wafs_blending_0p25_admin_msg" "${COMOUTwmo}/iscs_0p25_admin_msg" + if [[ "${SENDDBN_NTC}" == "YES" ]]; then + "${DBNROOT}/bin/dbn_alert" NTC_LOW WAFS "${job}" "${COMOUTwmo}/wifs_0p25_admin_msg" + "${DBNROOT}/bin/dbn_alert" NTC_LOW WAFS "${job}" "${COMOUTwmo}/iscs_0p25_admin_msg" fi fi -########################## -# Date dissemination -########################## - -if [ $SEND_US_WAFS = "YES" ] ; then - - ############################################################################################## - # - # checking any US WAFS product was sent due to No UK WAFS GRIB2 file or WAFS blending program - # (Alert once for all forecast hours) - # - if [ $SEND_AWC_US_ALERT = "NO" ] ; then - echo "WARNING! Missing UK data for WAFS GRIB2 0P25 blending. Send alert message to AWC ......" - make_NTC_file.pl NOXX10 KKCI $PDY$cyc NONE $FIXwafs/wafs_blending_0p25_admin_msg $PCOM/wifs_0p25_admin_msg - make_NTC_file.pl NOXX10 KWBC $PDY$cyc NONE $FIXwafs/wafs_blending_0p25_admin_msg $PCOM/iscs_0p25_admin_msg - if [ $SENDDBN_NTC = "YES" ] ; then - $DBNROOT/bin/dbn_alert NTC_LOW WAFS $job $PCOM/wifs_0p25_admin_msg - $DBNROOT/bin/dbn_alert NTC_LOW WAFS $job $PCOM/iscs_0p25_admin_msg - fi - - if [ $envir != prod ]; then - export maillist='nco.spa@noaa.gov' - fi - export maillist=${maillist:-'nco.spa@noaa.gov,ncep.sos@noaa.gov'} - export subject="WARNING! Missing UK data for WAFS GRIB2 0P25 blending, $PDY t${cyc}z f$fhr $job" - echo "*************************************************************" > mailmsg - echo "*** WARNING! Missing UK data for WAFS GRIB2 0P25 blending ***" >> mailmsg - echo "*************************************************************" >> mailmsg - echo "Missing data at $COMINuk:" >> mailmsg - cat missing_uk_files >> mailmsg - echo >> mailmsg - echo "Send alert message to AWC ...... " >> mailmsg - echo >> mailmsg - cat mailmsg > $COMOUT/${RUN}.t${cyc}z.f${fhr}.wafs_blend_0p25_usonly.emailbody - cat $COMOUT/${RUN}.t${cyc}z.f${fhr}.wafs_blend_0p25_usonly.emailbody | mail.py -s "$subject" $maillist -v - - export SEND_AWC_US_ALERT=YES - fi - ############################################################################################## - # - # Distribute US WAFS unblend Data to NCEP FTP Server (WOC) and TOC - # - echo "altering the unblended US WAFS products - $COMINus/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 " - echo "and $COMINus/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2.idx " - - if [ $SENDDBN = "YES" ] ; then - $DBNROOT/bin/dbn_alert MODEL WAFS_0P25_UBL_GB2 $job $COMINus/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 - $DBNROOT/bin/dbn_alert MODEL WAFS_0P25_UBL_GB2_WIDX $job $COMINus/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2.idx - fi - -# if [ $SENDDBN_NTC = "YES" ] ; then -# $DBNROOT/bin/dbn_alert NTC_LOW $NET $job $COMOUT/WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 -# fi - - - export SEND_US_WAFS=NO - -elif [ $SEND_UK_WAFS = "YES" ] ; then - ############################################################################################## - # - # checking any UK WAFS product was sent due to No US WAFS GRIB2 file - # (Alert once for all forecast hours) - # - if [ $SEND_AWC_UK_ALERT = "NO" ] ; then - echo "WARNING: Missing US data for WAFS GRIB2 0P25 blending. Send alert message to AWC ......" - make_NTC_file.pl NOXX10 KKCI $PDY$cyc NONE $FIXwafs/wafs_blending_0p25_admin_msg $PCOM/wifs_0p25_admin_msg - make_NTC_file.pl NOXX10 KWBC $PDY$cyc NONE $FIXwafs/wafs_blending_0p25_admin_msg $PCOM/iscs_0p25_admin_msg - if [ $SENDDBN_NTC = "YES" ] ; then - $DBNROOT/bin/dbn_alert NTC_LOW WAFS $job $PCOM/wifs_0p25_admin_msg - $DBNROOT/bin/dbn_alert NTC_LOW WAFS $job $PCOM/iscs_0p25_admin_msg - fi - - if [ $envir != prod ]; then - export maillist='nco.spa@noaa.gov' - fi - export maillist=${maillist:-'nco.spa@noaa.gov,ncep.sos@noaa.gov'} - export subject="WARNING! Missing US data for WAFS GRIB2 0P25 blending, $PDY t${cyc}z $fhr $job" - echo "*************************************************************" > mailmsg - echo "*** WARNING! Missing US data for WAFS GRIB2 0P25 blending ***" >> mailmsg - echo "*************************************************************" >> mailmsg - echo "Missing data at $COMINus:" >> mailmsg - echo WAFS_0p25_unblended_$PDY${cyc}f${fhr}.grib2 >> mailmsg - echo >> mailmsg - echo "Send alert message to AWC ...... " >> mailmsg - echo >> mailmsg - cat mailmsg > $COMOUT/${RUN}.t${cyc}z.f${fhr}.wafs_blend_0p25_ukonly.emailbody - cat $COMOUT/${RUN}.t${cyc}z.f${fhr}.wafs_blend_0p25_ukonly.emailbody | mail.py -s "$subject" $maillist -v - - export SEND_AWC_UK_ALERT=YES - fi - ############################################################################################## - # - # Distribute UK WAFS unblend Data to NCEP FTP Server (WOC) and TOC - # - echo "altering the unblended UK WAFS products - EGRR_WAFS_0p25_unblended_${PDY}_${cyc}z_t${fhr}.grib2" - - if [ $SENDDBN = "YES" ] ; then - $DBNROOT/bin/dbn_alert MODEL WAFS_UKMET_0P25_UBL_GB2 $job EGRR_WAFS_0p25_unblended_${PDY}_${cyc}z_t${fhr}.grib2 - fi - -# if [ $SENDDBN_NTC = "YES" ] ; then -# $DBNROOT/bin/dbn_alert NTC_LOW $NET $job EGRR_WAFS_0p25_unblended_${PDY}_${cyc}z_t${fhr}.grib2 -# fi - export SEND_UK_WAFS=NO - - -else - ############################################################################################## - # - # TOCGRIB2 Processing WAFS Blending GRIB2 (Icing, CB, GTG) - - # As in August 2020, no WMO header is needed for WAFS data at 1/4 deg - ## . prep_step - ## export pgm=$TOCGRIB2 - ## startmsg - - ## export FORT11=0p25_blended_${PDY}${cyc}f${fhr}.grib2 - ## export FORT31=" " - ## export FORT51=grib2.t${cyc}z.WAFS_0p25_blended_f${fhr} - - ## $TOCGRIB2 < $FIXwafs/grib2_blended_wafs_wifs_f${fhr}.0p25 >> $pgmout 2> errfile - - ## err=$?;export err ;err_chk - ## echo " error from tocgrib=",$err - - ############################################################################################## - # - # Distribute US WAFS unblend Data to NCEP FTP Server (WOC) and TOC - # - if [ $SENDCOM = YES ]; then - cpfs 0p25_blended_${PDY}${cyc}f${fhr}.grib2 $COMOUT/WAFS_0p25_blended_$PDY${cyc}f$fhr.grib2 - ## cp grib2.t${cyc}z.WAFS_0p25_blended_f${fhr} $PCOM/grib2.t${cyc}z.WAFS_0p25_blended_f${fhr} - fi +if [[ ! -z "$missing_uk_files" ]] ; then + subject="WARNING! Missing UK data for WAFS GRIB2 0P25 blending, ${PDY} t${cyc}z ${job}" + echo "*************************************************************" >mailmsg + echo "*** WARNING! Missing UK data for WAFS GRIB2 0P25 blending ***" >>mailmsg + echo "*************************************************************" >>mailmsg + echo >>mailmsg + echo "Send alert message to AWC ...... " >>mailmsg + echo >>mailmsg + for file in $missing_uk_files ; do + cat $file >>mailmsg + done + + cat mailmsg >"${COMOUT}/${RUN}.t${cyc}z.wafs_blend_0p25_ukmissing.emailbody" + cat "${COMOUT}/${RUN}.t${cyc}z.wafs_blend_0p25_ukmissing.emailbody" | mail.py -s "${subject}" "${MAILTO}" -v +fi - if [ $SENDDBN_NTC = "YES" ] ; then - # Distribute Data to NCEP FTP Server (WOC) and TOC - echo "No WMO header yet" - ## $DBNROOT/bin/dbn_alert NTC_LOW $NET $job $PCOM/grib2.t${cyc}z.WAFS_0p25_blended_f${fhr} - fi - if [ $SENDDBN = "YES" ] ; then - $DBNROOT/bin/dbn_alert MODEL WAFS_0P25_BL_GB2 $job $COMOUT/WAFS_0p25_blended_$PDY${cyc}f$fhr.grib2 - fi +if [[ ! -z "$missing_us_files" ]] ; then + subject="WARNING! Missing US data for WAFS GRIB2 0P25 blending, ${PDY} t${cyc}z ${job}" + echo "*************************************************************" >mailmsg + echo "*** WARNING! Missing US data for WAFS GRIB2 0P25 blending ***" >>mailmsg + echo "*************************************************************" >>mailmsg + echo >>mailmsg + echo "Send alert message to AWC ...... " >>mailmsg + echo >>mailmsg + for file in $missing_us_files ; do + cat $file >>mailmsg + done + + cat mailmsg >"${COMOUT}/${RUN}.t${cyc}z.wafs_blend_0p25_usmissing.emailbody" + cat "${COMOUT}/${RUN}.t${cyc}z.wafs_blend_0p25_usmissing.emailbody" | mail.py -s "${subject}" "${MAILTO}" -v fi -################################################################################ +if [[ ! -z "$no_blending_files" ]] ; then + subject="WARNING! Not blended for WAFS GRIB2 0P25 blending, ${PDY} t${cyc}z ${job}" + echo "*************************************************************" >mailmsg + echo "*** WARNING! Not blended for WAFS GRIB2 0P25 blending ***" >>mailmsg + echo "*************************************************************" >>mailmsg + echo >>mailmsg + echo "Send alert message to AWC ...... " >>mailmsg + echo >>mailmsg + for file in $no_blending_files ; do + cat $file >>mailmsg + done + + cat mailmsg >"${COMOUT}/${RUN}.t${cyc}z.wafs_blend_0p25_noblending.emailbody" + cat "${COMOUT}/${RUN}.t${cyc}z.wafs_blend_0p25_noblending.emailbody" | mail.py -s "${subject}" "${MAILTO}" -v +fi -exit 0 -# +err_chk diff --git a/scripts/exwafs_grib2_1p25.sh b/scripts/exwafs_grib2_1p25.sh index 2382134..e1ba63d 100755 --- a/scripts/exwafs_grib2_1p25.sh +++ b/scripts/exwafs_grib2_1p25.sh @@ -1,211 +1,168 @@ #!/bin/bash ###################################################################### -# UTILITY SCRIPT NAME : exwafs_grib2.sh +# UTILITY SCRIPT NAME : exwafs_grib2_1p25.sh # DATE WRITTEN : 07/15/2009 # -# Abstract: This utility script produces the WAFS GRIB2. The output -# GRIB files are posted on NCEP ftp server and the grib2 files -# are pushed via dbnet to TOC to WAFS (ICSC). -# This is a joint project of WAFC London and WAFC Washington. +# Abstract: This utility script produces the WAFS GRIB2. The output +# grib2 files pushed via dbnet to TOC to WAFS (ICSC). +# This is a joint project of WAFC London and WAFC Washington. # -# We are processing WAFS grib2 for fhr from 06 - 36 +# We are processing WAFS grib2 for fhr from 06 - 36 # with 3-hour time increment. # -# History: 08/20/2014 +# AWF is processed through fhr=72 +# +# History: 08/xx/2009 - First implementation +# 08/20/2014 # - ingest master file in grib2 (or grib1 if grib2 fails) # - output of icng tcld cat cb are in grib2 -# 02/21/2020 +# 02/21/2020 # - Prepare unblended icing severity and GTG tubulence # for blending at 0.25 degree -# 02/22/2022 -# - Add grib2 data requested by FAA +# 10/xx/2021 - Remove jlogfile +# 02/22/2022 +# - Add grib2 AWF data requested by FAA # - Stop generating grib1 data for WAFS +# 09/08/2024 - WAFS separation +# - Filename changes according to EE2 standard except for files sent to UK +# - dbn_alert subtype is changed from gfs to WAFS +# - Fix bugzilla 1213: Filename should use fHHH instead of FHH ##################################################################### -echo "-----------------------------------------------------" -echo "JWAFS_GRIB2 at 00Z/06Z/12Z/18Z GFS&WAFS postprocessing" -echo "-----------------------------------------------------" -echo "History: AUGUST 2009 - First implementation of this new script." -echo "Oct 2021 - Remove jlogfile" -echo "Feb 2022 - Add FAA data, stop grib1 data" -echo "May 2024 - WAFS separation" -echo " " -##################################################################### - -set -x -fhr=$1 -export fhr="$(printf "%03d" $(( 10#$fhr )) )" - -DATA=$DATA/$fhr -mkdir -p $DATA -cd $DATA - -########################################################## -# Wait for the availability of the gfs master pgrib file -########################################################## -# file name and forecast hour of GFS model data in Grib2 are 3 digits - -# 2D data -master2=$COMINgfs/gfs.${cycle}.master.grb2f${fhr} -master2i=$COMINgfs/gfs.${cycle}.master.grb2if${fhr} -# 3D data -wafs2=$COMIN/${RUN}.${cycle}.master.f$fhr.grib2 -wafs2i=$COMIN/${RUN}.${cycle}.master.f$fhr.grib2.idx - -######################################## -echo "HAS BEGUN!" -######################################## - -echo " ------------------------------------------" -echo " BEGIN MAKING GFS WAFS GRIB2 PRODUCTS" -echo " ------------------------------------------" - -set +x -echo " " -echo "#####################################" -echo " Process GRIB WAFS PRODUCTS " -echo " FORECAST HOURS 06 - 36." -echo "#####################################" -echo " " set -x +GFS_MASTER="${COMINgfs}/gfs.t${cyc}z.master.grb2f${fhr}" +WAFS_MASTER="${COMIN}/${RUN}.t${cyc}z.master.f${fhr}.grib2" -if [ $fhr -le 36 -a $fhr -gt 0 ] ; then - wafs_timewindow=yes +ifhr="$((10#${fhr}))" +if ((ifhr > 0 && ifhr <= 36)); then + wafs_timewindow="YES" else - wafs_timewindow=no + wafs_timewindow="NO" fi +cd "${DATA}" || err_exit "FATAL ERROR: Could not 'cd ${DATA}'; ABORT!" + #--------------------------- # 1) Grib2 data for FAA #--------------------------- -$WGRIB2 $master2 | grep -F -f $FIXwafs/grib2_gfs_awf_master.list | $WGRIB2 -i $master2 -grib tmpfile_wafsf${fhr} +cpreq "${GFS_MASTER}" ./gfs_master.grib2 +${WGRIB2} "./gfs_master.grib2" | grep -F -f "${FIXwafs}/wafs/grib2_gfs_awf_master.list" | ${WGRIB2} -i "./gfs_master.grib2" -grib "tmpfile_wafsf${fhr}" + # F006 master file has two records of 0-6 hour APCP and ACPCP each, keep only one -# FAA APCP ACPCP: included every 6 forecast hour (0, 48], every 12 forest hour [48, 72] (controlled by $FIXwafs/grib2_gfs_awf_master.list) -if [ $fhr -eq 6 ] ; then - $WGRIB2 tmpfile_wafsf${fhr} -not "(APCP|ACPCP)" -grib tmp.grb2 - $WGRIB2 tmpfile_wafsf${fhr} -match APCP -append -grib tmp.grb2 -quit - $WGRIB2 tmpfile_wafsf${fhr} -match ACPCP -append -grib tmp.grb2 -quit - mv tmp.grb2 tmpfile_wafsf${fhr} +# FAA APCP ACPCP: included every 6 forecast hour (0, 48], every 12 forest hour [48, 72] (controlled by ${FIXwafs}/wafs/grib2_gfs_awf_master.list) +if ((ifhr == 6)); then + ${WGRIB2} "tmpfile_wafsf${fhr}" -not "(APCP|ACPCP)" -grib tmp.grb2 + ${WGRIB2} "tmpfile_wafsf${fhr}" -match APCP -append -grib tmp.grb2 -quit + ${WGRIB2} "tmpfile_wafsf${fhr}" -match ACPCP -append -grib tmp.grb2 -quit + mv tmp.grb2 "tmpfile_wafsf${fhr}" fi + # U V will have the same grid message number by using -ncep_uv. # U V will have the different grid message number without -ncep_uv. -$WGRIB2 tmpfile_wafsf${fhr} \ - -set master_table 6 \ - -new_grid_winds earth -set_grib_type jpeg \ - -new_grid_interpolation bilinear -if ":(UGRD|VGRD):max wind" -new_grid_interpolation neighbor -fi \ - -new_grid latlon 0:288:1.25 90:145:-1.25 ${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2 -$WGRIB2 -s ${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2 > ${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2.idx +${WGRIB2} "tmpfile_wafsf${fhr}" \ + -set master_table 6 \ + -new_grid_winds earth -set_grib_type jpeg \ + -new_grid_interpolation bilinear -if ":(UGRD|VGRD):max wind" -new_grid_interpolation neighbor -fi \ + -new_grid latlon 0:288:1.25 90:145:-1.25 "${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2" +${WGRIB2} -s "${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2" >"${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2.idx" # For FAA, add WMO header. The header is different from WAFS -export pgm=$TOCGRIB2 +cpreq "${FIXwafs}/wafs/grib2_gfs_awff${fhr}.45" gfs_wmo_header45 + +export pgm="${TOCGRIB2}" +# Clean out any existing output files . prep_step -startmsg -export FORT11=${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2 + +export FORT11="${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2" export FORT31=" " -export FORT51=grib2.wafs.t${cyc}z.awf_grid45.f${fhr} -$TOCGRIB2 < $FIXwafs/grib2_gfs_awff${fhr}.45 >> $pgmout 2> errfile -err=$?;export err ;err_chk -echo " error from tocgrib=",$err +export FORT51="grib2.wafs.t${cyc}z.awf_grid45.f${fhr}" +${TOCGRIB2} >"${pgmout}" 2>errfile +export err=$? +err_chk + +# Check if TOCGRIB2 succeeded in creating the output file +if [[ ! -f "${FORT51}" ]]; then + err_exit "FATAL ERROR: '${pgm}' failed to create '${FORT51}', ABORT!" +fi -if [ $wafs_timewindow = 'yes' ] ; then -#--------------------------- -# 2) traditional WAFS fields -#--------------------------- - # 3D data from $wafs2, on exact model pressure levels - $WGRIB2 $wafs2 | grep -F -f $FIXwafs/grib2_wafs.gfs_master.list | $WGRIB2 -i $wafs2 -grib tmpfile_wafsf${fhr} - # 2D data from $master2 - tail -5 $FIXwafs/grib2_wafs.gfs_master.list > grib2_wafs.gfs_master.list.2D - $WGRIB2 $master2 | grep -F -f grib2_wafs.gfs_master.list.2D | $WGRIB2 -i $master2 -grib tmpfile_wafsf${fhr}.2D +if [[ "${wafs_timewindow}" == "YES" ]]; then + #--------------------------- + # 2) traditional WAFS fields + #--------------------------- + # 3D data from "./wafs_master.grib2", on exact model pressure levels + cpreq "${WAFS_MASTER}" ./wafs_master.grib2 + ${WGRIB2} "./wafs_master.grib2" | grep -F -f "${FIXwafs}/wafs/grib2_wafs.gfs_master.list" | ${WGRIB2} -i "./wafs_master.grib2" -grib "tmpfile_wafsf${fhr}" + # 2D data from "./gfs_master.grib2" + tail -5 "${FIXwafs}/wafs/grib2_wafs.gfs_master.list" >grib2_wafs.gfs_master.list.2D + ${WGRIB2} "./gfs_master.grib2" | grep -F -f grib2_wafs.gfs_master.list.2D | ${WGRIB2} -i "./gfs_master.grib2" -grib "tmpfile_wafsf${fhr}.2D" # Complete list of WAFS data - cat tmpfile_wafsf${fhr}.2D >> tmpfile_wafsf${fhr} - # WMO header - cp $FIXwafs/grib2_wafsf${fhr}.45 wafs_wmo_header45 + cat tmpfile_wafsf${fhr}.2D >>tmpfile_wafsf${fhr} + # U V will have the same grid message number by using -ncep_uv. # U V will have the different grid message number without -ncep_uv. - $WGRIB2 tmpfile_wafsf${fhr} \ - -set master_table 6 \ - -new_grid_winds earth -set_grib_type jpeg \ - -new_grid_interpolation bilinear -if ":(UGRD|VGRD):max wind" -new_grid_interpolation neighbor -fi \ - -new_grid latlon 0:288:1.25 90:145:-1.25 gfs.t${cyc}z.wafs_grb45f${fhr}.grib2 - $WGRIB2 -s gfs.t${cyc}z.wafs_grb45f${fhr}.grib2 > gfs.t${cyc}z.wafs_grb45f${fhr}.grib2.idx + ${WGRIB2} "tmpfile_wafsf${fhr}" \ + -set master_table 6 \ + -new_grid_winds earth -set_grib_type jpeg \ + -new_grid_interpolation bilinear -if ":(UGRD|VGRD):max wind" -new_grid_interpolation neighbor -fi \ + -new_grid latlon 0:288:1.25 90:145:-1.25 "gfs.t${cyc}z.wafs_grb45f${fhr}.grib2" + ${WGRIB2} -s "gfs.t${cyc}z.wafs_grb45f${fhr}.grib2" >"gfs.t${cyc}z.wafs_grb45f${fhr}.grib2.idx" # For WAFS, add WMO header. Processing WAFS GRIB2 grid 45 for ISCS and WIFS - export pgm=$TOCGRIB2 + cpreq "${FIXwafs}/wafs/grib2_wafsf${fhr}.45" wafs_wmo_header45 + + export pgm="${TOCGRIB2}" + # Clean out any existing output files . prep_step - startmsg - export FORT11=gfs.t${cyc}z.wafs_grb45f${fhr}.grib2 + + export FORT11="gfs.t${cyc}z.wafs_grb45f${fhr}.grib2" export FORT31=" " - export FORT51=grib2.wafs.t${cyc}z.grid45.f${fhr} - $TOCGRIB2 < wafs_wmo_header45 >> $pgmout 2> errfile - err=$?;export err ;err_chk - echo " error from tocgrib=",$err + export FORT51="grib2.wafs.t${cyc}z.grid45.f${fhr}" + ${TOCGRIB2} >"${pgmout}" 2>errfile + export err=$? + err_chk + + # Check if TOCGRIB2 succeeded in creating the output file + if [[ ! -f "${FORT51}" ]]; then + err_exit "FATAL ERROR: '${pgm}' failed to create '${FORT51}', ABORT!" + fi fi # wafs_timewindow -if [ $SENDCOM = "YES" ] ; then - - ############################## - # Post Files to COM - ############################## +# Send data to COM +if [[ "${SENDCOM}" == "YES" ]]; then # FAA data - cpfs ${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2 $COMOUT/${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2 - cpfs ${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2.idx $COMOUT/${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2.idx + cpfs "${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2" "${COMOUT}/${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2" + cpfs "${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2.idx" "${COMOUT}/${RUN}.t${cyc}z.awf_grid45.f${fhr}.grib2.idx" # WAFS data - if [ $wafs_timewindow = 'yes' ] ; then - cpfs gfs.t${cyc}z.wafs_grb45f${fhr}.grib2 $COMOUT/gfs.t${cyc}z.wafs_grb45f${fhr}.grib2 - cpfs gfs.t${cyc}z.wafs_grb45f${fhr}.grib2.idx $COMOUT/gfs.t${cyc}z.wafs_grb45f${fhr}.grib2.idx + if [[ "${wafs_timewindow}" == "YES" ]]; then + cpfs "gfs.t${cyc}z.wafs_grb45f${fhr}.grib2" "${COMOUT}/gfs.t${cyc}z.wafs_grb45f${fhr}.grib2" + cpfs "gfs.t${cyc}z.wafs_grb45f${fhr}.grib2.idx" "${COMOUT}/gfs.t${cyc}z.wafs_grb45f${fhr}.grib2.idx" fi - ############################## - # Post Files to PCOM - ############################## - - cpfs grib2.wafs.t${cyc}z.awf_grid45.f${fhr} $PCOM/grib2.wafs.t${cyc}z.awf_grid45.f${fhr} + cpfs "grib2.wafs.t${cyc}z.awf_grid45.f${fhr}" "${COMOUTwmo}/grib2.wafs.t${cyc}z.awf_grid45.f${fhr}" - if [ $wafs_timewindow = 'yes' ] ; then - cpfs grib2.wafs.t${cyc}z.grid45.f${fhr} $PCOM/grib2.wafs.t${cyc}z.grid45.f${fhr} + if [[ "${wafs_timewindow}" == "YES" ]]; then + cpfs "grib2.wafs.t${cyc}z.grid45.f${fhr}" "${COMOUTwmo}/grib2.wafs.t${cyc}z.grid45.f${fhr}" fi fi -###################### -# Distribute Data -###################### - -if [ $SENDDBN = "YES" ] ; then - -# -# Distribute Data to WOC -# - if [ $wafs_timewindow = 'yes' ] ; then - $DBNROOT/bin/dbn_alert MODEL WAFS_1P25_GB2 $job $COMOUT/gfs.t${cyc}z.wafs_grb45f${fhr}.grib2 -# -# Distribute Data to TOC TO WIFS FTP SERVER (AWC) -# - $DBNROOT/bin/dbn_alert NTC_LOW $NET $job $PCOM/grib2.wafs.t${cyc}z.grid45.f${fhr} +# Alert via DBN +if [[ "${SENDDBN}" == "YES" ]]; then + if [[ "${wafs_timewindow}" == "YES" ]]; then + # Distribute Data to WOC + "${DBNROOT}/bin/dbn_alert" MODEL WAFS_1P25_GB2 "${job}" "${COMOUT}/gfs.t${cyc}z.wafs_grb45f${fhr}.grib2" fi -# -# Distribute data to FAA -# - $DBNROOT/bin/dbn_alert NTC_LOW $NET $job $PCOM/grib2.wafs.t${cyc}z.awf_grid45.f${fhr} - - fi -################################################################################ -# GOOD RUN -set +x -echo "**************JOB EXWAFS_GRIB2.SH COMPLETED NORMALLY ON THE IBM" -echo "**************JOB EXWAFS_GRIB2.SH COMPLETED NORMALLY ON THE IBM" -echo "**************JOB EXWAFS_GRIB2.SH COMPLETED NORMALLY ON THE IBM" -set -x -################################################################################ - -echo "HAS COMPLETED NORMALLY!" - -exit 0 - -############## END OF SCRIPT ####################### +if [[ "$SENDDBN_NTC" == "YES" ]] ; then + if [[ "${wafs_timewindow}" == "YES" ]]; then + # Distribute Data to TOC TO WIFS FTP SERVER (AWC) + "${DBNROOT}/bin/dbn_alert" NTC_LOW "${NET}" "${job}" "${COMOUTwmo}/grib2.wafs.t${cyc}z.grid45.f${fhr}" + fi + # Distribute data to FAA + "${DBNROOT}/bin/dbn_alert" NTC_LOW "${NET}" "${job}" "${COMOUTwmo}/grib2.wafs.t${cyc}z.awf_grid45.f${fhr}" +fi diff --git a/scripts/exwafs_upp.sh b/scripts/exwafs_upp.sh index fb068ac..cb1c844 100755 --- a/scripts/exwafs_upp.sh +++ b/scripts/exwafs_upp.sh @@ -1,129 +1,113 @@ #!/bin/bash +###################################################################### +# UTILITY SCRIPT NAME : exwafs_upp.sh +# DATE WRITTEN : 07/22/2024 +# +# Abstract: This script runs the offline UPP based on GFS model output +# and creates the WAFS master grib2 file +# +# History: 07/22/2024 +# - initial version, for WAFS separation +# - Add additional levels of icing and turbulence than prior to WAFS separation ##################################################################### -echo "-----------------------------------------------------" -echo " exwafs_upp.sh" -echo " Jun 24 - Mao - script for wafs upp" -echo "-----------------------------------------------------" -##################################################################### + set -x -cd $DATA - -msg="HAS BEGUN on `hostname`" -postmsg "$msg" - -POSTGPSH=${POSTGPSH:-$USHwafs/wafs_upp.sh} -PREFIX=${PREFIX:-${RUNupp}.t${cyc}z} - -SUFFIX=".nc" -export MODEL_OUT_FORM=${MODEL_OUT_FORM:-netcdfpara} - -export PGBOUT=wafsfile # For UPP Fortran code -export PGIOUT=wafsifile - -############################################################ -# Post Analysis Files before starting the Forecast Post -############################################################ -if [ $fhr = "anl" ]; then -#---------------------------------- - export VDATE=${PDY}${cyc} - - loganl=$COMINgfs/${PREFIX}.atmanl${SUFFIX} - - if test -f $loganl ; then - - [[ -f flxfile ]] && rm flxfile ; [[ -f atmfile ]] && rm atmfile - ln -fs $COMINgfs/${PREFIX}.atmanl${SUFFIX} atmfile - export ATMINP=atmfile - ln -fs $COMINgfs/${PREFIX}.sfcanl${SUFFIX} flxfile - export FLXINP=flxfile - -########################## WAFS U/V/T analysis start ########################## -# U/V/T on ICAO pressure levels for WAFS verification - if [[ $RUNupp = gfs ]] ; then - - #For MDL2P.f, WAFS pressure levels are different from master file - export POSTGPVARS="KPO=56,PO=84310.,81200.,78190.,75260.,72430.,69680.,67020.,64440.,61940.,59520.,57180.,54920.,52720.,50600.,48550.,46560.,44650.,42790.,41000.,39270.,37600.,35990.,34430.,32930.,31490.,30090.,28740.,27450.,26200.,25000.,23840.,22730.,21660.,20650.,19680.,18750.,17870.,17040.,16240.,15470.,14750.,14060.,13400.,12770.,12170.,11600.,11050.,10530.,10040.,9570.,9120.,8700.,8280.,7900.,7520.,7170.,popascal=.true.," - - export PostFlatFile=$PARMwafs/postxconfig-NT-GFS-WAFS-ANL.txt - export CTLFILE=$PARMwafs/postcntrl_gfs_wafs_anl.xml - - $POSTGPSH - export err=$? - - if [ $err -ne 0 ] ; then - echo " *** GFS POST WARNING: WAFS output failed for analysis, err=$err" - else - - # Need to be saved for WAFS U/V/T verification, - # resolution higher than WAFS 1.25 deg for future compatibility - wafsgrid="latlon 0:1440:0.25 90:721:-0.25" - $WGRIB2 $PGBOUT -set_grib_type same -new_grid_winds earth \ - -new_grid_interpolation bilinear -set_bitmap 1 \ - -new_grid $wafsgrid ${PGBOUT}.tmp - - if test $SENDCOM = "YES" - then - cpfs ${PGBOUT}.tmp $COMOUT/$RUN.t${cyc}z.0p25.anl.grib2 - $WGRIB2 -s ${PGBOUT}.tmp > $COMOUT/$RUN.t${cyc}z.0p25.anl.grib2.idx - fi - rm $PGBOUT ${PGBOUT}.tmp - fi - fi - fi -########################## WAFS U/V/T analysis end ########################## -else -########################## WAFS forecast start ########################## - export VDATE=`${NDATE} +${fhr} ${PDY}${cyc}` - [[ -f flxfile ]] && rm flxfile ; [[ -f atmfile ]] && rm atmfile - ln -fs $COMINgfs/${PREFIX}.atmf${fhr}${SUFFIX} atmfile - export ATMINP=atmfile - ln -fs $COMINgfs/${PREFIX}.sfcf${fhr}${SUFFIX} flxfile - export FLXINP=flxfile - - # Generate WAFS products on ICAO standard level. - # Do not need to be sent out to public, WAFS package will process the data. - if [[ $RUNupp = gfs || $RUNupp = gefs ]] ; then - - #For MDL2P.f, WAFS pressure levels are different from master file - export POSTGPVARS="KPO=58,PO=97720.,90810.,84310.,81200.,78190.,75260.,72430.,69680.,67020.,64440.,61940.,59520.,57180.,54920.,52720.,50600.,48550.,46560.,44650.,42790.,41000.,39270.,37600.,35990.,34430.,32930.,31490.,30090.,28740.,27450.,26200.,25000.,23840.,22730.,21660.,20650.,19680.,18750.,17870.,17040.,16240.,15470.,14750.,14060.,13400.,12770.,12170.,11600.,11050.,10530.,10040.,9570.,9120.,8700.,8280.,7900.,7520.,7170.,popascal=.true.," - - run=`echo $RUNupp | tr '[a-z]' '[A-Z]'` - # Extend WAFS u/v/t up to 120 hours - if [ $fhr -le 48 ] ; then - export PostFlatFile=$PARMwafs/postxconfig-NT-${run}-WAFS.txt - export CTLFILE=$PARMwafs/postcntrl_${RUNupp}_wafs.xml - else - export PostFlatFile=$PARMwafs/postxconfig-NT-GFS-WAFS-EXT.txt - export CTLFILE=$PARMwafs/postcntrl_gfs_wafs_ext.xml - fi - - # gtg has its own configurations - #cp $HOMEwafs/sorc/ncep_post.fd/post_gtg.fd/gtg.config.$RUNupp . - #cp $HOMEwafs/sorc/ncep_post.fd/post_gtg.fd/imprintings.gtg_${RUNupp}.txt . - #cp $HOMEwafs/sorc/ncep_post.fd/post_gtg.fd/gtg.input.$RUNupp . - cp $PARMwafs/gtg.config.$RUNupp gtg.config - cp $PARMwafs/gtg_imprintings.txt gtg_imprintings.txt - - $POSTGPSH - - export err=$? - - if [ $err -ne 0 ] ; then - echo " *** GFS POST WARNING: WAFS output failed for f${fhr}, err=$err" - else - if [ -e $PGBOUT ] ; then - if [ $SENDCOM = "YES" ] ; then - cpfs $PGBOUT $COMOUT/$RUN.t${cyc}z.master.f$fhr.grib2 - $WGRIB2 -s $PGBOUT > $PGIOUT # WAFS products exist from ush/gfs_nceppost.sh before running anything else - cpfs $PGIOUT $COMOUT/$RUN.t${cyc}z.master.f$fhr.grib2.idx - fi - fi - fi - fi - [[ -f wafsfile ]] && rm wafsfile ; [[ -f wafsifile ]] && rm wafsifile +POSTGRB2TBL=${POSTGRB2TBL:-"${g2tmpl_ROOT}/share/params_grib2_tbl_new"} +MPIRUN=${MPIRUN:-"mpiexec -l -n 126 -ppn 126 --cpu-bind depth --depth 1"} + +if [[ "${fhr}" == "anl" ]]; then # Analysis + + VDATE="${PDY}${cyc}" + ATMINP="${COMINgfs}/gfs.t${cyc}z.atmanl.nc" + FLXINP="${COMINgfs}/gfs.t${cyc}z.sfcanl.nc" + PostFlatFile="${PARMwafs}/upp/postxconfig-NT-GFS-WAFS-ANL.txt" + +else # Forecast + + VDATE=$(${NDATE} +${fhr} ${PDY}${cyc}) + ATMINP="${COMINgfs}/gfs.t${cyc}z.atmf${fhr}.nc" + FLXINP="${COMINgfs}/gfs.t${cyc}z.sfcf${fhr}.nc" + ifhr="$((10#${fhr}))" + if ((ifhr <= 48)); then + PostFlatFile="${PARMwafs}/upp/postxconfig-NT-GFS-WAFS.txt" + else + PostFlatFile="${PARMwafs}/upp/postxconfig-NT-GFS-WAFS-EXT.txt" + fi + +fi + +cd "${DATA}" || err_exit "FATAL ERROR: Could not 'cd ${DATA}'; ABORT!" + +# Copy required inputs to local directory +cpreq "${ATMINP}" ./atmfile +cpreq "${FLXINP}" ./flxfile +cpreq "${POSTGRB2TBL}" . +cpreq "${PostFlatFile}" ./postxconfig-NT.txt +cpreq "${PARMwafs}/upp/nam_micro_lookup.dat" ./eta_micro_lookup.dat +if [[ "${fhr}" != "anl" ]]; then + cpreq "${PARMwafs}/upp/gtg.config.gfs" gtg.config + cpreq "${PARMwafs}/upp/gtg_imprintings.txt" gtg_imprintings.txt +fi + +# Create the itag file +rm -f itag +cat >itag <>${pgmout} 2>errfile +export err=$? +err_chk + +# Check if UPP succeeded in creating the master file +if [[ ! -f "${PGBOUT}" ]]; then + err_exit "FATAL ERROR: UPP failed to create '${PGBOUT}', ABORT!" +fi +# Copy relevant files to COMOUT +if [[ "${fhr}" == "anl" ]]; then # U/V/T analysis interpolated file for verification (EVS) + + # Interpolate to 0.25-degree grid + ${WGRIB2} "${PGBOUT}" \ + -set_grib_type same -new_grid_winds earth \ + -new_grid_interpolation bilinear -set_bitmap 1 \ + -new_grid latlon 0:1440:0.25 90:721:-0.25 "${PGBOUT}.0p25" + export err=$? + ((err != 0)) && err_exit "FATAL ERROR: 'wgrib2' failed to interpolate '${PGBOUT}' to 0.25-deg grid, ABORT!" + + # Copy interpolated file to COMOUT and index the file + if [[ "${SENDCOM}" == "YES" ]]; then + cpfs "${PGBOUT}.0p25" "${COMOUT}/${RUN}.t${cyc}z.0p25.anl.grib2" + ${WGRIB2} -s "${PGBOUT}.0p25" >"${COMOUT}/${RUN}.t${cyc}z.0p25.anl.grib2.idx" + fi + +else # Forecast WAFS master files (including hazard aviation data if forecast hour is 48 or less) + + # Copy master files to COMOUT and index the file + if [[ "${SENDCOM}" == "YES" ]]; then + cpfs "${PGBOUT}" "${COMOUT}/${RUN}.t${cyc}z.master.f${fhr}.grib2" + ${WGRIB2} -s "${PGBOUT}" >"${COMOUT}/${RUN}.t${cyc}z.master.f${fhr}.grib2.idx" + fi fi -########################### WAFS forecast end ########################### -echo "PROGRAM IS COMPLETE!!!!!" diff --git a/sorc/CMakeLists.txt b/sorc/CMakeLists.txt index 7035817..a6ebccc 100644 --- a/sorc/CMakeLists.txt +++ b/sorc/CMakeLists.txt @@ -2,7 +2,7 @@ cmake_minimum_required(VERSION 3.19) # Get the version from the VERSION file. #file(STRINGS "VERSION" pVersion) -set(pVersion 1.0.0) +set(pVersion 7.0.0) project( wafs diff --git a/sorc/build_all.sh b/sorc/build_all.sh index 9fa5ca3..f19a7e1 100755 --- a/sorc/build_all.sh +++ b/sorc/build_all.sh @@ -3,7 +3,7 @@ set -eu # Get the root of the cloned WAFS directory -readonly DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}" )" )/.." && pwd -P) +readonly DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}")")/.." && pwd -P) # User Options export BUILD_TYPE=${BUILD_TYPE:-"Release"} @@ -22,12 +22,15 @@ mkdir -p "${DIR_ROOT}/sorc/logs" echo "building ... wafs" rm -f "${DIR_ROOT}/sorc/logs/log.wafs" -./build_wafs.sh >& "${DIR_ROOT}/sorc/logs/log.wafs" 2>&1 +./build_wafs.sh >&"${DIR_ROOT}/sorc/logs/log.wafs" 2>&1 echo "building ... upp" rm -f "${DIR_ROOT}/sorc/logs/log.upp" -./build_upp.sh >& "${DIR_ROOT}/sorc/logs/log.upp" 2>&1 +./build_upp.sh >&"${DIR_ROOT}/sorc/logs/log.upp" 2>&1 -echo "... done!" +echo "building ... done!" + +echo "listing executables ..." +ls -l "${DIR_ROOT}/exec" exit diff --git a/sorc/build_upp.sh b/sorc/build_upp.sh index 419196b..482f79b 100755 --- a/sorc/build_upp.sh +++ b/sorc/build_upp.sh @@ -3,7 +3,7 @@ set -eu # Get the root of the cloned WAFS directory -readonly DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}" )" )/.." && pwd -P) +readonly DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}")")/.." && pwd -P) # Check WAFS/exec folder exists if [[ ! -d "${DIR_ROOT}/exec" ]]; then @@ -11,21 +11,34 @@ if [[ ! -d "${DIR_ROOT}/exec" ]]; then fi # upp_v8.3.0: -cd "${DIR_ROOT}/sorc/upp.fd" - -# copy UPP parm/ to the main vertical structure -cp -r parm "${DIR_ROOT}/parm/upp" +cd "${DIR_ROOT}/sorc/wafs_upp.fd" + +# copy WAFS specific UPP parm/ files to the main vertical structure +mkdir -p "${DIR_ROOT}/parm/upp" +upp_parm_files=(nam_micro_lookup.dat \ + postcntrl_gfs_wafs_anl.xml \ + postcntrl_gfs_wafs_ext.xml \ + postcntrl_gfs_wafs.xml \ + postxconfig-NT-GFS-WAFS-ANL.txt \ + postxconfig-NT-GFS-WAFS-EXT.txt \ + postxconfig-NT-GFS-WAFS.txt \ + gtg_imprintings.txt ) +for upp_parm_file in "${upp_parm_files[@]}"; do + rm -f "${DIR_ROOT}/parm/upp/${upp_parm_file}" + cp "parm/${upp_parm_file}" "${DIR_ROOT}/parm/upp/${upp_parm_file}" +done +rm -f "${DIR_ROOT}/parm/upp/gtg.config.gfs" +cp "sorc/post_gtg.fd/gtg.config.gfs" "${DIR_ROOT}/parm/upp/gtg.config.gfs" # copy GTG code to UPP -cp sorc/post_gtg.fd/*f90 sorc/ncep_post.fd/. -cp sorc/post_gtg.fd/gtg.config.gfs parm/gtg.config.gfs +cp -f sorc/post_gtg.fd/*f90 sorc/ncep_post.fd/. # Build upp executable file -cd "${DIR_ROOT}/sorc/upp.fd/sorc" +cd "${DIR_ROOT}/sorc/wafs_upp.fd/sorc" ./build_ncep_post.sh # Copy upp to WAFS/exec -rm -rf "${DIR_ROOT}/exec/ncep_post" -cp "${DIR_ROOT}/sorc/upp.fd/exec/ncep_post" "${DIR_ROOT}/exec/ncep_post" +rm -rf "${DIR_ROOT}/exec/wafs_upp.x" +cp "${DIR_ROOT}/sorc/wafs_upp.fd/exec/ncep_post" "${DIR_ROOT}/exec/wafs_upp.x" exit diff --git a/sorc/build_wafs.sh b/sorc/build_wafs.sh index 96d77fb..9fed25b 100755 --- a/sorc/build_wafs.sh +++ b/sorc/build_wafs.sh @@ -3,25 +3,23 @@ set -eu # Get the root of the cloned WAFS directory -readonly DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}" )" )/.." && pwd -P) +readonly DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}")")/.." && pwd -P) # User Options BUILD_TYPE=${BUILD_TYPE:-"Release"} CMAKE_OPTS=${CMAKE_OPTS:-} +MACHINE_ID=${MACHINE_ID:-"wcoss2"} COMPILER=${COMPILER:-"intel"} BUILD_DIR=${BUILD_DIR:-"${DIR_ROOT}/sorc/build/wafs"} INSTALL_PREFIX=${INSTALL_PREFIX:-"${DIR_ROOT}/sorc/install/wafs"} #==============================================================================# -# Detect machine (sets MACHINE_ID) -source "${DIR_ROOT}/ush/detect_machine.sh" - # Load modules -source "${DIR_ROOT}/ush/module-reset.sh" +module reset source "${DIR_ROOT}/versions/build.ver" module use "${DIR_ROOT}/modulefiles" -module load wafs_"${MACHINE_ID}.${COMPILER}" +module load "wafs_${MACHINE_ID}.${COMPILER}" module list # Collect BUILD Options diff --git a/sorc/checkout_upp.sh b/sorc/checkout_upp.sh index 52f1823..6ac0c7c 100755 --- a/sorc/checkout_upp.sh +++ b/sorc/checkout_upp.sh @@ -5,5 +5,7 @@ set -eu # Get the root of the cloned WAFS directory readonly DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}" )" )/.." && pwd -P) +cd "${DIR_ROOT}" + # Checkout upp and gtg code -git submodule update --init --recursive +git submodule update --init --recursive diff --git a/sorc/upp.fd b/sorc/upp.fd deleted file mode 160000 index 81b72dc..0000000 --- a/sorc/upp.fd +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 81b72dca8e338b33e1aeacceb247edb77b531974 diff --git a/sorc/wafs_upp.fd b/sorc/wafs_upp.fd new file mode 160000 index 0000000..35030d8 --- /dev/null +++ b/sorc/wafs_upp.fd @@ -0,0 +1 @@ +Subproject commit 35030d84ab56906165767d127a6b23ea5450512f diff --git a/ush/detect_machine.sh b/ush/detect_machine.sh deleted file mode 100755 index 683ee0d..0000000 --- a/ush/detect_machine.sh +++ /dev/null @@ -1,92 +0,0 @@ -#!/bin/bash - -# The authoritative copy of this script lives in the ufs-weather-model at: -# https://github.com/ufs-community/ufs-weather-model/blob/develop/tests/detect_machine.sh -# If any local modifications are made or new platform support added, -# please consider opening an issue and a PR to the ufs-weather-model -# so that this copy remains in sync with its authoritative source -# -# Thank you for your contribution - -# If the MACHINE_ID variable is set, skip this script. -[[ -n ${MACHINE_ID:-} ]] && return - -# First detect w/ hostname -case $(hostname -f) in - - adecflow0[12].acorn.wcoss2.ncep.noaa.gov) MACHINE_ID=acorn ;; ### acorn - alogin0[12].acorn.wcoss2.ncep.noaa.gov) MACHINE_ID=acorn ;; ### acorn - clogin0[1-9].cactus.wcoss2.ncep.noaa.gov) MACHINE_ID=wcoss2 ;; ### cactus01-9 - clogin10.cactus.wcoss2.ncep.noaa.gov) MACHINE_ID=wcoss2 ;; ### cactus10 - dlogin0[1-9].dogwood.wcoss2.ncep.noaa.gov) MACHINE_ID=wcoss2 ;; ### dogwood01-9 - dlogin10.dogwood.wcoss2.ncep.noaa.gov) MACHINE_ID=wcoss2 ;; ### dogwood10 - - gaea5[1-8]) MACHINE_ID=gaea ;; ### gaea51-58 - gaea5[1-8].ncrc.gov) MACHINE_ID=gaea ;; ### gaea51-58 - - hfe0[1-9]) MACHINE_ID=hera ;; ### hera01-09 - hfe1[0-2]) MACHINE_ID=hera ;; ### hera10-12 - hecflow01) MACHINE_ID=hera ;; ### heraecflow01 - - s4-submit.ssec.wisc.edu) MACHINE_ID=s4 ;; ### s4 - - fe[1-8]) MACHINE_ID=jet ;; ### jet01-8 - tfe[12]) MACHINE_ID=jet ;; ### tjet1-2 - - Orion-login-[1-4].HPC.MsState.Edu) MACHINE_ID=orion ;; ### orion1-4 - - [Hh]ercules-login-[1-4].[Hh][Pp][Cc].[Mm]s[Ss]tate.[Ee]du) MACHINE_ID=hercules ;; ### hercules1-4 - - login[1-4].stampede2.tacc.utexas.edu) MACHINE_ID=stampede ;; ### stampede1-4 - - login0[1-2].expanse.sdsc.edu) MACHINE_ID=expanse ;; ### expanse1-2 - - discover3[1-5].prv.cube) MACHINE_ID=discover ;; ### discover31-35 - *) MACHINE_ID=UNKNOWN ;; # Unknown platform -esac - -if [[ ${MACHINE_ID} == "UNKNOWN" ]]; then - case ${PW_CSP:-} in - "aws" | "google" | "azure") MACHINE_ID=noaacloud ;; - *) PW_CSP="UNKNOWN" - esac -fi - -# Overwrite auto-detect with MACHINE if set -MACHINE_ID=${MACHINE:-${MACHINE_ID}} - -# If MACHINE_ID is no longer UNKNNOWN, return it -if [[ "${MACHINE_ID}" != "UNKNOWN" ]]; then - return -fi - -# Try searching based on paths since hostname may not match on compute nodes -if [[ -d /lfs/h3 ]]; then - # We are on NOAA Cactus or Dogwood - MACHINE_ID=wcoss2 -elif [[ -d /lfs/h1 && ! -d /lfs/h3 ]]; then - # We are on NOAA TDS Acorn - MACHINE_ID=acorn -elif [[ -d /mnt/lfs1 ]]; then - # We are on NOAA Jet - MACHINE_ID=jet -elif [[ -d /scratch1 ]]; then - # We are on NOAA Hera - MACHINE_ID=hera -elif [[ -d /work ]]; then - # We are on MSU Orion or Hercules - if [[ -d /apps/other ]]; then - # We are on Hercules - MACHINE_ID=hercules - else - MACHINE_ID=orion - fi -elif [[ -d /gpfs && -d /ncrc ]]; then - # We are on GAEA. - MACHINE_ID=gaea -elif [[ -d /data/prod ]]; then - # We are on SSEC's S4 - MACHINE_ID=s4 -else - echo WARNING: UNKNOWN PLATFORM 1>&2 -fi diff --git a/ush/mkwfsgbl.sh b/ush/mkwfsgbl.sh index c1d5376..c8847ba 100755 --- a/ush/mkwfsgbl.sh +++ b/ush/mkwfsgbl.sh @@ -1,71 +1,55 @@ #!/bin/bash +################################################################################ # UTILITY SCRIPT NAME : mkwfsgbl.sh # AUTHOR : Mary Jacobs # DATE WRITTEN : 11/06/96 # # Abstract: This utility script produces the GFS WAFS -# bulletins. +# bulletins. # -# Input: 2 arguments are passed to this script. +# Input: 2 arguments are passed to this script. # 1st argument - Forecast Hour - format of 2I -# 2nd argument - In hours 12-30, the designator of +# 2nd argument - In hours 12-30, the designator of # a or b. # # Logic: If we are processing hours 12-30, we have the # added variable of the a or b, and process # accordingly. The other hours, the a or b is dropped. # -echo "History: SEPT 1996 - First implementation of this utility script" -echo "History: AUG 1999 - Modified for implementation on IBM SP" -echo " - Allows users to run interactively" -# +# History: SEPT 1996 - First implementation of this utility script" +# AUG 1999 - Modified for implementation on IBM SP" +# - Allows users to run interactively" +# SEP 2007 - BAG - Put in fix on 20070925 to force the percision of U and V winds +# to default to 1 through the use of the grib_wafs.namelist file. +################################################################################ set -x -fhr="$1" -sets_key=$2 -num=$# - -if test $num -ge 2 -then - echo " Appropriate number of arguments were passed" - set -x - if [ -z "$DATA" ] - then - export DATA=`pwd` - cd $DATA - setpdy.sh - . PDY - fi -else - echo "" - echo "Usage: mkwfsgbl.sh \$hour [a|b]" - echo "" - exit 16 -fi +fhr=${1:-?"fhr is missing"} +sets_key=${2:-?"sets_key is missing"} + +set +x echo " ------------------------------------------" echo " BEGIN MAKING gfs WAFS PRODUCTS" echo " ------------------------------------------" echo "Enter Make WAFS utility." +set -x + +ifhr=$((10#$fhr)) +fhr3=$(printf "%03i" "${ifhr}") +GFS_MASTER="${COMINgfs}/gfs.t${cyc}z.master.grb2f${fhr3}" ############################## # Copy Input Field to $DATA ############################## -if test ! -f pgrbf${fhr} -then - fhr3="$(printf "%03d" $(( 10#$fhr )) )" - -# To solve Bugzilla #408: remove the dependency of grib1 files in gfs wafs job in next GFS upgrade -# Reason: It's not efficent if simply converting from grib2 to grib1 (costs 6 seconds with 415 records) -# Solution: Need to grep 'selected fields on selected levels' before CNVGRIB (costs 1 second with 92 records) -# ln -s $COMINgfs/gfs.${cycle}.pgrb2.1p00.f$fhr3 pgrb2f${fhr} -# $WGRIB2 pgrb2f${fhr} | grep -F -f $FIXwafs/grib_wafs.grb2to1.list | $WGRIB2 -i pgrb2f${fhr} -grib pgrb2f${fhr}.tmp - masterfile=$COMINgfs/gfs.${cycle}.master.grb2f${fhr3} - $WGRIB2 $masterfile | grep -F -f $FIXwafs/grib_wafs.grb2to1.list | $WGRIB2 -i $masterfile -grib masterf$fhr - +if [[ ! -f "pgrbf${fhr}" ]]; then + + cpreq "${GFS_MASTER}" "./gfs_masterf${fhr}.grib2" + ${WGRIB2} "./gfs_masterf${fhr}.grib2" | grep -F -f "${FIXwafs}/wafs/grib_wafs.grb2to1.list" | ${WGRIB2} -i "./gfs_masterf${fhr}.grib2" -grib "masterf${fhr}" + # Change data input from 1p00 files to master files export opt1=' -set_grib_type same -new_grid_winds earth ' export opt21=' -new_grid_interpolation bilinear ' @@ -75,32 +59,27 @@ then export opt27=":(APCP|ACPCP):" export opt28=' -new_grid_interpolation budget -fi ' export grid1p0="latlon 0:360:1.0 90:181:-1.0" - $WGRIB2 masterf$fhr $opt1 $opt21 $opt24 $opt25 $opt26 $opt27 $opt28 \ - -new_grid $grid1p0 pgb2file_${fhr}1p00 + ${WGRIB2} "masterf${fhr}" ${opt1} ${opt21} ${opt24} ${opt25} ${opt26} ${opt27} ${opt28} \ + -new_grid ${grid1p0} "pgb2file_${fhr}1p00" # trim RH vaule larger than 100. - $WGRIB2 pgb2file_${fhr}1p00 -not_if ':RH:' -grib pgrb2f${fhr}.tmp \ - -if ':RH:' -rpn "10:*:0.5:+:floor:1000:min:10:/" -set_grib_type same \ - -set_scaling -1 0 -grib_out pgrb2f${fhr}.tmp - - $CNVGRIB -g21 pgrb2f${fhr}.tmp pgrbf${fhr} + ${WGRIB2} "pgb2file_${fhr}1p00" -not_if ':RH:' -grib "pgrb2f${fhr}.tmp" \ + -if ':RH:' -rpn "10:*:0.5:+:floor:1000:min:10:/" -set_grib_type same \ + -set_scaling -1 0 -grib_out "pgrb2f${fhr}.tmp" + + ${CNVGRIB} -g21 "pgrb2f${fhr}.tmp" "pgrbf${fhr}" fi -# -# BAG - Put in fix on 20070925 to force the percision of U and V winds -# to default to 1 through the use of the grib_wafs.namelist file. -# -$COPYGB -g3 -i0 -N$FIXwafs/grib_wafs.namelist -x pgrbf${fhr} tmp -mv tmp pgrbf${fhr} -$GRBINDEX pgrbf${fhr} pgrbif${fhr} +${COPYGB} -g3 -i0 -N${FIXwafs}/wafs/grib_wafs.namelist -x "pgrbf${fhr}" tmp +mv tmp "pgrbf${fhr}" +${GRBINDEX} "pgrbf${fhr}" "pgrbif${fhr}" ############################## # Process WAFS ############################## -if test $fhr -ge '12' -a $fhr -le '30' -then - sets=$sets_key +if ((ifhr >= 12 && ifhr <= 30)); then + sets=${sets_key} set +x echo "We are processing the primary and secondary sets of hours." echo "These sets are the a and b of hours 12-30." @@ -110,7 +89,8 @@ else unset sets fi -export pgm=wafs_makewafs +export pgm="wafs_makewafs.x" + . prep_step export FORT11="pgrbf${fhr}" @@ -118,30 +98,22 @@ export FORT31="pgrbif${fhr}" export FORT51="xtrn.wfsgfs${fhr}${sets}" export FORT53="com.wafs${fhr}${sets}" -startmsg -$EXECwafs/wafs_makewafs.x < $FIXwafs/grib_wfsgfs${fhr}${sets} >>$pgmout 2>errfile -export err=$?;err_chk - - -############################## -# Post Files to PCOM -############################## +${EXECwafs}/${pgm} <"${FIXwafs}/wafs/grib_wfsgfs${fhr}${sets}" >>"${pgmout}" 2>errfile +export err=$? +err_chk -if test "$SENDCOM" = 'YES' -then - cpfs xtrn.wfsgfs${fhr}${sets} $PCOM/xtrn.wfsgfs${cyc}${fhr}${sets}.$jobsuffix +if [[ ! -f "xtrn.wfsgfs${fhr}${sets}" ]]; then + err_exit "FATAL ERROR: '${pgm}' failed to create 'xtrn.wfsgfs${fhr}${sets}'" fi -############################## -# Distribute Data -############################## +# Send data to COM +if [[ "${SENDCOM}" == "YES" ]]; then + cpfs "xtrn.wfsgfs${fhr}${sets}" "${COMOUTwmo}/xtrn.wfsgfs${cyc}${fhr}${sets}.${jobsuffix}" +fi -if [ "$SENDDBN_NTC" = 'YES' ] ; then - $DBNROOT/bin/dbn_alert GRIB_LOW gfs $job $PCOM/xtrn.wfsgfs${cyc}${fhr}${sets}.$jobsuffix +# Alert via DBN +if [[ "${SENDDBN_NTC}" == "YES" ]]; then + "${DBNROOT}/bin/dbn_alert" GRIB_LOW gfs "${job}" "${COMOUTwmo}/xtrn.wfsgfs${cyc}${fhr}${sets}.${jobsuffix}" else - echo "xtrn.wfsgfs${cyc}${fhr}${sets}.$job file not posted to db_net." + echo "xtrn.wfsgfs${cyc}${fhr}${sets}.${job} file not posted to db_net." fi - -echo "Wafs Processing $fhr hour completed normally" - -exit diff --git a/ush/module-reset.sh b/ush/module-reset.sh deleted file mode 100755 index c5838ec..0000000 --- a/ush/module-reset.sh +++ /dev/null @@ -1,115 +0,0 @@ -#!/bin/bash -set -u - -if [[ $MACHINE_ID = jet* ]] ; then - # We are on NOAA Jet - if ( ! eval module help > /dev/null 2>&1 ) ; then - source /apps/lmod/lmod/init/bash - fi - export LMOD_SYSTEM_DEFAULT_MODULES=contrib - module reset - -elif [[ $MACHINE_ID = hera* ]] ; then - # We are on NOAA Hera - if ( ! eval module help > /dev/null 2>&1 ) ; then - source /apps/lmod/lmod/init/bash - fi - export LMOD_SYSTEM_DEFAULT_MODULES=contrib - module reset - -elif [[ $MACHINE_ID = orion* ]] ; then - # We are on Orion - if ( ! eval module help > /dev/null 2>&1 ) ; then - source /apps/lmod/lmod/init/bash - fi - export LMOD_SYSTEM_DEFAULT_MODULES=contrib - module reset - -elif [[ $MACHINE_ID = hercules* ]] ; then - # We are on Hercules - if ( ! eval module help > /dev/null 2>&1 ) ; then - source /apps/other/lmod/lmod/init/bash - fi - export LMOD_SYSTEM_DEFAULT_MODULES=contrib - module reset - -elif [[ $MACHINE_ID = s4* ]] ; then - # We are on SSEC Wisconsin S4 - if ( ! eval module help > /dev/null 2>&1 ) ; then - source /usr/share/lmod/lmod/init/bash - fi - export LMOD_SYSTEM_DEFAULT_MODULES=license_intel - module reset - -elif [[ $MACHINE_ID = wcoss2 ]]; then - # We are on WCOSS2 - module reset - -elif [[ $MACHINE_ID = cheyenne* ]] ; then - # We are on NCAR Cheyenne - if ( ! eval module help > /dev/null 2>&1 ) ; then - source /glade/u/apps/ch/modulefiles/default/localinit/localinit.sh - fi - module purge - -elif [[ $MACHINE_ID = stampede* ]] ; then - # We are on TACC Stampede - if ( ! eval module help > /dev/null 2>&1 ) ; then - source /opt/apps/lmod/lmod/init/bash - fi - module purge - -elif [[ $MACHINE_ID = gaea* ]] ; then - # We are on GAEA. - if ( ! eval module help > /dev/null 2>&1 ) ; then - # We cannot simply load the module command. The GAEA - # /etc/profile modifies a number of module-related variables - # before loading the module command. Without those variables, - # the module command fails. Hence we actually have to source - # /etc/profile here. - source /etc/profile - __ms_source_etc_profile=yes - else - __ms_source_etc_profile=no - fi - module purge - # clean up after purge - unset _LMFILES_ - unset _LMFILES_000 - unset _LMFILES_001 - unset LOADEDMODULES - module load modules - if [[ -d /opt/cray/ari/modulefiles ]] ; then - module use -a /opt/cray/ari/modulefiles - fi - if [[ -d /opt/cray/pe/ari/modulefiles ]] ; then - module use -a /opt/cray/pe/ari/modulefiles - fi - if [[ -d /opt/cray/pe/craype/default/modulefiles ]] ; then - module use -a /opt/cray/pe/craype/default/modulefiles - fi - if [[ -s /etc/opt/cray/pe/admin-pe/site-config ]] ; then - source /etc/opt/cray/pe/admin-pe/site-config - fi - if [[ "$__ms_source_etc_profile" == yes ]] ; then - source /etc/profile - unset __ms_source_etc_profile - fi - -elif [[ $MACHINE_ID = expanse* ]]; then - # We are on SDSC Expanse - if ( ! eval module help > /dev/null 2>&1 ) ; then - source /etc/profile.d/modules.sh - fi - module purge - module load slurm/expanse/20.02.3 - -elif [[ $MACHINE_ID = discover* ]]; then - # We are on NCCS discover - export SPACK_ROOT=/discover/nobackup/mapotts1/spack - export PATH=$PATH:$SPACK_ROOT/bin - . $SPACK_ROOT/share/spack/setup-env.sh - -else - echo WARNING: UNKNOWN PLATFORM 1>&2 -fi diff --git a/ush/wafs_grib2_0p25_blending.sh b/ush/wafs_grib2_0p25_blending.sh new file mode 100755 index 0000000..a78dcb3 --- /dev/null +++ b/ush/wafs_grib2_0p25_blending.sh @@ -0,0 +1,150 @@ +#!/bin/bash + +################################################################################ +# UTILITY SCRIPT NAME : wafs_grib2_0p25_blending.sh +# DATE WRITTEN : 04/02/2020 +# +# Abstract: This script looks for US and UK WAFS Grib2 products at 1/4 deg, +# waits unblended UK data for specified period of time, and blends +# whenever UK data becomes available. After the waiting time window +# expires, the script sends out US data only if UK data doesn't arrive +# +# History: 04/02/2020 - First implementation of this new script +# 10/xx/2021 - Remove jlogfile +# 05/25/2022 - Add ICAO new milestone Nov 2023 +# 09/08/2024 - WAFS separation +# - Filename changes according to EE2 standard except for files sent to UK +# - dbn_alert subtype is changed from gfs to WAFS +# - Fix bugzilla 1213: Filename should use fHHH instead of FHH. +# - Parallel run for each forecast hour, changed from sequential run. +# - Fix bugzilla 1593: Improve email notification for missing UK WAFS data. +# - Extend waiting time window from 15 to 25 minutes +################################################################################ + +set -x + +fhr=$1 + +mkdir -p "${DATA}/${fhr}" +cd "${DATA}/${fhr}" || err=1 +if (( err != 0 )); then + echo "FATAL ERROR: Could not 'cd ${DATA}/${fhr}'" + echo "WAFS blending 0p25 program failed at " ${PDY}${cyc}F${fhr} > ../no_blending_files.$fhr + exit 1 +fi + +############################################### +# Specify Timeout Behavior for WAFS blending +############################################### +# SLEEP_TIME - Amount of time (secs) to wait for a input file before exiting +# SLEEP_INT - Amount of time (secs) to wait between checking for input files +SLEEP_TIME=${SLEEP_TIME:-1500} +SLEEP_INT=${SLEEP_INT:-10} +SLEEP_LOOP_MAX=$((SLEEP_TIME / SLEEP_INT)) + +########################## +# look for UK WAFS data. +########################## +MISSING_UK_WAFS="NO" + +for ((ic = 1; ic <= SLEEP_LOOP_MAX; ic++)); do + # Three(3) unblended UK files for each cycle+fhour: icing, turb, cb + ukfiles=$(find "${COMINuk}" -name "egrr_wafshzds_unblended_*_0p25_${PDY:0:4}-${PDY:4:2}-${PDY:6:2}T${cyc}:00Z_t${fhr}.grib2" | wc -l) + if ((ukfiles >= 3)); then + echo "Found all 3 UK WAFS GRIB2 files, continue ..." + break + fi + + if ((ic == SLEEP_LOOP_MAX)); then + products="cb ice turb" + for prod in ${products}; do + ukfile="egrr_wafshzds_unblended_${prod}_0p25_${PDY:0:4}-${PDY:4:2}-${PDY:6:2}T${cyc}:00Z_t${fhr}.grib2" + if [[ ! -f "${COMINuk}/${ukfile}" ]]; then + echo "WARNING: UK WAFS GRIB2 file '${ukfile}' not found after waiting over ${SLEEP_TIME} seconds" + echo "Missing ${COMINuk}/${ukfile}" >> ../missing_uk_files.$fhr + fi + done + echo "WARNING: UK WAFS GRIB2 unblended data is not completely available, no blending" + MISSING_UK_WAFS="YES" + break + else + sleep "${SLEEP_INT}" + fi +done + +########################## +# look for US WAFS data. +########################## +MISSING_US_WAFS="NO" + +if [[ ! -f "${COMINus}/WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2.idx" ]]; then + # Blending ecflow run is triggered by the finishing of the upstream JWAFS_GRIB2_0P25 f048 + # - In real time ecflow, blending starts at T+4:30, unblended upstream products are all available. + # It doesn't need to sleep/wait + # - In non-real time ecflow, blending has no time tigger, unblended f048 is not guaranteed to be the last finished one. + # It needs to sleep/wait. All upstream jobs can be finished within 60 seconds when f048 is finished. + # + # Standalone blending doesn't need to sleep/wait since unblended upstream grib2_0p25 runs standalone first. + sleep 60 + if [[ ! -f "${COMINus}/WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2.idx" ]]; then + echo "WARNING: missing US unblended data - ${COMINus}/WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2" + echo "Missing ${COMINus}/WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2" > ../missing_us_files.$fhr + MISSING_US_WAFS="YES" + fi +fi + +########################## +# Blending or unblended +########################## +if [[ "${MISSING_UK_WAFS}" == "YES" ]] && [[ "${MISSING_US_WAFS}" == "YES" ]]; then + cat ../missing_uk_files.$fhr ../missing_us_files.$fhr > ../no_blending_files.$fhr + rm ../missing_uk_files.$fhr ../missing_us_files.$fhr +elif [[ "${MISSING_UK_WAFS}" == "YES" ]]; then + echo "turning back on dbn alert for unblended US WAFS product" + # Avoid duplicate dbn_alert of unblended grib2 file which was done in the upstream grib2_0p25 job, fix bugzilla 1226 + # "${DBNROOT}/bin/dbn_alert" MODEL WAFS_0P25_UBL_GB2 "${job}" "${COMINus}/WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2" + "${DBNROOT}/bin/dbn_alert" MODEL WAFS_0P25_UBL_GB2_WIDX "${job}" "${COMINus}/WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2.idx" +else + # retrieve UK products + # Three(3) unblended UK files for each cycle+fhour: icing, turb, cb + cat "${COMINuk}/egrr_wafshzds_unblended_"*"_0p25_${PDY:0:4}-${PDY:4:2}-${PDY:6:2}T${cyc}:00Z_t${fhr}.grib2" >"EGRR_WAFS_0p25_unblended_${PDY}_${cyc}z_t${fhr}.grib2" + + if [[ "${MISSING_US_WAFS}" == "YES" ]]; then + if [ $SENDDBN = "YES" ] ; then + "$DBNROOT/bin/dbn_alert" MODEL WAFS_UKMET_0P25_UBL_GB2 "${job}" "EGRR_WAFS_0p25_unblended_${PDY}_${cyc}z_t${fhr}.grib2" + fi + else + # pick up US data + cpreq "${COMINus}/WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2" . + + # run blending code + export pgm="wafs_blending_0p25.x" + + . prep_step + + ${EXECwafs}/${pgm} "WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2" \ + "EGRR_WAFS_0p25_unblended_${PDY}_${cyc}z_t${fhr}.grib2" \ + "0p25_blended_${PDY}${cyc}f${fhr}.grib2 >f${fhr}.out" + + err=$? + if (( err != 0 )); then + echo "turning back on dbn alert for unblended US WAFS product" + "${DBNROOT}/bin/dbn_alert" MODEL WAFS_0P25_UBL_GB2_WIDX "${job}" "${COMINus}/WAFS_0p25_unblended_${PDY}${cyc}f${fhr}.grib2.idx" + echo "WAFS blending 0p25 program failed at " ${PDY}${cyc}F${fhr} > ../no_blending_files.$fhr + else + # Distribute US WAFS unblend Data to NCEP FTP Server (WOC) and TOC + if [[ "${SENDCOM}" == "YES" ]]; then + cpfs "0p25_blended_${PDY}${cyc}f${fhr}.grib2" "${COMOUT}/WAFS_0p25_blended_${PDY}${cyc}f${fhr}.grib2" + fi + + if [[ "${SENDDBN_NTC}" == "YES" ]]; then + # Distribute Data to NCEP FTP Server (WOC) and TOC + echo "No WMO header yet" + fi + + if [[ "${SENDDBN}" == "YES" ]]; then + "${DBNROOT}/bin/dbn_alert" MODEL WAFS_0P25_BL_GB2 "${job}" "${COMOUT}/WAFS_0p25_blended_${PDY}${cyc}f${fhr}.grib2" + fi + fi + fi +fi diff --git a/ush/wafs_upp.sh b/ush/wafs_upp.sh deleted file mode 100755 index 2658a5e..0000000 --- a/ush/wafs_upp.sh +++ /dev/null @@ -1,43 +0,0 @@ -#!/bin/bash - -set -x - -# making the time stamp format for ncep post -export YY=`echo $VDATE | cut -c1-4` -export MM=`echo $VDATE | cut -c5-6` -export DD=`echo $VDATE | cut -c7-8` -export HH=`echo $VDATE | cut -c9-10` - -run=`echo $RUNupp | tr '[a-z]' '[A-Z]'` -cat > itag < outpost_gfs_${VDATE}_${CTL} - -export ERR=$? -export err=$ERR - -exit $err diff --git a/versions/build.ver b/versions/build.ver old mode 100755 new mode 100644 diff --git a/versions/run.ver b/versions/run.ver index 6685f91..e6ad531 100644 --- a/versions/run.ver +++ b/versions/run.ver @@ -2,26 +2,23 @@ export wafs_ver=v7.0.0 export gfs_ver=v16.3 export radarl2_ver=v1.2 +export bufr_dump_ver=1.1.2 +export util_shared_ver=1.4.0 -export intel_ver=19.1.3.304 export PrgEnvintel_ver=8.1.0 +export intel_ver=19.1.3.304 export craype_ver=2.7.10 +export craympich_ver=8.1.9 +export craypals_ver=1.1.3 -export libjpeg_ver=9c -export prod_util_ver=2.0.14 -export prod_envir_ver=2.0.6 -export grib_util_ver=1.2.3 -export wgrib2_ver=2.0.7 - -#For make_NTC_file.pl in blending -export util_shared_ver=1.4.0 +export cfp_ver=2.0.4 -# For GCIP -export bufr_dump_ver=1.1.2 +export prod_envir_ver=2.0.6 -# For UPP +export libjpeg_ver=9c export hdf5_ver=1.10.6 export netcdf_ver=4.7.4 export g2tmpl_ver=1.9.1 -export craympich_ver=8.1.9 -export craypals_ver=1.1.3 +export prod_util_ver=2.0.14 +export grib_util_ver=1.2.3 +export wgrib2_ver=2.0.7