Bugfixes on previous PR that was merged prior to testing

[aerorahul]

#44 was merged without a review while testing was still in progress.

During the testing of #44, a few bugs were uncovered. They are fixed in this PR.
Additionally, this PR:

• allows user to setup a ecflow case for any arbitrary date and experiment ID. dev/ecf/setup_ecf.sh YYYYMMDDHH expid will result in ecf/def/wafsexpid.def. This allows one to run any number of suites from a single clone of the WAFS package.
• removes grib1 processing of forecast hours 00, 06, 54, 66. It was noticed that the exwafs_grib.sh script has no condition to handle these forecast hours.
• upp.fd is cloned as wafs_upp.fd per EE2 requirement

Throughout testing, the wafs_grib2_1p25 jobs for forecast hours 06-36 consistently fail during the execution of tocgrib2. This behavior is reproducible. Please see attached screenshot @YaliMao-NOAA can you please assist?

A snippet from a failed log looks like:

12 + /apps/ops/prod/libs/intel/19.1.3.304/grib_util/1.2.3/bin/tocgrib2
12 + export err=30
12 + err=30
12 + err_chk

-------------------------------------------------------------
-- FATAL ERROR: Job wafs_grib2_1p25_f006_00.150736122.cbqs01 failed, ERROR IN /apps/ops/prod/libs/intel/19.1.3.304/grib_util/1.2.3/bin/tocgrib2 RETURN CODE 30
-- ABNORMAL EXIT at Fri Aug 30 21:57:44 UTC 2024 on nid001589
-------------------------------------------------------------

The DATA directory for the above failed job is at:
/lfs/h2/emc/stmp/rahul.mahajan/wafsx001/tmp/wafs_grib2_1p25_f006_00.150736122.cbqs01

The log file is at:
/lfs/h2/emc/stmp/rahul.mahajan/wafsx001/output/wafs_grib2_1p25_f006_00.o150736122

The relevant paths for review are available at:
HOMEwafs: /lfs/h2/emc/eib/noscrub/rahul.mahajan/eibWork/EE2/wafs/wafs.v7.0.0
COM: /lfs/h2/emc/ptmp/rahul.mahajan/wafsx001/com
OUTPUT: /lfs/h2/emc/stmp/rahul.mahajan/wafsx001/output
DATA: /lfs/h2/emc/stmp/rahul.mahajan/wafsx001/tmp

[YaliMao-NOAA]

I found a root issue which came from UPP. UPP failed to generate WAFS products successfully. I am going to point it out inline where the issue is.

[YaliMao-NOAA]

@aerorahul Ecflow server failed to start. Error message:
ERROR: Cannot locate directory /lfs/h2/emc/ptmp/yali.mao/ecflow_test/ecflow/com

A simple fix:
instead of this line: mkdir -p ${ECF_ROOT}
change it to: mkdir -p ${ECF_COMDIR}

[aerorahul]

@YaliMao-NOAA
I ran a second test with the updates and all jobs successfully succeeded.

[YaliMao-NOAA]

@aerorahul
Should HOMEwafs line be removed from all job scripts?
export HOMEwafs=${HOMEwafs:-${NWROOT}/wafs.${wafs_ver}}

[aerorahul]

@aerorahul Should HOMEwafs line be removed from all job scripts? export HOMEwafs=${HOMEwafs:-${NWROOT}/wafs.${wafs_ver}}

There is no HOMEwafs in any of the j-jobs (and there shouldn't be). They are only present in the dev/driver scripts.