Add missing arch_tars dependencies

[AntonMFernando-NOAA]

Description

• The gfs_arch_tars job currently does not depend on gempak jobs, even though it archives data produced by them. This PR will introduce that dependency. Additionally, there are several missing dependencies for cleanup when the arch_tar job is not executed. Nearly all of the job's dependencies need to be replicated for cleanup in case arch_tar doesn't run. This PR will address this problem as well.

  Resolves Missing gfs_arch_tars dependencies #3294

Type of change

• Bug fix (fixes something broken)
• New feature (adds functionality)
• Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

• Is this a breaking change (a change in existing functionality)? NO
• Does this change require a documentation update? NO
• Does this change require an update to any of the following submodules? NO (If YES, please add a link to any PRs that are pending.)
  • EMC verif-global
  • GDAS
  • GFS-utils
  • GSI
  • GSI-monitor
  • GSI-utils
  • UFS-utils
  • UFS-weather-model
  • wxflow

How has this been tested?

• Cycled test on WCOSS2

Checklist

• Any dependent changes have been merged and published
• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have documented my code, including function, input, and output descriptions
• My changes generate no new warnings
• New and existing tests pass with my changes
• This change is covered by an existing CI test or a new one has been added
• Any new scripts have been added to the .github/CODEOWNERS file with owners
• I have made corresponding changes to the system documentation if necessary

[DavidHuber-NOAA]

I ran a pair of tests on Hera, both C96_atm3dvar_extended with DO_GEMPAK=NO (there are some issues with gempak jobs on Hera). One of the tests ran with HPSSARCH=YES and one with HPSSARCH=NO to test the dependencies. Visual inspection of the resulting XMLs show that the arch_vrfy, arch_tars, and cleanup jobs all have the correct dependencies.

I then made some adjustments to the XML to open the throttle to test if I overlooked any dependencies. This was achieved by changing the parameters cyclethrottle to 8 and taskthrottle to 125. All jobs succeeded on their first pass, the vrfy jobs all ran after all of the product jobs completed and before the metp and tars jobs, and the cleanup jobs all ran last for their associated RUN.

It would be good to repeat this operation with the other CI tests (especially the hybvar tests). When CI testing for this PR is launched on Hercules and Hera, I will inspect the XMLs to verify all jobs have the correct dependencies.

[WalterKolczynski-NOAA]

It would be good to repeat this operation with the other CI tests (especially the hybvar tests). When CI testing for this PR is launched on Hercules and Hera, I will inspect the XMLs to verify all jobs have the correct dependencies.

I'm doing Hercules first, as that seems to be the machine where the cleanup was running prematurely the most often.

[DavidHuber-NOAA]

Looking through the C96C48_hybatmDA, C48mx500_3DVarAOWCDA, and C48mx500_hybAOWCDA XMLs generated by the Hercules CI, I think all of the dependencies are correct.

[emcbot]

CI Passed on Hercules in Build# 1
Built and ran in directory /work2/noaa/global/CI/HERCULES/3319

Experiment C48_ATM_342a7861 Completed 1 Cycles: *SUCCESS* at Fri Feb 14 15:09:22 CST 2025
Experiment C96_S2SWA_gefs_replay_ics_342a7861 Completed 1 Cycles: *SUCCESS* at Fri Feb 14 15:46:10 CST 2025
Experiment C48mx500_hybAOWCDA_342a7861 Completed 2 Cycles: *SUCCESS* at Fri Feb 14 15:52:01 CST 2025
Experiment C96C48_hybatmDA_342a7861 Completed 3 Cycles: *SUCCESS* at Fri Feb 14 16:46:41 CST 2025
Experiment C96_atm3DVar_342a7861 Completed 3 Cycles: *SUCCESS* at Fri Feb 14 16:52:36 CST 2025
Experiment C48_S2SW_342a7861 Completed 1 Cycles: *SUCCESS* at Fri Feb 14 17:10:53 CST 2025
Experiment C48mx500_3DVarAOWCDA_342a7861 Completed 2 Cycles: *SUCCESS* at Fri Feb 14 18:17:46 CST 2025
Experiment C48_S2SWA_gefs_342a7861 Completed 1 Cycles: *SUCCESS* at Fri Feb 14 18:24:25 CST 2025

[DavidHuber-NOAA]

@TerrenceMcGuinness-NOAA I think that the Jenkins client may have crashed on Hera. Could you check on it?

[DavidHuber-NOAA]

It looks like Terry is out of the office today and tomorrow, so I will launch CI manually via generate_workflows.sh on Hera.

[DavidHuber-NOAA]

Tests are being run in /scratch1/NCEPDEV/global/David.Huber/para_3319/.

[DavidHuber-NOAA]

Visually inspected the XMLs for cases C48mx500_3DVarAOWCDA, C48mx500_hybAOWCDA, C96C48_hybatmDA, C96C48_hybatmaerosnowDA, and C96C48_ufs_hybatmDA. All arch_tars, earc_tars, and arch_cleanup dependencies look correct.

[DavidHuber-NOAA]

All tests passed on Hera. Also, no jobs ran more than once.

[DavidHuber-NOAA]

Looks good!