Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_gdasapp_atm_jjob_var test incomplete with upcoming GW PR 2420 #1011

Closed
DavidNew-NOAA opened this issue Mar 29, 2024 · 3 comments
Closed
Assignees

Comments

@DavidNew-NOAA
Copy link
Collaborator

Due to new changes in the upcoming Global Workflow PR (#2420)[https://github.com/NOAA-EMC/global-workflow/pull/2420/files], the test_gdasapp_atm_jjob_var suite will fail on the run and final jobs. There are two reasons for this:

  1. JGLOBAL_ATM_ANALYSIS_RUN is now called JGLOBAL_ATM_ANALYSIS_VARIATIONAL, causing test_gdasapp_atm_jjob_var_run test to fail.

  2. There is a new rocoto job, gdasatmanlfv3inc that needs its own test between test_gdasapp_atm_jjob_var_run and test_gdasapp_atm_jjob_var_final, otherwise test_gdasapp_atm_jjob_var_final will fail, because it relies on this new job.

@DavidNew-NOAA DavidNew-NOAA self-assigned this Mar 29, 2024
@RussTreadon-NOAA
Copy link
Contributor

Orion test

Install GDASApp develop at fadffb0 inside a working copy of g-w feature/jediinc2fv3. Initial run of test_gdasapp had the following failures

        1743 - test_gdasapp_fv3jedi_fv3inc (Failed)
        1777 - test_gdasapp_atm_jjob_var_run (Failed)
        1778 - test_gdasapp_atm_jjob_var_final (Failed)

test_gdasapp_atm_jjob_var_run failed for the reason stated in this issue. Make the following changes in the working copy of GDASApp develop

On branch develop
Your branch is up to date with 'origin/develop'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   test/atm/global-workflow/CMakeLists.txt
        modified:   test/atm/global-workflow/jjob_var_run.sh

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        test/atm/global-workflow/jjob_var_inc.sh

These changes are as follows:

  • jjob_var_run.sh: replace JGLOBAL_ATM_ANALYSIS_RUN with JGLOBAL_ATM_ANALYSIS_VARIATIONAL
  • CMakeLists.txt: add test_gdasapp_atm_jjob_var_inc
  • jjob_var_inc.sh: new script to run JGLOBAL_ATM_ANALYSIS_FV3_INCREMENT

The above local modifications are in /work2/noaa/da/rtreadon/git/global-workflow/jediinc2fv3/sorc/gdas.cd.

After making the above changes the atm_jjob_var tests run to completion.

(gdasapp) Orion-login-4:/work2/noaa/da/rtreadon/git/global-workflow/jediinc2fv3/sorc/gdas.cd/build$ ctest -R test_gdasapp_atm_jjob_var
Test project /work2/noaa/da/rtreadon/git/global-workflow/jediinc2fv3/sorc/gdas.cd/build
    Start 1776: test_gdasapp_atm_jjob_var_init
1/4 Test #1776: test_gdasapp_atm_jjob_var_init ....   Passed   44.36 sec
    Start 1777: test_gdasapp_atm_jjob_var_run
2/4 Test #1777: test_gdasapp_atm_jjob_var_run .....   Passed  170.16 sec
    Start 1778: test_gdasapp_atm_jjob_var_inc
3/4 Test #1778: test_gdasapp_atm_jjob_var_inc .....   Passed   74.18 sec
    Start 1779: test_gdasapp_atm_jjob_var_final
4/4 Test #1779: test_gdasapp_atm_jjob_var_final ...   Passed   42.29 sec

100% tests passed, 0 tests failed out of 4

Total Test time (real) = 331.65 sec

test_gdasapp_fv3jedi_fv3inc failed during the test reference check

OOPS_STATS Run end                                  - Runtime:      5.13 sec,  Memory: total:     1.42 Gb, per task: min =   218.64 Mb, max =   259.60 Mb
Run: Finishing gdasapp::fv3inc with status = 0
terminate called after throwing an instance of 'oops::TestReferenceFloatMismatchError'
  what():  Test reference Float mismatch @ Line:8
Test Val : 1.2364032020428795e-09
Ref  Val : 5.6141089843750000e+04
Delta    : 5.6141089843748763e+04
Relative tolerance: 2.8070544921875619e+01
Absolute tolerance: 0.0000000000000000e+00
Test Line: 'specific_humidity                            | Min:+1.2364032020428795e-09 Max:+2.1250871941447258e-02 RMS:+5.1335857648077696e-03'
Ref Line : 'surface_pressure                             | Min:+5.6141089843750000e+04 Max:+1.0346660156250000e+05 RMS:+9.8890740204548158e+04'
srun: error: Orion-03-28: task 0: Aborted
srun: Terminating StepId=17424760.0
slurmstepd: error: *** STEP 17424760.0 ON Orion-03-28 CANCELLED AT 2024-04-12T13:14:52 ***

@DavidNew-NOAA
Copy link
Collaborator Author

Just addressed the test_gdasapp_fv3jedi_fv3inc issue with #1039 . Thanks for feedback

CoryMartin-NOAA pushed a commit that referenced this issue Apr 15, 2024
This PR addresses issue
[#1011](#1011), related to the
failure of the "gdasatmanlvar" jjob test due to a change in the name of
the "gdasatmanlvar" run script, and, with the upcoming Global Workflow
PR [#2420](NOAA-EMC/global-workflow#2420), the
impending failure of the "gdasatmanlfinal" jjob. This fixes the script
name in the "gdasatmanlvar" test and adds a new test for
"gdasatmanlfv3inc" which will fix the issure with "gdasatmanlfinal".

The "gdasatmanlfv3inc" and "gdasatmanlfinal" won't pass yes in GW
develop, but will after
[#2420](NOAA-EMC/global-workflow#2420) merges GW
feature/jediinc2fv3.

This PR is just a verbatim copy of @RussTreadon-NOAA 's work, taken from
his comment
[here](#1011 (comment)).
I re-ran the new tests, and they also passed for me in GW
feature/jediinc2fv3.
@DavidNew-NOAA
Copy link
Collaborator Author

Closing this PR with #1041 now merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants