Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability for JEDI-to-FV3 increment converter to process ensembles #1022

Merged
merged 27 commits into from
Apr 11, 2024

Conversation

DavidNew-NOAA
Copy link
Collaborator

@DavidNew-NOAA DavidNew-NOAA commented Apr 4, 2024

Major changes to JEDI-to-FV3 increment converter:

  1. Allow processing of ensembles using the same YAML template format that the FV3-JEDI LGETKF app uses. Alternatively, a list of background/increments can be specified.

  2. Fix incorrect calculation of FV3 increment, for variables other than delp and hydrostatic_delz, related to two issues:
    a. When adding the JEDI increment to the background to get the analysis, some mixing-ratio-type variables become
    negative and are set to zero by FV3-JEDI. However, when we then subtract the analysis and background after the
    variable change to get delp and hydrostatic_delz, we don't get the same increment that we started with for those
    mixing-ratio fields.
    b. For ensemble DA, the deterministic background and JEDI increment are not the same resolution, so the background
    and analysis discussed above must be interpolated to half resolution before being subtracted but after the JEDI
    increment is interpolated to full resolution and added to the background. Because of these two interpolations, similar
    to above, we don't get the same increment as we started with.

I now use Atlas to manually set all variables other than delp and hydrostatic_delz back to their original values, which solved both issues.

Additionally, I make the following other changes:

  1. Rename fv3jedi_fv3inc.yaml to fv3jedi_fv3inc_variational.yaml and rewrite according to the new YAML format.
  2. Rewrite the gdasapp_test_fv3jedi_fv3inc CTest input YAML according to the new YAML format.
  3. Create a new fv3jedi_fv3inc_lgetkf.yaml for an eventual use in converting the ensemble job increment in Global Workflow.

Testing these new features yields the following results:

  1. For delp and hydrostatic_delz in ensemble DA, the FV3 increment produced by the deterministic variational analysis in Global Workflow (branch feature/jediinc2fv3) is zero diff with develop, which has the previous version of the OOPS-based increment converter. I tested the other increment variables by writing the FV3 increment out to the native cubed sphere and compared with the JEDI increment. They are zero diff.
  2. The gdasapp_test_fv3jedi_fv3inc passes with the same test reference used in develop.
  3. Using a special Global Workflow branch to test the lgetkf analysis, the YAML templating works, and the resulting FV3 increment variables, delp and hydrostatic_delz, differ from the one values produced by develop (using jediinc2fv3.py) within a reasonable relative error, similar to that when I first tested the increment converter against jediinc2fv3.py with the variational analysis.

@DavidNew-NOAA DavidNew-NOAA marked this pull request as ready for review April 4, 2024 23:55
@DavidNew-NOAA
Copy link
Collaborator Author

Actually I have one more idea for a testing strategy, then will re-open...

@DavidNew-NOAA
Copy link
Collaborator Author

After a ton of debugging and testing, I'm finally re-opening this PR

@CoryMartin-NOAA CoryMartin-NOAA merged commit 005adcd into develop Apr 11, 2024
7 checks passed
@CoryMartin-NOAA CoryMartin-NOAA deleted the feature/lgetkf-jediinc2fv3 branch April 11, 2024 13:26
danholdaway added a commit that referenced this pull request Apr 15, 2024
* upstream/develop:
  remove seviri from gdas_prototype_3d yaml (#1043)
  Fix GW jjob tests for upcoming GW PR #2420 (#1041)
  Fix test output for fv3jedi_fv3inc.h (#1039)
  Run g-w linker script before ctest for prepoceanobs task (#1034)
  Update femps and fv3-jedi-lm (#1036)
  Add ability for JEDI-to-FV3 increment converter to process ensembles (#1022)
  Add AVHRR/NOAA-15/18/19 assimilation to end-to-end GDASApp validation (#997)
  Catch error when trying to copy missing obs files from DATA to ROTDIR in prepoceanobs (#1028)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants