Merge GFSDA v16 changes into GSI develop

[emilyhcliu]

Merge GFS v16.3.4 DA changes into GSI develop.
This will be a bit complicated and need some advise.

Apparently, we can not do the two-dot syntax since the structure in the develop is very different from GFS.v16.3 release.

@aerorahul Do you have any suggestion on how to proceed with the merging the GFS.v16.3.4 DA changes into GSI develop?
I have three options for now
The First one is from @RussTreadon-NOAA.

Use git diff develop...release/gfsda.v16 (triple dots)

This should extract the list of differences from the point that we branched out gfsda.v16.3 from the develop.
Checking these changes and manually added them to the develop.

The second one is just my imagination and is a question for @aerorahul
Do you think we can try do auto merge for the GSI/src directory only between the develop and gfsda.v16.3?

The third one is open to any suggestion from @aerorahul

[RussTreadon-NOAA]

@emilyhcliu , a git merge release/gfsda.v16 into a forked copy of develop may work well for you.

I did git merge release/gfsda.v16 into a local copy of develop on Cactus. A small number of files are listed as modified and merged. A longer list of files is unmerged with conflicts. Most of the conflicts are due to files in release/gfsda.v16.3 no longer existing in develop. These conflicts can be removed from the local copy of develop. We do, however, need to ensure that these release/gfsda.v16.3 changes are properly captured in develop of the repos in which these files now reside.

You know what you changed in release/gfsda.v16.3 so it may be relatively easy for you to sort out what to keep and what to toss.

[emilyhcliu]

@RussTreadon-NOAA Thanks for the tips. I will do a straight forward git merge, double-check the auto merge ones and look through the conflicts.

[RussTreadon-NOAA]

Sounds good, @emilyhcliu . You may list me as a reviewer when we get to the PR step.

[RussTreadon-NOAA]

@emilyhcliu Looking at open GSI issues, it seems closure of this issue will address GSI issue #518. What do you think @aerorahul ?

[aerorahul]

@emilyhcliu Looking at open GSI issues, it seems closure of this issue will address GSI issue #518. What do you think @aerorahul ?

yes, I think it will.

[RussTreadon-NOAA]

@emilyhcliu Looking at open GSI issues, it seems closure of this issue will address GSI issue #518. What do you think @aerorahul ?

yes, I think it will.

I agree. Thanks for the confirmation!

[RussTreadon-NOAA]

@emilyhcliu , fyi

The fix submodule in NOAA-EMC/GSI develop has changed. It now points at NOAA-EMC/GSI-fix. GSI-fix only contains ASCII GSI fix files. GSI binary files are staged on disk on select machines and also available from ftp.emc.ncep.noaa.gov.

A diff of /lfs/h1/ops/prod/packages/gfs.v16.3.4/sorc/gsi.fd/fix/ and NOAA-EMC/GSI-fix develop returns the following files with differences:

diff /lfs/h1/ops/prod/packages/gfs.v16.3.4/sorc/gsi.fd/fix/global_anavinfo.l127.txt develop/global_anavinfo.l127.txt
diff /lfs/h1/ops/prod/packages/gfs.v16.3.4/sorc/gsi.fd/fix/global_convinfo.txt develop/global_convinfo.txt
diff /lfs/h1/ops/prod/packages/gfs.v16.3.4/sorc/gsi.fd/fix/global_satinfo.txt develop/global_satinfo.txt

We need to get the GFS v16.3.4 updates to these ASCII GSI fix files into NOAA-EMC/GSI-fix develop. Doing so requires the following:

• open a GSI-fix issue
• create a personal fork of GSI-fix
• create a branch in your fork
• commit the updated GFS v16.3.4 ASCII fix files to your forked branch
• document the changes in the issue
• open a GSI-fix PR

You may assign the PR to me.

[RussTreadon-NOAA]

As @ADCollard pointed out NOAA-EMC/GSI-fix already has the updated global_convinfo.txt and global_satinfo.txt from GFS v16.3.5. We want to keep these files in NOAA-EMC/GSI-fix.

A closer examination of global_anavinfo.l127.txt in GFS v16.3.4 and v16.3.5 shows it to be behind NOAA-EMC/GSI-fix. The snapshot of global_analvinfo.l127.txt in NOAA-EMC/GSI-fix adds a section for EFSOI

state_vector_efsoi::
!var     level  itracer source     funcof
 u       127      0     met_guess    u
 v       127      0     met_guess    v
 tv      127      0     met_guess    tv
 q       127      1     met_guess    q
 ps        1      0     met_guess    prse
::

We want to retain this section in NOAA-EMC/GSI-fix.

So we do not need to update NOAA-EMC/GSI-fix. The current head of NOAA-EMC/GSI-fix is consistent with the operational GFS.

Sorry @emilyhcliu for the false alarm. Thank you again @ADCollard for catching my mistake.

[emilyhcliu]

@RussTreadon-NOAA I am working on JEDI stuffs today. I will work on merging the v16.3 change with develop this week.

[emilyhcliu]

@RussTreadon-NOAA
I did a drill run to merge the gfsda.v16 with GSI develop.
Here is the documentation

I resolved the conflicts and committed to my local branch. The merged code compiled without problems
Next steps:

• Do a real merge to a branch and create PR
• Update regression for all-sky radiance assimilation
• Run single-cycle test

[emilyhcliu]

A branch was created for the merge.
Files changed between the release/gfsda.v16 and the GSI develop is listed in this document

[KateFriedman-NOAA]

@emilyhcliu @RussTreadon-NOAA Are there any updates on this? Thanks!

[RussTreadon-NOAA]

@emilyhcliu @RussTreadon-NOAA Are there any updates on this? Thanks!

None that I am aware of.

[emilyhcliu]

@RussTreadon-NOAA I am running the regression test one more time. I will turn this draft PR to a real one tonight.