Lightning observation processing tasks are needed for RRFS

[christinaholtNOAA]

Description

RRFS runs an observation pre-processing step on Vaisala lightening data in netcdf format. Specifically, the process_lightening task runs the process_Lightning.exe from rrfs-utl on files named like ${YYJJJHH}${n}0005r on Jet.

Please reference the exregional_process_lightning.sh script in RRFS_dev1 for specifics.

Solution

Use the RRFS_dev1 version of this task as a guide to a solution. Include any necessary default configuration settings, add j-jobs, and ex-scripts, and hook them into the workflow.

This task has no dependencies other than data retrieval, so should be able to stand alone as a full workflow without other components.

Requirements**

• All contributions must be machine independent and follow the style and guidelines of the SRW App.
• The contributed scripts must be tested, and new tests should be contributed for SRW. These WE2E tests can be short and only include data retrieval and running the task.

Acceptance Criteria (Definition of Done)

Matches answers obtained in RRFS. Please reach out to someone at NOAA GSL to help with this test case.

[EdwardSnyder-NOAA]

Daniel's PR-540 encompasses a lot of work the lightning obs processing task would need. So, I merged my changes for this task into his PR. The lightning task does work but per the acceptance criteria, I would need to match the results from a RRFS test case. @christinaholtNOAA, would you be able to help me get a case so I can test the lightning obs data against?

[christinaholtNOAA]

These changes work in Daniel's PR, but plans are to pull these changes out into a separate small PR.

Need to be able to test one cycle, and having trouble. Will open PR to discuss that problem.

[MatthewPyle-NOAA]

Eddie reported an issue with a missing item (fv3_grid_spec) in his testing. Believe this is actually a fixed file - need to sort out why it is reaching the working directory in RRFS_dev1 but not for this test case.

[christinaholtNOAA]

Iterating through PR questions/comments.

Needs dataset for running .
Tested with a wrapper. Still need to add changes for default config settings.
Request to add a new PR for the rrfs_utl fix .

[christinaholtNOAA]

Preparing changes for re-review. Eddy plans to stage some test cases under the test data area, and use the July 2022 path.

fv3_grid_spec files are technically pre-generated files that come out of the forecast step. Soft-linked C403 mosaic halo file...

There are 2 different grid spec files. There's one that's in the input, but also one that comes out of the forecast. Let's ask Ming Hu about the differences.

Don't worry about support for io_layouts other than 1 for now.

[EdwardSnyder-NOAA]

Ming suggested to use the fv3_grid_spec under expt_name/expt_date instead of the one found under the INPUT directory.

The changes have been addressed and the PR is ready for review.

[christinaholtNOAA]

No changes to the workflow, so it shouldn't be an issue with XML/YAML. Can go after the variables PR.