Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/msienkie/prep qc modifications #174

Merged
merged 38 commits into from
Jul 25, 2022

Conversation

gmao-msienkie
Copy link
Contributor

@gmao-msienkie gmao-msienkie commented Apr 18, 2022

Requesting changes for PAQC to be merged into 'develop' and future tags.

The impetus for this request is to add the NCEP changes in CQCVAD need to process the new 'Level 2' data - increases in array sizes. Should be zero diff prior to the new data start (3/30/2022).

There is a small bug fix in fix_ascat.f90 to correctly implement a quality control check. I believe this program is only needed when running with gdas_prepbufr files during the period before 2011/05/09 12z when the ASCATW data was given a purge flag.

Modified the formatting in a few routines - gsidiag_bin2txt.f90, scanbuf2.f

Add some new routines used in MERRA2 flag_NPbuoy.f, twindow.f (twindow.rc, twindow_m2.rc) Also added utility program scanbuf0_accum.f (like scanbuf0 but only prints cumulative counts at the end).

Merged in NCEP changes to bring code up to date with latest NCEP (obsproc_prep.v5.5.0) - prevents.f, cqcbufr.f cqcvad.f profcqc.f

Modified ancient copy of 'read_prepbufr' code in radcor directory to work with current prepbufr format files.

Modified oiqcbufr.f to use 10-digit dates (yyyymmddhh) with the BUFR library . This avoids future problems with the shifting time window for 8-digit dates.

gmao-msienkie and others added 30 commits October 17, 2019 17:34
number of winds in SATWND file for use with MERRA2
both GOES-16 and GOES-17 winds will be reformatted and passed to MERRA2/FPIT
time window changes for two digit BUFR years - prevents.f, profcqc.f
modification for debug compile - cqcbufr.f

prevents.f also added increase in bufr record size (from CVS)
profcqc.f also added increase in max number of records from a station (from a
previous NCEP update in CVS)
aircraft data that is using the current operational table.
… newer

BUFR formats into formats used in MERRA2/RPIT 5_12 to enable them to be
assimilated in runs with the older DAS tags.
Try to bring this branch up to date with current 5_27 changes
…kie/PAQC_modifications_v5_27_1_p3

Bringing minor branch into branch with collection of PAQC fixes
Update branch with current DAS version
cumulative count of each subset at the end rather than counting subset
chunks as they are read in.
…rray

(Probably doesn't make any difference, since gross limits so large.  Also
ASCAT is likely no longer flagged.)
…mperature

and wind observtions) for fixed buoys with 90N (wrong) latitude
@gmao-msienkie gmao-msienkie requested review from a team as code owners April 18, 2022 18:19
@gmao-msienkie
Copy link
Contributor Author

Sorry about including all the miscellaneous files - if necessary I can pull some of these off onto separate requests. After the CQCVAD problem I thought it would be a good idea to catch up with changes that I had been accumulating, to bring us up to date with NCEP

@gmao-msienkie gmao-msienkie added the Non 0-diff The changes in this pull request are non-zero-diff label Apr 18, 2022
tclune
tclune previously approved these changes Apr 18, 2022
Copy link
Collaborator

@tclune tclune left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cmake changes ok

@gmao-msienkie
Copy link
Contributor Author

Technically the changes are non-zero diff. It is extremely unlikely for the modification in fix_ascat to trigger a change in the output and more than likely that the data formerly unflagged would be used in the assimilation.
The changes in cqcvad.f to increase the array size are only non-zero after the start of the new data at the end of March 2022.

@gmao-msienkie
Copy link
Contributor Author

I ran some tests with this code and compared with results from running with code from a 5_29_4 tag. It looks like the cqcvad.f already had changes to increase array sizes versus the code in MERRA2. The BUFR files resulting from running gmao_prepqc were zero diff. It would still be good to bring in the various prepqc changes in this tag so we can be prepared for possible future changes.

@rtodling
Copy link
Collaborator

Meta, I am bring this in, ok?

@gmao-msienkie
Copy link
Contributor Author

I modified CQCVAD to increase the maximum number of reports, and I tested the code to see there are zero diffs for files with less than 50000 levels and that files with more than that number of levels are handled correctly by the new code.
I merged the current develop branch and pushed my changes to GitHub.

Copy link
Collaborator

@rtodling rtodling left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you Meta.

@rtodling rtodling merged commit 3efd8a3 into develop Jul 25, 2022
@gmao-msienkie gmao-msienkie deleted the feature/msienkie/prepQC_modifications branch January 31, 2023 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug fix Non 0-diff The changes in this pull request are non-zero-diff
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants