Added support of the NSSL CCPP suite - WoFS_v0 and WE2E tests #708
Conversation
There was a problem hiding this comment.
Thank you for this PR. My understanding from Curtis' spreadsheet is that the WoFS suite should use Noah LSM (instead of NoahMP), which will require PRs to fv3atm and ufs-weather-model to either change or add a suite and associated regression test.
Regarding the name of the suite, FV3_WoFS seems reasonable to me, but I am open to alternatives.
Yes, the v1nssl suite is a stand-in at the moment. We will probably just replace that suite in fv3atm with FV3_WoFS (or whatever) and then also convert the current RT items to use the new suite. The question I have is about the gravity wave drag settings. Our current suite (FV3_RRFS_v1nssl_lsmnoah, for the daily NSSL-FV3 runs) uses Noah LSM and has cires_ugwd. I know basically nothing about these schemes, (e.g., SSGWD/TOFD seems to refer to ugwpv1_gsldrag?) I'm also not really clear what effect GFS_radiation_surface and mynnsfc_wrapper have vs. just sfc_diff. We can test whatever is recommended there. |
|
@MicroTed Can you please restate your questions? They were not clear to me. My understanding is that the suite with the NSSL mp will use cires_ugwp (same as RRFSv1_beta). Do you have a concern about that? Regarding the choice of surface layer scheme, are you saying that you have only tested the NSSL mp wit the GFS surface layer scheme, and you have little experience about its behavior when used with the MYNN surface layer scheme? |
|
@ligiabernardet Sorry -- I put a lot into one comment. The NSSL suite does include cires_ugwp, so that is fine. One question is, what is the difference between UGWv0 and SSGWD/TOFD? I guess these are namelist options, like do_ugwp and do_gsl_drag_tofd? |
…iton files for suite FV3_WoFS_v0
...rn_mdls_suites_community/config.grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_WoFS_v0.sh
Outdated
Show resolved
Hide resolved
...ds_extrn_mdls_suites_community/config.grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_WoFS_v0.sh
Outdated
Show resolved
Hide resolved
...ds_extrn_mdls_suites_community/config.grid_RRFS_CONUS_3km_ics_HRRR_lbcs_RAP_suite_WoFS_v0.sh
Outdated
Show resolved
Hide resolved
|
@ywangwof For the 3 new WE2E tests that you're including, config.grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_WoFS_v0.sh is there a reason to use different external models with the different resolutions (FV3GFS vs. NAM vs. HRRR/RAP)? |
|
No a specific reason. I just want to test the possible resolution 25 km/13 km/3 km and external models (FV3GFS/NAM/HRRR) combinations. Do you think I should add 9 test configurations, or just 3 tests with the same external model? After today's discussion, I learned that the workflow E2E tests are not a set of comprehensive tests. Considering the main purpose of the Warn-on-Forecast project, I would like to add two test configurations only with this PR config.grid_RRFS_SUBCONUS_3km_ics_HRRR_lbcs_RAP_suite_WoFS_v0.sh Any further suggestion or comments? Thanks. |
@ywangwof Sure. Is it just to be able to test on a smaller domain? FYI I'm going to add the 3km grid over Indianapolis probably tomorrow. |
|
@ywangwof I didn't mean to delay merging of this PR by my grid suggestion. Let's get this in on whichever grids are currently available in the App. We can then switch the tests to other grids if that makes more sense. Do the tests config.grid_RRFS_SUBCONUS_3km_ics_HRRR_lbcs_RAP_suite_WoFS_v0.sh work right now? If so, I can approve. |
|
config.grid_RRFS_SUBCONUS_3km_ics_HRRR_lbcs_RAP_suite_WoFS_v0.sh works well. |
|
@ywangwof I am ok with having only HRRR/RAP support in the App for now since it's better than nothing. I will approve based on that. Could you put the changes that are needed to the namelist to get FV3GFS to ICs/LBCs to work (that you sent in an email) in this PR or in a new issue so we know what needs to be done? Thanks. |
|
In order for FV3GFS ICs/LBCs to work with WoFs suite, we should set |
I think there is a simple fix for this by setting the surface_value to zero for water_nc in the field_table (it needs to be zero, anyway, for just this circumstance where water_nc is not in the IC) |
@ligiabernardet My thinking was that the RT for v1nssl already tests the microphysics, so it wasn't clear that we need an additional RT for the wofs_v0 suite. If needed, I have changes to retool the v1nssl reg. test for wofs_v0. I'm in the field this week, so I'm behind on a few things. |
* Add Gaea as a supported platform for the regional_workflow (ufs-community#734) * Updates to port regional workflow to gaea * Temp change with -v as batch option * new fixes for gaea/slurm * Updated time for make lbcs * added TEST data directory path * Update gaea.sh * fixes for PR * Add more parameters to CSV file containing WE2E test info (ufs-community#740) ## DESCRIPTION OF CHANGES: The script/function `get_WE2Etest_names_subdirs_descs.sh` (which is called from `run_WE2E_tests.sh` if needed) creates a CSV (Comma-Separated Value) file named `WE2E_test_info.csv` that contains information about the WE2E tests. Currently, this CSV file contains only 3 columns: the test name, the names of any alternate names for the test, and the test description. In order to have a more complete summary of the WE2E tests, this PR modifies `get_WE2Etest_names_subdirs_descs.sh` so that additional information is included in the CSV file. This additional information consists of the values of the following experiment variables for each test: ``` PREDEF_GRID_NAME CCPP_PHYS_SUITE EXTRN_MDL_NAME_ICS EXTRN_MDL_NAME_LBCS DATE_FIRST_CYCL DATE_LAST_CYCL CYCL_HRS INCR_CYCL_FREQ FCST_LEN_HRS LBC_SPEC_INTVL_HRS NUM_ENS_MEMBERS ``` In addition, the script uses this information to calculate the number of times each test calls the forecast model (e.g. if the test uses 3 different cycle dates, then the forecast model will be called 3 times; if it is an ensemble test for a single cycle, the test will call the forecast model as many times as the number of ensemble members). ## TESTS CONDUCTED: The script `run_WE2E_tests.sh` was called that in turn calls `get_WE2Etest_names_subdirs_descs.sh`. This created a new CSV file that contained the new fields (columns). The CSV file was imported into Google Sheets (using "|" as the field/column separator) and looked correct. ## DOCUMENTATION: The documentation is for the most part already within the `get_WE2Etest_names_subdirs_descs.sh`. This PR slightly modifies that documentation to update it. * Update directory structure of NCO mode (ufs-community#743) * update vertical structure of NCO mode * update sample script for nco * Fix typo on write component of new RRFS CONUS * Default CCPP physics option is FV3_GFS_v16 (ufs-community#746) * Updated the default CCPP physics option to FV3_GFS_v16 * Updated the default CCPP physics option to FV3_GFS_v16 in config_defaults.sh Co-authored-by: Natalie Perlin <Natalie@Natalies-MacBook-Air.local> * Adds an alternative python workflow generation path (ufs-community#698) * Workflow in python starting to work. * Use new python_utils package structure. * Some bug fixes. * Use uppercase TRUE/FALSE in var_dfns * Use config.sh by default. * Minor bug fixes. * Remove config.yaml * Update to the latest develop * Remove quotes from numbers in predef grid. * Minor bug fix. * Move validity checker to the bottom of setup * Add more unit tests. * Update with python_utils changes. * Update to latest develop additions (Need to re-run regression test) * Use set_namelist and fill_jinja_template as python functions. * Replace sed regex searches with python re. * Use python realpath. * Construct settings as dictionary before passing to fill_jinja and set_namelist * Use yaml for setting predefined grid parameters. * Use xml parser for ccpp phys suite definition file. * Remove more run_command calls. * Simplify some func argument processing. * Move different config format parsers to same file. * Use os.path.join for the sake of macosx * Remove remaining func argument processing via os.environ. * Minor bug fix in set_extrn_mdl_params.sh * Add suite defn in test_data. * Minor fixes on unittest on jet. * Simplify boolean condition checks. * Include old in renaming of old directories * Fix conflicting yaml !join tag for paths and strings. * Bug fix with setting sfcperst dict. * Imitate "readlink -m" with os.path.realpath instead of os.readlink * Don't use /tmp as that is shared by multiple users. * Bug fix with cron line, maintain quotes around TRUE/FALSE. * Update to latest develop (untested) * Bug fix with existing cron line and quotes. * Bug fix with case-sensitive MACHINE name, and empty EXPT_DIR. * Update to latest develop * More updates. * Bug fix thanks to @willmayfield! Check both starting/ending characters are brackets for shell variable to be considered an array. * Make empty EXPT_BASEDIR workable. * Update to latest develop * Update in predef grid. * Check f90nml as well. Co-authored-by: Daniel Abdi <dabdi@Orion-login-2.HPC.MsState.Edu> * Fix typo and crontab issue on wcoss dell in workflow python scripts (ufs-community#750) * Fix typo and failure on wcoss * fix new line issue on wcoss dell * remove capture_output * Get USER from environment Co-authored-by: Daniel Abdi <daniel.abdi@noaa.gov> * Add new WE2E configs (ufs-community#748) ## DESCRIPTION OF CHANGES: Added two new WE2E config files for the Sub-CONUS Indianapolis domain to support the upcoming SRW release. In addition, modified the external data used in the `config.specify_EXTRN_MDL_SYSBASEDIR_ICS_LBCS.sh` to match more common datasets used in the WE2E testing process. ## TESTS CONDUCTED: Successfully ran the new WE2E tests (`config.SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_HRRR.sh`, `config.SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta.sh`) and `config.specify_EXTRN_MDL_SYSBASEDIR_ICS_LBCS.sh` on NOAA Parallel Works AWS instance. ## DEPENDENCIES: None. ## DOCUMENTATION: No documentation changes are required. * Added a fixed WoF grid and the python tool to determine the write component parameters (ufs-community#733) * Added a fixed WoF grid and the python tool to determine the write component parameters * Update set_predef_grid_params.sh * Renamed file as recommended and removed unused lines * Modified comment Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com> Co-authored-by: WYH@MBP <yunheng.wang@4374-ywang.winstorm.nssl> * Replace env with modulefiles in scripts (ufs-community#752) * change env to mod * update we2e script * WE2E script improvements for usability (ufs-community#745) ## DESCRIPTION OF CHANGES: * Modifications to `run_WE2E_tests.sh`: * Add examples to help/usage statement * Modifications to `check_expts_status.sh`: * Add arguments list that can be processed by `process_args` * Add new optional arguments: `num_log_lines`, `verbose` * Include a help/usage message ## TESTS CONDUCTED: * Ran `run_WE2E_tests.sh --help` from the command line and got the expected help message. * Ran `check_expts_status.sh --help` from the command line and got the expected help message. * Used `run_WE2E_tests.sh` to run a set of 2 WE2E tests -- works as expected. * Used `check_expts_status` to check on the status of the 2 tests run above and got the expected status message. ## DEPENDENCIES: PR #[241](ufs-community/ufs-srweather-app#241) ## DOCUMENTATION: A lot of this PR is documentation in the scripts. There is an accompanying documentation PR #[241](ufs-community/ufs-srweather-app#241) into ufs-srweather-app. * Standardize static data across Tier-1 platforms; fix and improve IC and LBC data retrieval (ufs-community#744) * Bug fixes (grid size + suppress screen output from module load) (ufs-community#756) ## DESCRIPTION OF CHANGES: 1) Adjust y-direction size of write-component grid of `SUBCONUS_Ind_3km` predefined grid from 195 to 197 (this was just an oversight in PR ufs-community#725 ). 2) Redirect output of module load in launch script (`launch_FV3LAM_wflow.sh`) to `/dev/null` to avoid unwanted screen output (which was introduced in PR #[238](ufs-community/ufs-srweather-app#238) in ufs-srweather-app and is about how to load the `regional_workflow` environment and is not relevant in this context). ## TESTS CONDUCTED: 1) Plotted the `SUBCONUS_Ind_3km` grid to ensure it has correct size (it does). 2) Manually ran `launch_FV3LAM_wflow.sh` from the command line to verify that screen output is suppressed (it is). * Update default SPP ISEED array in config_defaults.sh to use unique values (ufs-community#759) * Modify RRFS North America 3- and 13-km domain configuration and WE2E test. * Modify default ISEED values for SPP * Fix grid in WE2E test * Update workflow python scripts (ufs-community#760) * update python scripts * Change output file name of run_post to meet NCO standards (ufs-community#758) * change output file name * change variable name * update python script * remove duplicates * add a check for empty variables * move variable to common area * clean up unnecessary comments * update scripts * remove duplicate * update python scripts * fix user-staged dir path issue in python script * Add POST_OUTPUT_DOMAIN_NAME to WE2E tests for new grids (ufs-community#763) * Add new var to we2e tests for new grids * rename we2e tests for custom grid * remove unnecessary $ Co-authored-by: Mark Potts <33099090+mark-a-potts@users.noreply.github.com> Co-authored-by: gsketefian <31046882+gsketefian@users.noreply.github.com> Co-authored-by: Chan-Hoo.Jeon-NOAA <60152248+chan-hoo@users.noreply.github.com> Co-authored-by: Natalie Perlin <68030316+natalie-perlin@users.noreply.github.com> Co-authored-by: Natalie Perlin <Natalie@Natalies-MacBook-Air.local> Co-authored-by: danielabdi-noaa <52012304+danielabdi-noaa@users.noreply.github.com> Co-authored-by: Daniel Abdi <dabdi@Orion-login-2.HPC.MsState.Edu> Co-authored-by: Daniel Abdi <daniel.abdi@noaa.gov> Co-authored-by: EdwardSnyder-NOAA <96196752+EdwardSnyder-NOAA@users.noreply.github.com> Co-authored-by: Yunheng Wang <47898913+ywangwof@users.noreply.github.com> Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com> Co-authored-by: WYH@MBP <yunheng.wang@4374-ywang.winstorm.nssl> Co-authored-by: Michael Kavulich <kavulich@ucar.edu>
|
A new PR #772 to branch release/public-v2 was issued. So this PR is closed and waiting for a final solution for restart reproducibility when blending > 0 with the ufs-weather-model repository. Just note that the restart reproducibility issue is not unique to this CCPP suite (WoFS_v0), but is a common bug for all regional workflows. |
DESCRIPTION OF CHANGES:
Added support for the NSSL CCPP suite, WoFS_v0 in the develop branch.
TESTS CONDUCTED:
Tested on Odin using Intel 2020 and Jet with the predefined grid RRFS_CONUS_3km and RRFS_SUBCONUS_3km.
DEPENDENCIES:
It depends on the fv3atm PR #514.
DOCUMENTATION:
Will study the document to update when the CCPP suite name is finalized.
MODIFIED AND NEW FILES:
ISSUES:
CONTRIBUTORS (optional):
Ted Mansell (@MicroTed) provided the initial runtime files for this workflow.