Updates to aerosol RTs - new dust and fcsav #1170
Conversation
These changes brings the regression tests closer to the p8c target configuration. Changes include: - Updating fractional clay and sand to the SOILGRIDSv2 data - REF: https://soil.copernicus.org/articles/7/217/2021/ - Silt isn't actually used in the scheme. Set to /dev/null:0.0 to skip reading a file and instead fill with zeros - SSM is no longer used as the albedo drag partition describes the same information to the scheme. Set to /dev/null:1.0 so that all values are unity - `DU_RDRAG` now uses a three year climatology of the Chappel and Webb 2016 albedo drag partition derived from the latest 3 years of MODIS BRDF Albedo monthly averages - `DU_UTHRES` now uses a more robust threshold velocity derived from known historical dust point observations using a data driven approach from the combination of a met reanalysis, SOILGRIDSv2, and a RandomForest ensemble
This updates the `alpha` parameter to be consistent with the latest dust code changes and inputs Note that gamma here doesn't matter because the `ssm=1` now in the `AERO_ExtData.rc` file
This fix adds the environment vairable FSCAV_AERO so that the wet scavaging coeffients are not by default set to zero ensuring that the convective parameterizations are fully tested when aerosols are used.
|
@DeniseWorthen Can you point to where I should make this modification in default_vars.sh? I am not entirely familiar with this. |
|
@bbakernoaa You could add them here ufs-weather-model/tests/default_vars.sh Line 1247 in cb84926 All the coupled P8 tests will then use them. EDIT: I see now that |
This was rolled back because it is now in default_vars.sh
should it be here though? ufs-weather-model/tests/default_vars.sh Line 922 in cb84926 |
The earlier section is for FV3 standalone. If you scroll back, you will see it is in the block |
|
@DeniseWorthen Thanks just making sure. |
|
@bbakernoaa please bring these up to date with respective authoritative repositories
|
|
@bbakernoaa During the call, I was trying to understand whether we can combine your update to the gocart files that you are showing above at the time we update the ICs. Or, are the other updates listed here (to tests/parm) required at the same time as the update to input-data. |
|
@DeniseWorthen The updates to Maybe I just don't understand the question completely. |
|
Are you asking if we do cp /scratch1/NCEPDEV/rstprod/nexus_emissions/FENGSHA/FENGSHA_Albedo_drag_v1.nc /scratch1/NCEPDEV/nems/emc.nemspara/RT/NEMSfv3gfs/input-data-20211210/GOCART/ExtData/dust/
cp /scratch1/NCEPDEV/rstprod/nexus_emissions/FENGSHA/FENGSHA_SOILGRIDS2019_GEFSv12_v1.2.nc /scratch1/NCEPDEV/nems/emc.nemspara/RT/NEMSfv3gfs/input-data-20211210/GOCART/ExtData/dust/
cp /scratch1/NCEPDEV/rstprod/nexus_emissions/FENGSHA/randomforestensemble_uthres.nc /scratch1/NCEPDEV/nems/emc.nemspara/RT/NEMSfv3gfs/input-data-20211210/GOCART/ExtData/dust/
will this change the current RTs? If so no. This will not impact the current regression tests since we did not change the tests/parm/gocart here. This would not overwrite anything in the |
|
I'll be creating a new input-data directory for the P8c ICs as part of PR #909. I was hoping to combine your update to the input-data directory and avoid having to make another new input-data directory later. You have a fix for scavenging, which I understand is independent (a namelist/bug fix). But you're also updating two files in tests/parm. One looks like a tweak to a number (the scaling factor). Does the other change (tests/parm/gocart/AERO_ExtData.rc) depend on having the new input-data files in place? If the change to the AERO_ExtData.rc depends on the input data, I think I need to make that change as part of my PR to update to the P8c ICs. |
|
@DeniseWorthen I think I understand what you are asking now. Yes this can be part of your other PR #909. This wouldn't be a problem and should be done before this change for the RTs so that the data is there. |
|
@bbakernoaa Great. Please make a PR to my PR #909 with the changes you need in tests/parm/gocart |
|
@bbakernoaa Sorry--I may be confused here. There are no existing files w/ those names in the current input data. But neither are there files which you appear to be changing (eg FENGSHA_DUST_INPUTS_v0.6.nc) in the current directory. The only dust file currently present is gocart.dust_source.v5a.x1152_y721.nc. You are not replacing that file, just adding additional dust files. So I think there is nothing that needs to be done in my PR, correct? |
|
@bbakernoaa The three files FENGSHA_Albedo_drag_v1.nc, FENGSHA_SOILGRIDS2019_GEFSv12_v1.2.nc and randomforestensemble_uthres.nc are under /scratch1/NCEPDEV/nems/emc.nemspara/RT/NEMSfv3gfs/input-data-20220414/GOCART/ExtData/dust. Also if you'd like to change the FSCAV_AERO with same value for all the cpld aerosol tests and, you just need change in defaultvars.sh. You don't need to change it in each test unless the test is using a different setting. Thanks |
|
@DeniseWorthen @junwang-noaa @rmontuoro @bbakernoaa I am fine with this PR. Thanks. |
|
We probably have everything needed for the manually updating the INTEL develop-20220517. The auto-bl files haven't been moved (but I can see them) and I created baselines in order to test the new default vars. We can either manually combine those two directories and run auto-rt (after barry commits the new default-vars), or do the whole auto-bl. |
Much faster copying the files over yourself and running the AutoRT RT task. |
|
OK, let me get stuff moved. I've pushed the default_var changes now. |
|
@MinsukJi-NOAA I think we need to re-run the GNU on Cheyenne as well, because I changed the c96 resources which are used by both the cpld_debug test and the GNU-based noaero tests. |
@DeniseWorthen Okay. I will create both rt labels when the baseline is ready. |
|
@MinsukJi-NOAA I've moved everything. I think we're ready for cheyenne.intel and gnu auto-RT. |
on-behalf-of @ufs-community <brian.curtis@noaa.gov>
|
The cpl_mpi_p8 test has failed; I messed up the default vars for this test. I will need to re-run. A couple of other tests are just waiting in the Q. |
|
Automated RT Failure Notification |
|
@rmontuoro @lipan-NOAA @DeniseWorthen Please review the PR as it passed all regression tests and is ready for commit. |
tests/rt.sh
Outdated
| @@ -531,10 +531,9 @@ mkdir ${LOG_DIR} | |||
| if [[ $ROCOTO == true ]]; then | |||
|
|
|||
| ROCOTO_XML=${PATHRT}/rocoto_workflow.xml | |||
| ROCOTO_STATE=${PATHRT}/rocoto_workflow.state | |||
There was a problem hiding this comment.
Why is this line removed?
tests/rt.sh
Outdated
| ROCOTO_DB=${PATHRT}/rocoto_workflow.db | ||
|
|
||
| rm -f $ROCOTO_XML $ROCOTO_DB $ROCOTO_STATE *_lock.db |
There was a problem hiding this comment.
Why is $ROCOTO_STATE removed?
There was a problem hiding this comment.
@DusanJovic-NOAA Honestly I don't know when that was changed. It may have come from when I pulled in the latest changes from the develop branch. Looking back a the log history I didn't modify that. I have no problem adding it back in if that is what we should do.
There was a problem hiding this comment.
@DusanJovic-NOAA it seems to me that these lines were not there until @SamuelTrahanNOAA 's PR 1195.
There was a problem hiding this comment.
@bbakernoaa Please revert those changes. It is okay to make these changes at this point in the commit since they do not affect the regression test results.
|
@bbakernoaa Can you please address @DusanJovic-NOAA 's question on ROCOTO modification above? |
|
Additional tests were run for The run directories are
|
|
@bbakernoaa The resource increase is caused by these changes in |
|
This is undoubtedly where the "issue" is as it was really the only thing that was modified. But this isn't a problem with the model at all. It is behaving and performing exactly as expected. After thinking about it, the "issue" is in the resolution of the input files. I created higher resolution files so that they could be used throughout the applications including RRFS-SMOKE as the FENGSHA scheme is also apart of that implementation. I didn't think about the increased resources in memory usage when I did this. If the increased resources are a large issue then I can create some lower resolution files that would be the same resolution that was previously used. |
|
@bbakernoaa I believe someone mentioned at the coupled tag-up an issue about fields being stored at full resolution in the model (all months, all vertical levels) but it wasn't clear to me at the time if the comment applied to fields used by the aerosol scheme or not. Do you think it applies in this case at all? In other words, when these files are read in, what is the dimensionality of the fields which are internal to the model? |
|
@DeniseWorthen I think that people are getting confused about exactly what is happening. The ExtData child component reads the data in at the native file input resolution. This is not stored internal to the dust child component of GOCART. Rather when the data is passed to the dust component it is regridded to the native model resolution. Like I said if we need to reduce it back to the original memory footprint I can preprocess a the inputs with a degradation in the input resolution to back it down to what was there previously. This isn't a model problem but rather a RT requirement problem. |
|
@bbakernoaa @rmontuoro @yangfanglin @SMoorthi-emc It looks to me the timing increase is during integration, not the initialization. test cpld_control_p8_change1-2 ( with the changes in this PR) |
PR Checklist
This PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR. Please consult the ufs-weather-model wiki if you are unsure how to do this.
This PR has been tested using a branch which is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR
An Issue describing the work contained in this PR has been created either in the subcomponent(s) or in the ufs-weather-model. The Issue should be created in the repository that is most relevant to the changes in contained in the PR. The Issue and the dependent sub-component PR
are specified below.
Results for one or more of the regression tests change and the reasons for the changes are understood and explained below.
New or updated input data is required by this PR. If checked, please work with the code managers to update input data sets on all platforms.
Description
First there are new settings for dust aerosols that need to be updated in the
tests/parm/gocart/AERO_ExtData.rc.Second the
tests/parm/gocart//DU2G_instance_DU.rcfile needs an updated alpha scaling factor for the dust emission. Changes from0.7to0.04Adds convective wet scavenging for coupled model runs with aerosols. This doesn't impact other schemes and is done by exporting the variable
FSCAV_AEROin the tests/tests/cpld* RT configuration files.There are new input files that will need to be copied to the INPUT root outside of this repository.
cp /scratch1/NCEPDEV/rstprod/nexus_emissions/FENGSHA/FENGSHA_Albedo_drag_v1.nc /scratch1/NCEPDEV/nems/emc.nemspara/RT/NEMSfv3gfs/input-data-20211210/GOCART/ExtData/dust/cp /scratch1/NCEPDEV/rstprod/nexus_emissions/FENGSHA/FENGSHA_SOILGRIDS2019_GEFSv12_v1.2.nc /scratch1/NCEPDEV/nems/emc.nemspara/RT/NEMSfv3gfs/input-data-20211210/GOCART/ExtData/dust/cp /scratch1/NCEPDEV/rstprod/nexus_emissions/FENGSHA/randomforestensemble_uthres.nc /scratch1/NCEPDEV/nems/emc.nemspara/RT/NEMSfv3gfs/input-data-20211210/GOCART/ExtData/dust/Issue(s) addressed
Testing
How were these changes tested? What compilers / HPCs was it tested with? Are the changes covered by regression tests? (If not, why? Do new tests need to be added?) Have regression tests and unit tests (utests) been run? On which platforms and with which compilers? (Note that unit tests can only be run on tier-1 platforms)