Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add system and unit tests for making fsurdat with all crops everywhere #2081

Merged
merged 36 commits into from
Sep 19, 2023
Merged

Add system and unit tests for making fsurdat with all crops everywhere #2081

merged 36 commits into from
Sep 19, 2023

Conversation

samsrabin
Copy link
Collaborator

@samsrabin samsrabin commented Jul 31, 2023 •
edited by ekluzek
Loading

Description of changes

Add unit test for Python code that makes a surface dataset with (a) all natural land moved into cropland and (b) cropland split evenly among all CFTs. This special file is used to generate prescribed crop maturity requirements, with the idea being that we want a value for every crop in every gridcell regardless of whether the crop is currently there.

Previously, this was done using make_fsurdat_all_crops_everywhere.py. However, I realized that this functionality fits well in the existing fsurdat_modifier tool, which conveniently has its test infrastructure all set up already. I am thus moving this capability there by adding the evenly_split_cropland option, which in combination with setting PCT_CROP = 100.0 does the same thing as make_fsurdat_all_crops_everywhere.py.

Also adds a new 10x15 RXCROPMATURITY test, in aux_clm and clm_pymods suites.

Specific notes

Contributors other than yourself, if any: None

CTSM Issues Fixed:
Fixes #2079

Are answers expected to change (and if so in what way)? No

Any User Interface Changes (namelist or namelist defaults changes)? No

Testing performed:

  • test_sys_fsurdat_modifier.py and test_unit_fsurdat_modifier.py, which both got new and/or updated tests, both pass.
  • test_unit_singlept_data_surfdata*.py scripts, which also exercise fsurdat modification code, also pass.
  • clm_pymods tests all pass.

samsrabin added 2 commits July 28, 2023 18:15
@samsrabin
make_fsurdat_all_crops_everywhere.py: Improve efficiency.
de43fd1 
Only write the variables we need to change, instead of the entire dataset.
@samsrabin
make_fsurdat_all_crops_everywhere.py: Automatic naming of output file…
2428d76 
… to reflect input file.
@samsrabin samsrabin added the testing additions or changes to tests label Jul 31, 2023
@samsrabin samsrabin self-assigned this Jul 31, 2023
@samsrabin
Reformatting with black.
0a5a9e8
@samsrabin
Added previous commit to .git-blame-ignore-revs.
e803501
@samsrabin
Added evenly_split_cropland option to fsurdat_modifier.py.
10289ad 
This could also be accomplished by using dom_pft to specify an equal value for all crop PFTs. However, this method is more robust to user error (we don't have to rely on the user correctly writing out a long string of numbers) and code updates (if more crop PFTs are added).
@samsrabin
RXCROPMATURITY now uses fsurdat_modifier.
a7290fb 
Instead of make_fsurdat_all_crops_everywhere.py, which has been removed.
@samsrabin samsrabin changed the title Add unit test for make_fsurdat_all_crops_everywhere.py Add unit test for making fsurdat with all crops everywhere Aug 2, 2023
@samsrabin samsrabin force-pushed the make-fsurdat-ace-dev branch from a891b19 to e613405 Compare August 2, 2023 22:39
@samsrabin
Copy link
Collaborator Author

Just waiting for Cheyenne to come back so I can test RXCROPMATURITY with the changes.

@samsrabin samsrabin changed the title Add unit test for making fsurdat with all crops everywhere Add systems and unit tests for making fsurdat with all crops everywhere Aug 2, 2023
@samsrabin samsrabin changed the title Add systems and unit tests for making fsurdat with all crops everywhere Add system and unit tests for making fsurdat with all crops everywhere Aug 2, 2023
@samsrabin samsrabin mentioned this pull request Aug 4, 2023
Closed
@samsrabin samsrabin marked this pull request as ready for review August 7, 2023 15:13
@samsrabin samsrabin requested a review from slevis-lmwg August 9, 2023 20:05
ekluzek
ekluzek approved these changes Aug 10, 2023
View reviewed changes
Copy link
Collaborator

@ekluzek ekluzek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is great. It adds a useful feature for general use in these scripts with a nicely named option evenly_split_cropland which is self descriptive.

ekluzek added 2 commits August 10, 2023 00:32
@ekluzek
Merge tag 'ctsm5.1.dev133' into make-fsurdat-ace-dev
33d553b 
FATES API update to facilitate fates refactor

This updates a number of FATES type names and module use statements
which correspond with a refactoring effort that moves FATES
patches and cohorts into their own respective modules.

With the FATES update is a minor science update, so there are
changes to answers for FATES.

This also incorporates a minor update to a more recent version
of the ccs config external.
@ekluzek
Merge branch 'make-fsurdat-ace-dev' of github.com:samsrabin/CTSM into…
cee3c26 
… make-fsurdat-ace-dev
@samsrabin samsrabin added PR status: ready PR: this is ready to merge in, with all tests satisfactory and reviews complete tag: simple bfb labels Sep 7, 2023
samsrabin added a commit that referenced this pull request Sep 19, 2023
@samsrabin
Merge 5 bit-for-bit pull requests:
2e2434d 
* Add system and unit tests for making fsurdat with all crops everywhere (#2081)
* Rework master_list* files etc. (#2087)
* Fixes to methane Tech Note (#2091)
* Add is_doy_in_interval() function (#2158)
* Avoid using subprocess.run() in FSURDATMODIFYCTSM (#2125)

Closes issues:
* Add unit test for making fsurdat with all crops everywhere (#2079)
* Rework master_list_(no)?fates.rst? (#2083)
* conda run -n can fail if a conda environment is already active (#2109)
* conda fails to load for SystemTests (#2111)
@samsrabin samsrabin merged commit cdcc35e into ESCOMP:master Sep 19, 2023
samsrabin added a commit to samsrabin/CTSM that referenced this pull request Sep 19, 2023
@samsrabin
Merge tag 'ctsm5.1.dev142' into newcropcals-rxsowwindow
8537757 
b4b changes to Python scripts, history lists, tech note, and clm_time_manager.

* Add system and unit tests for making fsurdat with all crops everywhere (ESCOMP#2081)
* Rework master_list* files etc. (ESCOMP#2087)
* Fixes to methane Tech Note (ESCOMP#2091)
* Add is_doy_in_interval() function (ESCOMP#2158)
* Avoid using subprocess.run() in FSURDATMODIFYCTSM (ESCOMP#2125)

Closes issues:
* Add unit test for making fsurdat with all crops everywhere (ESCOMP#2079)
* Rework master_list_(no)?fates.rst? (ESCOMP#2083)
* conda run -n can fail if a conda environment is already active (ESCOMP#2109)
* conda fails to load for SystemTests (ESCOMP#2111)
samsrabin added a commit to samsrabin/CTSM that referenced this pull request Sep 20, 2023
@samsrabin
Merge tag 'ctsm5.1.dev142' into newcropcals-rxsowwindow
ff687dc 
b4b changes to Python scripts, history lists, tech note, and clm_time_manager.

* Add system and unit tests for making fsurdat with all crops everywhere (ESCOMP#2081)
* Rework master_list* files etc. (ESCOMP#2087)
* Fixes to methane Tech Note (ESCOMP#2091)
* Add is_doy_in_interval() function (ESCOMP#2158)
* Avoid using subprocess.run() in FSURDATMODIFYCTSM (ESCOMP#2125)

Closes issues:
* Add unit test for making fsurdat with all crops everywhere (ESCOMP#2079)
* Rework master_list_(no)?fates.rst? (ESCOMP#2083)
* conda run -n can fail if a conda environment is already active (ESCOMP#2109)
* conda fails to load for SystemTests (ESCOMP#2111)
samsrabin added a commit to samsrabin/CTSM that referenced this pull request Sep 21, 2023
@samsrabin
Merge tag 'ctsm5.1.dev142' into newcropcals-rxsowwindow
1382c5d 
b4b changes to Python scripts, history lists, tech note, and clm_time_manager.

* Add system and unit tests for making fsurdat with all crops everywhere (ESCOMP#2081)
* Rework master_list* files etc. (ESCOMP#2087)
* Fixes to methane Tech Note (ESCOMP#2091)
* Add is_doy_in_interval() function (ESCOMP#2158)
* Avoid using subprocess.run() in FSURDATMODIFYCTSM (ESCOMP#2125)

Closes issues:
* Add unit test for making fsurdat with all crops everywhere (ESCOMP#2079)
* Rework master_list_(no)?fates.rst? (ESCOMP#2083)
* conda run -n can fail if a conda environment is already active (ESCOMP#2109)
* conda fails to load for SystemTests (ESCOMP#2111)
samsrabin added a commit to samsrabin/CTSM that referenced this pull request Sep 27, 2023
@samsrabin
Merge tag 'ctsm5.1.dev142' into hist-anomalies
1e1b604 
b4b changes to Python scripts, history lists, tech note, and clm_time_manager.

* Add system and unit tests for making fsurdat with all crops everywhere (ESCOMP#2081)
* Rework master_list* files etc. (ESCOMP#2087)
* Fixes to methane Tech Note (ESCOMP#2091)
* Add is_doy_in_interval() function (ESCOMP#2158)
* Avoid using subprocess.run() in FSURDATMODIFYCTSM (ESCOMP#2125)

Closes issues:
* Add unit test for making fsurdat with all crops everywhere (ESCOMP#2079)
* Rework master_list_(no)?fates.rst? (ESCOMP#2083)
* conda run -n can fail if a conda environment is already active (ESCOMP#2109)
* conda fails to load for SystemTests (ESCOMP#2111)
samsrabin added a commit to samsrabin/CTSM that referenced this pull request Oct 2, 2023
@samsrabin
Merge tag 'ctsm5.1.dev142' into tillage
d43c617 
b4b changes to Python scripts, history lists, tech note, and clm_time_manager.

* Add system and unit tests for making fsurdat with all crops everywhere (ESCOMP#2081)
* Rework master_list* files etc. (ESCOMP#2087)
* Fixes to methane Tech Note (ESCOMP#2091)
* Add is_doy_in_interval() function (ESCOMP#2158)
* Avoid using subprocess.run() in FSURDATMODIFYCTSM (ESCOMP#2125)

Closes issues:
* Add unit test for making fsurdat with all crops everywhere (ESCOMP#2079)
* Rework master_list_(no)?fates.rst? (ESCOMP#2083)
* conda run -n can fail if a conda environment is already active (ESCOMP#2109)
* conda fails to load for SystemTests (ESCOMP#2111)
samsrabin added a commit to samsrabin/CTSM that referenced this pull request Oct 3, 2023
@samsrabin
Merge tag 'ctsm5.1.dev142' into tillage-and-residues3
0d36638 
b4b changes to Python scripts, history lists, tech note, and clm_time_manager.

* Add system and unit tests for making fsurdat with all crops everywhere (ESCOMP#2081)
* Rework master_list* files etc. (ESCOMP#2087)
* Fixes to methane Tech Note (ESCOMP#2091)
* Add is_doy_in_interval() function (ESCOMP#2158)
* Avoid using subprocess.run() in FSURDATMODIFYCTSM (ESCOMP#2125)

Closes issues:
* Add unit test for making fsurdat with all crops everywhere (ESCOMP#2079)
* Rework master_list_(no)?fates.rst? (ESCOMP#2083)
* conda run -n can fail if a conda environment is already active (ESCOMP#2109)
* conda fails to load for SystemTests (ESCOMP#2111)
samsrabin added a commit to samsrabin/CTSM that referenced this pull request Oct 4, 2023
@samsrabin
Merge tag 'ctsm5.1.dev142' into hillslope_hydrology-ssr
595d95d 
b4b changes to Python scripts, history lists, tech note, and clm_time_manager.

* Add system and unit tests for making fsurdat with all crops everywhere (ESCOMP#2081)
* Rework master_list* files etc. (ESCOMP#2087)
* Fixes to methane Tech Note (ESCOMP#2091)
* Add is_doy_in_interval() function (ESCOMP#2158)
* Avoid using subprocess.run() in FSURDATMODIFYCTSM (ESCOMP#2125)

Closes issues:
* Add unit test for making fsurdat with all crops everywhere (ESCOMP#2079)
* Rework master_list_(no)?fates.rst? (ESCOMP#2083)
* conda run -n can fail if a conda environment is already active (ESCOMP#2109)
* conda fails to load for SystemTests (ESCOMP#2111)
samsrabin added a commit to samsrabin/CTSM that referenced this pull request Oct 5, 2023
@samsrabin
Merge tag 'ctsm5.1.dev142' into soil-gas-diffusivity-bugfix
d369f8d 
b4b changes to Python scripts, history lists, tech note, and clm_time_manager.

* Add system and unit tests for making fsurdat with all crops everywhere (ESCOMP#2081)
* Rework master_list* files etc. (ESCOMP#2087)
* Fixes to methane Tech Note (ESCOMP#2091)
* Add is_doy_in_interval() function (ESCOMP#2158)
* Avoid using subprocess.run() in FSURDATMODIFYCTSM (ESCOMP#2125)

Closes issues:
* Add unit test for making fsurdat with all crops everywhere (ESCOMP#2079)
* Rework master_list_(no)?fates.rst? (ESCOMP#2083)
* conda run -n can fail if a conda environment is already active (ESCOMP#2109)
* conda fails to load for SystemTests (ESCOMP#2111)
@samsrabin samsrabin mentioned this pull request Nov 21, 2023
Open
samsrabin added a commit to samsrabin/CTSM that referenced this pull request Dec 23, 2023
@samsrabin
Merge tag 'ctsm5.1.dev142' into tillage-mike-merge142
19c4790 
b4b changes to Python scripts, history lists, tech note, and clm_time_manager.

* Add system and unit tests for making fsurdat with all crops everywhere (ESCOMP#2081)
* Rework master_list* files etc. (ESCOMP#2087)
* Fixes to methane Tech Note (ESCOMP#2091)
* Add is_doy_in_interval() function (ESCOMP#2158)
* Avoid using subprocess.run() in FSURDATMODIFYCTSM (ESCOMP#2125)

Closes issues:
* Add unit test for making fsurdat with all crops everywhere (ESCOMP#2079)
* Rework master_list_(no)?fates.rst? (ESCOMP#2083)
* conda run -n can fail if a conda environment is already active (ESCOMP#2109)
* conda fails to load for SystemTests (ESCOMP#2111)

# Conflicts:
#	src/biogeochem/CNBalanceCheckMod.F90
#	src/biogeochem/CNCIsoFluxMod.F90
#	src/biogeochem/CNDriverMod.F90
#	src/biogeochem/CNPhenologyMod.F90
#	src/biogeochem/CNProductsMod.F90
#	src/biogeochem/CNVegCarbonFluxType.F90
#	src/biogeochem/CNVegNitrogenFluxType.F90
#	src/biogeochem/EDBGCDynMod.F90
#	src/main/clm_initializeMod.F90
#	src/main/controlMod.F90
#	src/soilbiogeochem/SoilBiogeochemDecompCascadeBGCMod.F90
samsrabin added a commit to samsrabin/CTSM that referenced this pull request Dec 23, 2023
@samsrabin
Merge tag 'ctsm5.1.dev142' into tillage-mike-merge142
9cdc75a 
b4b changes to Python scripts, history lists, tech note, and clm_time_manager.

* Add system and unit tests for making fsurdat with all crops everywhere (ESCOMP#2081)
* Rework master_list* files etc. (ESCOMP#2087)
* Fixes to methane Tech Note (ESCOMP#2091)
* Add is_doy_in_interval() function (ESCOMP#2158)
* Avoid using subprocess.run() in FSURDATMODIFYCTSM (ESCOMP#2125)

Closes issues:
* Add unit test for making fsurdat with all crops everywhere (ESCOMP#2079)
* Rework master_list_(no)?fates.rst? (ESCOMP#2083)
* conda run -n can fail if a conda environment is already active (ESCOMP#2109)
* conda fails to load for SystemTests (ESCOMP#2111)
@slevis-lmwg slevis-lmwg mentioned this pull request Feb 15, 2024
Closed
@samsrabin samsrabin added simple bfb bit-for-bit labels Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ekluzek ekluzek ekluzek approved these changes

@slevis-lmwg slevis-lmwg slevis-lmwg approved these changes

Assignees

@samsrabin samsrabin

Labels
bfb bit-for-bit PR status: ready PR: this is ready to merge in, with all tests satisfactory and reviews complete testing additions or changes to tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add unit test for making fsurdat with all crops everywhere
3 participants
@samsrabin @ekluzek @slevis-lmwg