Implement ELM v2 data architecture for mass and energy state and flux variables #2742
Conversation
|
Design document for this work is on the Land/Energy NGD Confluence page, at |
|
Given the number of conflicts, this is a case where you should merge master to the branch to help resolve them. |
|
Although you'll have to do all your testing again then. |
Adds subroutines to initialize and clean all the new column and vegetation level data structures. also renames a few physical_properties types to improve data structure naming consistency [BFB]
Created a new module for the state and flux variables at the column level (ColumnDataType). Needed because col_pp is referenced inside HistFileMod, and initialization of the history fields for state and flux variables references HistFileMod. Implementation will be extended later to other sub-grid levels. [BFB]
An incremental commit in preparation for major update associated with tsoisno variable. [BFB]
Includes migration of history initialization and restart for soil and snow temperatures. [BFB]
Continued migration of column-level energy state variables, with history, cold start, and restart. [BFB]
Continued migration of column-level temperature variables. [BFB]
Add remaining temperature-related variables to new data type, and update the use of these variables throughout the code. Unused variables removed (e.g. dt_grnd_col) or treated as constant (e.g. beta_col). Also modified the instantiation of urbanparams_type, moving this out of clm_instMod.F90, and into UrbanParamsType.F90. This fixes some issues with circular use referencing. [BFB]
Added initialization, history, and restart capability for vegetation energy state structure. Also introduced a new module for vegetation data structures. Tested against land developer test suite [BFB]
Complete migration of all vegetation (pft) level energy state variables from the vegetation_vars structure in veg_es structure. Requires a change to the BeTR submodule for data passing to use new data structures. With that change, all land developer test suite tests pass BFB. [BFB]
Add topounit-to-gridcell scaling factor, and ability to accept history file variables defined at the topounit level. [BFB]
Created a new data structure for the state and flux variables that are defined at topounit level. Revised all references to these data structures in "use" statements. [BFB]
Completed the addition of new history variables for the v2 data architecture at the topounit level. All previous history field names and other metadata are maintained in the new code, so this change is BFB with master. At a later date, ELM team may want to add additional history outputs, or remove some that are duplicates with different names as part of the code cleanup process. [BFB]
|
I can make the needed commit for the build fail. |
|
Its your PR so you can declare those diffs to be acceptable and ask for them to be blessed. But you should know why they happened. |
|
The diffs are between coupler history files output at the end of the runs. "Ln9" is 9 timesteps. "Ld2" is 2 days". |
|
Regarding the DIFFS: every other time I have seen DIFFS in FSDS they propagate quickly to many other energy variables. The fact that this impacts only FSDS suggests that somehow the DIFF is not propagating. I'd be glad to explore further, but it seems like the simulation is progressing BFB in subsequent time steps. |
|
@rljacob just saw your message about time steps. Is there information in the DIFF output that indicates on which timestep the reported DIFF occurs? |
|
I'm going by the name of the files that were compared which is in the teststatus output on cdash. You can also download a tar file of all the log files by clicking on the gold box next to the test name. Questions to ask yourself: why these cases and not other F or coupled cases? |
|
Based on what we know now, I think these DIFFS are acceptable, and would like to have them blessed. I would like to understand why they occur, and can set this as a new task to dig into these compsets and provide an answer. There is something special about how these compsets are working compared to all the other tests, and I'd work with the people most knowledgeable about these tests to sort it out. I would not ask to bless and proceed, even for small DIFFS, if this was propagating to the 30+ other coupler history variables that are typically impacted when a DIFF appears in FSDS. |
|
Ok. @jqyin add your commit for the build fix, re-merge to next and wait for another round of testing. |
Implement ELM v2 data architecture for mass and energy state and flux variables Re-merge to fix build error [BFB]
Implement ELM v2 data architecture for mass and energy state and flux variables Re-merge to fix the floating-point exception for F cases [BFB]
|
@thorntonpe the build and runtime fails are now clear, but SMS_D_Ld1.ne30_oECv3_ICG.A_WCYCL1850S_CMIP6.sandiatoss3_intel.allactive-v1cmip6 and SMS_D_Ln5.ne4_ne4.FC5AV1C-L.sandiatoss3_intel.cam-cosplite_nhtfrq5 show large DIFFs on 18 fields, e.g., CLMMODIS (ncol,time) t_index = 1 1 CLMODIS (ncol,cosp_tau_modis,cosp_prs,time) t_index = 1 1 more details on https://my.cdash.org/viewTest.php?onlyfailed&buildid=1620355 |
|
So after fixing the build error you now have more diffs then before? |
|
The large diffs are in COSP (satellite simulator) fields so they are diagnostic and don't change answers. Why would a land model change do that? The FSDS diff is only in runs with the gnu compiler. Only runs that have cam output show any diffs. Most tests only compare coupler history with baselines. No diffs there. |
|
@rljacob , yes, I am also confused by this. I'm looking at the list of diffs, all are *MODIS fields - I don't know anything about how those are generated, and I'm not sure what the connection - if any - to the land model is. Who is the expert on these *MODIS fields? Is it possible these diffs are coming from another merged PR? |
|
There was a bug fix for the MODIS simulator that was recently merged, and then reverted. Possible that the baseline and test case have inconsistent versions of this code change? |
|
I don't see anything in the recent history of next involving MODIS. |
|
One way or another, that seems a very likely culprit, given the diffs. I don't know enough about how the baselines are generated for testing on next to be able to track this down. |
|
Ok if you go back a week PR #2770 caused MODIS changes and has been sitting on next since then. That's the reason. |
|
So is there anything that I need to do for integration of this PR? |
|
@rljacob I agree. The same DIFFs on prod_next starts on Mar. 7 https://my.cdash.org/index.php?project=ACME_Climate&date=2019-03-07. Looks like it's due to PR #2770. After the fix for debug mode F cases, it shows up now on integration next. |
Implement ELM v2 data architecture for mass and energy state and flux variables Migrates mass and energy state and flux variables from CLM4.5 data structures, which mixed variables at multiple subgrid levels, to ELM v2 data structures, which separate the variables according to subgrid level. For example, carbon state variables are moved from a single type into three distinct types, one for gridcell-level, one for column-level, and one for vegetation (patch)-level variables. Methods acting on these types are also defined independently for each subgrid level. Migrates all energy, water, carbon, nitrogen, and phosphorus state and flux variables and data-type methods. Leaves open the issue of how to structure subroutine calling interfaces. The original code uses a variety of methods. Here, an example is provided for the module biogeochem/CNCStateUpdate1Mod.F90 and all its upstream calls, based on explicit references to data type instances at the clm_driver() level, and passed references at all lower levels. Branch tests bit-for-bit against the E3SM land developer test suite. [BFB]
|
Merged #2742 |
updates for compatibility with XML to html tools These are updates to cime based classes in support of the XML to HTML auto-generation tools in https://github.com/NCAR/CESM_xml2html. Also minor update to DOCN config_component.xml to correct invalid ASCII character. Test suite: scripts_regression_tests.py B_CheckCode Test baseline: N/A Test namelist changes: N/A Test status: bit for bit Fixes [CIME Github issue #] User interface changes?: None Update gh-pages html (Y/N)?: N Code review:jedwards
updates for compatibility with XML to html tools These are updates to cime based classes in support of the XML to HTML auto-generation tools in https://github.com/NCAR/CESM_xml2html. Also minor update to DOCN config_component.xml to correct invalid ASCII character. Test suite: scripts_regression_tests.py B_CheckCode Test baseline: N/A Test namelist changes: N/A Test status: bit for bit Fixes [CIME Github issue #] User interface changes?: None Update gh-pages html (Y/N)?: N Code review:jedwards
…ng-add-logging Automatically Merged using E3SM Pull Request AutoTester PR Title: Make nudging log which file it's reading PR Author: bartgol PR LABELS: enhancement, I/O, AT: AUTOMERGE, nudging
Migrates mass and energy state and flux variables from CLM4.5 data structures, which mixed variables at multiple subgrid levels, to ELM v2 data structures, which separate the variables according to subgrid level. For example, carbon state variables are moved from a single type into three distinct types, one for gridcell-level, one for column-level, and one for vegetation (patch)-level variables. Methods acting on these types are also defined independently for each subgrid level. Migrates all energy, water, carbon, nitrogen, and phosphorus state and flux variables and data-type methods.
Leaves open the issue of how to structure subroutine calling interfaces. The original code uses a variety of methods. Here, an example is provided for the module biogeochem/CNCStateUpdate1Mod.F90 and all its upstream calls, based on explicit references to data type instances at the clm_driver() level, and passed references at all lower levels.
Branch tests bit-for-bit against the E3SM land developer test suite.
[BFB]