Tomas fixes to address failure to integrate in dev/14.3.0 #2116
Conversation
There was a problem hiding this comment.
I made some corrections on top of these updates in the bugfix/tomas-gchp branch because hese updates cause non-TOMAS fullchem simulations to crash when reading HISTORY.rc. This is because the TOMAS species are listed by default in the SpeciesConc collection but are not defined in non-TOMAS simulations.
Additionally, GCHP TOMAS simulations are still crashing in integration tests at Harvard (using gcc 10.2.0 as compiler build type = Debug). GCHP compiles successfully for TOMAS, but crashes with a FPE error:
SOACOND WARNING: SOA mass is being discarded
Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.
#0 0x150dbb028b4f in ???
#1 0xdaa415 in __error_mod_MOD_nan_dble
at /n/holylfs05/LABS/jacob_lab/msulprizio/IntTests/GCHP_14.3.0-alpha.9/CodeDir/src/GCHP_GridComp/GEOSChem_GridComp/geos-chem/GeosUtil/error_mod.F90:160
#2 0x93da1c in __tomas_mod_MOD_aerophys
at /n/holylfs05/LABS/jacob_lab/msulprizio/IntTests/GCHP_14.3.0-alpha.9/CodeDir/src/GCHP_GridComp/GEOSChem_GridComp/geos-chem/GeosCore/tomas_mod.F90:735
#3 0x942ed0 in __tomas_mod_MOD_do_tomas
at /n/holylfs05/LABS/jacob_lab/msulprizio/IntTests/GCHP_14.3.0-alpha.9/CodeDir/src/GCHP_GridComp/GEOSChem_GridComp/geos-chem/GeosCore/tomas_mod.F90:261
#4 0x4d7504 in __chemistry_mod_MOD_do_chemistry
at /n/holylfs05/LABS/jacob_lab/msulprizio/IntTests/GCHP_14.3.0-alpha.9/CodeDir/src/GCHP_GridComp/GEOSChem_GridComp/geos-chem/GeosCore/chemistry_mod.F90:633
...
The complete log files are attached here:
|
Hi @msulprizio, thanks for these updates! I have encountered this issue with the FPE coming from tomas_mod.F90 at line 735 with debug on - but was able to address the issue by using a spun-up restart file that included the TOMAS tracers. Do you have a restart that includes TOMAS tracers? I can send one if that would be of help. |
|
Hi @BettyCroft! If you have a more recent restart file for the TOMAS simulation, please do send that to us and I can retest that simulation. |
|
Hi @msulprizio - yes I'd be happy to send that file along to you now. Is there a preferred method that you would suggest for sending? |
|
Hi @msulprizio! I've placed a restart that includes TOMAS tracers in a Box folder and sent you an invite (https://wustl.box.com/s/psgj840gbjflgdo0xtx91g2qt2nu4gg9) - please send me know if this enables the GCHP-TOMAS simulation to integrate successfully. |
|
Hi @msulprizio - I have added responses to each of your comments above. Please let me know if you have any additional concerns/discussion. Thanks again! |
|
Hi @msulprizio! Could you send a quick status update? Are the TOMAS updates going into dev/14.3.0? I noticed that some of my recent commits at (c3c6b71) are not there (particularly the updates to the aerodiag subroutine in tomas_mod.F90) - whereas other earlier TOMAS updates are in there. |
|
@BettyCroft That is correct. This pull request has not been merged into dev/14.3.0 yet. I have downloaded your spun-up restart file and am integration testing now. If I can get things to run, then we may still be able to merge this in before the 14.3.0 release as a zero-diff update (with respect to the fullchem benchmark). Otherwise, it will have to wait until 14.3.1. |
|
Thanks for this update @msulprizio! This approach sounds excellent. |
…4.3.0 Signed-off-by: Melissa Sulprizio <mpayer@seas.harvard.edu>
|
This pull request is now superseded by #2136. |
Name and Institution (Required)
Name: Betty Croft
Institution: WashU/Dalhousie
Confirm you have reviewed the following documentation
yes
Describe the update
These updates are additional bug fixes for TOMAS to integrate successfully in debug mode in dev/14.3.0. This should be merged alongside:
Expected changes
TOMAS in both GCHP and GC-Classic is expected to integrate successfully in debug mode in dev/14.3.0.
Reference(s)
Related Github Issue(s)
These updates address issue #1741