floating invalid with F20TRC5AV1C-04P2 and -bgc bgc -nutrient_comp_pathway eca

[ndkeen]

This testing is on cori-knl using a master of Jul 23rd.

This test passes: SMS_PT_D_Ln5.ne30_ne30.F20TRC5AV1C-04P2
However, if I do the following in the casedir:

./xmlchange --id CLM_BLDNML_OPTS --val "-bgc bgc -nutrient_comp_pathway eca" 

And rebuild, I get the following error:

 71: 2aaba8508000-2aabab679000 rwxs 000forrtl: error (65): floating invalid
 71: Image              PC                Routine            Line        Source             
 71: e3sm.exe           000000000C69BF8E  Unknown               Unknown  Unknown
 71: e3sm.exe           000000000BF35120  Unknown               Unknown  Unknown
 71: e3sm.exe           000000000743B0A3  allocationmod_mp_        2415  AllocationMod.F90
 71: e3sm.exe           0000000007FA2FA0  soillittdecompmod         410  SoilLittDecompMod.F90
 71: e3sm.exe           0000000007A94EC9  ecosystemdynmod_m         590  EcosystemDynMod.F90
 71: e3sm.exe           00000000058E6B84  clm_driver_mp_clm        1012  clm_driver.F90
 71: e3sm.exe           000000000AAEB3E3  Unknown               Unknown  Unknown
 71: e3sm.exe           000000000AAA2CC0  Unknown               Unknown  Unknown
 71: e3sm.exe           000000000AAA42B4  Unknown               Unknown  Unknown
 71: e3sm.exe           000000000AA75BD4  Unknown               Unknown  Unknown
 71: e3sm.exe           00000000058C0697  clm_driver_mp_clm         560  clm_driver.F90
 71: e3sm.exe           000000000588CC8F  lnd_comp_mct_mp_l         509  lnd_comp_mct.F90
 71: e3sm.exe           0000000000466A13  component_mod_mp_         737  component_mod.F90
 71: e3sm.exe           0000000000430284  cime_comp_mod_mp_        2602  cime_comp_mod.F90
 71: e3sm.exe           000000000044EA0D  MAIN__                    133  cime_driver.F90

/global/cscratch1/sd/ndk/acme_scratch/cori-knl/m06-jul22/SMS_PT_D_Ln5.ne30_ne30.F20TRC5AV1C-04P2.cori-knl_intel.20190724_091649_m056xe

I also tested with ony -bgc bgc, which completed, but it doesn't let me build with only -nutrient_comp_pathway eca (ie I needed both). Can someone confirm if this is a valid thing to test? That is, a valid way to configure? Note that I started with the following options: -bgc bgc -nutrient cnp -nutrient_comp_pathway eca -soil_decomp century -methane -nitrif_denitrif which fails in the same way as above -- then narrowed down which options were causing the error.

[ndkeen]

I tested this again with master of Nov 6th and I see same issue.

[qzhu-lbl]

Hi Noel, I think Jinyun's right, his PR https://github.com/E3SM-Project/E3SM/pull/3303/files fixed the nan issue for col_plant_pdemand_vr, should also fix the floating invalid issue you got in line 2415 AllocationMod.F90. dsolutionp_dt(c,j) = gross_pmin_vr(c,j) -potential_immob_p_vr(c,j) - & col_plant_pdemand_vr(c,j) + biochem_pmin_vr_col(c,j) + & primp_to_labilep_vr_col(c,j) + pdep_to_sminp(c) * ndep_prof(c,j)

…

On Wed, Dec 4, 2019 at 3:35 PM noel ***@***.***> wrote: I tested this again with master of Nov 6th and I see same issue. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#3088?email_source=notifications&email_token=AC3AHXQCATMXBRQ5B5CTY4TQXA5FVA5CNFSM4IGUVWWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF64V5I#issuecomment-561892085>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC3AHXQFHDKZGZNKODT4PZLQXA5FVANCNFSM4IGUVWWA> .

-- Research Scientist Climate Sciences Department Lawrence Berkeley National Laboratory 1 Cyclotron Rd, Berkeley, CA, USA 94720 Senior Fellow Berkeley Institute of Data Science University of California, Berkeley 190 Doe Library, Berkeley, CA 94720

[qzhu-lbl]

Hi Noel,

Could you please try

qzhu-lbl/lnd/plant_pdemand_init_fixnan

see if the error is still there?

[ndkeen]

I tried the same experiment again and it failed in the same way with master of July 7th.

First run SMS_PT_D_Ln5.ne30_ne30.F20TRC5AV1C-04P2
then do ./xmlchange --id CLM_BLDNML_OPTS --val "-bgc bgc -nutrient_comp_pathway eca" 
and rebuild

[ndkeen]

With master of Nov19th, I tried this again. I don't see the same error, but instead, the same error as I found here #2044

 6: At line 726 of file /global/cscratch1/sd/ndk/wacmy/m49-nov19/components/eam/src/physics/clubb/grid_class.F90
 6: Fortran runtime error: Index '-1' of dimension 1 of array 'gr%dzm' below lower bound of 0
 6:

I don't know if there were changes that allowed code to run further (and hit this error), or if this new error is complicating things here.

[ndkeen]

It looks like the compset no longer available, but I tried the following:
First run SMS_PT_D_Ln5.ne30_ne30.F2010-CICE.cori-knl_intel, then
do ./xmlchange --id ELM_BLDNML_OPTS --val "-bgc bgc -nutrient_comp_pathway eca"

and I get yet a different error:

0: (shr_stream_findBounds) ERROR: LVD not found, all data is before yearFirst

[ndkeen]

I'm going to close the issue as it's no longer the same error as original and not clear it's even supported.