Regression Testing Updates: GNU/Intel into one RT. New logs/ dir in tests/ #1718
Conversation
…odel into gnuintel2
…ull RT suite on hera
|
To prove tests don't change baselines for this PR will be manual due to the fact that the baselines directory is changing a bit. I have used the following script to compare baselines: #! /bin/bash
TESTDIR=/scratch1/NCEPDEV/stmp4/Brian.Curtis/FV3_RT/REGRESSION_TEST_GOOD/
TESTS=`ls ${TESTDIR}`
BLDIR=/scratch1/NCEPDEV/nems/emc.nemspara/RT/NEMSfv3gfs/develop-20230413/
#echo $TESTDIRS
for i in $TESTS
do
TEST=${i%_*}
CMPLR=${i##*_}
CMPDIR=${BLDIR}${CMPLR^^}/${TEST}
echo "Comparing files for ${CMPLR^^} test: ${TEST}"
for j in `ls ${CMPDIR}`
do
if [[ ${j} == "RESTART" ]]; then
for k in `ls ${CMPDIR}/$j`
do
if [[ ${k##*.} =~ nc* ]]; then
echo "--> USING NCCMP ON FILE: ${j}/${k} "
nccmp -d -f -g -B --Attribute=checksum --warn=format ${TESTDIR}/$i/$j/$k ${CMPDIR}/$j/$k || echo -e "\n"
else
echo "--> USING CMP ON FILE: ${j}/${k}"
cmp ${TESTDIR}/$i/$j/$k ${CMPDIR}/$j/$k || echo -e "\n"
fi
done
elif [[ ${j##*.} =~ nc* ]]; then
echo "--> USING NCCMP ON FILE: ${j}"
nccmp -d -f -g -B --Attribute=checksum --warn=format ${TESTDIR}/$i/$j ${CMPDIR}/$j || echo -e "\n"
else
echo "--> USING CMP ON FILE: ${j}"
cmp ${TESTDIR}/$i/$j ${CMPDIR}/$j || echo -e "\n"
fi
done
doneAnd here is it's results: |
|
@BrianCurtis-NOAA I'm going to get started with jenkins-ci on this PR |
|
Attached are the jenkins ci logs; the tests failed and we're looking into this. |
If you need me to look through what you're doing, i'd be happy to. |
|
I'm trying to test ORT manually against this PR on Hera, and receiving unbound variable error for "TEST_NAME". I get this error with both intel and gnu. Edit* (I should add this occurs on the std_base) |
|
@FernandoAndrade-NOAA ORTs passed when I ran them manually against this PR, following Brian's changes. I think you can try jenkins-ci again, when you're able. |
|
@BrianCurtis-NOAA update on Orion. All the RTs are complete except for a failure with regional_atmaq_faster case. The rest are good. However, it looks like Orion is down currently so I'm unable to troubleshoot the failed case at the moment. The baselines were created and copied over fine on Orion, so it may be a good idea to skip the logs on for this HPC, if Orion remains down the rest of the day. @jkbk2004 fyi. |
@zach1221 Sounds like unexpected orion network issue. all orion baselines were created ok, right? I agree we may move on to merge unless orion is back online next a few hours. |
|
Yes, the baselines created fine. I'm not certain why that last regional_atmaq case failed in the RT/matching. As soon as I started to dig through the logs I was kicked out. |
|
I think its justified to skip Orion, but I recommend creating an issue to check why once Orion is back and addressing anything in that issue. |
|
@BrianCurtis-NOAA I agree. I'll get the issue created now, so that when Orion is back up I can ensure everything is tracked and cleaned up. |
|
Ok, we have approvals. I'm proceeding with merging. |
|
Looks like Orion just came back up. Checking that last atmaq test. |
|
regional_atmaq_faster passed on Orion, after extending the wallclock, so looks like it was just timing out. I'll push the orion RT logs and then merge. |
Description
rt.conf updates:
rt.sh updates:
bl_date.conf:
Log updates:
Baseline storage updates:
opnReqTest updates:
./opnReqTest -e -n regional_controlPR template update:
What I have tested:
What I need to test:
Input data additions/changes
Anticipated changes to regression tests:
This should not change answers to any test, but the whole regression testing system is getting name changes, so all baselines will need to be regenerated with the new code changes and new baseline-dirs will need to be created.
Subcomponents involved:
Combined with PR's (If Applicable):
Commit Queue Checklist:
Linked PR's and Issues:
Testing Day Checklist:
Testing Log (for CM's):