Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert verification to solely use METplus configuration files (feature/ens_design_RRFS) #695

Merged
merged 7 commits into from
Mar 22, 2022
Merged

Convert verification to solely use METplus configuration files (feature/ens_design_RRFS) #695

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1439830
Create feature/ens_design_RRFS branch and move original MET and METpl…
michelleharrold Jan 15, 2022
7d89b94
Updated METplus conf files to remove dependency on project-based MET …
michelleharrold Feb 4, 2022
49a9922
Sync with develop.
michelleharrold Feb 25, 2022
bc2bf09
Updates to METplus conf files and run scripts for transitioning away …
michelleharrold Mar 3, 2022
ed84f7d
Merge remote-tracking branch 'upstream/develop' into feature/ens_desi…
michelleharrold Mar 7, 2022
7d3d358
Removed MET configuration files; changed envar ACCUM back to acc in s…
michelleharrold Mar 7, 2022
26b61ec
Removed old METplus conf files and removed OBS PROB info from Grid-St…
michelleharrold Mar 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions scripts/exregional_run_ensgridvx.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,11 +146,11 @@ export LOG_SUFFIX
#
if [ ${VAR} == "APCP" ]; then
acc="${ACCUM}h" # for stats output prefix in EnsembleStatConfig
${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michelleharrold I know we discussed error checking in a meeting before, but I wanted to understand the details (which I've forgotten) of why we don't want to use error-checking here. By error-checking, I mean adding to this call a print_err_msg_exit statement if it fails, e.g.

${METPLUS_PATH}/ush/run_metplus.py \
    -c ${METPLUS_CONF}/common.conf \
    -c ${METPLUS_CONF}/EnsembleStat_${VAR}${acc}.conf || print_err_msg_exit "\
Call to "run_metplus.py" failed with exit code $?.
"

I and other reviewers are now encouraging use of error-checking so the ex-script doesn't exit with a false success status. But for some of the verification ex-scripts, I remember you and/or @jwolff-ncar said we should not have the script fail even if the return code from METplus is nonzero. Can you explain again the reason for that? Also, are some of the other vx sripts well-behaved in the sense that we can do error-checking with them? Thanks.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gsketefian - good question. I agree not having the error checking for the verification can cause some frustration when a job fails but is called successful by the workflow. @jwolff-ncar and I decided not to implement error checking for the vx jobs because we run all the forecast lead times through the verification in one job, and there are circumstances where some observations may not present (which is okay). This would cause the verification job to error out prematurely if a forecast/obs combo was not present. I am definitely open to brainstorming ways around this because having more robust error checking would certainly make testing and monitoring easier.

-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/EnsembleStat_${VAR}${acc}.conf
else
${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/EnsembleStat_${VAR}.conf
fi
Expand Down
4 changes: 2 additions & 2 deletions scripts/exregional_run_ensgridvx_mean.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,11 +155,11 @@ export LOG_SUFFIX
#
if [ ${VAR} == "APCP" ]; then
export acc="${ACCUM}h"
${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/GridStat_${VAR}${acc}_mean.conf
else
${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/GridStat_${VAR}_mean.conf
fi
Expand Down
4 changes: 2 additions & 2 deletions scripts/exregional_run_ensgridvx_prob.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,11 +155,11 @@ export LOG_SUFFIX
#
if [ ${VAR} == "APCP" ]; then
export acc="${ACCUM}h"
${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/GridStat_${VAR}${acc}_prob.conf
else
${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/GridStat_${VAR}_prob.conf
fi
Expand Down
4 changes: 2 additions & 2 deletions scripts/exregional_run_enspointvx.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,11 +137,11 @@ export MODEL
export NET
export NUM_ENS_MEMBERS

${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/EnsembleStat_conus_sfc.conf

${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/EnsembleStat_upper_air.conf

Expand Down
4 changes: 2 additions & 2 deletions scripts/exregional_run_enspointvx_mean.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,11 +138,11 @@ export MET_CONFIG
export MODEL
export NET

${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_conus_sfc_mean.conf

${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_upper_air_mean.conf
#
Expand Down
4 changes: 2 additions & 2 deletions scripts/exregional_run_enspointvx_prob.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,11 +138,11 @@ export MET_CONFIG
export MODEL
export NET

${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_conus_sfc_prob.conf

${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_upper_air_prob.conf
#
Expand Down
8 changes: 4 additions & 4 deletions scripts/exregional_run_gridstatvx.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,13 +160,13 @@ export NET
#
if [ ${VAR} == "APCP" ]; then
export acc="${ACCUM}h" # for stats output prefix in GridStatConfig
${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/${VAR}_${acc}.conf
-c ${METPLUS_CONF}/GridStat_${VAR}${acc}.conf
else
${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/${VAR}.conf
-c ${METPLUS_CONF}/GridStat_${VAR}.conf
fi

#
Expand Down
4 changes: 2 additions & 2 deletions scripts/exregional_run_pointstatvx.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,11 +141,11 @@ export MET_CONFIG
export MODEL
export NET

${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_conus_sfc.conf

${METPLUS_PATH}/ush/master_metplus.py \
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_upper_air.conf

Expand Down
Loading