cprnc does not FAIL if fields are missing

[gold2718]

While cprnc notes if fields cannot be compared, it does not FAIL a comparison if a field (NetCDF variable) goes missing. I can see allowing a new field but if we delete a field, should that count as an answer change? If not cprnc itself, then should a comparison test FAIL in this case?

[quantheory]

I think that adding a variable and deleting a variable should both be considered FAIL. Adding a field can also be a mistake, albeit one that's usually less bad than accidentally removing a field.

Plus, if only deleting a variable was considered to FAIL, we would lose symmetry. It would be counter-intuitive if tag A failed baseline tests against tag B, but tag B passed baseline tests against tag A. Or to put it differently, baseline comparison should define an equivalence relation between tags.

[billsacks]

Feeling is that this should be reported, but maybe labeled differently from a FAIL?

[billsacks]

Decision in CSEG meeting: We'll have cprnc report a warning if fields have been either added or removed.

[billsacks]

I have addressed the cprnc side of this issue in #442

After that change comes to master, we'll still need to do the following:

1. Make changes to the system test scripts to parse the new output
2. Update and rebuild cprnc on all test platforms. Note that many of these likely still point to the old svn repository; for these, we'll need to add a clone of cime, and update the cprnc path in config_machines.xml to point to the new location.
3. Run some tests that trigger the new behavior; make sure these are treated properly by the test system

Adding @fischer-ncar to the cc list

[mnlevy1981]

Note that many of these likely still point to the old svn repository; for these, we'll need to add a clone of cime, and update the cprnc path in config_machines.xml to point to the new location.

I think we should do the following:

1. Rename the existing cprnc checkout as cprnc.old (or delete it altogether)
2. svn co https://github.com/CESM-Development/cime/trunk/tools/cprnc

No need to clone the entire CIME repo just to access cprnc when github gives us SVN hooks

[billsacks]

This issue was moved to ESMCI#870