Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failures in K_TestCimeCase for CESM #2057

Closed
billsacks opened this issue Nov 14, 2017 · 4 comments
Closed

Failures in K_TestCimeCase for CESM #2057

billsacks opened this issue Nov 14, 2017 · 4 comments
Labels
st: wontfix tp: unit tests ty: Bug

Comments

@billsacks
Copy link
Member

Running scripts_regression_tests from the head of master (da29a73) I get failures in these three tests in K_TestCimeCase (running on yellowstone, with cime_model = cesm):

(1)

test_cime_case_prereq (__main__.K_TestCimeCase) ... job case.run depends on ['prereq_test']
Submit job case.run
dependencies: ['prereq_test']
job case.st_archive depends on ['prereq_test', '0']
Submit job case.st_archive
dependencies: ['prereq_test', '0']
ERROR

(2) test_cime_case_test_walltime_mgmt_6 (__main__.K_TestCimeCase) ... FAIL

(3) test_cime_case_test_walltime_mgmt_7 (__main__.K_TestCimeCase) ... FAIL

I notice that the other test_cime_case_test_walltime_mgmt tests are skipped for CESM; should these be skipped, too?
@jedwards4b
Copy link
Contributor

These fail on yellowstone (only) due to the way that the lsf system handles time with hh:mm instead of hh:mm:ss - I've known about this for a while, but since it was only this machine I have ignored it.

@billsacks
Copy link
Member Author

Okay, if it's just yellowstone I'm okay closing this as wontfix.

@billsacks billsacks mentioned this issue Nov 14, 2017
Merged
jgfouca added a commit that referenced this issue Nov 14, 2017
@jgfouca
Merge pull request #2060 from billsacks/compare_two_comparison_unit_t…
559c50a 
…ests

Add unit tests of compare_two's handling of pass/fail in comparison

Previously, I had figured that it was sufficient to ensure that
_component_compare_test was actually being called, figuring that the
tests of _component_compare_test belong elsewhere. But, since this is
such a critical aspect of the system_test_compare_two infrastructure,
I'm adding some tests covering _component_compare_test here.

These new tests cover the logic related to in-test comparisons in
system_tests_compare_two and system_tests_common. However, they will
NOT cover the logic in hist_utils: I'm stubbing out the actual call into
hist_utils, under the assumption that this is - or should be - covered
by other tests. But I'm not sure if hist_utils is actually covered
sufficiently by unit tests. If it isn't, it should be.

This also required some minor refactoring of system_tests_common in
order to allow stubbing out the call into hist_utils. (This refactoring
would not have been needed if we allowed use of the mock module: see
#2056.)

Test suite: scripts_regression_tests on yellowstone
Passes other than the issues documented in #2057 and #2059

Also ensured that comparison failures are still reported correctly by
running a REP test that forces a comparison failure (due to missing
cprnc).
Test baseline: n/a
Test namelist changes: none
Test status: bit for bit

Fixes: Partially addresses #1640

User interface changes?: none

Update gh-pages html (Y/N)?: N

Code review:
@billsacks
Copy link
Member Author

@jedwards4b is this one also yellowstone-specific? It doesn't look to have anything to do with time formats:

======================================================================
ERROR: test_cime_case_prereq (__main__.K_TestCimeCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./scripts_regression_tests.py", line 1472, in test_cime_case_prereq
    self.assertTrue(prereq_name in prereq_substr, "Dependencies added, but not the user specified one")
UnboundLocalError: local variable 'prereq_substr' referenced before assignment

@billsacks billsacks mentioned this issue Nov 15, 2017
Merged
@jedwards4b
Copy link
Contributor

Yes see #2048

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
st: wontfix tp: unit tests ty: Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jedwards4b @billsacks