Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

problems with branch identification and coverage #594

Closed
nedbat opened this issue Aug 2, 2017 · 2 comments
Closed

problems with branch identification and coverage #594

nedbat opened this issue Aug 2, 2017 · 2 comments
Labels
duplicate This issue or pull request already exists

Comments

@nedbat
Copy link
Owner

nedbat commented Aug 2, 2017

Originally reported by Anonymous


See pytest-cov for history.

I have code that contains branches that are provably executed from output but show up in the coverage as missing. Working with the pytest-cov folks, I reduced the problem to a very simple yet verbose group of unit tests.

There are 4 tests (foo to foobar inclusive) that test the branch detection and coverage. All of the functions are the same with additional yet fixed complexity wrapping them. foo() is the basic if block. fooa() is an if block with the foo() if block contained within it. Other complexity is added (the for block seems to break things) to make the test fairly complete.

Ideally, the coverage should be 100% since it is basically the same if block. However, the ideal case is not the result. There are then sna() and snafu() that have changed the basic if block to if-else blocks and they do pass.

Here is the command line to run the test:
python3 -m pytest --cov=cov_test --cov-branch --cov-report term-missing PyScripts/cov_test.py

It produces these results:

============================= test session starts ==============================
platform linux -- Python 3.5.2, pytest-3.1.3, py-1.4.34, pluggy-0.4.0
rootdir: /home/niessner, inifile:
plugins: cov-2.5.1
collected 1 item s

PyScripts/cov_test.py .

----------- coverage: platform linux, python 3.5.2-final-0 -----------
Name                    Stmts   Miss Branch BrPart  Cover   Missing
-------------------------------------------------------------------
PyScripts/cov_test.py      48      0     20      2    97%   18->21, 27->30


=========================== 1 passed in 0.03 seconds ===========================

Here is the test module:

def foo(states:[bool]):
    if all(states):
        print ('foo')
        pass
    return True

def foob (states:[bool]):
    if True:
        if all(states):
            print ('foob')
            pass
        pass
    return True

def fooba (states:[bool]):
    for i in range(10):
        if all(states):
            print ('fooba')
            pass
        pass
    return True

def foobar (states:[bool]):
    if True:
        for i in range(10):
            if all(states):
                print ('fooba')
                pass
            pass
        pass
    return True

def sna (states:[bool]):
    for i in range(10):
        if all(states):
            print ('sna')
        else: print ('ans')
        pass
    return True

def snafu (states:[bool]):
    if True:
        for i in range(10):
            if all(states):
                print ('snafu')
            else: print ('ufans')
            pass
        pass
    return True

def test():
    assert foo([True,False])
    assert foo([True,True])
    assert foob([True,False])
    assert foob([True,True])
    assert fooba([True,False])
    assert fooba([True,True])
    assert foobar([True,False])
    assert foobar([True,True])
    assert sna([True,False])
    assert sna([True,True])
    assert snafu([True,False])
    assert snafu([True,True])
    return

@nedbat
Copy link
Owner Author

nedbat commented Aug 2, 2017

This is because lines 21 and 30 are "pass", which are optimized away by the Python compiler. If you change those lines to something with an effect (a = 1), the coverage will go to 100%.

Unfortunately, Python doesn't have an option to completely disable the peephole optimizer, so there is no way to keep them. Comment on Python bug 2506 if you feel strongly about it: http://bugs.python.org/issue2506

@nedbat
Copy link
Owner Author

nedbat commented Aug 2, 2017

Duplicate of #198.

@nedbat nedbat closed this as completed Aug 2, 2017
@nedbat nedbat added major bug Something isn't working labels Jun 23, 2018
@nedbat nedbat added duplicate This issue or pull request already exists and removed 4.4 bug Something isn't working labels Aug 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

1 participant