pytest.mark.parametrize fails with lambdas #1111

mithrandi · 2015-10-04T15:09:50Z

For example, the following:

import pytest
test = pytest.mark.parametrize('a', [1, 2, 3])(lambda a: None)

fails like this:

INTERNALERROR> Traceback (most recent call last):                                                                                                                                                                                     [1/950]
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/main.py", line 84, in wrap_session
INTERNALERROR>     doit(config, session)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/main.py", line 122, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 521, in __call__
INTERNALERROR>     return self._docall(self.methods, kwargs)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 528, in _docall
INTERNALERROR>     firstresult=self.firstresult).execute()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 394, in execute
INTERNALERROR>     res = method(*args)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/main.py", line 142, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 521, in __call__
INTERNALERROR>     return self._docall(self.methods, kwargs)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 528, in _docall
INTERNALERROR>     firstresult=self.firstresult).execute()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 393, in execute
INTERNALERROR>     return wrapped_call(method(*args), self.execute)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 113, in wrapped_call
INTERNALERROR>     return call_outcome.get_result()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 138, in get_result
INTERNALERROR>     py.builtin._reraise(*ex)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 123, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 394, in execute
INTERNALERROR>     res = method(*args)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/runner.py", line 65, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/runner.py", line 72, in runtestprotocol
INTERNALERROR>     rep = call_and_report(item, "setup", log)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/runner.py", line 121, in call_and_report
INTERNALERROR>     report = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 521, in __call__
INTERNALERROR>     return self._docall(self.methods, kwargs)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 528, in _docall
INTERNALERROR>     firstresult=self.firstresult).execute()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 393, in execute
INTERNALERROR>     return wrapped_call(method(*args), self.execute)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 109, in wrapped_call
INTERNALERROR>     wrap_controller.send(call_outcome)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/skipping.py", line 157, in pytest_runtest_makereport
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 138, in get_result
INTERNALERROR>     py.builtin._reraise(*ex)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 123, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 394, in execute
INTERNALERROR>     res = method(*args)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/runner.py", line 227, in pytest_runtest_makereport
INTERNALERROR>     style=item.config.option.tbstyle)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/python.py", line 618, in _repr_failure_py
INTERNALERROR>     style=style)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/main.py", line 392, in _repr_failure_py
INTERNALERROR>     return excinfo.value.formatrepr()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/python.py", line 1540, in formatrepr
INTERNALERROR>     lines, _ = inspect.getsourcelines(function)
INTERNALERROR>   File "/usr/lib/python2.7/inspect.py", line 690, in getsourcelines
INTERNALERROR>     lines, lnum = findsource(object)
INTERNALERROR>   File "/usr/lib/python2.7/inspect.py", line 526, in findsource
INTERNALERROR>     file = getfile(object)
INTERNALERROR>   File "/usr/lib/python2.7/inspect.py", line 420, in getfile
INTERNALERROR>     'function, traceback, frame, or code object'.format(object))
INTERNALERROR> TypeError: <MarkDecorator 'parametrize' {'args': ('a', [1, 2, 3], <function <lambda> at 0x7fbd7d56aaa0>), 'kwargs': {}}> is not a module, class, method, function, traceback, frame, or code object

This reproducer may seem rather odd by itself, but something like this may arise with a decorator that returns a lambda.

I have confirmed this failure on pytest 2.7.0, 2.7.3, 2.8.0, and 2.8.1.

The text was updated successfully, but these errors were encountered:

nicoddemus · 2015-10-04T15:18:22Z

Thanks for the report @mithrandi! 😄

RonnyPfannschmidt · 2016-06-25T15:48:07Z

this looks like a lambda is mistaken for a positional arggument

Stranger6667 · 2016-07-26T05:51:07Z

Now the code above produces the following output:

cannot collect 'test' because it is not a function.

And it is not failure anymore. Tested with latest pytest version from repository on Python 2.7.12 & 3.5.1.

RonnyPfannschmidt · 2016-07-26T09:41:33Z

@mithrandi can thos one be closed based on the changes in master

The-Compiler · 2016-07-26T10:25:47Z

@RonnyPfannschmidt If something still fails but in a different way, I don't think it should be closed 😉

RonnyPfannschmidt · 2016-07-26T11:31:38Z

@The-Compiler as far as i can tell however we now have a normal report of the object there and the markdecorator invocation does consider a lambda an argument, not afunction that should be decorated

so i think with the change in reporting, everything works as designed

nicoddemus · 2016-07-26T11:51:56Z

I don't think we should support defining test functions using lambdas at all, I at least can't think of a valid use case for it. So I agree with @RonnyPfannschmidt that the error message is now appropriate, which I think was the heart of this issue.

@mithrandi what do you think?

mithrandi · 2016-07-26T14:31:53Z

The use case is that you are decorating a test function with pytest.mark.parametrize that was already decorated with another decorator that returns a lambda. As the user of these decorators, you don't really have control over the implementation of the decorators, and while you may be able to convince the implementor of the other decorator that they shouldn't use lambdas, the requirement seems a bit strange. Consider something simple like:

from functools import wraps
def enhance(f):
    return wraps(f)(lambda *a, **kw: f(enhancement, *a, **kw))

RonnyPfannschmidt · 2016-07-26T14:48:53Z

hmm, interesting, can you perhaps link to the library in question?

mithrandi · 2016-07-26T15:25:17Z

Unfortunately the exact circumstances where this came up are lost to me in the mists of time; I vaguely recall I was investigating something to do with https://github.com/HypothesisWorks/hypothesis-python but nothing beyond that.

RonnyPfannschmidt · 2016-07-26T15:31:25Z

i see, i propose closing then

The-Compiler · 2016-07-26T15:39:32Z

I can imagine there are other libaries doing this kind of thing. I'm going to be honest: I think pytest should support it, and it might be quite easy, but I'm not volunteering to do it - too much on my plate right now, sorry.

nicoddemus · 2016-07-26T15:54:41Z

In light of that, I agree with @The-Compiler that we should at least try to support it, if it's easy. What do you think @RonnyPfannschmidt? If so, I propose closing this one and opening another issue, like "Allow lambdas to be used as test functions" or so.

RonnyPfannschmidt · 2016-07-27T04:19:42Z

Nope, if the function name is <lambda>, the decorator is broken

Handling those different in marks means we are certain to break testsuites for bad reason

nicoddemus · 2016-07-27T10:51:58Z

Nope, if the function name is , the decorator is broken

I'm not sure that's the case here, at least with the example given by @mithrandi:

>>> from functools import wraps
>>> def enhance(f):
...     return wraps(f)(lambda *a, **kw: f(10, *a, **kw))
...
>>>
>>> @enhance
... def foo(v, x):
...     print v, x
...
>>> foo
<function foo at 0x0000000002AB55F8>
>>> foo(30)
10 30

Note that the name of the decorated function is foo, not lambda.

Is there anything special about lambdas that would prevent them from being collected as tests? I didn't take a look at the code, just wondering.

RonnyPfannschmidt · 2016-07-27T11:09:19Z

@nicoddemus the code in the marker specifically triggers for lambdas and ignores only them without checking into a wrapped function

nicoddemus · 2016-07-27T11:12:58Z

Oh I see, it complicates the marker code, not the collection code. Got it.

The-Compiler · 2016-08-05T10:22:49Z

Since we aren't sure if we should do this at all (I still think we should if possible...), this definitely shouldn't block 3.0.

RonnyPfannschmidt · 2017-01-18T15:01:50Z

closing as it seems we cant reproduce, please reopen if a reproducer happens to get known

mithrandi changed the title ~~pytest.mark.parametrize fails with lambdas~~ pytest.mark.parametrize fails with lambdas Oct 4, 2015

nicoddemus added type: bug problem that needs to be addressed topic: parametrize related to @pytest.mark.parametrize labels Oct 4, 2015

RonnyPfannschmidt mentioned this issue Jun 25, 2016

fix marker interaction in a backward incompatible way #1670

Closed

12 tasks

nicoddemus added this to the 3.0 milestone Jun 26, 2016

The-Compiler removed this from the 3.0 milestone Aug 5, 2016

nicoddemus closed this as completed Sep 14, 2017

pytestbot mentioned this issue Mar 14, 2018

Question: Can someone explain the below uses of @pytest.mark.usefixtures #3308

Closed

pytestbot mentioned this issue Apr 28, 2018

pytest.parametrize fails on test function with non-signature-preserving decorator on Python 3 #3435

Closed

pytestbot mentioned this issue Jun 6, 2018

metafunc.parametrize(..., scope="session") fails when mixed with other parametrizations #3542

Open

pytestbot mentioned this issue Jun 30, 2018

multiple mark.parametrize with tuples: ValueError: indirect given to [...] #3644

Closed

pytestbot mentioned this issue Jun 30, 2018

validate arguments of parametrize mark #3645

Open

pytestbot mentioned this issue Jul 9, 2018

Error in arguments to pytest.mark.parameterize fails all tests #3674

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pytest.mark.parametrize fails with lambdas #1111

pytest.mark.parametrize fails with lambdas #1111

mithrandi commented Oct 4, 2015

nicoddemus commented Oct 4, 2015

RonnyPfannschmidt commented Jun 25, 2016

Stranger6667 commented Jul 26, 2016

RonnyPfannschmidt commented Jul 26, 2016

The-Compiler commented Jul 26, 2016

RonnyPfannschmidt commented Jul 26, 2016

nicoddemus commented Jul 26, 2016 •

edited

Loading

mithrandi commented Jul 26, 2016

RonnyPfannschmidt commented Jul 26, 2016

mithrandi commented Jul 26, 2016

RonnyPfannschmidt commented Jul 26, 2016

The-Compiler commented Jul 26, 2016

nicoddemus commented Jul 26, 2016

RonnyPfannschmidt commented Jul 27, 2016

nicoddemus commented Jul 27, 2016

RonnyPfannschmidt commented Jul 27, 2016

nicoddemus commented Jul 27, 2016

The-Compiler commented Aug 5, 2016

RonnyPfannschmidt commented Jan 18, 2017

pytest.mark.parametrize fails with lambdas #1111

pytest.mark.parametrize fails with lambdas #1111

Comments

mithrandi commented Oct 4, 2015

nicoddemus commented Oct 4, 2015

RonnyPfannschmidt commented Jun 25, 2016

Stranger6667 commented Jul 26, 2016

RonnyPfannschmidt commented Jul 26, 2016

The-Compiler commented Jul 26, 2016

RonnyPfannschmidt commented Jul 26, 2016

nicoddemus commented Jul 26, 2016 • edited Loading

mithrandi commented Jul 26, 2016

RonnyPfannschmidt commented Jul 26, 2016

mithrandi commented Jul 26, 2016

RonnyPfannschmidt commented Jul 26, 2016

The-Compiler commented Jul 26, 2016

nicoddemus commented Jul 26, 2016

RonnyPfannschmidt commented Jul 27, 2016

nicoddemus commented Jul 27, 2016

RonnyPfannschmidt commented Jul 27, 2016

nicoddemus commented Jul 27, 2016

The-Compiler commented Aug 5, 2016

RonnyPfannschmidt commented Jan 18, 2017

nicoddemus commented Jul 26, 2016 •

edited

Loading