Run each file containing a unit test class as subprocess

[wolfc01]

Hello,

Consider the following very simple example in a file named test.py:

import unittest

class Test(unittest.TestCase):
  """test """
...

if __name__ == '__main__':
  unittest.main()

before using pytest, i used to test the above on a build server as following python test.py which effectively runs the test in its own process. Then each unittest is strictly separated from other unittests, as the OS frees up all resources after the test has run.

Because of several requirements which I don't want to explain here, I need to have each test as depicted above to run in it's own process, and use the process exit code for pass(0)/fail(1).

Is above possible using pytest?
Best regards,
Carl.

[pytestbot]

GitMate.io thinks possibly related issues are #2023 (subprocess tests hang), #1484 (running py.test), #2544 (Run tests truly programmatically), #20 (Running a test module as a script.), and #2187 (Should test runs using lastfailed still scan all test files?).

[The-Compiler]

It sounds like you want the pytest-forked plugin.

[wolfc01]

Hello,
Thanks for the reply, I just tried it.

The following code framework shows briefly what I want to do:

class Test(unittest.TestCase):
  """test """

  def test_1(self):
    """a test"""

p = os.fork()
if p==0: #child
  unittest.main()
else:#parent
  pid, res = os.waitpid(p, 0)
  assert(res==0)

Above runs perfectly when i run python test.py, but not using pytest, or pytest --forked.

Best regards
Carl.

[wolfc01]

Oops, accidentally closed, sorry.

[RonnyPfannschmidt]

you can just pass each file to a new py.test call if thats your issue,
alternatively some kind of plugin would be needed as py.test currently doesn't do that kind of orchestration on its own

[wolfc01]

Hello Ronny,

you can just pass each file to a new py.test call if thats your issue,

Unfortunately not. On a buildserver, i just want want to run pytest on the entire tree, that saves maintenance and lowers the chance on human errors.

alternatively some kind of plugin would be needed

I think so yes. Is it easy to write such a plugin?

Best regards
Carl.

[RonnyPfannschmidt]

@wolfc01 potentially, it depends on the use case you plan to implement - and how detailed you need your reports

[wolfc01]

Hello Ronny,

quite simple:

• run a number of python files containing unittests classes one after the another using subprocess
• for each subprocess: sum all abs(exit codes) , collect all stdout and stderror, and dump entire output to a file.
  May be it is better to create a simple runner doing the above and not using pytest, but i'm not sure what gooedies of pytest I loose then...

Best regards
Carl.

[RonnyPfannschmidt]

since you want to run unittest only tests, you dont loose much of pytest
it highly likely that your reporting needs will be met better by wireing up a simple runner

[wolfc01]

Ronny,

I Agree.

Best regards,
Carl.