Allow running commands before and after running tests in all environments

[pytoxbot]

• Bitbucket: https://bitbucket.org/hpk42/tox/issue/275
• Originally reported by: @brechtm
• Originally created at: 2015-10-08T15:31:43.889

This is only relevant for detox. Beside envlist, we could introduce setupenv and breakdownenv.

This could be useful to combine the coverage data for projects with a unified Python 2/3 codebase. In this case coverage can never be 100% due to Python 2/3-specific code paths, so it makes sense to combine them.

[pytoxbot]

Original comment by @hynek

I’ve tried to understand what runtest_pre/post does by jumping through the commits and it seems to allow per-env hooks and not hooks for “before anything happens” and ”after everything is done”. Am I missing something? Are there any docs I haven’t found?

[pytoxbot]

Original comment by sallner

Add tests for tox_runtest_pre/post.

Resolves issue #275.

Supersedes pull request #175.

→ <>

[pytoxbot]

Original comment by @schinckel

I have a new pull request #193 which does the per-venv stuff related to this. I'm not 100% sure where the "overall" hooks should go though.

[pytoxbot]

Original comment by @schinckel

I'm not sure it's only relevant to detox: I have a similar patch (that I'll update to match the requirements detailed from PR175) that allows me to report each venv as a seperate "build" back to bitbucket.

[pytoxbot]

Original comment by @Itxaka

@hpk42 that sounds great! I will try to do the diff this afternoon, as my patch for that part....well, lets say Im not too sure its the proper way of doing it as I kind of did a trial and error approach as we didnt had that much time to implement it properly.

[pytoxbot]

Original comment by @hpk42

@Itxaka i am slowly getting back to tox issues FWIW. There are a few people who do good PRs but not yet others who review and merge PRs, handle other's issues, do releases etc.

As to this issue here it guess it makes sense to introduce and document hooks. I am happy about a PR (referencing this issue275) -- you could start with just putting up your tox-plus as a PR unchanged to see what you changed. We can take it from there.

[pytoxbot]

Original comment by @Itxaka

As this repo moves a bit slow, and we needed this at work, I half implemented it on https://github.com/Itxaka/tox-plus as plugin hooks so there could be plugins hijacking it to do that stuff.
Would be willing to port it to tox but my last patch has been sitting there ignored, so thats why I went with a fork :(

[pytoxbot]

Original comment by @aldanor

Yea, this would be very useful in general -- some setup like coverage erase and whatnot before running the envs, and finalization/cleanup like coverage combine / coverage report afterwards.

Should it be completely unconditional though? E.g., you may want to run tox -e lint in which case it doesn't really make sense to run coverage setup/teardown. How would you set that up?

Maybe the better naming would be something like before_test and after_test (aking to the commonly naming scheme on Travis)?