Introduce global timeouts for tests #80
Comments
|
In JUnit 4, if a Test super class specifies a Timeout Rule, then any Test subclass automatically inherits that and it becomes a kind of global timeout - as long as the Test super class is extended of course. This is very much needed in JUnit 5 as a replacement for this JUnit 4 feature to catch rogue tests that keep running and tie up the local build / build server unnecessarily as well as catch actual bugs. |
|
I can't imagine using "assertTimeoutPreemptively" all over the code. Global timeout is very needed. |
|
FYI: I posted an answer on Stack Overflow outlining how to implement a non-preemptive global timeout extension here: https://stackoverflow.com/a/50233807/388980 |
|
While this is a good step, a real timeout would be great, since it would, for example, prevent hanging tests from hanging on the build servers, etc. - which is why we used a timeout rule in Junit4 for all tests, just to make sure they would never hang and block a resource there. Surrounding all test code with assertTimeoutPreemptively is, as everyone hopefully agrees, not really a good solution. Honestly, it's not even a good solution for one method, since it's clunky. An extension and annotation would be cleaner. Currently, this is the thing that worries me most for switching from JUnit4 to 5 (since we had multiple situations - for various reasons - in the past where tests did not finish properly... We could prevent that with the Timeout rule, but now...?). |
|
Tentatively slated for the 5.3 backlog for team discussion |
|
FYI: #20 is potentially a blocker for this issue, but I have not yet labeled this issue as blocked since we have not yet decided how we wish to implement global timeouts. |
|
I agree with @FlorianSchaetz and @deepakab03. We like to cancel our integration tests if they take too much time (e.g. waiting for an event from a queue). We don't want to surround all of our tests with assertTimeoutPreemptively(). |
|
This is a potential blocker for Hadoop's migration to JUnit5. |
|
Any updates on this one? Having |
|
It's on the agenda for 5.5 together with #20. |
|
@yb172 in the meantime, if you're using Gradle 5.0+ you can set timeouts on your Test tasks to prevent command line/CI builds from hanging indefinitely. |
Good to know. Thanks for sharing! |
|
in progress |
|
Almost finished... Go, Marc, go! 🚴 |
This commit introduces declarative timeouts for testable and lifecycle methods. The API surface consists of a new `@Timeout` annotation that can be applied to `@Test`, `@TestTemplate`,` @TestFactory`, and lifecycle methods. In addition, it can be applied to a test class which has the same meaning as declaring it on all of its testable, but not its lifecycle methods. Moreover, default timeouts can be defined using configuration parameters for all of the above methods, all lifecycle methods, all testable methods, or each method type individually. The timeout is implemented in a preemptive manner, but without having to execute the test in a different thread. Instead, a single-thread `ExecutorService` is used to schedule an interrupt of the original test thread. This way, `ThreadLocal` will continue to work which is important for integration tests that use frameworks like Spring and its `@Transactional` support. Resolves #80.
|
Thank you! We in the Hadoop team owe you a beer or ice cream of your choice |
|
Beer-flavored ice cream?! 🍺🍨 |
|
if you can find some |
|
Is there anyway we can have preemptive global timeout. The current global time out will hang if the program runs infinitely. The old timeout rule works in Junit4 is really helpful to abort the specific unit test in some scenarios. |
That's already the case.
If you have a use case that JUnit Jupiter does not work with, please open a new issue with further details. |
Feature Request
Currently, in JUnit 4 it is not possible to have global timeouts for the tests. The
Timeoutrule does not guarantee that any@Aftermethod will be called. Seejunit-team/junit4#1231 (comment)
and the recently updated
https://github.com/junit-team/junit/wiki/Timeout-for-tests
It would be good to have a global timeout in JUnit Jupiter. Even better, it would be great if it would be possible to set different global timeouts for
@BeforeEach@Test, and@AfterEach, which could be overridden (if needed) directly in the test files via annotations.Related Issues
The text was updated successfully, but these errors were encountered: