Tests failing because Logback appenders don't exist #457
Comments
|
In addition, we could also create a Jupiter extension that would automagically reset the Logback logging system assuming the default @ExtendWith(ResetLogbackLoggingExtension.class)
class MyTest {
// ...
}And if there is a need to customize the logging configuration, then it could be registered programmatically: class MyTest {
@RegisterExtension
static final ResetLogbackLoggingExtension resetLogbackExtension =
ResetLogbackLoggingExtension.builder()
.withLoggingConfiguration("my-custom-test-logback-config.xml")
.build()
// ...
}The extension would reset Logback after all tests have run, e.g. in a |
|
And now for some strange reason, doing a |
|
Never mind, it still fails in Gitpod.... |
sleberknight
added a commit
that referenced
this issue
Feb 6, 2024
* Update InMemoryAppenderExtension so that it will reset the Logback logging system if it doesn't find the expected appenders. It only tries once, which will fix the case when a previously-executed test corrupts the logging system, like DropwizardAppExtension and DropwizardClientExtension both do by stopping and detaching ALL appenders. Fixes #457
dsingley
pushed a commit
that referenced
this issue
Feb 6, 2024
* Update InMemoryAppenderExtension so that it will reset the Logback logging system if it doesn't find the expected appenders. It only tries once, which will fix the case when a previously-executed test corrupts the logging system, like DropwizardAppExtension and DropwizardClientExtension both do by stopping and detaching ALL appenders. Fixes #457
This was referenced Feb 6, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
After the last few commits were merged, we suddenly started seeing
InMemoryAppenderTestandLoggingComparisonListenerTestfail. The errors look like:This happens in the GitHub Actions CI environment, and also when running tests from the command line in Gitpod, i.e, via
mvn test. But this does not happen when running the tests using Maven from the command line on my MacBook Pro or from within IntelliJ. All the tests pass when running them using the Java Test Runner in VSCode in Gitpod.The problem is that
InMemoryAppenderTestandLoggingComparisonListenerTestboth rely on specific Logback appenders, since they are testing Logback and appending behavior. But in the GitHub Actions environment and from the command line in Gitpod (both of which are Linux), the appenders are simply gone when these tests run.There are two factors at play here. First, Maven's Surefire test plugin runs tests in a non-deterministic manner (as it should since we don't want tests to require execution in a specific order). Second, some tests are using Dropwizard testing extensions like
DropwizardAppExtensionandDropwizardClientExtension, both of which - rather unfortunately - mess with the Logback logging configuration once tests have executed. They do this presumably because they actually create and start a Dropwizard application for testing purposes. Both of these extensions use theDropwizardTestSupportclass, which does the following in its privatestopIfRequiredmethod:The key part is the call to
detachAndStopAllAppenders. When you combine that with the non-deterministic test execution order, if tests that use a Dropwizard extension execute beforeInMemoryAppenderTestorLoggingComparisonListenerTest, then the appenders will have been removed already, and thus our tests fail.I don't think it's great for the Dropwizard extensions to detach and stop all appenders. I'd rather they not mess with the Logback logging configuration at all during tests. But I doubt that's going to change. So, the solution here is that we need to ensure that the expected appender exists. And specifically, we need to change
InMemoryAppenderExtensionso that if it sees that the appender does not exist in itsbeforeEachmethod, it resets the Logback logging configuration.For this issue, make the fix in
InMemoryAppenderExtension. But later the reset mechanism can be exposed so that any test could do this. And this would allow tests that use the useful but troublesome Dropwizard extensions to reset the logging configuration in a@AfterAllmethod. This would have the side-benefit of ensuring tests can log. For many years now, it has annoyed me that if a test using a Dropwizard extension runs early in the test execution, none of the subsequent tests log anything (because all the appenders were removed).Here is output from a recent test run in GitHub Actions:
The text was updated successfully, but these errors were encountered: