Executing Cucumber tests on Junit5 using the JUnit Platform Suite Engine creates empty html or json reports

[hartjm]

Issue

When executing Cucumber tests on Junit5 using the JUnit Platform Suite Engine an empty html and/or json report file is created upon test completion.

The cause of the issue is that 2 instances of CucumberTestEngine get created as part of the process with the second instance created after the tests have completed. The creation of this second instance instantiates a new version the plugins and the constructor of the HtmlFormatter overwrites the already created file.

The first instance of CucumberTestEngine is created indirectly via the creation of the SuiteTestEngine and the second directly via the classloader.

To Reproduce

The simplest example can be found by git cloning into the repo https://github.com/bonigarcia/mastering-junit5 and building the junit5-cucumber sub-project using Java 8 and maven. Once built follow these steps:

1. Update junit-platform.properties to add in the following property cucumber.plugin=html:target/foo.html
2. Run the CucumberTest via maven or IDE (IntelliJ in my case).
3. Observe the file foo.html created in target folder - size zero bytes.
4. Set a breakpoints in the class HtmlFormatter on line 45.
5. Debug the test and observe that when the breakpoint is hit on line 45 (first time) the html report is created and the contents good
6. Set a new breakpoint on line 17 of HtmlFormatter (in the constructor)
7. Run the code until the new breakpoint is hit - a second instance of CucumberTestEngine has been created after the execution of SuiteTestEngine
8. Run the test to the end and the good report file has now been overwritten by the instantiation of a new HtmlFormatter.

Expected behavior
Html reports are generated successfully when using the JUnit Platform Suite Engine to execute cucumber tests

Context & Motivation
I'd like to be able to migrate fully to Junit5 for all testing frameworks and preserve the current generation of html reports feature

Your Environment

• Cucumber 7.1.0
• Windows 10
• Apache Maven 3.8.1
• OpenJDK Java 1.80_312

[mpkorstanje]

Interesting.

As a work around you can use an annotation on CucumberTest to configure any plugins instead of the property file.

@Suite
@IncludeEngines("cucumber")
@SelectClasspathResource("io/github/bonigarcia")
@ConfigurationParameter(key = GLUE_PROPERTY_NAME, value = "io.github.bonigarcia")
@ConfigurationParameter(key = PLUGIN_PROPERTY_NAME, value = "html:target/foo.html")
public class CucumberTest {

}