Three ways to run multiple JUnit 5 test classes

Running multiple JUnit 5 test classes with @Suite, Eclipse, and Maven.

As a Java developer, you often have the need to execute multiple JUnit test classes on your local machine. This blog post describes three ways for doing so:

• JUnit 5 Suites
• Eclipse “Run As”
• Maven Surefire Plugin

A project with the examples below can be found on GitHub.

JUnit 5 Suites

The @Suite annotation turns a class into a test suite, allowing it to choose test classes to be executed with the @SelectClasses annotation:

@Suite
@SelectClasses({ ATest.class, BTest.class })
public class SuiteWithClasses {
}

Apart from cherry-picking single classes, you can also use @SelectPackages for choosing whole packages:

@Suite
// also runs tests from sub-packages:
@SelectPackages("systems.enji.junit.suite")
public class SuiteWithPackages {
}

Test selection based on JUnit Tags is also possible:

@Suite
@SelectPackages("systems.enji.junit.suite")
// the following annotation without @SelectPackages would not run any tests:
@IncludeTags({"custom-tag-1", "custom-tag-2"})
public class SuiteWithTags {
}

More elaborate inclusions and exclusions (using pattern matching) can be implemented with other annotations from the org.junit.platform.suite.api package.

The simplest way to use JUnit 5 with these annotations is to add the following dependencies to your POM:

  <dependencies>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter</artifactId>
      <version>5.8.2</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.platform</groupId>
      <artifactId>junit-platform-suite</artifactId>
      <version>1.8.2</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

The possibility to create test suites was initially missing in JUnit 5, which meant that the JUnit “Vintage” engine had to be used whenever test suites were required in the context of JUnit 5 test runs. In September 2021, the @Suite annotation was added to version 1.8 of the JUnit Platform (which corresponds to version 5.8 of JUnit Jupiter). The junit-platform-suite module now shows up as top-level platform module in JUnit’s dependency diagram.

Eclipse “Run As”

By right-clicking a package in the Package Explorer and selecting the context menu entry “Run As -> JUnit Test”, Eclipse executes all JUnit tests inside this package. Note that

• both normal JUnit test classes and suites are executed, so if a test is referenced by a suite in the same package, this test will be executed twice
• sub-packages are not included in the execution
• this context menu entry is not available when selecting multiple packages or classes

In simple scenarios, this is a convenient approach to running multiple test classes in the absence of Junit test suites.

Maven Surefire Plugin

By default, the “test” goal of the Maven Surefire Plugin

mvn test

executes all test classes that match one of the following patterns:

• Test*.java
• *Test.java
• *Tests.java
• *TestCase.java

Just like the Eclipse IDE, Surefire executes both normal test classes and suites, which might lead to duplicate test runs if not configured properly.

For filtering tests according to their JUnit tags, the tag names can be provided as command-line parameter:

mvn test -Dgroups="custom-tag-1, custom-tag-2"

Test reports (TXT and XML) are generated inside the “target/surefire-reports” directory. For HTML reports, you also need the Maven Surefire Report Plugin. There is no integration of test results into the “JUnit” view of Eclipse, which means that this way of running tests is less comfortable for developers (but reasonable for build servers).

Even though the Surefire plugin is integrated into Maven, executing JUnit 5 tests requires a more recent version, so you have to explicitly add the following dependency to your pom:

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.22.2</version>
      </plugin>
    </plugins>
  </build>