TL;DR: By using a centralized JUnit Test Suite and shared Hibernate EntityManagerFactory, we drastically reduced integration test times in a large Gradle multi-module build from ∼15 minutes to ∼3 minutes.

As a project grows, integration testing can quickly become a bottleneck. It can end up becoming a monster of a test suite that takes a long time to execute. When faced with this dilemma, most projects will either limit the amount of integration tests or decrease the frequency with which they are executed. Both of these options are bad for the long-term stability of the system. This post is about how we are effectively running almost 3,000 tests with hundreds of integration tests in a multi-module Gradle project with 400 modules in only 3-4 minutes.

Running a profiler on the test code, it quickly became obvious that the biggest bottleneck of the tests was setting up EntityManagerFactory instances. The first part of the puzzle was being able to share the EntityManagerFactory across JUnit contexts, which was shown in yesterday's post. This post creates a reusable Hibernate TestBase that all integration tests can depend on. The setup in this post only has to be expanded to be shared across the entire test base.

In the classic JUnit + Gradle setup, each module's test suite is executed in its own JVM instance. Gradle does this by default to ensure isolation—tests in one module can't accidentally affect those in another, and each module's classpath is cleanly defined. However, this isolation comes at a cost: in a 400-module project, every module must set up its own EntityManagerFactory, effectively creating hundreds of connection pools and repeatedly initializing the same infrastructure.

Normally, there's no way to share tests or runtime state between modules. But by introducing a Suite, we can combine multiple test packages into a single context. With a bit of Gradle configuration, it's possible to run all integration tests within a single JVM process and reuse shared resources across modules.

In order to use the Suite features, a dependency on the following packages is needed:

  • testImplementation 'org.junit.platform:junit-platform-suite-api'
  • testRuntimeOnly 'org.junit.platform:junit-platform-suite-engine'

With those dependencies, the test Suite is set up using:

@Suite
@SuiteDisplayName("Test Suite")
@SelectPackages("dev.mhh.project")
@Execution(ExecutionMode.CONCURRENT)
class TestSuite { }

This will run the test suite for the single module. In order to run tests across all modules, the following dependencies need to be added for each other module:

  • testImplementation project(':project') to be able to use all the classes of the project.
  • testImplementation project(':project').sourceSets.test.output loads in all test classes, so the Suite can run them.
  • (Optional) testImplementation project(':project').sourceSets.testFixtures.output if you are using test fixtures.
  • (Optional) implementation project(':project') without this dependency, JaCoCo will not be able to report test coverage of the tests.

Here's an example of a dedicated test-suite module's build.gradle:

dependencies {
    rootProject.subprojects.findAll { it.name != 'test-suite' && it.name != 'dependency-platform' }.forEach {
        implementation it
        testImplementation it
        testImplementation it.sourceSets.testFixtures.output
        testImplementation it.sourceSets.test.output
    }

    testImplementation 'org.junit.platform:junit-platform-suite-api'
    testRuntimeOnly 'org.junit.platform:junit-platform-suite-engine'
}

In order to ensure parallel execution, you might have to add a junit-platform.properties file to the src/test/resources directory:

junit.jupiter.execution.parallel.enabled=true
junit.jupiter.execution.parallel.mode.default=concurrent
junit.jupiter.execution.parallel.mode.classes.default=concurrent
junit.jupiter.execution.parallel.config.strategy=dynamic

This setup is dynamic, but if you know your infrastructure, it might be worth tuning the number of parallel threads. We have had a lot of success with 4 threads in a system where we expect our build agents to have 6 vCPUs.

Ensure that you have properly handled the data isolation of your tests. If you are using the TestBase shared in yesterday's post, it is already handled.

Using this setup, we reduced the test times significantly and are now running thousands of tests in a few minutes. If you put effort into optimizing your tests, it should serve you well for multi-year projects with even more tests.