This README.md and other documentation is specific to a branch / release

---

API
Engine
Maven Plugin

A multithreaded Java test engine for parameterized class / scenario / integration testing based on JUnit 5 Platform.

Purpose

Parameterized Testing

Parameterized testing is traditionally test method oriented ...

• For each test method, execute the test method with the list of test arguments.

for (TestMethod testMethod : TestMethods) {
   for (TestArgument testArgument : TestArguments) {
      testMethod(testArgument)
   }   
}

Verifyica swaps the paradigm to be test argument oriented ...

• For each test argument, execute the list of test methods with the test argument.

for (TestArgument testArgument : TestArguments) {
   for (TestMethod testMethod : TestMethods) {
      testMethod(testArgument)
   }
}

Scenario Testing

Unit testing is traditionally used where test methods in a test class are isolated and independent of each other ...

• for a test class
  • all test methods are executed
  • i.e. a failure of one test method doesn't affect the testing of other test methods

Verifyica uses a dependent test method paradigm ...

• for a test class
  • order the test methods
  • for a test argument
    • execute the test methods sequentially
    • if a test method fails, remaining test methods are skipped

Notes

• default test method ordering is by test method name (or display name if defined), then by @Verifyica.Order annotations.

• if a specific test method is selected, run all test methods that are ordered before the selected test method

• IntelliJ has test method selection issues if multiple test methods are selected

  • In these scenarios, all test methods will be executed

Features

• Purpose built for integration testing using testcontainers-java
  • i.e. for each test argument, execute a set of test methods
• Scenario based testing
  • if a test method fails, remaining test methods will be marked as skipped
  • default test method execution based on @Verifyica.Order annotation
  • additional annotation @Verifyica.DependsOn can be used to define dependency order
• Multithreaded test class / test argument testing
  • configurable constraints
  • test arguments can be tested multithreaded
  • test methods are always tested sequentially for a test argument
• Virtual thread support (Java 21+)
• Properties file driven configuration
• Test class filtering
  • class name
  • class tags
• Test class interceptors (extensions)
• Engine interceptors (extensions)
  • set up / tear down global / external resources
• Cleanup of AutoClosable test classes / test arguments
• Cleanup of AutoClosable objects in Context maps
• Object / resource sharing via contexts
  • EngineContext
  • ClassContext
  • ArgumentContext
• LockManager
  • locking semantics
• IntelliJ support
• Maven support
  • via the Verifyica Maven Plugin
• JUnit5 ConsoleLauncher support

High Level Design

for (TestClass testClass : TestClasses) {

    instantiate test class instance

    execute @Verifyica.Prepare methods (superclass then subclass)

    for (TestArgument testArgument : TestArguments) {
    
       execute @Verifyica.BeforeAll methods (superclass then subclass)
      
       for (test method : Test methods) {
       
          execute @Verifyica.BeforeEach methods (superclass then subclass)
          
              execute test method
          
          execute @Verifyica.AfterEach methods (subclass then superclass)
       }
       
       execute @Verifyica.AfterAll methods (subclass then superclass)
    }

    execute @Verificya.Conclude methods (subclass then superclass)
   
    destroy test class instance
}

Examples

Examples:

• Examples

Example tests using testcontainers-java

• KafkaTest.java
• MongoDBTest.java
• NginxTest.java

Various other examples used for testing:

• Tests

Core Requirements

Java

Verifyica is compiled against/targets Java 8, but Java 21+ is recommended for virtual thread support.

Maven

When using Maven, the Verifyica Maven Plugin is required.

Plugin XML

Configure the Maven Surefire plugin to include standard JUnit tests (or exclude Verifyica tests) ...

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.5.2</version>
    <configuration>
        <!-- Only include JUnit tests -->
        <includes>
            <include>%regex[.*org.junit.*]</include>
        </includes>
        <systemPropertyVariables>
            <junit.jupiter.extensions.autodetection.enabled>true
            </junit.jupiter.extensions.autodetection.enabled>
        </systemPropertyVariables>
    </configuration>
</plugin>

Add the Verifyica Maven plugin ...

<plugin>
    <groupId>org.verifyica</groupId>
    <artifactId>verifyica-maven-plugin</artifactId>
    <version>VERSION</version>
    <executions>
        <execution>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Add the Verifyica API and Engine artifacts ...

Dependency XML

<dependency>
    <groupId>org.verifyica</groupId>
    <artifactId>verifyica-api</artifactId>
    <version>VERSION</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.verifyica</groupId>
    <artifactId>verifyica-engine</artifactId>
    <version>VERSION</version>
    <scope>test</scope>
</dependency>

Notes

• Verifyica test classes must be excluded from the Maven Surefire plugin

Ancillary Requirements

To perform integration testing, you need a test context/environment. Typically this is created using testcontainers-java and Docker (or another compatible container environment.)

Alternatively, tests can be performed on an external environment/resource. Ideally the external environment/resource would have an API to initialize and destroy it.

An EngineInterceptor can be used to initialize and destroy an external environment/resource before and after execution of tests.

Documentation

• Annotations
• API
• Interceptors
• Locking
• Configuration
• Filters
• Wiki

Notes

• Documentation is specific to a release

Contributing

See Contributing for details.

License

Apache License 2.0, see LICENSE.

Code of Conduct

See Code of Conduct for details.

DCO

See DCO for details.

Support

YourKit supports open source projects with innovative and intelligent tools for monitoring and profiling Java and .NET applications.

YourKit is the creator of YourKit Java Profiler, YourKit .NET Profiler, and YourKit YouMonitor.

---

Copyright (C) 2024-present Verifyica project authors and contributors