Skip to content

[tests] Rework test process and document it dropping use of prioer profile for straight property usage #2820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Feb 20, 2023
10 changes: 6 additions & 4 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,18 +40,20 @@ jobs:
- name: Set argLine command line option
if: ${{ matrix.java == '11' }}
run: echo 'ARG_LINE=-D"argLine=--illegal-access=permit"' >> $GITHUB_ENV
- name: Run all tests
if: ${{ matrix.os == 'ubuntu-latest' && matrix.java == '11' }}
run: echo 'ARG_LINE=-D"excludedGroups="' >> $GITHUB_ENV
- name: Skip tests that require illegal reflective access
if: ${{ matrix.os == 'ubuntu-latest' && matrix.java != '11' }}
run: echo 'ARG_LINE=-D"excludedGroups=RequireIllegalAccess"' >> $GITHUB_ENV
- name: Skip tests that require illegal reflective access
if: ${{ matrix.os != 'ubuntu-latest' && matrix.java != '11' }}
run: echo 'ARG_LINE=-D"excludedGroups=TestcontainersTests,RequireIllegalAccess"' >> $GITHUB_ENV
- name: Set env command line option
if: ${{ matrix.os == 'ubuntu-latest' }}
run: echo 'ENV_GITHUB="-Denv.GITHUB"' >> $GITHUB_ENV
- name: Active Profiles
run: ./mvnw help:active-profiles
- name: Test with Maven
if: ${{ matrix.os != 'windows-latest' }}
run: ./mvnw test -B -D"license.skip=true" -D"jacoco.skip=true" $ENV_GITHUB $ARG_LINE
run: ./mvnw test -B -D"license.skip=true" -D"jacoco.skip=true" $ARG_LINE
- name: Test with Maven
if: ${{ matrix.os == 'windows-latest' && matrix.java == '11' }}
run: ./mvnw test -B -D"license.skip=true" -D"jacoco.skip=true" -D"argLine=--illegal-access=permit"
Expand Down
41 changes: 41 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,44 @@ Mybatis-core is now being auto formatted. Given nature of some code logic with
- ```// @formatter:on``` to end the block of unformatted code

If comment sections need same behaviour such as javadocs, note that the entire block must be around entire comment as direct usage does not properly indicate that formatter treats it all as one comment block regardless.

Tests
-----

Mybatis-3 code runs more expressive testing depending on jdk usage and platform.

By default, we set ```<excludedGroups>TestcontainersTests</excludedGroups>``` which will exclude a subset of tests with @Tag('TestcontainersTests'). Further, if pre jdk 16, we will further exclude record classes from executions further reducing tests.

When using jdk 16+, we adjust the rule to ```<excludedGroups>TestcontainersTests,RequireIllegalAccess</excludedGroups>```.

When we run on ci platform, we further make adjustments as needed. See [here](.github/workflows/ci.yaml) for details.

As of 2/20/2023, using combined system + jdk will result in given number of tests ran. This will change as tests are added or removed over time.

without adjusting settings (ie use as is, platform does not matter)

- any OS + jdk 11 = 1730 tests
- any OS + jdk 17 = 1710 tests
- any OS + jdk 19 = 1710 tests
- any OS + jdk 20 = 1710 tests
- any OS + jdk 21 = 1710 tests

our adjustments for GH actions where platform does matter

- windows + jdk 11 = 1730 tests
- windows + jdk 17 = 1710 tests
- windows + jdk 19 = 1710 tests
- windows + jdk 20 = 1710 tests
- windows + jdk 21 = 1710 tests

- linux + jdk 11 = 1765 tests
- linux + jdk 17 = 1745 tests
- linux + jdk 19 = 1745 tests
- linux + jdk 20 = 1745 tests
- linux + jdk 21 = 1745 tests

- mac + jdk 11 = 1730 tests
- mac + jdk 17 = 1710 tests
- mac + jdk 19 = 1710 tests
- mac + jdk 20 = 1710 tests
- mac + jdk 21 = 1710 tests
18 changes: 4 additions & 14 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,8 @@

<properties>
<clirr.comparisonVersion>3.4.6</clirr.comparisonVersion>
<!-- Add slow test groups here and annotate classes similar to @Tag('groupName'). -->
<!-- Excluded groups are ran on github ci, to force here, pass -d"excludedGroups=" -->
<excludedGroups>TestcontainersTests</excludedGroups>
<maven.compiler.testCompilerArgument>-parameters</maven.compiler.testCompilerArgument>
<module.name>org.mybatis</module.name>
Expand Down Expand Up @@ -335,7 +337,7 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemProperties>
<systemPropertyVariables>
<property>
<name>derby.stream.error.file</name>
<value>${project.build.directory}/derby.log</value>
Expand All @@ -344,7 +346,7 @@
<name>derby.system.home</name>
<value>${project.build.directory}</value>
</property>
</systemProperties>
</systemPropertyVariables>
</configuration>
</plugin>
<plugin>
Expand Down Expand Up @@ -418,18 +420,6 @@
</build>

<profiles>
<profile>
<!-- Run slow tests only on github ci, to force run otherwise use -D"env.GITHUB" -->
<id>github-ci</id>
<activation>
<property>
<name>env.GITHUB</name>
</property>
</activation>
<properties>
<excludedGroups />
</properties>
</profile>
<profile>
<id>pre16</id>
<activation>
Expand Down