Add a builder and runner for the full connectors TCK

[arjantijms]

The connectors TCK that can be downloaded directly omits a few important tests (it is described in the TCK why it omits them, so it's on purpose and no mistake).

This build script and runner tries to bring back all tests.

Note; I couldn't find what magic the full platform scripts use to deploy an MDB archive, on which some of the extra tests depend. I now deploy this archive manually in the runner, but it remains a mystery.

The reproduction of the TCK works by downloading the binary platform TCK, which contains the war and ear files used by the tests, as well as the source platform TCK which contains the scripts to build the connectors TCK. It also downloads the reduced connector binary TCK which contains a jar file called "connector.jar", which is missing from the binary platform TCK. I couldn't find how to build this jar separately.

An alternative approach would be just downloading the platform TCK source and fully building it locally.

A few tests are failing when run as part of the full set of tests, but succeed when run individually. The platform TCK seems to contain some code to counter this, but I also could not find it.

[dmatej]

Hmm, seems something is wrong ...

mvn clean install -Ptck -pl :jakarta-connectors-full-tck,:glassfish-external-tck-connectors-full

[INFO]      [exec] [javatest.batch] Completed running 153 tests.
[INFO]      [exec] [javatest.batch] Number of Tests Passed      = 147
[INFO]      [exec] [javatest.batch] Number of Tests Failed      = 6
[INFO]      [exec] [javatest.batch] Number of Tests with Errors = 0

[INFO]      [exec] [javatest.batch] FAILED........com/sun/ts/tests/connector/localTx/msginflow/MDBClient.java#testActivationSpecImplRAA_from_servlet
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/connector/localTx/msginflow/MDBClient.java#testAppExceptionNotSupported_from_servlet
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/connector/localTx/msginflow/MDBClient.java#testAppExceptionRequired_from_servlet
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/connector/localTx/msginflow/MDBClient.java#testContextSetupCompleted_from_servlet
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/connector/localTx/msginflow/MDBClient.java#testEJBExceptionNotSupported_from_servlet
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/connector/localTx/msginflow/MDBClient.java#testEJBExceptionRequired_from_servlet
[INFO]      [exec] [javatest.batch] FAILED........com/sun/ts/tests/connector/localTx/msginflow/MDBClient.java#testEndpointActivationName_from_servlet
[INFO]      [exec] [javatest.batch] FAILED........com/sun/ts/tests/connector/localTx/msginflow/MDBClient.java#testGetEndpoinClass_from_servlet
[INFO]      [exec] [javatest.batch] FAILED........com/sun/ts/tests/connector/localTx/msginflow/MDBClient.java#testIBAnnoASConfigProp_from_servlet
[INFO]      [exec] [javatest.batch] FAILED........com/sun/ts/tests/connector/localTx/msginflow/MDBClient.java#testIBAnnoMsgTransactedUsingXid_from_servlet
[INFO]      [exec] [javatest.batch] FAILED........com/sun/ts/tests/signaturetest/connector/ConnectorSigTest.java#signatureTest

[INFO] Reactor Summary for TCK: Install Jakarta Connectors full TCK 7.0.9-SNAPSHOT:
[INFO]
[INFO] TCK: Install Jakarta Connectors full TCK ........... SUCCESS [02:56 min]
[INFO] TCK: Connectors .................................... SUCCESS [06:25 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  09:24 min
[INFO] Finished at: 2023-09-03T21:42:08+02:00

[dmatej]

It should not pass when the TCK failed ...

[arjantijms]

It should not pass when the TCK failed ...

Indeed it should not. I'll check what went wrong there.

[arjantijms]

@dmatej it seems to be an issue with all Ant based TCKs, good that you found it. Another case to always check the failed outcome of tests as well.

Can you check again? I fixed the two connectors ones, and will do a separate PR for the other ones.

[dmatej]

it seems to be an issue with all Ant based TCKs, good that you found it.

In TckRunner I resolved that by parsing the junit xml report; another option is to add set -e, but that would cause also that script fails with every error - sometimes it is not desired and means further changes in bash scripts (if I want to tolerate failure, add || true).

[dmatej]

[INFO]      [exec] [javatest.batch] Test results: passed: 147; failed: 6
...
[INFO]      [exec] BUILD FAILED
...
[INFO] Reactor Summary for TCK: Install Jakarta Connectors full TCK 7.0.9-SNAPSHOT:
[INFO]
[INFO] TCK: Install Jakarta Connectors full TCK ........... SUCCESS [04:06 min]
[INFO] TCK: Connectors .................................... FAILURE [10:01 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  14:12 min
[INFO] Finished at: 2023-09-07T12:47:11+02:00

Now it is better, except those failures :-)