Introduces native.surefire.skip to skip unit tests with -Dnative

[zakkak]

This patch allows native integration tests to run without also running
the corresponding unit tests. To achieve this one needs to issue:

mvn -Dnative -Dnative.surefire.skip verify
or
mvn -Dnative -Dnative.surefire.skip -pl integration-tests -amd verify
or
mvn -Dnative -Dnative.surefire.skip -pl integration-tests/artemis-core verify
etc.

[zakkak]

As discussed in https://groups.google.com/d/msg/quarkus-dev/zoMscmKGXo4/xYOdXB4CCQAJ

I hope I did not miss any native integration tests, I used git grep 'name>native<' integration-tests to find the patched pom.xml files.
Sorry for the long and boring PR :)

[gsmet]

Could you explain what's the rationale of this? Why do you need this? Thanks!

[zakkak]

Could you explain what's the rationale of this? Why do you need this? Thanks!

It's an effort to reduce the time spent in native-tests CI jobs.
In quarkus' github workflow the unit tests of the "integration tests" are currently ran in both linux-jvm-tests and native-tests jobs, so they can be safely skipped in the latter, saving some time.

Inspecting the log archive from [1] we get the following numbers per native-tests group,
where in X seconds out of ~Y ( Z% ):

• X are the seconds spent only on unit tests (/IT/ {next;} in awk skips over integration tests
• Y is the total seconds spent in the build-with-maven step of the job
• Z is X*100/Y

$ for i in Native\ Tests\ -\ */; do echo -n "$i\t"; awk '/IT/ {next;} /Time elapsed/ {sum+=$14} /] Total time/ {total+=$5*60} END {print sum " seconds out of ~"total" ( ~"sum*100/total"% )" }' $i/*; done 
Native Tests - Amazon/  50.288 seconds out of ~600 ( ~8.38133% )
Native Tests - Cache/   29.721 seconds out of ~900 ( ~3.30233% )
Native Tests - Data1/   62.602 seconds out of ~1680 ( ~3.72631% )
Native Tests - Data2/   68.245 seconds out of ~2100 ( ~3.24976% )
Native Tests - Data3/   58.306 seconds out of ~1740 ( ~3.35092% )
Native Tests - Data4/   47.887 seconds out of ~1500 ( ~3.19247% )
Native Tests - Data5/   39.493 seconds out of ~1200 ( ~3.29108% )
Native Tests - gRPC/    59.673 seconds out of ~2040 ( ~2.92515% )
Native Tests - HTTP/    51.792 seconds out of ~1500 ( ~3.4528% )
Native Tests - Main/    32.825 seconds out of ~540 ( ~6.0787% )
Native Tests - Messaging/       57.275 seconds out of ~1260 ( ~4.54563% )
Native Tests - Misc1/   60.54 seconds out of ~2321.58 ( ~2.60771% )
Native Tests - Misc2/   20.797 seconds out of ~900 ( ~2.31078% )
Native Tests - Misc3/   16.671 seconds out of ~420 ( ~3.96929% )
Native Tests - Misc4/   14.771 seconds out of ~600 ( ~2.46183% )
Native Tests - Security1/       42.237 seconds out of ~1080 ( ~3.91083% )
Native Tests - Security2/       117.713 seconds out of ~1680 ( ~7.00673% )
Native Tests - Security3/       37.152 seconds out of ~540 ( ~6.88% )
Native Tests - Spring/  43.213 seconds out of ~1200 ( ~3.60108% )

Note also that a similar native-tests job is being used in Mandrel [2] and will probably be used in Graal at some point in the future [3].

[1] https://github.com/quarkusio/quarkus/pull/10787/checks?check_run_id=879134957
[2] https://github.com/graalvm/mandrel/blob/f9289aa21c68cd8e2b32e0666a6ce3d72d5a1172/.github/workflows/quarkus.yml#L135
[3] oracle/graal#2349

[geoand]

This was brought up in https://groups.google.com/g/quarkus-dev/c/zoMscmKGXo4/m/U5vq7NjzCAAJ

[gastaldi]

I like this approach. Any objections on getting this merged?

[geoand]

Not from me

[zakkak]

Just a note here:

Is the maven plugin expected to be used outside of this repo to generate integration tests?
If so then the corresponding integration-tests/pom.xml would also need to be patched with https://github.com/quarkusio/quarkus/pull/10791/files#diff-32e5ef896402deb1c95ac6b572035786, but I am not sure how to do this.

[geoand]

I am not sure what you mean, can you explain a little further?

[zakkak]

If I understand correctly, in https://github.com/quarkusio/quarkus/pull/10791/files#diff-02eb384f391430108d12ba7ff54c4a11 I am patching a template used by the create-extension mojo.

This patch relies on:

<native.surefire.skip>${skipTests}</native.surefire.skip>

being present in integration-tests/pom.xml to skip surefire tests both with -DskipTests and -Dnative.surefire.skip.
Without the above, it only skips them with -Dnative.surefire.skip and not with -DskipTests, which might be unexpected.

So if the create-extension mojo is used outside of quarkusio/quarkus, -DskipTests won't skip surefire tests for native image integration tests and that might be an issue.

[geoand]

Hm... I think you are right. @aloubyansky would know more for sure :)

[aloubyansky]

@zakkak yes, create-extension can be used to create a standalone extension project.

[zakkak]

@aloubyansky in this case where is the best place to add <native.surefire.skip>${skipTests}</native.surefire.skip>?
Should it be in devtools/maven/src/main/resources/create-extension-templates/parent-pom.xml, devtools/maven/src/main/resources/create-extension-templates/integration-test-pom.xml or somewhere else?

[aloubyansky]

integration-test-pom.xml i think.

[zakkak]

I opened #10932 to further discuss this.