[Quarkus 3.0.0.CR2] Performance regression introduced in Camel Quarkus 3.0.0-M1

[aldettinger]

A performance regression was spotted on the Camel Quarkus 3.0.0-M1 staging release.
After investigation it comes from quarkusio/quarkus#32289.

The fix will probably to revert to default io.netty.noUnsafe value. Let's see.

Some more notes:
We spot the performance regression when commit aa942c2 upgrades Quarkus to 3.0.0.Alpha4

 Camel Quarkus version  |  Duration  |   JVM req/s [%increase]   |  Native req/s [%increase]  |  Status  |
----------------------------------------------------------------------------------------------------------
                2.16.0  |       30m  |  38804.35 req/s [+0.00%]  |   22357.99 req/s [+0.00%]  |                                OK  |
        3.0.0-SNAPSHOT  |       30m  |  38857.07 req/s [+0.14%]  |   21166.75 req/s [-5.33%]  |  Potential performance regression  |

Looking at the logs, there is a message introduced in 3.0.0-SNAPSHOT:

2023-03-29 19:32:02,511 INFO  [io.net.uti.int.PlatformDependent] (Thread-1)
Your platform does not provide complete low-level API for accessing direct buffers reliably.
Unless explicitly requested, heap buffer will always be preferred to avoid potential system instability.

Whereas, we don't see regression just before the upgrade at commit 3143c76:

                2.16.0  |       10m  |  39046.96 req/s [+0.00%]  |   22398.94 req/s [+0.00%]  |      OK  |
        3.0.0-SNAPSHOT  |       10m  |  39299.79 req/s [+0.65%]  |   22559.46 req/s [+0.72%]  |      OK  |

With alpha4, native-image is passed with a new option "-J-Dio.netty.noUnsafe=true".
This behavior has been introduced in below Quarkus commit:
quarkusio/quarkus@ca155ca

When discarding this change with -Dquarkus.native.additional-build-args=-J-Dio.netty.noUnsafe=false in PerfRegressionCommand.java:

                2.16.0  |       10m  |  39030.75 req/s [+0.00%]  |   22435.33 req/s [+0.00%]  |      OK  |
        3.0.0-SNAPSHOT  |       10m  |  38671.56 req/s [-0.92%]  |   22119.79 req/s [-1.41%]  |      OK  |

The regression is gone.

Highly likely, it could come from a coupled effect where disabling Unsafe and having no cleaners would cause a netty performance degradation. More details below:
https://github.com/netty/netty/blob/9728d62b406df45513ee693dcfdf727957bc8020/common/src/main/java/io/netty/util/internal/PlatformDependent.java#L213

The fix would probably be to find a way to return to the original io.netty.noUnsafe value for both JVM and native.

[aldettinger]

Ok, the fix has been validated against quarkus-main:

                Camel Quarkus Throughput Performance Increase Compared to Previous Version                
 Camel Quarkus version  |  Duration  |   JVM req/s [%increase]   |  Native req/s [%increase]  |  Status  |
----------------------------------------------------------------------------------------------------------
                2.16.0  |       10m  |  38193.94 req/s [+0.00%]  |   22215.51 req/s [+0.00%]  |      OK  |
        3.0.0-SNAPSHOT  |       10m  |  38691.42 req/s [+1.30%]  |   22466.17 req/s [+1.13%]  |      OK  |