Startup performance issue with OpenAPI

[loicmathieu]

Describe the bug
Startup performance degradation between releases observed specifficaly when using the OpenAPI extention.

Expected behavior
Startup performance should be better or at least consistent between releases.

Actual behavior
Switching Quarkus versions with the stock demo OpenAPI application from the tutorial, we can see the following augmentation time increasing releases after releases:

• 0.15.0 (smallrye-openapi:1.1.3) => Quarkus augmentation completed in 1336ms
• 0.14.0 (smallrye-openapi:1.1.1) => Quarkus augmentation completed in 1258ms
• 0.13.0 (smallrye-openapi:1.1.1) => Quarkus augmentation completed in 1109ms
• 0.12.0 (smallrye-openapi:1.1.1) => Quarkus augmentation completed in 1020ms

In a "real world apps" with the following features :
groal, cdi, flyway, hibernate-orm, hibernate-validator, jdbc-postgresql, narayana-jta, resteasy, resteasy-jsonb, smallrye-health, smallrye-metrics, smallrye-openapi, swagger-ui we can see a big degradation in the Quarkus augmentation time between the release 0.13 and 0.14 and a medium one between 0.14 and 0.15:

• 0.15 => Quarkus augmentation completed in 2561ms
• 0.14 => Quarkus augmentation completed in 2154ms
• 0.13 => Quarkus augmentation completed in 1445ms

I didn't have time to dig deeper but if someone point me to some changes between 0.13 and 0.14 that can be related or at least to what is done at augmentation time for OpenAPI (we didn't see this kind of startup performance degradation without it) I can investiguate.

To Reproduce
Steps to reproduce the behavior:

1. Create an API with documentation (the more the best)
2. Try with current version
3. Try with version 0.13

Environment (please complete the following information):

• Output of uname -a or ver: Linux 5.0.0-15-generic Automatic Hibernate bytecode enhancement #16-Ubuntu SMP Mon May 6 17:41:33 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
• Output of java -version: openjdk version "11.0.3" 2019-04-16
• GraalVM version (if different from Java): not tested with GraalVM
• Quarkus version or git rev: 0.15.0

[kenfinnigan]

Are you able to verify similar degradation with JDK 8?

[loicmathieu]

From my "real world apps" with Java 8 :

• 0.15 => Quarkus augmentation completed in 4256ms
• 0.13 => Quarkus augmentation completed in 2009ms

The degradation seems to be worse but as I make realy quick test with some stuff running on my computer I don't think that the differences between 8 and 11 are relevant. But what is relevent is that still in Java 8 startup time have a huge regression between Quarksu 0.13 and Quarkus 0.15 when using the OpenAPI extention

[kenfinnigan]

As far as I can tell the only changes to OpenAPI between those releases are:

• 760adaf#diff-f7426777fbbaee3e12175eae8d11f0c4
• c2aca15#diff-f7426777fbbaee3e12175eae8d11f0c4
• b874f88#diff-f7426777fbbaee3e12175eae8d11f0c4

Not sure if 760adaf#diff-6b97e6937f293937c85b7a9ae73919f7R73 is the problem

[loicmathieu]

I more suspect something in this big PR b874f88#diff-f7426777fbbaee3e12175eae8d11f0c4 as I don't see anything in the two others that can cause a severe performance degradation.
I will have a quick look today if I find some times ...

[gsmet]

@loicmathieu could you maybe attach a profiler and see what you can get from it?

[loicmathieu]

@gsmet profiling startup is hard, I try using the startup profiler of visualvm but cannot see anything relevant (no OpenAPI specific hotpoint). I will try to investiguate a little more tomorrow if I find some time.

[kenfinnigan]

#2569 will hopefully help with startup when OpenAPI is involved as well

[loicmathieu]

Hi, as this commit is merged to master I recompile Quarkus from master to see if it changes something but unfortunatly not ... Still the same startup time ...

[loicmathieu]

I retry today with the master with #2569 and it seems to help, I think that tommorow my computer was misbehave.
It will be greate if some others can make the same test with real application with OpenAPI and version 0.13, 015 and a fresh build from master to check if we can close this issue or not.

[gsmet]

If you saw some improvements, let's close it. We can still reopen a new one if not.

[loicmathieu]

Hi,
With 0.16 release I see a real boost of performance in quarkus augmentation.
From the 2 projects I am currently working on:

• Quarkus augmentation completed in 3715ms => Quarkus augmentation completed in 1877ms
• Quarkus augmentation completed in 2804ms => Quarkus augmentation completed in 1550ms

And removing openapi extention only improve the performance of the augmentation of 70ms so it's negligeable!

So good works! And no need to re-open it ;)