OOM in tests since Quarkus 3.13.0 #42303

mschorsch · 2024-08-03T04:48:00Z

Describe the bug

Since Quarkus 3.13.0 we get the following exception in our tests:

    java.lang.RuntimeException: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkus.deployment.steps.ConfigGenerationBuildStep#generateConfigClass threw an exception: java.lang.OutOfMemoryError: Java heap space
        at org.objectweb.asm.MethodWriter.visitFrameEnd(MethodWriter.java:1850)
        at org.objectweb.asm.MethodWriter.computeAllFrames(MethodWriter.java:1648)
        at org.objectweb.asm.MethodWriter.visitMaxs(MethodWriter.java:1548)
        at io.quarkus.gizmo.MethodCreatorImpl.write(MethodCreatorImpl.java:140)
        at io.quarkus.gizmo.ClassCreator.writeTo(ClassCreator.java:233)
        at io.quarkus.gizmo.ClassCreator.close(ClassCreator.java:257)
        at io.quarkus.deployment.configuration.RunTimeConfigurationGenerator$GenerateOperation.run(RunTimeConfigurationGenerator.java:523)
        at io.quarkus.deployment.steps.ConfigGenerationBuildStep.generateConfigClass(ConfigGenerationBuildStep.java:318)
        at java.base/java.lang.invoke.LambdaForm$DMH/0x0000732419437800.invokeVirtual(LambdaForm$DMH)
        at java.base/java.lang.invoke.LambdaForm$MH/0x0000732419492400.invoke(LambdaForm$MH)
        at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:157)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at io.quarkus.builder.BuildContext$$Lambda/0x00007324193e5458.run(Unknown Source)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.runWith(Thread.java:1596)
        at java.base/java.lang.Thread.run(Thread.java:1583)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)
        at io.quarkus.test.junit.QuarkusTestExtension.throwBootFailureException(QuarkusTestExtension.java:634)
        at io.quarkus.test.junit.QuarkusTestExtension.interceptTestClassConstructor(QuarkusTestExtension.java:718)
        at java.base/java.util.Optional.orElseGet(Optional.java:364)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

        Caused by:
        java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
                [error]: Build step io.quarkus.deployment.steps.ConfigGenerationBuildStep#generateConfigClass threw an exception: java.lang.OutOfMemoryError: Java heap space
                at org.objectweb.asm.MethodWriter.visitFrameEnd(MethodWriter.java:1850)
                at org.objectweb.asm.MethodWriter.computeAllFrames(MethodWriter.java:1648)
                at org.objectweb.asm.MethodWriter.visitMaxs(MethodWriter.java:1548)
                at io.quarkus.gizmo.MethodCreatorImpl.write(MethodCreatorImpl.java:140)
                at io.quarkus.gizmo.ClassCreator.writeTo(ClassCreator.java:233)
                at io.quarkus.gizmo.ClassCreator.close(ClassCreator.java:257)
                at io.quarkus.deployment.configuration.RunTimeConfigurationGenerator$GenerateOperation.run(RunTimeConfigurationGenerator.java:523)
                at io.quarkus.deployment.steps.ConfigGenerationBuildStep.generateConfigClass(ConfigGenerationBuildStep.java:318)
                at java.base/java.lang.invoke.LambdaForm$DMH/0x0000732419437800.invokeVirtual(LambdaForm$DMH)
                at java.base/java.lang.invoke.LambdaForm$MH/0x0000732419492400.invoke(LambdaForm$MH)
                at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
                at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:157)
                at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
                at java.base/java.lang.reflect.Method.invoke(Method.java:580)
                at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
                at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
                at io.quarkus.builder.BuildContext$$Lambda/0x00007324193e5458.run(Unknown Source)
                at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
                at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
                at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
                at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
                at java.base/java.lang.Thread.runWith(Thread.java:1596)
                at java.base/java.lang.Thread.run(Thread.java:1583)
                at org.jboss.threads.JBossThread.run(JBossThread.java:483)
            at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:335)
            at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:254)
            at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:58)
            at io.quarkus.test.junit.QuarkusTestExtension.doJavaStart(QuarkusTestExtension.java:219)
            at io.quarkus.test.junit.QuarkusTestExtension.ensureStarted(QuarkusTestExtension.java:601)
            at io.quarkus.test.junit.QuarkusTestExtension.beforeAll(QuarkusTestExtension.java:651)
            ... 1 more

            Caused by:
            io.quarkus.builder.BuildException: Build failure: Build failed due to errors
                [error]: Build step io.quarkus.deployment.steps.ConfigGenerationBuildStep#generateConfigClass threw an exception: java.lang.OutOfMemoryError: Java heap space
                at org.objectweb.asm.MethodWriter.visitFrameEnd(MethodWriter.java:1850)
                at org.objectweb.asm.MethodWriter.computeAllFrames(MethodWriter.java:1648)
                at org.objectweb.asm.MethodWriter.visitMaxs(MethodWriter.java:1548)
                at io.quarkus.gizmo.MethodCreatorImpl.write(MethodCreatorImpl.java:140)
                at io.quarkus.gizmo.ClassCreator.writeTo(ClassCreator.java:233)
                at io.quarkus.gizmo.ClassCreator.close(ClassCreator.java:257)
                at io.quarkus.deployment.configuration.RunTimeConfigurationGenerator$GenerateOperation.run(RunTimeConfigurationGenerator.java:523)
                at io.quarkus.deployment.steps.ConfigGenerationBuildStep.generateConfigClass(ConfigGenerationBuildStep.java:318)
                at java.base/java.lang.invoke.LambdaForm$DMH/0x0000732419437800.invokeVirtual(LambdaForm$DMH)
                at java.base/java.lang.invoke.LambdaForm$MH/0x0000732419492400.invoke(LambdaForm$MH)
                at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
                at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:157)
                at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
                at java.base/java.lang.reflect.Method.invoke(Method.java:580)
                at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
                at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
                at io.quarkus.builder.BuildContext$$Lambda/0x00007324193e5458.run(Unknown Source)
                at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
                at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
                at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
                at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
                at java.base/java.lang.Thread.runWith(Thread.java:1596)
                at java.base/java.lang.Thread.run(Thread.java:1583)
                at org.jboss.threads.JBossThread.run(JBossThread.java:483)
                at io.quarkus.builder.Execution.run(Execution.java:123)
                at io.quarkus.builder.BuildExecutionBuilder.execute(BuildExecutionBuilder.java:79)
                at io.quarkus.deployment.QuarkusAugmentor.run(QuarkusAugmentor.java:159)
                at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:331)
                ... 6 more

                Caused by:
                java.lang.OutOfMemoryError: Java heap space
                    at org.objectweb.asm.MethodWriter.visitFrameEnd(MethodWriter.java:1850)
                    at org.objectweb.asm.MethodWriter.computeAllFrames(MethodWriter.java:1648)
                    at org.objectweb.asm.MethodWriter.visitMaxs(MethodWriter.java:1548)
                    at io.quarkus.gizmo.MethodCreatorImpl.write(MethodCreatorImpl.java:140)
                    at io.quarkus.gizmo.ClassCreator.writeTo(ClassCreator.java:233)
                    at io.quarkus.gizmo.ClassCreator.close(ClassCreator.java:257)
                    at io.quarkus.deployment.configuration.RunTimeConfigurationGenerator$GenerateOperation.run(RunTimeConfigurationGenerator.java:523)
                    at io.quarkus.deployment.steps.ConfigGenerationBuildStep.generateConfigClass(ConfigGenerationBuildStep.java:318)
                    at java.base/java.lang.invoke.LambdaForm$DMH/0x0000732419437800.invokeVirtual(LambdaForm$DMH)
                    at java.base/java.lang.invoke.LambdaForm$MH/0x0000732419492400.invoke(LambdaForm$MH)
                    at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
                    at java.base/java.lang.reflect.Method.invoke(Method.java:580)
                    at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
                    at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
                    at io.quarkus.builder.BuildContext$$Lambda/0x00007324193e5458.run(Unknown Source)
                    at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
                    at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
                    at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
                    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
                    at java.base/java.lang.Thread.runWith(Thread.java:1596)
                    at java.base/java.lang.Thread.run(Thread.java:1583)
                    at org.jboss.threads.JBossThread.run(JBossThread.java:483)

Expected behavior

No response

Actual behavior

No response

How to Reproduce?

~~I didn't managed to create a reproducer yet.~~

#42303 (comment)

Output of `uname -a` or `ver`

Linux

Output of `java -version`

Java 21

Quarkus version or git rev

Quarkus 3.13.0

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

Gradle 8.9

Additional information

No response

The text was updated successfully, but these errors were encountered:

gsmet · 2024-08-03T07:40:27Z

Could you be affected by #42298 ?

If not, I think we will need to have a look at the heap dump. Add -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/your/path to the Surefire argLine to get a heap dump.

Might be a good idea to share it with me (don't make it public). You can send me a private link to gsmet at redhat dot com.

gsmet · 2024-08-03T08:37:27Z

@mschorsch might be worth trying #42304

gsmet · 2024-08-03T09:33:42Z

OK, I don't think #42304 will make any difference so I think getting heap dumps is in order.

gsmet · 2024-08-03T09:34:55Z

What would really help is if you could somehow identify which test class is running when you get the OOM and add a breakpoint there in 3.12.3 and get a heap dump too so that we could compare both.

(But interested in the OOM heap dump first as there might be something obvious in it)

mschorsch · 2024-08-03T10:50:58Z

@mschorsch might be worth trying #42304

I tried it but it didn't seem to fix the issue.

mschorsch · 2024-08-03T10:53:57Z

If I run the tests individually or only a subset of all tests then I don't get an OOM.

gsmet · 2024-08-03T10:56:12Z

Yeah, I think we will need a heap dump here. There's something leaking somewhere.

mschorsch · 2024-08-03T11:55:00Z

@gsmet

Might be a good idea to share it with me (don't make it public). You can send me a private link to gsmet at redhat dot com.

I am not sure if I am authorized to provide a heap dump. I have to clarify this beforehand. Sorry.

Could you be affected by #42298 ?

But good (or bad :) ) news, yes, this seems the same issue. I've replaced @QuarkusTestResource(OidcWiremockTestResource::class) with @QuarkusTestResource(OidcWiremockTestResource::class, restrictToAnnotatedClass = true) and with Quarkus 3.12.3 I also get an OOM in exactly the same test as in Quarkus 3.13.0

gsmet · 2024-08-03T12:18:04Z

Ah OK, good. I'm busy on something else this week-end. If you had the time to assemble a small reproducer, that would be very helpful for me to have a look on Monday.

Thanks!

gsmet · 2024-08-03T12:19:09Z

(And yes, totally agree that sharing a heap dump can be sensitive but given the problem looks identified, if you have a small reproducer, I'll be able to track it without a heap dump)

mschorsch · 2024-08-04T14:32:20Z

@gsmet I managed to create a reproducer: oom-reproducer.zip

./gradlew test --console=plain --info

You can add the following argument to get a heap dump:

tasks.test {
    jvmArgs("-XX:+HeapDumpOnOutOfMemoryError")
}

gsmet · 2024-08-05T07:32:16Z

Thanks for the reproducer.

I end up with an OOM with Gradle but not with Maven, probably because of Gradle parallelizing tests (for now I don't think it's Gradle specific, just that it manifests more easily with Gradle).

From what I can see, the problem is a huge QuarkusClassLoader leak. What I don't understand is exactly why it manifests only when starting the test resources per test.

mschorsch · 2024-08-06T07:05:14Z

I end up with an OOM with Gradle but not with Maven, probably because of Gradle parallelizing tests (for now I don't think it's Gradle specific, just that it manifests more easily with Gradle).

I think so too.

mschorsch · 2024-08-06T10:17:57Z

@gsmet As a temporary workaround for the upcoming Quarkus 3.13.1 you could change restrictToAnnotatedClass in @WithTestResource back to false, this would restore the previous behaviour.

geoand · 2024-08-06T10:22:14Z

That indeed fixes the problem.

I am looking into the root cause of this, but it's likely complex

gsmet · 2024-08-06T10:39:02Z

Except this annotation was solely created to change the default so...

I think maybe we should undeprecate @QuarkusTestResource for now.

geoand · 2024-08-06T10:40:13Z

Let's wait until I find the exact cause of this

Fixes: quarkusio#42303

geoand · 2024-08-06T11:12:15Z

Can you try #42338 please?

It worked for me.

Fixes: quarkusio#42303

Fix ClassLoader leak in KeycloakDevServicesProcessor

mschorsch · 2024-08-06T12:57:01Z

@geoand I've build your branch and tested against our application. The OOM ist still there.

geoand · 2024-08-06T13:03:38Z

But it's not in the reproducer, is it?

Because that's what I used.

The reason I am asking is because there might be a different leak and in that case we would need a separate reproducer.

mschorsch · 2024-08-06T13:06:57Z

I've tested against our real application not against the reproducer.

    java.lang.RuntimeException: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkus.smallrye.openapi.deployment.SmallRyeOpenApiProcessor#build threw an exception: java.lang.OutOfMemoryError: Java heap space
        at java.base/jdk.internal.misc.Unsafe.allocateUninitializedArray(Unsafe.java:1380)
        at java.base/java.lang.StringConcatHelper.newArray(StringConcatHelper.java:511)
        at java.base/java.lang.String.join(String.java:3515)
        at java.base/java.lang.System$2.join(System.java:2551)
        at java.base/java.util.StringJoiner.toString(StringJoiner.java:174)
        at java.base/java.lang.reflect.Executable.sharedToGenericString(Executable.java:173)
        at java.base/java.lang.reflect.Method.toGenericString(Method.java:487)
        at java.desktop/java.beans.MethodRef.find(MethodRef.java:78)
        at java.desktop/java.beans.MethodRef.get(MethodRef.java:62)
        at java.desktop/java.beans.PropertyDescriptor.getWriteMethod(PropertyDescriptor.java:304)
        at io.smallrye.openapi.api.util.MergeUtil.lambda$mergeObjects$1(MergeUtil.java:100)
        at io.smallrye.openapi.api.util.MergeUtil$$Lambda/0x00007a33acaab998.test(Unknown Source)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
        at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
        at io.smallrye.openapi.api.util.MergeUtil.mergeObjects(MergeUtil.java:101)
        at io.smallrye.openapi.runtime.scanner.dataobject.AnnotationTargetProcessor.processField(AnnotationTargetProcessor.java:185)
        at io.smallrye.openapi.runtime.scanner.dataobject.AnnotationTargetProcessor.process(AnnotationTargetProcessor.java:74)
        at io.smallrye.openapi.runtime.scanner.OpenApiDataObjectScanner.lambda$depthFirstGraphSearch$1(OpenApiDataObjectScanner.java:279)
        at io.smallrye.openapi.runtime.scanner.OpenApiDataObjectScanner$$Lambda/0x00007a33aca148d8.accept(Unknown Source)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
        at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
        at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1939)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)

        [error]: Build step io.quarkus.arc.deployment.ArcProcessor#generateResources threw an exception: java.lang.reflect.UndeclaredThrowableException
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:860)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1583)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)
    Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
        at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
        at io.quarkus.arc.processor.BeanProcessor.generateResources(BeanProcessor.java:429)
        at io.quarkus.arc.deployment.ArcProcessor.generateResources(ArcProcessor.java:537)
        at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:854)
        ... 7 more
    Caused by: java.lang.OutOfMemoryError: Java heap space
        at java.base/java.util.HashSet.<init>(HashSet.java:171)
        at java.base/java.util.LinkedHashSet.<init>(LinkedHashSet.java:181)
        at io.quarkus.gizmo.BytecodeCreatorImpl.findActiveResultHandles(BytecodeCreatorImpl.java:1638)
        at io.quarkus.gizmo.BytecodeCreatorImpl.allocateLocalVariables(BytecodeCreatorImpl.java:1618)
        at io.quarkus.gizmo.MethodCreatorImpl.write(MethodCreatorImpl.java:138)
        at io.quarkus.gizmo.ClassCreator.writeTo(ClassCreator.java:233)
        at io.quarkus.gizmo.ClassCreator.close(ClassCreator.java:257)
        at io.quarkus.arc.processor.ContextInstancesGenerator.generate(ContextInstancesGenerator.java:103)
        at io.quarkus.arc.processor.BeanProcessor$8.call(BeanProcessor.java:421)
        at io.quarkus.arc.processor.BeanProcessor$8.call(BeanProcessor.java:416)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.runWith(Thread.java:1596)
        ... 2 more
        at io.quarkus.test.junit.QuarkusTestExtension.throwBootFailureException(QuarkusTestExtension.java:643)
        at io.quarkus.test.junit.QuarkusTestExtension.interceptTestClassConstructor(QuarkusTestExtension.java:727)
        at java.base/java.util.Optional.orElseGet(Optional.java:364)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

        Caused by:
        java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
                [error]: Build step io.quarkus.smallrye.openapi.deployment.SmallRyeOpenApiProcessor#build threw an exception: java.lang.OutOfMemoryError: Java heap space
                at java.base/jdk.internal.misc.Unsafe.allocateUninitializedArray(Unsafe.java:1380)
                at java.base/java.lang.StringConcatHelper.newArray(StringConcatHelper.java:511)
                at java.base/java.lang.String.join(String.java:3515)
                at java.base/java.lang.System$2.join(System.java:2551)
                at java.base/java.util.StringJoiner.toString(StringJoiner.java:174)
                at java.base/java.lang.reflect.Executable.sharedToGenericString(Executable.java:173)
                at java.base/java.lang.reflect.Method.toGenericString(Method.java:487)
                at java.desktop/java.beans.MethodRef.find(MethodRef.java:78)
                at java.desktop/java.beans.MethodRef.get(MethodRef.java:62)
                at java.desktop/java.beans.PropertyDescriptor.getWriteMethod(PropertyDescriptor.java:304)
                at io.smallrye.openapi.api.util.MergeUtil.lambda$mergeObjects$1(MergeUtil.java:100)
                at io.smallrye.openapi.api.util.MergeUtil$$Lambda/0x00007a33acaab998.test(Unknown Source)
                at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
                at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
                at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
                at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
                at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
                at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
                at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
                at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
                at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
                at io.smallrye.openapi.api.util.MergeUtil.mergeObjects(MergeUtil.java:101)
                at io.smallrye.openapi.runtime.scanner.dataobject.AnnotationTargetProcessor.processField(AnnotationTargetProcessor.java:185)
                at io.smallrye.openapi.runtime.scanner.dataobject.AnnotationTargetProcessor.process(AnnotationTargetProcessor.java:74)
                at io.smallrye.openapi.runtime.scanner.OpenApiDataObjectScanner.lambda$depthFirstGraphSearch$1(OpenApiDataObjectScanner.java:279)
                at io.smallrye.openapi.runtime.scanner.OpenApiDataObjectScanner$$Lambda/0x00007a33aca148d8.accept(Unknown Source)
                at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
                at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
                at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
                at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1939)
                at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
                at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)

                [error]: Build step io.quarkus.arc.deployment.ArcProcessor#generateResources threw an exception: java.lang.reflect.UndeclaredThrowableException
                at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:860)
                at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
                at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
                at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
                at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
                at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
                at java.base/java.lang.Thread.run(Thread.java:1583)
                at org.jboss.threads.JBossThread.run(JBossThread.java:483)
        Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
                at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
                at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
                at io.quarkus.arc.processor.BeanProcessor.generateResources(BeanProcessor.java:429)
                at io.quarkus.arc.deployment.ArcProcessor.generateResources(ArcProcessor.java:537)
                at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
                at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:854)
                ... 7 more
        Caused by: java.lang.OutOfMemoryError: Java heap space
                at java.base/java.util.HashSet.<init>(HashSet.java:171)
                at java.base/java.util.LinkedHashSet.<init>(LinkedHashSet.java:181)
                at io.quarkus.gizmo.BytecodeCreatorImpl.findActiveResultHandles(BytecodeCreatorImpl.java:1638)
                at io.quarkus.gizmo.BytecodeCreatorImpl.allocateLocalVariables(BytecodeCreatorImpl.java:1618)
                at io.quarkus.gizmo.MethodCreatorImpl.write(MethodCreatorImpl.java:138)
                at io.quarkus.gizmo.ClassCreator.writeTo(ClassCreator.java:233)
                at io.quarkus.gizmo.ClassCreator.close(ClassCreator.java:257)
                at io.quarkus.arc.processor.ContextInstancesGenerator.generate(ContextInstancesGenerator.java:103)
                at io.quarkus.arc.processor.BeanProcessor$8.call(BeanProcessor.java:421)
                at io.quarkus.arc.processor.BeanProcessor$8.call(BeanProcessor.java:416)
                at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
                at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
                at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
                at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
                at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
                at java.base/java.lang.Thread.runWith(Thread.java:1596)
                ... 2 more
            at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:335)
            at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:254)
            at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:58)
            at io.quarkus.test.junit.QuarkusTestExtension.doJavaStart(QuarkusTestExtension.java:219)
            at io.quarkus.test.junit.QuarkusTestExtension.ensureStarted(QuarkusTestExtension.java:610)
            at io.quarkus.test.junit.QuarkusTestExtension.beforeAll(QuarkusTestExtension.java:660)
            ... 1 more

            Caused by:
            io.quarkus.builder.BuildException: Build failure: Build failed due to errors
                [error]: Build step io.quarkus.smallrye.openapi.deployment.SmallRyeOpenApiProcessor#build threw an exception: java.lang.OutOfMemoryError: Java heap space
                at java.base/jdk.internal.misc.Unsafe.allocateUninitializedArray(Unsafe.java:1380)
                at java.base/java.lang.StringConcatHelper.newArray(StringConcatHelper.java:511)
                at java.base/java.lang.String.join(String.java:3515)
                at java.base/java.lang.System$2.join(System.java:2551)
                at java.base/java.util.StringJoiner.toString(StringJoiner.java:174)
                at java.base/java.lang.reflect.Executable.sharedToGenericString(Executable.java:173)
                at java.base/java.lang.reflect.Method.toGenericString(Method.java:487)
                at java.desktop/java.beans.MethodRef.find(MethodRef.java:78)
                at java.desktop/java.beans.MethodRef.get(MethodRef.java:62)
                at java.desktop/java.beans.PropertyDescriptor.getWriteMethod(PropertyDescriptor.java:304)
                at io.smallrye.openapi.api.util.MergeUtil.lambda$mergeObjects$1(MergeUtil.java:100)
                at io.smallrye.openapi.api.util.MergeUtil$$Lambda/0x00007a33acaab998.test(Unknown Source)
                at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
                at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
                at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
                at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
                at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
                at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
                at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
                at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
                at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
                at io.smallrye.openapi.api.util.MergeUtil.mergeObjects(MergeUtil.java:101)
                at io.smallrye.openapi.runtime.scanner.dataobject.AnnotationTargetProcessor.processField(AnnotationTargetProcessor.java:185)
                at io.smallrye.openapi.runtime.scanner.dataobject.AnnotationTargetProcessor.process(AnnotationTargetProcessor.java:74)
                at io.smallrye.openapi.runtime.scanner.OpenApiDataObjectScanner.lambda$depthFirstGraphSearch$1(OpenApiDataObjectScanner.java:279)
                at io.smallrye.openapi.runtime.scanner.OpenApiDataObjectScanner$$Lambda/0x00007a33aca148d8.accept(Unknown Source)
                at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
                at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
                at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
                at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1939)
                at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
                at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)

                [error]: Build step io.quarkus.arc.deployment.ArcProcessor#generateResources threw an exception: java.lang.reflect.UndeclaredThrowableException
                at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:860)
                at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
                at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
                at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
                at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
                at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
                at java.base/java.lang.Thread.run(Thread.java:1583)
                at org.jboss.threads.JBossThread.run(JBossThread.java:483)
            Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
                at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
                at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
                at io.quarkus.arc.processor.BeanProcessor.generateResources(BeanProcessor.java:429)
                at io.quarkus.arc.deployment.ArcProcessor.generateResources(ArcProcessor.java:537)
                at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
                at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:854)
                ... 7 more
            Caused by: java.lang.OutOfMemoryError: Java heap space
                at java.base/java.util.HashSet.<init>(HashSet.java:171)
                at java.base/java.util.LinkedHashSet.<init>(LinkedHashSet.java:181)
                at io.quarkus.gizmo.BytecodeCreatorImpl.findActiveResultHandles(BytecodeCreatorImpl.java:1638)
                at io.quarkus.gizmo.BytecodeCreatorImpl.allocateLocalVariables(BytecodeCreatorImpl.java:1618)
                at io.quarkus.gizmo.MethodCreatorImpl.write(MethodCreatorImpl.java:138)
                at io.quarkus.gizmo.ClassCreator.writeTo(ClassCreator.java:233)
                at io.quarkus.gizmo.ClassCreator.close(ClassCreator.java:257)
                at io.quarkus.arc.processor.ContextInstancesGenerator.generate(ContextInstancesGenerator.java:103)
                at io.quarkus.arc.processor.BeanProcessor$8.call(BeanProcessor.java:421)
                at io.quarkus.arc.processor.BeanProcessor$8.call(BeanProcessor.java:416)
                at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
                at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
                at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
                at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
                at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
                at java.base/java.lang.Thread.runWith(Thread.java:1596)
                ... 2 more
                at io.quarkus.builder.Execution.run(Execution.java:123)
                at io.quarkus.builder.BuildExecutionBuilder.execute(BuildExecutionBuilder.java:79)
                at io.quarkus.deployment.QuarkusAugmentor.run(QuarkusAugmentor.java:159)
                at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:331)
                ... 6 more

                Caused by:
                java.lang.OutOfMemoryError: Java heap space
                    at java.base/jdk.internal.misc.Unsafe.allocateUninitializedArray(Unsafe.java:1380)
                    at java.base/java.lang.StringConcatHelper.newArray(StringConcatHelper.java:511)
                    at java.base/java.lang.String.join(String.java:3515)
                    at java.base/java.lang.System$2.join(System.java:2551)
                    at java.base/java.util.StringJoiner.toString(StringJoiner.java:174)
                    at java.base/java.lang.reflect.Executable.sharedToGenericString(Executable.java:173)
                    at java.base/java.lang.reflect.Method.toGenericString(Method.java:487)
                    at java.desktop/java.beans.MethodRef.find(MethodRef.java:78)
                    at java.desktop/java.beans.MethodRef.get(MethodRef.java:62)
                    at java.desktop/java.beans.PropertyDescriptor.getWriteMethod(PropertyDescriptor.java:304)
                    at io.smallrye.openapi.api.util.MergeUtil.lambda$mergeObjects$1(MergeUtil.java:100)
                    at io.smallrye.openapi.api.util.MergeUtil$$Lambda/0x00007a33acaab998.test(Unknown Source)
                    at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
                    at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
                    at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
                    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
                    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
                    at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
                    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
                    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
                    at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
                    at io.smallrye.openapi.api.util.MergeUtil.mergeObjects(MergeUtil.java:101)
                    at io.smallrye.openapi.runtime.scanner.dataobject.AnnotationTargetProcessor.processField(AnnotationTargetProcessor.java:185)
                    at io.smallrye.openapi.runtime.scanner.dataobject.AnnotationTargetProcessor.process(AnnotationTargetProcessor.java:74)
                    at io.smallrye.openapi.runtime.scanner.OpenApiDataObjectScanner.lambda$depthFirstGraphSearch$1(OpenApiDataObjectScanner.java:279)
                    at io.smallrye.openapi.runtime.scanner.OpenApiDataObjectScanner$$Lambda/0x00007a33aca148d8.accept(Unknown Source)
                    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
                    at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
                    at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
                    at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1939)
                    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
                    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)

mschorsch · 2024-08-06T13:08:39Z

If the repeoducer is now working then there are more leaks ...

geoand · 2024-08-06T13:17:17Z

Right, which is why we would need a reproducer to surfaces it / them - there is absolutely no way to track down a leak without being able to reproduce it reliably

Fixes: quarkusio#42303 (cherry picked from commit d30d070)

mschorsch · 2024-08-07T11:55:06Z

@geoand I've created a new issue #42355 with updated reproducer.

geoand · 2024-08-07T11:56:05Z

Thanks!

Fixes: quarkusio#42303

mschorsch added the kind/bug Something isn't working label Aug 3, 2024

quarkus-bot bot added the triage/needs-triage label Aug 3, 2024

mschorsch changed the title ~~OOM in Tests since Quarkus 3.13.0~~ OOM in tests since Quarkus 3.13.0 Aug 3, 2024

geoand added area/config area/core and removed triage/needs-triage labels Aug 5, 2024

geoand added a commit to geoand/quarkus that referenced this issue Aug 6, 2024

Fix ClassLoader leak in KeycloakDevServicesProcessor

9e456ef

Fixes: quarkusio#42303

geoand mentioned this issue Aug 6, 2024

Fix ClassLoader leak in KeycloakDevServicesProcessor #42338

Merged

geoand added a commit to geoand/quarkus that referenced this issue Aug 6, 2024

Fix ClassLoader leak in KeycloakDevServicesProcessor

d30d070

Fixes: quarkusio#42303

geoand closed this as completed in #42338 Aug 6, 2024

geoand added a commit that referenced this issue Aug 6, 2024

Merge pull request #42338 from geoand/#42303

39ec432

Fix ClassLoader leak in KeycloakDevServicesProcessor

quarkus-bot bot added this to the 3.14 - main milestone Aug 6, 2024

gsmet modified the milestones: 3.14 - main, 3.13.1 Aug 6, 2024

gsmet pushed a commit to gsmet/quarkus that referenced this issue Aug 6, 2024

Fix ClassLoader leak in KeycloakDevServicesProcessor

d26ad3f

Fixes: quarkusio#42303 (cherry picked from commit d30d070)

mschorsch mentioned this issue Aug 7, 2024

OOM in tests since Quarkus 3.13.0 (Part 2) #42355

Closed

mschorsch mentioned this issue Aug 8, 2024

Executing tests in Quarkus 3.13+ are extremely slow #42390

Closed

afalhambra-hivemq mentioned this issue Aug 22, 2024

OOM in tests (Part 3) #42471

Closed

danielsoro pushed a commit to danielsoro/quarkus that referenced this issue Sep 20, 2024

Fix ClassLoader leak in KeycloakDevServicesProcessor

cc351d8

Fixes: quarkusio#42303

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OOM in tests since Quarkus 3.13.0 #42303

OOM in tests since Quarkus 3.13.0 #42303

mschorsch commented Aug 3, 2024 •

edited

Loading

gsmet commented Aug 3, 2024

gsmet commented Aug 3, 2024

gsmet commented Aug 3, 2024

gsmet commented Aug 3, 2024 •

edited

Loading

mschorsch commented Aug 3, 2024

mschorsch commented Aug 3, 2024 •

edited

Loading

gsmet commented Aug 3, 2024

mschorsch commented Aug 3, 2024

gsmet commented Aug 3, 2024

gsmet commented Aug 3, 2024

mschorsch commented Aug 4, 2024 •

edited

Loading

gsmet commented Aug 5, 2024

mschorsch commented Aug 6, 2024

mschorsch commented Aug 6, 2024

geoand commented Aug 6, 2024

gsmet commented Aug 6, 2024

geoand commented Aug 6, 2024

geoand commented Aug 6, 2024

mschorsch commented Aug 6, 2024

geoand commented Aug 6, 2024

mschorsch commented Aug 6, 2024

mschorsch commented Aug 6, 2024

geoand commented Aug 6, 2024

mschorsch commented Aug 7, 2024

geoand commented Aug 7, 2024

OOM in tests since Quarkus 3.13.0 #42303

OOM in tests since Quarkus 3.13.0 #42303

Comments

mschorsch commented Aug 3, 2024 • edited Loading

Describe the bug

Expected behavior

Actual behavior

How to Reproduce?

Output of uname -a or ver

Output of java -version

Quarkus version or git rev

Build tool (ie. output of mvnw --version or gradlew --version)

Additional information

gsmet commented Aug 3, 2024

gsmet commented Aug 3, 2024

gsmet commented Aug 3, 2024

gsmet commented Aug 3, 2024 • edited Loading

mschorsch commented Aug 3, 2024

mschorsch commented Aug 3, 2024 • edited Loading

gsmet commented Aug 3, 2024

mschorsch commented Aug 3, 2024

gsmet commented Aug 3, 2024

gsmet commented Aug 3, 2024

mschorsch commented Aug 4, 2024 • edited Loading

gsmet commented Aug 5, 2024

mschorsch commented Aug 6, 2024

mschorsch commented Aug 6, 2024

geoand commented Aug 6, 2024

gsmet commented Aug 6, 2024

geoand commented Aug 6, 2024

geoand commented Aug 6, 2024

mschorsch commented Aug 6, 2024

geoand commented Aug 6, 2024

mschorsch commented Aug 6, 2024

mschorsch commented Aug 6, 2024

geoand commented Aug 6, 2024

mschorsch commented Aug 7, 2024

geoand commented Aug 7, 2024

mschorsch commented Aug 3, 2024 •

edited

Loading

Output of `uname -a` or `ver`

Output of `java -version`

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

gsmet commented Aug 3, 2024 •

edited

Loading

mschorsch commented Aug 3, 2024 •

edited

Loading

mschorsch commented Aug 4, 2024 •

edited

Loading