-
Notifications
You must be signed in to change notification settings - Fork 106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Corrupted message in Kafka using Java 11 with native #152
Comments
Thanks we will investigate |
I have managed to reproduce the issue locally. micronaut-kafka/kafka/src/main/java/io/micronaut/configuration/kafka/graal/KafkaSubstitutions.java Line 41 in 47aaeb5
From what I've seen, this will only get triggered when using Java 9+, which would explain why it works on 8 and not on 11. It expects to get a CRC32C implementation, but this would supply CRC32. |
@graemerocher No problem at all! Hopefully it will solve the issue. If not, I'll dig a bit further. |
Sure @graemerocher I will try it using the snapshot and I let you guys know. |
Wait a bit the snapshot is not published yet |
Ok the snapshot is published |
@ivangfr were you able to verify the issue is resolved with the snapshot? |
@graemerocher I am having a problem to build the native image
That is the I have in
In
|
exit status |
Note you will need a reference to the Kafka snapshot: implementation("io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT") And make sure you change |
Thanks @graemerocher . I fixed the wrong reference version to I've just compiled the However, once the 1st request comes, it's throwing the following exception
|
Ok interesting. Can probably be worked around by adding I wonder why https://github.com/micronaut-projects/micronaut-kafka/blob/master/kafka/src/main/java/io/micronaut/configuration/kafka/graal/KafkaSubstitutions.java#L40 is not working Stand by let me try a few things |
@ivangfr I went through the steps to reproduce the issue with your sample and 014b1f8 seems to fix it I had to apply the following patch to get it working in your example (which assumes using Subject: [PATCH] Use micronaut 2.0 rc1 and kafka snapshot
---
.../consumer-api/build.gradle | 25 ++++++++++++-------
.../src/main/docker/Dockerfile.native | 4 +--
.../producer-api/build.gradle | 25 ++++++++++++-------
.../src/main/docker/Dockerfile.native | 4 +--
4 files changed, 36 insertions(+), 22 deletions(-)
diff --git a/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle b/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle
index afbf4a9..5358f2d 100644
--- a/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle
+++ b/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle
@@ -10,6 +10,7 @@ group "com.mycompany.consumerapi"
repositories {
jcenter()
+ mavenLocal()
}
configurations {
@@ -18,11 +19,17 @@ configurations {
}
ext {
- set('micronautVersion', '2.0.0.M3')
+ set('micronautVersion', '2.0.0.RC1')
set('lombokVersion', '1.18.12')
set('logbackVersion', '1.2.3')
}
+configurations.all {
+ resolutionStrategy {
+ force "io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT"
+ }
+}
+
dependencies {
//-- IMPORTANT!
// Lombok annotation processor must place before the Micronaut processors in the build configuration
@@ -30,31 +37,31 @@ dependencies {
compileOnly("org.projectlombok:lombok:$lombokVersion")
annotationProcessor("org.projectlombok:lombok:$lombokVersion")
//--
- annotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ annotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
annotationProcessor("io.micronaut:micronaut-inject-java")
annotationProcessor("io.micronaut:micronaut-validation")
annotationProcessor("io.micronaut:micronaut-graal")
- compileOnly(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ compileOnly(platform("io.micronaut:micronaut-bom:$micronautVersion"))
compileOnly("org.graalvm.nativeimage:svm")
- implementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ implementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
implementation("io.micronaut:micronaut-management")
- implementation("io.micronaut.configuration:micronaut-micrometer-core")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-core")
implementation("io.opentracing.brave:brave-opentracing")
implementation("io.micronaut:micronaut-tracing")
- implementation("io.micronaut.configuration:micronaut-micrometer-registry-prometheus")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-registry-prometheus")
implementation("io.micronaut:micronaut-inject")
implementation("io.micronaut:micronaut-validation")
implementation("io.micronaut:micronaut-runtime")
implementation("io.micronaut:micronaut-http-server-netty")
implementation("io.micronaut:micronaut-http-client")
- implementation("io.micronaut.kafka:micronaut-kafka")
+ implementation("io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT")
implementation("io.micronaut.cache:micronaut-cache-caffeine")
runtimeOnly("io.zipkin.brave:brave-instrumentation-http")
runtimeOnly("io.zipkin.reporter2:zipkin-reporter")
runtimeOnly("ch.qos.logback:logback-classic:$logbackVersion")
- testAnnotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testAnnotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testAnnotationProcessor("io.micronaut:micronaut-inject-java")
- testImplementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testImplementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testImplementation("org.junit.jupiter:junit-jupiter-api")
testImplementation("io.micronaut.test:micronaut-test-junit5")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine")
diff --git a/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native b/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native
index 08c605b..83bb4d9 100644
--- a/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native
+++ b/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native
@@ -1,6 +1,6 @@
-FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
+#FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
# For JDK 11
-#FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
+FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
RUN gu install native-image
COPY build/libs/consumer-api-*-all.jar /home/app/consumer-api/build/libs/consumer-api.jar
diff --git a/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle b/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle
index fa86250..3d0bd23 100644
--- a/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle
+++ b/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle
@@ -10,6 +10,7 @@ group "com.mycompany.producerapi"
repositories {
jcenter()
+ mavenLocal()
}
configurations {
@@ -18,11 +19,17 @@ configurations {
}
ext {
- set('micronautVersion', '2.0.0.M3')
+ set('micronautVersion', '2.0.0.RC1')
set('lombokVersion', '1.18.12')
set('logbackVersion', '1.2.3')
}
+configurations.all {
+ resolutionStrategy {
+ force "io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT"
+ }
+}
+
dependencies {
//-- IMPORTANT!
// Lombok annotation processor must place before the Micronaut processors in the build configuration
@@ -30,31 +37,31 @@ dependencies {
compileOnly("org.projectlombok:lombok:$lombokVersion")
annotationProcessor("org.projectlombok:lombok:$lombokVersion")
//--
- annotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ annotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
annotationProcessor("io.micronaut:micronaut-inject-java")
annotationProcessor("io.micronaut:micronaut-validation")
annotationProcessor("io.micronaut:micronaut-graal")
- compileOnly(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ compileOnly(platform("io.micronaut:micronaut-bom:$micronautVersion"))
compileOnly("org.graalvm.nativeimage:svm")
- implementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ implementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
implementation("io.micronaut:micronaut-management")
- implementation("io.micronaut.configuration:micronaut-micrometer-core")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-core")
implementation("io.opentracing.brave:brave-opentracing")
implementation("io.micronaut:micronaut-tracing")
- implementation("io.micronaut.configuration:micronaut-micrometer-registry-prometheus")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-registry-prometheus")
implementation("io.micronaut:micronaut-inject")
implementation("io.micronaut:micronaut-validation")
implementation("io.micronaut:micronaut-runtime")
implementation("io.micronaut:micronaut-http-server-netty")
implementation("io.micronaut:micronaut-http-client")
- implementation("io.micronaut.kafka:micronaut-kafka")
+ implementation("io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT")
implementation("io.micronaut.cache:micronaut-cache-caffeine")
runtimeOnly("io.zipkin.brave:brave-instrumentation-http")
runtimeOnly("io.zipkin.reporter2:zipkin-reporter")
runtimeOnly("ch.qos.logback:logback-classic:$logbackVersion")
- testAnnotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testAnnotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testAnnotationProcessor("io.micronaut:micronaut-inject-java")
- testImplementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testImplementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testImplementation("org.junit.jupiter:junit-jupiter-api")
testImplementation("io.micronaut.test:micronaut-test-junit5")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine")
diff --git a/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native b/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native
index c05986d..16b43d7 100644
--- a/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native
+++ b/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native
@@ -1,6 +1,6 @@
-FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
+#FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
# For JDK 11
-#FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
+FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
RUN gu install native-image
COPY build/libs/producer-api-*-all.jar /home/app/producer-api/build/libs/producer-api.jar
--
2.24.3 (Apple Git-128)
|
Perfect @graemerocher ! I am looking forward to |
Task List
Actual Behaviour
Hi, I've migrated my
graalvm-quarkus-micronaut-springboot
toJava 11
. All seems to work fine except for the Micronaut Producer-Consumer example.Before, both Producer and Consumer were using Java version
1.8
(inbuild.gradle
) andoracle/graalvm-ce:20.0.0-java8
as base image for the native images. This configuration was working fine.Then, I updated Producer and Consumer to Java version
11
andoracle/graalvm-ce:20.0.0-java11
as base image. I am able to build the application's native docker images but, at runtime, I have the errors below. It seems that the messages are been corrupted somehow.Logs
Producer
Consumer
Expected Behaviour
The message should be produced and consumed correctly as done using Java 8
Steps to Reproduce
Clone project
You will need Kafka & Zookeeper running. In thie README explains how to start them by running a
docker-compose.yml
file present ingraalvm-quarkus-micronaut-springboot/producer-consumer
folder;Go to Micronaut Procucer
build.gradle
and change the Java Version to 11Go to Micronaut Producer
Dockerfile.native
. Comment the Java8 line and uncomment the Java11Do the 2 steps described above, but for the Micronaut Consumer.
Environment Information
Example Application
graalvm-quarkus-micronaut-springboot
The text was updated successfully, but these errors were encountered: