Corrupted message in Kafka using Java 11 with native #152
Labels
Comments
|
Thanks we will investigate |
|
I have managed to reproduce the issue locally. From what I've seen, this will only get triggered when using Java 9+, which would explain why it works on 8 and not on 11. It expects to get a CRC32C implementation, but this would supply CRC32. |
graemerocher
added a commit
that referenced
this issue
Jun 12, 2020
|
@graemerocher No problem at all! Hopefully it will solve the issue. If not, I'll dig a bit further. |
|
Sure @graemerocher I will try it using the snapshot and I let you guys know. |
|
Wait a bit the snapshot is not published yet |
|
Ok the snapshot is published |
|
@ivangfr were you able to verify the issue is resolved with the snapshot? |
|
@graemerocher I am having a problem to build the native image That is the I have in In |
|
exit status |
|
Note you will need a reference to the Kafka snapshot: implementation("io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT")And make sure you change |
|
Thanks @graemerocher . I fixed the wrong reference version to I've just compiled the However, once the 1st request comes, it's throwing the following exception |
|
Ok interesting. Can probably be worked around by adding I wonder why https://github.com/micronaut-projects/micronaut-kafka/blob/master/kafka/src/main/java/io/micronaut/configuration/kafka/graal/KafkaSubstitutions.java#L40 is not working Stand by let me try a few things |
|
@ivangfr I went through the steps to reproduce the issue with your sample and 014b1f8 seems to fix it I had to apply the following patch to get it working in your example (which assumes using Subject: [PATCH] Use micronaut 2.0 rc1 and kafka snapshot
---
.../consumer-api/build.gradle | 25 ++++++++++++-------
.../src/main/docker/Dockerfile.native | 4 +--
.../producer-api/build.gradle | 25 ++++++++++++-------
.../src/main/docker/Dockerfile.native | 4 +--
4 files changed, 36 insertions(+), 22 deletions(-)
diff --git a/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle b/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle
index afbf4a9..5358f2d 100644
--- a/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle
+++ b/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle
@@ -10,6 +10,7 @@ group "com.mycompany.consumerapi"
repositories {
jcenter()
+ mavenLocal()
}
configurations {
@@ -18,11 +19,17 @@ configurations {
}
ext {
- set('micronautVersion', '2.0.0.M3')
+ set('micronautVersion', '2.0.0.RC1')
set('lombokVersion', '1.18.12')
set('logbackVersion', '1.2.3')
}
+configurations.all {
+ resolutionStrategy {
+ force "io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT"
+ }
+}
+
dependencies {
//-- IMPORTANT!
// Lombok annotation processor must place before the Micronaut processors in the build configuration
@@ -30,31 +37,31 @@ dependencies {
compileOnly("org.projectlombok:lombok:$lombokVersion")
annotationProcessor("org.projectlombok:lombok:$lombokVersion")
//--
- annotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ annotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
annotationProcessor("io.micronaut:micronaut-inject-java")
annotationProcessor("io.micronaut:micronaut-validation")
annotationProcessor("io.micronaut:micronaut-graal")
- compileOnly(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ compileOnly(platform("io.micronaut:micronaut-bom:$micronautVersion"))
compileOnly("org.graalvm.nativeimage:svm")
- implementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ implementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
implementation("io.micronaut:micronaut-management")
- implementation("io.micronaut.configuration:micronaut-micrometer-core")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-core")
implementation("io.opentracing.brave:brave-opentracing")
implementation("io.micronaut:micronaut-tracing")
- implementation("io.micronaut.configuration:micronaut-micrometer-registry-prometheus")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-registry-prometheus")
implementation("io.micronaut:micronaut-inject")
implementation("io.micronaut:micronaut-validation")
implementation("io.micronaut:micronaut-runtime")
implementation("io.micronaut:micronaut-http-server-netty")
implementation("io.micronaut:micronaut-http-client")
- implementation("io.micronaut.kafka:micronaut-kafka")
+ implementation("io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT")
implementation("io.micronaut.cache:micronaut-cache-caffeine")
runtimeOnly("io.zipkin.brave:brave-instrumentation-http")
runtimeOnly("io.zipkin.reporter2:zipkin-reporter")
runtimeOnly("ch.qos.logback:logback-classic:$logbackVersion")
- testAnnotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testAnnotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testAnnotationProcessor("io.micronaut:micronaut-inject-java")
- testImplementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testImplementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testImplementation("org.junit.jupiter:junit-jupiter-api")
testImplementation("io.micronaut.test:micronaut-test-junit5")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine")
diff --git a/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native b/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native
index 08c605b..83bb4d9 100644
--- a/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native
+++ b/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native
@@ -1,6 +1,6 @@
-FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
+#FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
# For JDK 11
-#FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
+FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
RUN gu install native-image
COPY build/libs/consumer-api-*-all.jar /home/app/consumer-api/build/libs/consumer-api.jar
diff --git a/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle b/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle
index fa86250..3d0bd23 100644
--- a/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle
+++ b/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle
@@ -10,6 +10,7 @@ group "com.mycompany.producerapi"
repositories {
jcenter()
+ mavenLocal()
}
configurations {
@@ -18,11 +19,17 @@ configurations {
}
ext {
- set('micronautVersion', '2.0.0.M3')
+ set('micronautVersion', '2.0.0.RC1')
set('lombokVersion', '1.18.12')
set('logbackVersion', '1.2.3')
}
+configurations.all {
+ resolutionStrategy {
+ force "io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT"
+ }
+}
+
dependencies {
//-- IMPORTANT!
// Lombok annotation processor must place before the Micronaut processors in the build configuration
@@ -30,31 +37,31 @@ dependencies {
compileOnly("org.projectlombok:lombok:$lombokVersion")
annotationProcessor("org.projectlombok:lombok:$lombokVersion")
//--
- annotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ annotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
annotationProcessor("io.micronaut:micronaut-inject-java")
annotationProcessor("io.micronaut:micronaut-validation")
annotationProcessor("io.micronaut:micronaut-graal")
- compileOnly(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ compileOnly(platform("io.micronaut:micronaut-bom:$micronautVersion"))
compileOnly("org.graalvm.nativeimage:svm")
- implementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ implementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
implementation("io.micronaut:micronaut-management")
- implementation("io.micronaut.configuration:micronaut-micrometer-core")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-core")
implementation("io.opentracing.brave:brave-opentracing")
implementation("io.micronaut:micronaut-tracing")
- implementation("io.micronaut.configuration:micronaut-micrometer-registry-prometheus")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-registry-prometheus")
implementation("io.micronaut:micronaut-inject")
implementation("io.micronaut:micronaut-validation")
implementation("io.micronaut:micronaut-runtime")
implementation("io.micronaut:micronaut-http-server-netty")
implementation("io.micronaut:micronaut-http-client")
- implementation("io.micronaut.kafka:micronaut-kafka")
+ implementation("io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT")
implementation("io.micronaut.cache:micronaut-cache-caffeine")
runtimeOnly("io.zipkin.brave:brave-instrumentation-http")
runtimeOnly("io.zipkin.reporter2:zipkin-reporter")
runtimeOnly("ch.qos.logback:logback-classic:$logbackVersion")
- testAnnotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testAnnotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testAnnotationProcessor("io.micronaut:micronaut-inject-java")
- testImplementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testImplementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testImplementation("org.junit.jupiter:junit-jupiter-api")
testImplementation("io.micronaut.test:micronaut-test-junit5")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine")
diff --git a/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native b/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native
index c05986d..16b43d7 100644
--- a/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native
+++ b/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native
@@ -1,6 +1,6 @@
-FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
+#FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
# For JDK 11
-#FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
+FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
RUN gu install native-image
COPY build/libs/producer-api-*-all.jar /home/app/producer-api/build/libs/producer-api.jar
--
2.24.3 (Apple Git-128)
|
|
Perfect @graemerocher ! I am looking forward to |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Task List
Actual Behaviour
Hi, I've migrated my
graalvm-quarkus-micronaut-springboottoJava 11. All seems to work fine except for the Micronaut Producer-Consumer example.Before, both Producer and Consumer were using Java version
1.8(inbuild.gradle) andoracle/graalvm-ce:20.0.0-java8as base image for the native images. This configuration was working fine.Then, I updated Producer and Consumer to Java version
11andoracle/graalvm-ce:20.0.0-java11as base image. I am able to build the application's native docker images but, at runtime, I have the errors below. It seems that the messages are been corrupted somehow.Logs
Producer
Consumer
Expected Behaviour
The message should be produced and consumed correctly as done using Java 8
Steps to Reproduce
Clone project
You will need Kafka & Zookeeper running. In thie README explains how to start them by running a
docker-compose.ymlfile present ingraalvm-quarkus-micronaut-springboot/producer-consumerfolder;Go to Micronaut Procucer
build.gradleand change the Java Version to 11Go to Micronaut Producer
Dockerfile.native. Comment the Java8 line and uncomment the Java11Do the 2 steps described above, but for the Micronaut Consumer.
Environment Information
Example Application
graalvm-quarkus-micronaut-springbootThe text was updated successfully, but these errors were encountered: