Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workaround for #14873 #14890

Merged
merged 8 commits into from
Feb 11, 2023
Merged

Workaround for #14873 #14890

merged 8 commits into from
Feb 11, 2023

Conversation

jsvd
Copy link
Member

@jsvd jsvd commented Feb 10, 2023

import ffi from JRuby 9.3.9.0 to overcome jruby/jruby#7579

Since JRuby ignores the files in "lib/jni" and uses the ones inside "lib/jruby.jar", we have to inject the older versions into the new JRuby's jruby.jar

fixes #14873

@jsvd
Copy link
Member Author

jsvd commented Feb 10, 2023

This fix isn't enough since JRuby ignores the files under "lib/jni" and uses the "jni" folder inside the jruby.jar. Two solutions are:

  1. modify the jruby.jar to include the older files from JRuby 9.3.9.0
  2. exploration: use -Djava.library.path= so JRuby looks at the path we want. it may require having to add logic to JvmOptionsParser.java to decide which directory to load from

@jsvd
Copy link
Member Author

jsvd commented Feb 11, 2023

To test:

./gradlew clean assembleTarDistribution

$ cd build
$ cat Dockerfile 
FROM oraclelinux:7
RUN yum update -y
RUN yum install java-11-openjdk.aarch64 -y

COPY logstash-8.8.0-SNAPSHOT.tar.gz /root
WORKDIR /root
RUN tar -zxf logstash-8.8.0-SNAPSHOT.tar.gz
WORKDIR /root/logstash-8.8.0-SNAPSHOT

$ docker run -it mytest ./vendor/jruby/bin/jruby -Xnative.verbose -w -d --dev -e 'system "date"' 
Successfully loaded native POSIX impl.
Sat Feb 11 01:53:00 UTC 2023

rubyUtils.gradle Outdated
Comment on lines 297 to 318
tasks.register("downloadAndInstallPreviousJRubyFFI", Copy) {
dependsOn=[downloadPreviousJRuby]
description "Install previous JRuby FFI files in the new JRuby directory"
inputs.file(previousJrubyTarPath)
outputs.dir("${projectDir}/vendor/jruby/lib/jni")
outputs.file("${projectDir}/vendor/jruby/lib/jruby.jar")
from tarTree(downloadPreviousJRuby.dest)
eachFile { f ->
f.path = f.path.replaceFirst("^jruby-${previousJRubyVersion}", '')
}
// Copy only the previous builds of libjffi that changed from JRuby 9.3.9.0 to 9.3.10.0
include "**/lib/jni/**/*"

includeEmptyDirs = false
into "${projectDir}/vendor/jruby/tmp"
doLast {
ant.jar(destfile: "${projectDir}/vendor/jruby/lib/jruby.jar", update: true) {
fileset(dir: "${projectDir}/vendor/jruby/tmp/lib")
}
delete files("${projectDir}/vendor/jruby/tmp")
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: minor re-arrange, return to only getting the FFI libs that are known to have changed, and output information about what we are injecting. I'm ok with filing this as a follow-up.

Suggested change
tasks.register("downloadAndInstallPreviousJRubyFFI", Copy) {
dependsOn=[downloadPreviousJRuby]
description "Install previous JRuby FFI files in the new JRuby directory"
inputs.file(previousJrubyTarPath)
outputs.dir("${projectDir}/vendor/jruby/lib/jni")
outputs.file("${projectDir}/vendor/jruby/lib/jruby.jar")
from tarTree(downloadPreviousJRuby.dest)
eachFile { f ->
f.path = f.path.replaceFirst("^jruby-${previousJRubyVersion}", '')
}
// Copy only the previous builds of libjffi that changed from JRuby 9.3.9.0 to 9.3.10.0
include "**/lib/jni/**/*"
includeEmptyDirs = false
into "${projectDir}/vendor/jruby/tmp"
doLast {
ant.jar(destfile: "${projectDir}/vendor/jruby/lib/jruby.jar", update: true) {
fileset(dir: "${projectDir}/vendor/jruby/tmp/lib")
}
delete files("${projectDir}/vendor/jruby/tmp")
}
}
tasks.register("downloadAndInstallPreviousJRubyFFI", Copy) {
dependsOn=[downloadPreviousJRuby]
description "Install previous JRuby FFI files in the new JRuby directory"
inputs.file(previousJrubyTarPath)
outputs.dir("${projectDir}/vendor/jruby/lib/jni")
outputs.file("${projectDir}/vendor/jruby/lib/jruby.jar")
includeEmptyDirs = false
from(tarTree(downloadPreviousJRuby.dest)) {
// Copy only the previous builds of libjffi that changed from JRuby 9.3.9.0 to 9.3.10.0
include "**/lib/jni/aarch64-Linux/libjffi-1.2.so"
include "**/lib/jni/arm-Linux/libjffi-1.2.so"
include "**/lib/jni/i386-Linux/libjffi-1.2.so"
include "**/lib/jni/x86_64-Linux/libjffi-1.2.so"
eachFile { f ->
f.path = f.path.replaceFirst("^jruby-${previousJRubyVersion}", '')
println "Installing Previous FFI: ${f.path}"
}
}
into "${projectDir}/vendor/jruby/tmp"
doLast {
ant.jar(destfile: "${projectDir}/vendor/jruby/lib/jruby.jar", update: true) {
fileset(dir: "${projectDir}/vendor/jruby/tmp/lib")
}
delete files("${projectDir}/vendor/jruby/tmp")
}
}

Copy link
Member

@yaauie yaauie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, and dipping down to ant is much more straight-forward than the zip utility method I was attempting.

build.gradle Outdated Show resolved Hide resolved
rubyUtils.gradle Show resolved Hide resolved
rakelib/compile.rake Outdated Show resolved Hide resolved
@jsvd jsvd requested a review from yaauie February 11, 2023 02:41
@jsvd jsvd marked this pull request as ready for review February 11, 2023 02:57
@jsvd jsvd merged commit 281ce70 into elastic:main Feb 11, 2023
@jsvd
Copy link
Member Author

jsvd commented Feb 11, 2023

@logstashmachine backport 8.7

@jsvd
Copy link
Member Author

jsvd commented Feb 11, 2023

@logstashmachine backport 8.6

@jsvd jsvd deleted the workaround_14873 branch February 11, 2023 12:21
github-actions bot pushed a commit that referenced this pull request Feb 11, 2023
import platform-specific ffi binaries from JRuby 9.3.9.0 to overcome jruby/jruby#7579

Since JRuby ignores the files in "lib/jni" and uses the ones inside "lib/jruby.jar" instead, we have to inject the older versions into the new JRuby's "jruby.jar"

fixes #14873

(cherry picked from commit 281ce70)
github-actions bot pushed a commit that referenced this pull request Feb 11, 2023
import platform-specific ffi binaries from JRuby 9.3.9.0 to overcome jruby/jruby#7579

Since JRuby ignores the files in "lib/jni" and uses the ones inside "lib/jruby.jar" instead, we have to inject the older versions into the new JRuby's "jruby.jar"

fixes #14873

(cherry picked from commit 281ce70)
mashhurs pushed a commit that referenced this pull request Feb 12, 2023
import platform-specific ffi binaries from JRuby 9.3.9.0 to overcome jruby/jruby#7579

Since JRuby ignores the files in "lib/jni" and uses the ones inside "lib/jruby.jar" instead, we have to inject the older versions into the new JRuby's "jruby.jar"

fixes #14873

(cherry picked from commit 281ce70)

Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
jsvd added a commit that referenced this pull request Feb 12, 2023
import platform-specific ffi binaries from JRuby 9.3.9.0 to overcome jruby/jruby#7579

Since JRuby ignores the files in "lib/jni" and uses the ones inside "lib/jruby.jar" instead, we have to inject the older versions into the new JRuby's "jruby.jar"

fixes #14873

(cherry picked from commit 281ce70)

Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
@andsel andsel mentioned this pull request Feb 13, 2023
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Logstash fails to start on OracleLinux7
3 participants