Logstash running natively on M1 requires signed version of jffi library

[robbavey]

Running logstash natively on an M1 equipped Mac requires the native jffi library to be signed, otherwise the following error occurs with a simple file input:

bin/logstash -e 'input { file { path => "/Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/test.txt" } } '

we see the following error:

[2021-11-30T17:31:49,511][INFO ][filewatch.observingtail  ][main][4fa55ec9f11c99b54f45a984f29681ce510082da14dce3639cbcd39099884d16] START, creating Discoverer, Watch with file and sincedb collections
warning: thread "[main]<file" terminated with exception (report_on_exception is true):
NotImplementedError: block device detection unsupported or native support failed to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
                     blockdev? at org/jruby/RubyFileTest.java:66
                    initialize at /Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.4.0/lib/filewatch/sincedb_collection.rb:20
  build_watch_and_dependencies at /Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.4.0/lib/filewatch/observing_base.rb:62
                    initialize at /Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.4.0/lib/filewatch/observing_base.rb:56
              start_processing at /Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.4.0/lib/logstash/inputs/file.rb:352
                           run at /Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.4.0/lib/logstash/inputs/file.rb:368
                   inputworker at /Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/logstash-core/lib/logstash/java_pipeline.rb:409
                   start_input at /Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/logstash-core/lib/logstash/java_pipeline.rb:400
warning: thread "Ruby-0-Thread-26: :1" terminated with exception (report_on_exception is true):

when running with -Djruby.native.verbose=true set in config/jvm.options we get the following error:

Failed to load native POSIX impl; falling back on Java impl. Stacktrace follows.
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider
	at jnr.ffi.provider.InvalidProvider$1.loadLibrary(InvalidProvider.java:49)
	at jnr.ffi.LibraryLoader.load(LibraryLoader.java:420)
	at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:307)
	at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:335)
	at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:41)
	at jnr.posix.MacOSPOSIX.<init>(MacOSPOSIX.java:16)
	at jnr.posix.POSIXFactory.loadMacOSPOSIX(POSIXFactory.java:153)
	at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:121)
	at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:93)
	at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:40)
	at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:34)
	at jnr.posix.LazyPOSIX.isNative(LazyPOSIX.java:404)
	at org.jruby.util.io.FilenoUtil.<init>(FilenoUtil.java:42)
	at org.jruby.Ruby.<init>(Ruby.java:294)
	at org.jruby.Ruby.newInstance(Ruby.java:706)
	at org.logstash.Logstash.<init>(Logstash.java:147)
	at org.logstash.Logstash.main(Logstash.java:69)
Caused by: java.lang.UnsatisfiedLinkError: could not get native definition for type `POINTER`, original error message follows: java.lang.UnsatisfiedLinkError: Unable to execute or load jffi binary stub from `/var/folders/nz/p6mxp5yd3dl0_c1_s755xlnc0000gn/T/`. Set `TMPDIR` or Java property `java.io.tmpdir` to a read/write path that is not mounted "noexec".
/Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/jffi15313360021452729204.dylib: dlopen(/Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/jffi15313360021452729204.dylib, 1): no suitable image found.  Did find:
	/Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/jffi15313360021452729204.dylib: code signature in (/Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/jffi15313360021452729204.dylib) not valid for use in process using Library Validation: Trying to load an unsigned library
	at com.kenai.jffi.internal.StubLoader.tempLoadError(StubLoader.java:551)
	at com.kenai.jffi.internal.StubLoader.loadFromJar(StubLoader.java:450)
	at com.kenai.jffi.internal.StubLoader.load(StubLoader.java:326)
	at com.kenai.jffi.internal.StubLoader.<clinit>(StubLoader.java:614)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:398)
	at com.kenai.jffi.Init.load(Init.java:68)
	at com.kenai.jffi.Foreign$InstanceHolder.getInstanceHolder(Foreign.java:50)
	at com.kenai.jffi.Foreign$InstanceHolder.<clinit>(Foreign.java:46)
	at com.kenai.jffi.Foreign.getInstance(Foreign.java:104)
	at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:242)
	at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
	at com.kenai.jffi.Type.resolveSize(Type.java:155)
	at com.kenai.jffi.Type.size(Type.java:138)
	at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:198)
	at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
	at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:77)
	at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:49)
	at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:73)
	at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:60)
	at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at java.base/java.lang.Class.newInstance(Class.java:584)
	at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:68)
	at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:57)
	at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
	at jnr.ffi.LibraryLoader.create(LibraryLoader.java:89)
	at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:292)
	at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:335)
	at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:41)
	at jnr.posix.MacOSPOSIX.<init>(MacOSPOSIX.java:16)
	at jnr.posix.POSIXFactory.loadMacOSPOSIX(POSIXFactory.java:153)
	at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:121)
	at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:93)
	at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:40)
	at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:34)
	at jnr.posix.LazyPOSIX.isNative(LazyPOSIX.java:404)
	at org.jruby.util.io.FilenoUtil.<init>(FilenoUtil.java:42)
	at org.jruby.Ruby.<init>(Ruby.java:294)
	at org.jruby.Ruby.newInstance(Ruby.java:706)
	at org.logstash.Logstash.<init>(Logstash.java:147)
	at org.logstash.Logstash.main(Logstash.java:69)

	at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:253)
	at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
	at com.kenai.jffi.Type.resolveSize(Type.java:155)
	at com.kenai.jffi.Type.size(Type.java:138)
	at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:198)
	at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
	at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:77)
	at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:49)
	at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:73)
	at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:60)
	at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at java.base/java.lang.Class.newInstance(Class.java:584)
	at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:68)
	at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:57)
	at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
	at jnr.ffi.LibraryLoader.create(LibraryLoader.java:89)
	at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:292)
	... 14 more

Looking at the debug stack trace, we can see there is a failure trying to load an unsigned library:

code signature in (/Users/robbavey/artifacts/logstash-8.1.0-SNAPSHOT/jffi15313360021452729204.dylib) not valid for use in process using Library Validation: Trying to load an unsigned library

Relates:
jnr/jnr-ffi#257
jnr/jffi#116

[robbavey]

Further investigation:

By using the codesign tool to manually signing the libjffi-1.2.jnilib library in jruby.jar and jruby-complete.jar

jar xf ./vendor/jruby/lib/jruby.jar jni/Darwin/libjffi-1.2.jnilib
codesign -s - jni/Darwin/libjffi-1.2.jnilib
jar uf ./vendor/jruby/lib/jruby.jar jni/Darwin/libjffi-1.2.jnilib
jar xf ./logstash-core/lib/jars/jruby-complete-9.2.20.0.jar jni/Darwin/libjffi-1.2.jnilib
codesign -s - jni/Darwin/libjffi-1.2.jnilib
jar uf ./logstash-core/lib/jars/jruby-complete-9.2.20.0.jar jni/Darwin/libjffi-1.2.jnilib

Logstash starts correctly, and gets past the error initially reported in the file input, the root cause of which appears to be the unsigned library

[robbavey]

Note, that the signed version of the jffi library has been released as part of jruby-9.3.2.0 - https://www.jruby.org/2021/12/02/jruby-9-3-2-0.html

[robbavey]

Fixed by upgrade to jruby-9.3.6.0 in #14304