Prebuilt binary do not load plugin

[Aludirk]

Version information:

go-ipfs version: 0.5.1
Repo version: 9
System version: amd64/darwin
Golang version: go1.13.10

Description:

I made an IPLD plugin for my own usage, it is based on the go-ipfs-example-plugin repo and I tune it so that it can run on IPFS v0.5.1

First, I made a local copy of go-ipfs v0.5.1 and made my plugin import this local copy. After build and install my own plugin to $IPFS_PATH/plugins and run by my build from source go-ipfs daemon, everything works perfectly

However, when I tried to import the official go-ipfs v0.5.1 rather than my local copy and run by prebuilt binary of macOS 64bit binary, the plugin just did not loaded in, and even did not call the Init of the plugin

By the way, both build from source go-ipfs and prebuilt binary are using the same $IPFS_PATH, and share the same config file

[welcome]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[Stebalien]

• Did you see any error messages?
• What go version are you using? You need to compile your plugin with go 1.13.10 (same version as go-ipfs).
• By "official version", I assume you mean one from dist.ipfs.io?

[Aludirk]

Ok, I tried one more time
I use exactly go1.13.10 to build my plugin with this time, still no luck to work with the macOS Binary

1. no error message from running ipfs commands
2. I have a print message on my plugin Init function, no such message print out when running ipfs commands
3. Even I tried to make a crash code on my plugin Init, no problem to run any ipfs commands

then I try one more time to use go1.13.10 to build my plugin with a local go-ipfs source code，everything is ok, it shows up the message in my Init code, also if I made some crash in my Init, the ipfs command crash immediately.

[Stebalien]

How are you installing the plugin exactly? What are you naming it? Is it marked as executable?

[Aludirk]

I tried one more time from scratch, and it still gets the same behaviour.

Setup

Assume work at ~/test:

For IPFS binary

1. wget https://dist.ipfs.io/go-ipfs/v0.5.1/go-ipfs_v0.5.1_darwin-amd64.tar.gz
2. tar xvfz go-ipfs_v0.5.1_darwin-amd64.tar.gz
3. export IPFS_PATH=~/test/.ipfs
4. go-ipfs/ipfs init

Just setup an ipfs v0.5.1 at ~/test/.ipfs, and version info is

go-ipfs version: 0.5.1
Repo version: 9
System version: amd64/darwin
Golang version: go1.13.10

Install Go1.13.10

1. wget https://dl.google.com/go/go1.13.10.darwin-amd64.tar.gz
2. tar -xzf go1.13.10.darwin-amd64.tar.gz
3. export PATH=~/test/go/bin:$PATH
4. export PATH=$GOPATH/bin:$PATH

For IPFS build from source

1. git clone https://github.com/ipfs/go-ipfs.git go-ipfs-source
2. cd go-ipfs-source/
3. git checkout v0.5.1
4. make build
5. export IPFS_PATH=~/test/go-ipfs-source/cmd/ipfs/.ipfs
6. cmd/ipfs/ipfs init
7. cd ..

Just setup an ipfs v0.5.1 from source at ~/test/go-ipfs-source/cmd/ipfs/.ipfs, and version info is

go-ipfs version: 0.5.1-8431e2e87
Repo version: 9
System version: amd64/darwin
Golang version: go1.13.10

For the testing plugin

1. git clone https://github.com/Aludirk/iscn-ipld.git
2. cd iscn-ipld
3. git checkout plugin-test
4. cd ..

Test

Test 1: ipfs v0.5.1 from source

1. cd iscn-ipld
2. ./set-target.sh ~/test/go-ipfs-source
3. export IPFS_PATH=~/test/go-ipfs-source/cmd/ipfs/.ipfs
4. make install
5. cd ../go-ipfs-source/cmd/ipfs
6. ls -al .ipfs/plugins and iscn-ipld.so should be found
7. ./ipfs version --all

********************************************************************************
Plugin INIT: ipld-iscn
********************************************************************************
go-ipfs version: 0.5.1-8431e2e87
Repo version: 9
System version: amd64/darwin
Golang version: go1.13.10

You should see the plugin is loaded.

Test 2: ipfs v0.5.1 binary

1. cd ../../../iscn-ipld
2. ./set-target.sh v0.5.1
3. export IPFS_PATH=~/test/.ipfs
4. make install
5. cd ..
6. ls -al .ipfs/plugins and iscn-ipld.so should be found
7. go-ipfs/ipfs version --all

go-ipfs version: 0.5.1
Repo version: 9
System version: amd64/darwin
Golang version: go1.13.10

You can see that the plugin Init did not run, even I inject some crash code in the Init function in ~/test/iscn-ipld/plugin/plugin.go, nothing happened. However if I use the crash version of plugin to repeat test#1, a crash will be happened even I just call ipfs version.

[Stebalien]

That's very strange.

• Could you try marking the plugin as non-executable (chmod a-x iscn-ipld.so)? The daemon should log an error if you do that.
• Try downloading and installing the example plugin: https://github.com/ipfs/go-ipfs-example-plugin. The "greeter" plugin should print to stdout when starting the daemon.
• What does go list -m all print when run inside your plugin?

[Aludirk]

For the non-executable test, it clearly shows that the prebuilt binary does not try to access the so file, no errors occurred. But for the build from source one, an error occurs:

ERROR plugin/loader loader/load_unix.go:38 non-executable file in plugins directory: <IPFS_PATH>/plugins/iscn-ipld.so

[Stebalien]

Ok, that means your build isn't even trying to load the plugins. My guess is that something is wrong with your IPFS_PATH environment variable.

Try running the pre-built go-ipfs binary with strace -f -e open ... to see if it's opening the right files.

[Aludirk]

In macOS, I use dtrace -n 'syscall::open*:entry { printf("%s %s", execname, copyinstr(arg0)); }' instead, and the results are as follow:

ipfs v0.5.1 from source

  2    168                       open:entry ipfs /dev/dtracehelper
  0    168                       open:entry ipfs /dev/tty
  0    168                       open:entry ipfs /dev/urandom
  0    168                       open:entry ipfs <home>/test/go-ipfs-source/cmd/ipfs/.ipfs/config
  3    168                       open:entry ipfs <home>/test/go-ipfs-source/cmd/ipfs/.ipfs/plugins
  3    168                       open:entry ipfs <home>/test/go-ipfs-source/cmd/ipfs/.ipfs/plugins/iscn-ipld.so

it worked as expected, it loads the plugin folder and file

ipfs v0.5.1 binary

  1    168                       open:entry ipfs /dev/dtracehelper
  1    168                       open:entry ipfs /dev/tty
  1    168                       open:entry ipfs /dev/urandom
  2    168                       open:entry ipfs <home>/test/.ipfs/config

it don't even try to load any of plugin folders

[Stebalien]

Could you run ls -laR ~/test/.ipfs/plugins?

[Aludirk]

ls -laR ~/test/.ipfs/plugins:

drwxr-xr-x  3 aludirk  staff        96 May 22 12:07 .
drwxr-xr-x  9 aludirk  staff       288 May 22 12:07 ..
-rwxr-xr-x  1 aludirk  staff  29350496 May 22 12:26 iscn-ipld.so

ls -la ~/test/.ipfs:

drwxr-xr-x   9 aludirk  staff   288 May 22 12:07 .
drwxr-xr-x   9 aludirk  staff   288 May 22 11:55 ..
drwxr-xr-x  34 aludirk  staff  1088 May 22 09:09 blocks
-rw-rw----   1 aludirk  staff  4808 May 22 09:09 config
drwxr-xr-x   9 aludirk  staff   288 May 22 09:09 datastore
-rw-------   1 aludirk  staff   190 May 22 09:09 datastore_spec
drwx------   2 aludirk  staff    64 May 22 09:09 keystore
drwxr-xr-x   3 aludirk  staff    96 May 22 12:07 plugins
-rw-r--r--   1 aludirk  staff     2 May 22 09:09 version

[Stebalien]

So... at this point, my remaining guess is some weird security feature where go-ipfs can't access files it didn't create.

What happens if you run the pre-built go-ipfs binary against the repo created with the one you built from source? What about the other way (run your go-ipfs build against ~/test/.ipfs/`)? I don't expect plugin loading to work, bug I'm curious to see what happens.

[Aludirk]

ipfs v0.5.1 from source

  0    168                       open:entry ipfs /dev/dtracehelper
  2    168                       open:entry ipfs /dev/tty
  0    168                       open:entry ipfs /dev/urandom
  2    168                       open:entry ipfs <home>/test/.ipfs/config
  0    168                       open:entry ipfs <home>/test/.ipfs/plugins
  0    168                       open:entry ipfs <home>/test/.ipfs/plugins/iscn-ipld.so

it tried to load and got an expected error since the plugin was built for the wrong target:

Error: error loading plugins: loading plugin <home>/test/.ipfs/plugins/iscn-ipld.so: plugin.Open("<home>/test/.ipfs/plugins/iscn-ipld"): plugin was built with a different version of package internal/cpu

ipfs v0.5.1 binary

  0    168                       open:entry ipfs /dev/dtracehelper
  1    168                       open:entry ipfs /dev/tty
  2    168                       open:entry ipfs /dev/urandom
  0    168                       open:entry ipfs <home>/test/go-ipfs-source/cmd/ipfs/.ipfs/config

as expectation, it only loads the config file but nothing about plugins

[Stebalien]

🤦 I know what's going on. The macos build was cross compiled while the linux build was compiled on linux. That means the linux build has CGO support while the macos build doesn't.

Plugins require cgo support, so we disable plugin support when CGO is disabled. We need to:

1. Change our build process to enable CGO on other platforms.
2. When building without CGO support, log an error when a plugin is installed and can't be loaded.

See: golang/go#19569