Unable to use function plugin with 1.2.0

[salmanazmat666]

Hi,
I'm trying to use a custom function plugin with kuiper v1.2.0. The plugin is built successfully, but when I use it in a rule, I get following error.

Create rule error : Parse SQL SELECT zee(v1) from demo error: error getting function zee: cannot open /home/salman/go/src/github.com/kuiper/_build/kuiper-1.2.0-linux-amd64/plugins/functions/zee.so: plugin.Open("/home/salman/go/src/github.com/kuiper/_build/kuiper-1.2.0-linux-amd64/plugins/functions/zee"): plugin was built with a different version of package internal/unsafeheader.

The function is just a copy of echo example that comes with the source code, the actual code can be found attached here. zee.txt

I get two different errors depending upon whether I compile the plugin using -trimpath flag or not.

Without -trimpath I get following error.

Create rule error : Parse SQL SELECT zee(v1) from demo error: error getting function zee: cannot open /home/salman/go/src/github.com/kuiper/_build/kuiper-1.2.0-linux-amd64/plugins/functions/zee.so: plugin.Open("/home/salman/go/src/github.com/kuiper/_build/kuiper-1.2.0-linux-amd64/plugins/functions/zee"): plugin was built with a different version of package internal/unsafeheader.

With -trimpath, I get following error

Create rule error : Parse SQL SELECT zee(v1) from demo error: error getting function zee: cannot open /home/salman/go/src/github.com/kuiper/_build/kuiper-1.2.0-linux-amd64/plugins/functions/zee.so: plugin.Open("/home/salman/go/src/github.com/kuiper/_build/kuiper-1.2.0-linux-amd64/plugins/functions/zee"): plugin was built with a different version of package github.com/emqx/kuiper/xstream/api..

This issue does not appear with kuiper 1.1.1 (I compiled without -trimpath), and the rule works correctly.
I'm using go1.15.7.

[ngjaying]

@salmanazmat666 Did you run in your local environement or Docker? I assume you have built kuiper with makefile in your local system. Then -trimpath is required.

Where did you put your plugin source file? The error 'plugin was built with a different version of package github.com/emqx/kuiper/xstream/api.' indicates the path to kuiper project is different from the main kuiper project and your plugin module. If possible, can you put your plugin into kuiper's project and build from kuiper root path?

If you do need a separate module, please also try to build from kuiper root path by adding your plugin module to kuiper's go.mod as below. Assume your plugin module name is github.com/sample/samplePlugin. And the project location is ../samplePlugin.

require(
   .........
   github.com/sample/samplePlugin v0.0.0 
)
replace github.com/sample/samplePlugin => ../samplePlugin    

Build in the kuiper root path

go build -trimpath --buildmode=plugin -o $eKuiperPath/_build/$build/plugins/functions/Zee.so ../samplePlugin/functions/zee.go

[salmanazmat666]

so putting plugin file into kuiper project and compiling from kuiper root path solved the issue.

[zeeshanshabbir93]

@ngjaying I am facing exactly same issue. I am following the plugin development tutorial. It doesn't mention that plugins source code should be in the Kuiper project. Instead it recommends using "replace" in the go.mod file of plugin module.This seems to work with v1.1.1 but not for v1.2.0. Can you please shed some light on this?

Also what is the recommended method for writing plugins for production environment? For us Kuiper will be running in docker container & we would want to add more plugins without modifying the Kuiper docker image.

[ngjaying]

@zeeshanshabbir93 I think you can either putting your plugin into Kuiper module or have a separate module.
I would recommend putting your plugin code into Kuiper module as it is the most stable way. If you do need a separate module, it seems more robust, to still build from the kuiper module as I mentioned in the first reply. There is an unresolved golang issue for building the plugin in another module. The tutorial will be updated.

Also what is the recommended method for writing plugins for production environment?

The recommended way is to build in the dev docker container with tag like "emqx/kuiper:1.2.0", export the plugin zip and use REST/CLI to install in the production docker container with tag like "emqx/kuiper:1.2.0-slim". In eKuiper's CI, we do build those shipped plugins by this way, and upload them to a central location like https://www.emqx.io/downloads/kuiper-plugins/1.0.0/debian/sinks/tdengine_amd64.zip for the installation of production systems.

[zeeshanshabbir93]

Thanks