Execute Cmd line transformer into a separate nvim instance

[asmodeus812]

Hello i would like to ask if / how would it be possible to feed the function transformer to be executed into a separate nvim instance, then fed into fzf, or if fzf allows to have a transformer argument similarly to the preview where we could pass in the action like we do for preview.

Say we have the following example, in the cmd line transformer each line could be expensive, however having it transform in the main instance, could lead to blocking even before the fzf window is opened.

        local fzf_fn = fzf_helpers.cmd_line_transformer("seq 1000", function(x)
            os.execute("sleep 10")
            local n = tonumber(x)
            return tostring(n * n)
        end)

        _G.test = coroutine.wrap(function()
            local action = fzf_helpers.choices_to_shell_cmd_previewer(function(items)
                return "seq " .. vim.fn.shellescape(tostring(items[1]))
            end)
            fzf.fzf(fzf_fn, "--preview=" .. action)
        end)

[cpkio]

What are you trying to achieve?

[asmodeus812]

I would like to process / transform the results of a big data set in a separate instance of nvim such that the current is not blocked by this processing. I was considering something like rpcrequest to an --embed nvim instance, for example.

This could be extended to the preview too, having the preview be computed in a separate child nvim instance, have that child instance server address / pipe to write to, and consume it by fzf

As it is now, the pipe preview reads/writes is the server address of the main nvim editor instance from whrere fzf was started. That could block if the processing is timeconsuming

[vijaymarupudi]

I see where you are coming from, I think this would have to be a different helper function because using a different process will dramatically change the expectations for how shared and mutable state would work (and the reduced ergonomics). It would be a bit involved, using a headless nvim like we do to handle communication with the main process, but the headless nvim itself would have to complete the processing. The function would have to be provided as a string in an argument. Reading the source code of the lua/fzf.lua file will help implement this, but unfortunately don't think I have the time to implement this anytime soon. Happy to merge it if a good quality implementation is provided!

[ibhagwan]

FYI, this is how fzf-lua’s multiprocess option works, it spawns a separate process and passes the arguments to the child as a lua string (now improved with the windows branch as a base64 encoded string), it’s also capable of reusing fzf-lua’s code without depending on the main process with a simple require overload, if a more involved function is required, function byte code is passed using string.dump:

string.dump({function})                                          *string.dump()*
        Returns a string containing a binary representation of the given
        function, so that a later |luaref-loadstring()| on this string returns a
        copy of the function. {function} must be a Lua function without
        upvalues.

and if string.dump isn’t sufficient it can communicate with the main instance over RPC for non-blocking small ops.

[vijaymarupudi]

I wasn't aware of string.dump, thanks for the tip!

[asmodeus812]

@ibhagwan yes i am aware indeed, i have read through the source. Besides icon transforms, what else is the multi process used for, by the way ?

Now that you have a lot of the shell escape functionality figured out in a more general way (being cross platform and all) would you consider contributing some of this effort eventually to this library ? Could be helpful for people wanting to build something small.

[ibhagwan]

@ibhagwan yes i am aware indeed, i have read through the source. Besides icon transforms, what else is the multi process used for, by the way ?

It’s used for everything that requires processing in a separate process, mostly icons but also ctags processing as another example, the code can take any lua string (or string.dump) as its processor.

Now that you have a lot of the shell escape functionality figured out in a more general way (being cross platform and all) would you consider contributing some of this effort eventually to this library ? Could be helpful for people wanting to build something small.

If you look at the contributors to this project I’ve done that in the past, I have over 20 contributions in this repo unfortunately I have no time for that anymore but that’s the beauty of open source, have it it :)