Internal failure executing runner: fork/exec /bin/zsh: no such file or directory - Trying to run runme in remote server

[jlewi]

I'm trying to run the runme grpc server inside a GCP Cloud Workstation and connect to it from a locally running vscode. So vscode and the runme extension is running entirely locally.

When I try to execute I command I get the error.

Internal failure executing runner: fork/exec /bin/zsh: no such file or directory

I believe this error message is indicating that the RunMe golang server is trying to fork/exec /bin/zsh and can't find it. I think the problem might be exec is missing.

jeremy-lewi:~$ which zsh
/bin/zsh

man exec
No manual entry for exec

Server logs clearly indicate its receiving the request

{"commandMode":"COMMAND_MODE_INLINE_SHELL","commands":["ls -la",""],"directory":"/Users/jlewi/git_triton","envs":["RUNME_ID=01J1VAAQ1CKE25TDPJCPKSYFPV","RUNME_RUNNER=v1","TERM=xterm-256color"],"knownId":"01J1VAAQ1CKE25TDPJCPKSYFPV","knownName":"echo-host","languageId":"sh","programName":"/bin/zsh","storeLastOutput":true,"tty":true,"winsize":{"cols":190,"rows":10}}}

It looks like RunMe uses the exec package

runme/internal/runner/go.go

Line 19 in 004c66d

command *exec.Cmd

So my hunch is that the exec is missing in my host system and therefore exec doesn't work.
I have no idea why that would be.

Don't know if this qualifies as a RunMe error. I'm going to try to confirm that's the case by trying to run the sample
https://github.com/stateful/runme/tree/004c66dc1eb498d0c829c47b29ee7841458a5338/examples/goexec

Related to #593

[jlewi]

Confirmed seems like thats the problem.

jeremy-lewi:~/git_runme/examples/goexec$ go run ./
2024/07/03 04:05:53 failed to start: fork/exec /usr/local/bin/bash: no such file or directory
exit status 1

[jlewi]

I just realized I didn't read that close enough and its trying to run bash and python; let me try changing it and seeing if that fixes the problem.

[jlewi]

So I was mistaken about the problem. I changed the program to

package main

import (
        "log"
        "os"
        "os/exec"
)

func main() {
        cmd := exec.Cmd{
                Path: "/sbin/apk",
                Args:   []string{"/sbin/apk", "version"},
                Stdin:  os.Stdin,
                Stdout: os.Stdout,
                Stderr: os.Stderr,
        }

        if err := cmd.Start(); err != nil {
                log.Fatalf("failed to start: %v", err)
        }

        if err := cmd.Wait(); err != nil {
                log.Fatalf("failed to wait: %v", err)
        }
}

That works successfully.
I think there might be a bug in the example program
https://github.com/stateful/runme/blob/004c66dc1eb498d0c829c47b29ee7841458a5338/examples/goexec/main.go

Based on the docs https://pkg.go.dev/os/exec#Cmd. I think Path must be specified and must also be arg0.

[jlewi]

This version also runs. Proving zsh is installed

package main

import (
        "log"
        "os"
        "os/exec"
)

func main() {
        cmd := exec.Cmd{
                Path: "/bin/zsh",
                Args:   []string{"/bin/zsh","-c",  "apk version"},
                Stdin:  os.Stdin,
                Stdout: os.Stdout,
                Stderr: os.Stderr,
        }

        if err := cmd.Start(); err != nil {
                log.Fatalf("failed to start: %v", err)
        }

        if err := cmd.Wait(); err != nil {
                log.Fatalf("failed to wait: %v", err)
        }
}

Gives output

jeremy-lewi:~/git_runme/examples/goexec$ go run ./
Installed:                                Available:

[jlewi]

@sourishkrout Can you point me at which Runner is likely invoked here?
https://github.com/stateful/runme/tree/004c66dc1eb498d0c829c47b29ee7841458a5338/internal/runner
I'd like to see if I can reproduce the problem in a simple go program.

Is there anyway to get more verbose logs to see what's invoked? I tried adding --log but not seeing any logs being written to the log file.

[sourishkrout]

Would you mind trying Runner v2 @jlewi? If you're going through the extension it's a matter of change the runme.server.runnerVersion setting (CMD+, in VS Code)? I'm going to have to work through your analysis a little closer but would love to understand if this is something we've fixed in Runner v2 already.

[sourishkrout]

Is there anyway to get more verbose logs to see what's invoked? I tried adding --log but not seeing any logs being written to the log file.

You can turn on runme.server.enableLogger in the extension which will show more logging in the bottom panel Output->Runme. Adding --dev flag to runme server will increase the log level.

[jlewi]

I tried with RunnerV2 and it works.

{"level":"info","ts":1720816708.8424606,"caller":"runnerv2service/service_resolve_program.go:17","msg":"running ResolveProgram in runnerService"}
{"level":"info","ts":1720816708.889515,"caller":"editorservice/service.go:103","msg":"Serialize"}
{"level":"info","ts":1720816708.9089472,"caller":"runnerv2service/service_execute.go:22","msg":"running Execute in runnerService","id":"01J2MAPB9CMXYDYMMNJ5SDQD2B"}
{"level":"info","ts":1720816708.9090204,"caller":"runnerv2service/service_execute.go:34","msg":"received initial request","id":"01J2MAPB9CMXYDYMMNJ5SDQD2B","req":"config:{program_name:\"/bin/zsh\" directory:\"/Users/jlewi/git_foyle/docs/content/en/docs/integrations\" language_id:\"shellscript\" env:\"RUNME_ID=01J2MANY1FRQC86GR1M8DZBEXG\" env:\"RUNME_RUNNER=v2alpha1\" env:\"TERM=\" commands:{items:\"echo \\\"hello world\\\"\" items:\"\"} mode:COMMAND_MODE_INLINE known_id:\"01J2MANY1FRQC86GR1M8DZBEXG\"} store_stdout_in_env:true"}
{"level":"info","ts":1720816708.912865,"logger":"NativeCommand","caller":"command/command_native.go:59","msg":"detected program path and arguments","id":"01J2MAPB9CMXYDYMMNJ5SDQD2B","instanceID":"01J2MAPB9G6X0PQSTG1SJ95XA9","program":"/bin/zsh","args":["-c","set -e -o pipefail\n\n/home/user/runme env dump --insecure > /home/user/tmp/runme-93292827/.env_start\n__cleanup() {\nrv=$?\n/home/user/runme env dump --insecure > /home/user/tmp/runme-93292827/.env_end\nexit $rv\n}\ntrap -- \"__cleanup\" EXIT\necho \"hello world\"\n\n"]}
{"level":"info","ts":1720816708.912925,"logger":"NativeCommand","caller":"command/command_native.go:81","msg":"starting a native command","id":"01J2MAPB9CMXYDYMMNJ5SDQD2B","instanceID":"01J2MAPB9G6X0PQSTG1SJ95XA9","config":"program_name:\"/bin/zsh\" arguments:\"-c\" arguments:\"set -e -o pipefail\\n\\n/home/user/runme env dump --insecure > /home/user/tmp/runme-93292827/.env_start\\n__cleanup() {\\nrv=$?\\n/home/user/runme env dump --insecure > /home/user/tmp/runme-93292827/.env_end\\nexit $rv\\n}\\ntrap -- \\\"__cleanup\\\" EXIT\\necho \\\"hello world\\\"\\n\\n\" directory:\"/Users/jlewi/git_foyle/docs/content/en/docs/integrations\" commands:{items:\"echo \\\"hello world\\\"\" items:\"\"} mode:COMMAND_MODE_INLINE"}
{"level":"info","ts":1720816731.60801,"caller":"runnerv2service/service_resolve_program.go:17","msg":"running ResolveProgram in runnerService"}
{"level":"info","ts":1720816731.6730247,"caller":"runnerv2service/service_execute.go:22","msg":"running Execute in runnerService","id":"01J2MAQ1GSQR523FFSAZ63EY1C"}
{"level":"info","ts":1720816731.67309,"caller":"runnerv2service/service_execute.go:34","msg":"received initial request","id":"01J2MAQ1GSQR523FFSAZ63EY1C","req":"config:{program_name:\"/bin/zsh\" directory:\"/Users/jlewi/git_foyle/docs/content/en/docs/integrations\" language_id:\"shellscript\" env:\"RUNME_ID=01J2MANY1FRQC86GR1M8DZBEXG\" env:\"RUNME_RUNNER=v2alpha1\" env:\"TERM=\" commands:{items:\"echo \\\"hello world\\\"\" items:\"\"} mode:COMMAND_MODE_INLINE known_id:\"01J2MANY1FRQC86GR1M8DZBEXG\"} store_stdout_in_env:true"}
{"level":"info","ts":1720816731.6762297,"logger":"NativeCommand","caller":"command/command_native.go:59","msg":"detected program path and arguments","id":"01J2MAQ1GSQR523FFSAZ63EY1C","instanceID":"01J2MAQ1GW2SHD04Q5Z7F08CJW","program":"/bin/zsh","args":["-c","set -e -o pipefail\n\n/home/user/runme env dump --insecure > /home/user/tmp/runme-1746784384/.env_start\n__cleanup() {\nrv=$?\n/home/user/runme env dump --insecure > /home/user/tmp/runme-1746784384/.env_end\nexit $rv\n}\ntrap -- \"__cleanup\" EXIT\necho \"hello world\"\n\n"]}
{"level":"info","ts":1720816731.6762695,"logger":"NativeCommand","caller":"command/command_native.go:81","msg":"starting a native command","id":"01J2MAQ1GSQR523FFSAZ63EY1C","instanceID":"01J2MAQ1GW2SHD04Q5Z7F08CJW","config":"program_name:\"/bin/zsh\" arguments:\"-c\" arguments:\"set -e -o pipefail\\n\\n/home/user/runme env dump --insecure > /home/user/tmp/runme-1746784384/.env_start\\n__cleanup() {\\nrv=$?\\n/home/user/runme env dump --insecure > /home/user/tmp/runme-1746784384/.env_end\\nexit $rv\\n}\\ntrap -- \\\"__cleanup\\\" EXIT\\necho \\\"hello world\\\"\\n\\n\" directory:\"/Users/jlewi/git_foyle/docs/content/en/docs/integrations\" commands:{items:\"echo \\\"hello world\\\"\" items:\"\"} mode:COMMAND_MODE_INLINE"}

[sourishkrout]

I tried with RunnerV2 and it works.

That's great news!