(remote proxy) encountered error while forwarding messages: stderr_task failed

[aleclarson]

Check for existing issues

• Completed

Describe the bug / provide steps to reproduce it

While trying to connect Zed to a remote Zed on another macOS Intel machine, I get a timeout during the "Starting Proxy..." message.

Environment

Zed: v0.161.1 (Zed Preview)
OS: macOS 11.7.8
Memory: 4 GiB
Architecture: x86_64

If applicable, add mockups / screenshots to help explain present your vision of the feature

No response

If applicable, attach your Zed.log file to this issue.

Zed.log

2024-11-11T20:38:11.633271-05:00 [ERROR] (remote proxy) encountered error while forwarding messages: stderr_task failed

Caused by:
    stderr closed, terminating...
2024-11-11T20:38:11.634064-05:00 [ERROR] (remote proxy) exiting due to error: stderr_task failed
2024-11-11T20:38:11.634765-05:00 [ERROR] proxy process terminated unexpectedly
2024-11-11T20:38:11.634988-05:00 [INFO] aborting reconnect, because not in state that allows reconnecting
2024-11-11T20:38:15.283209-05:00 [ERROR] failed to establish connection: Timeout detected
2024-11-11T20:38:15.285776-05:00 [ERROR] Timeout detected

[ConradIrwin]

@aleclarson This error message usually indicates that the run command is failing to boot. Do you have anything in ~/.local/share/zed/logs with more information?

[aleclarson]

~/.local/share/zed doesn't exist on either machine

[ConradIrwin]

Hmm. What OS is on the remote side? If it's macOS then ~/Library/Logs/Zed/ is the directory you want. If it's linux and you have XDG_* environment variables set, then it will depend on those.

The way this should work is that:

• Over ssh we run .zed_server/zed-remote-server-preview-0.161.1 proxy --identifier X.
• The proxy runs .zed_server/zed-remote-server-preview-0.161.1 run --lots-of-args-here.

While Zed is trying to SSH into the machine, can you run ps ax | grep zed-remote I want to see if the proxy command is running, what the identifier is, and if the run command is running.

After doing that, if you run .zed_server/zed-remote-server-preview-0.161.1 proxy --identifier X does it print out some binary data? (it tries to talk protobuf over stdout) or does something else happen?

Happy to debug together if you want: https://calendly.com/ConradIrwin/pairing

[Tom-Boscher]

I have the same issue with macOS Monterey (host) and Raspberry Pi OS Lite (client)

Here is my logs:

2024-11-13T18:20:26.267871+01:00 [ERROR] proxy process terminated unexpectedly
2024-11-13T18:20:26.268014+01:00 [INFO] aborting reconnect, because not in state that allows reconnecting
2024-11-13T18:20:31.152511+01:00 [ERROR] failed to establish connection: Timeout detected
2024-11-13T18:20:31.153065+01:00 [ERROR] Timeout detected

ps ax | grep zed-remote during connection give me:

77969   ??  S      0:00.01 ssh -o ControlMaster=no -o ControlPath=/var/folders/vf/szyq6yw97hq73rtm09v25g_80000gn/T/zed-ssh-session8fX2lg/ssh.sock ssh://pi@raspberrypi.local sh -c 'gunzip -f .zed_server/zed-remote-server-preview-0.161.1-download-77356.gz && chmod 755 .zed_server/zed-remote-server-preview-0.161.1-download-77356 && mv .zed_server/zed-remote-server-preview-0.161.1-download-77356 .zed_server/zed-remote-server-preview-0.161.1'

[ConradIrwin]

@Tom-Boscher What does uname -sm report on your Raspberry Pi? I wonder if we're erroneously downloading a 64-bit binary that cannot work on a 32-bit ARM machine.

[Tom-Boscher]

@Tom-Boscher What does uname -sm report on your Raspberry Pi? I wonder if we're erroneously downloading a 64-bit binary that cannot work on a 32-bit ARM machine.

Linux armv6l

[Tom-Boscher]

The Raspberry Pi Zero W is indeed using a 32 bits processor

[ConradIrwin]

Ok. Let me see if we can figure out whether or not we have a 64-bit arm processor from just the string we get back.

The extra bad news for you is that we don't yet ship 32bit arm builds. For now you may be able to build your own to make it work, but I appreciate this is a bit of a pain.

[ConradIrwin]

Actually, and in further bad news, you cannot currently build zed's remote server for arm32 due to bytecodealliance/wasmtime#1173.

In the future it might be reasonable to have a version without wasm time on 32-bit platforms. This would mean extensions don't work, but that might be a fine compromise.

[aleclarson]

While Zed is trying to SSH into the machine, can you run ps ax | grep zed-remote I want to see if the proxy command is running, what the identifier is, and if the run command is running.

It returns nothing on both machines while Starting proxy... is displayed in Zed UI.

What OS is on the remote side? If it's macOS then ~/Library/Logs/Zed/ is the directory you want.

OS: macOS 14.4.1
Memory: 20 GiB
Architecture: x86_64

Logs in ~/Library/Logs/Zed/ are not being written to when running open remote command.

[aleclarson]

It's working for me with Zed Preview v0.162.0

[aleclarson]

After installing Zed stable v0.162.1, I'm experiencing the same issue, I believe.

When I run ps ax | grep zed while trying to connect, I get:

ssh -N -o ControlPersist=no -o ControlMaster=yes -o ControlPath=/var/folders/g0/mlf9ffl54_52_z20b_bv85wh0000gn/T/zed-ssh-sessionQCFoGX/ssh.sock ssh://minion.local

My ~/.ssh/config contains:

Host minion.local
  User alec
  IdentityFile ~/.ssh/minion
  IdentitiesOnly yes

…and I can confirm ssh minion.local works as expected from the terminal.

The zed: open log command displays the same output shared in the original bug report.

[ConradIrwin]

@aleclarson what is the output of uname -sm on minion.local?

[aleclarson]

Darwin x86_64

[ConradIrwin]

Hmm. That should work just fine, is there anything in ~/Library/Logs/Zed/ on minion.local? I'm looking particularly for panic reports.

If not, happy to try and figure this out on a video call: https://calendly.com/conradirwin/pairing