Problem creating sync

[jennifgcrl]

Hi, I'm running into a problem with mutagen sync create:

Local machine:

• MacOS, m2, latest
• mutagen installed by brew, version 0.17.2

Remote:

• arch linux, up to date
• openssh 9.3p1

Here's the command output:

% mutagen sync create --name=name ./folder server:folder
Copying agent...
Error: unable to connect to beta: unable to connect to endpoint: unable to dial agent endpoint: unable to install agent: unable to copy agent binary: unable to run SCP process: EOF

ssh server works, as does scp -r ./folder server:folder.

I've tried variations root@server:folder, root@server:22:folder, root@server:22:~/folder, root@server:22:/root/folder, and they all run into the same error message.

[jennifgcrl]

Tried @0.16 and mutagen-beta (again via brew), same error message.

[jennifgcrl]

Some logs:

% MUTAGEN_LOG_LEVEL=debug mutagen daemon run
2023-07-19 02:55:57.053015 [I] [license] Licensing manager starting
2023-07-19 02:55:57.053145 [I] [license] No existing key found
2023-07-19 02:55:57.053159 [I] [license] No existing license token found
2023-07-19 02:55:57.053174 [I] [license] License renewal timer triggered
2023-07-19 02:55:57.053186 [I] [forward] Looking for existing sessions
2023-07-19 02:55:57.053259 [I] [forward] Session manager initialized
2023-07-19 02:55:57.053266 [I] [sync] Looking for existing sessions
2023-07-19 02:55:57.053306 [I] [sync] Session manager initialized
2023-07-19 02:55:58.661908 [I] [sync.sync_7eJeTJrT] Connecting to alpha endpoint
2023-07-19 02:55:58.661977 [I] [sync.sync_7eJeTJrT] Connecting to beta endpoint
2023-07-19 02:55:58.662018 [D] [sync.sync_7eJeTJrT.alpha.watching] Attempting to establish recursive watch
2023-07-19 02:55:58.663596 [D] [sync.sync_7eJeTJrT.alpha.watching] Watch successfully established
2023-07-19 02:55:58.663649 [D] [sync.sync_7eJeTJrT.alpha.watching] Attempting to enable accelerated scanning
2023-07-19 02:55:58.689778 [D] [sync.sync_7eJeTJrT.alpha.watching] Accelerated scanning now available
2023-07-19 02:55:58.689797 [D] [sync.sync_7eJeTJrT.alpha] Saving cache to disk
2023-07-19 02:55:58.765042 [E] [sync.sync_7eJeTJrT.beta] /bin/bash: line 1: .mutagen/agents/0.17.2/mutagen-agent: No such file or directory
2023-07-19 02:55:59.541889 [I] [sync.sync_7eJeTJrT] Beta connection failure: unable to connect to endpoint: unable to dial agent endpoint: unable to install agent: unable to copy agent binary: unable to run SCP process: EOF
2023-07-19 02:55:59.541938 [D] [sync.sync_7eJeTJrT.alpha.watching] Watching terminated

Looks like it does succeed in putting a 7.0M .mutagen-agenthashhashhash file on the remote. Moving this file to .mutagen/agent/0.17.2/mutagen-agent and chmod +x makes everything work.

[xenoscopic]

Hey @jennifgcrl, thanks for reporting the problem. I've seen a similar issue happen with Go-based SSH servers, but I haven't been able to reproduce it myself. The fact that this is happening with OpenSSH is interesting. Is there anything unique about the sshd_config on the remote?

Oh, also, is there anything special that you're doing to access this host from the client, e.g. ProxyCommand directives in your ssh_config? This should all work without any issues, but I'm really not sure what's causing this problem.

In the meantime, the workaround you've found (manually installing the agent binary) is the suggestion that I usually offer.

[colinmarc]

I have a similar issue, with a slightly different result:

mutagen sync create ~/foo/bar remote:~/foo/bar
Installing agent...
Error: unable to connect to beta: unable to connect to endpoint: unable to dial agent endpoint: unable to install agent: unable to invoke agent installation: remote error: zsh:1: permission denied: ./.mutagen-agenta9daefdd-6f26-4a9f-b0ef-f79fada2187b

% MUTAGEN_LOG_LEVEL=debug mutagen daemon run
2023-10-21 22:25:10.239708 [I] [license] Licensing manager starting
2023-10-21 22:25:10.239914 [I] [license] No existing key found
2023-10-21 22:25:10.239931 [I] [license] No existing license token found
2023-10-21 22:25:10.239950 [I] [license] License renewal timer triggered
2023-10-21 22:25:10.239966 [I] [forward] Looking for existing sessions
2023-10-21 22:25:10.240054 [I] [forward] Session manager initialized
2023-10-21 22:25:10.240066 [I] [sync] Looking for existing sessions
2023-10-21 22:25:10.240129 [I] [sync] Loading session sync_EDFX896SYrbjeCrwRiLlRxsJgxpT03krbOoGFNLzWkh
2023-10-21 22:25:10.241604 [I] [sync.sync_EDFX896S] Session loaded
2023-10-21 22:25:10.241617 [I] [sync] Session manager initialized
2023-10-21 22:25:14.892445 [I] [sync.sync_j8GnhfNw] Connecting to alpha endpoint
2023-10-21 22:25:14.892539 [I] [sync.sync_j8GnhfNw] Connecting to beta endpoint
2023-10-21 22:25:14.892734 [D] [sync.sync_j8GnhfNw.alpha.watching] Attempting to establish recursive watch
2023-10-21 22:25:14.894906 [D] [sync.sync_j8GnhfNw.alpha.watching] Watch successfully established
2023-10-21 22:25:14.894992 [D] [sync.sync_j8GnhfNw.alpha.watching] Attempting to enable accelerated scanning
2023-10-21 22:25:15.370522 [E] [sync.sync_j8GnhfNw.beta] zsh:1: no such file or directory: .mutagen/agents/0.17.2/mutagen-agent
2023-10-21 22:25:16.260802 [D] [sync.sync_j8GnhfNw.alpha.watching] Accelerated scanning now available
2023-10-21 22:25:16.260823 [D] [sync.sync_j8GnhfNw.alpha] Saving cache to disk
2023-10-21 22:25:23.578415 [I] [sync.sync_j8GnhfNw] Beta connection failure: unable to connect to endpoint: unable to dial agent endpoint: unable to install agent: unable to invoke agent installation: remote error: zsh:1: permission denied: ./.mutagen-agenta9daefdd-6f26-4a9f-b0ef-f79fada2187b
2023-10-21 22:25:23.578563 [D] [sync.sync_j8GnhfNw.alpha.watching] Watching terminated

I'm using tailscale SSH (which as you mentioned, is a go-based ssh server).

[mafredri]

I implemented a fix for Go-based SSH servers using pkg/sftp in pkg/sftp#567.

A way mutagen could address this issue is by changing the file mode after upload.

I believe OpenSSH respects the umask that has been configured during file create, so it’s possible the executable bit is being stripped that way. A chmod would fix that issue as well.