SSH Tunnel Authentication failed

[adam-paterson]

I'm trying to use Sequel Pro to connect to a DB, up until this morning this was working as expected but today I get an authentication error. I have no issues when using warden shell

Used command:  /usr/bin/ssh -v -N -S none -o ControlMaster=no -o ExitOnForwardFailure=yes -o ConnectTimeout=10 -o NumberOfPasswordPrompts=3 -o TCPKeepAlive=no -o ServerAliveInterval=60 -o ServerAliveCountMax=1 tunnel.warden.test -L 52173:REDACTED_db_1:3306

OpenSSH_7.9p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/adam_paterson/.ssh/config
debug1: Reading configuration data /Users/adam_paterson/.magento-cloud/ssh/session.config
debug1: Reading configuration data /Users/adam_paterson/.magento-cloud/.session/sess-cli-default/ssh/config
debug1: /Users/adam_paterson/.ssh/config line 21: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: /etc/ssh/ssh_config line 52: Applying options for tunnel.warden.test
debug1: Control socket " none" does not exist
debug1: Connecting to 127.0.0.1 [127.0.0.1] port 2222.
debug1: fd 3 clearing O_NONBLOCK
debug1: Connection established.
debug1: identity file /Users/adam_paterson/.warden/tunnel/ssh_key type 0
debug1: identity file /Users/adam_paterson/.warden/tunnel/ssh_key-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.9
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.1
debug1: match: OpenSSH_8.1 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 127.0.0.1:2222 as 'user'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:wY7AcNN3Xlu5+6ccvduSdX12EFPgUaVivIoO6BFZD78
debug1: Host '[127.0.0.1]:2222' is known and matches the ECDSA host key.
debug1: Found key in /Users/adam_paterson/.ssh/known_hosts:60
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: Will attempt key: user@tunnel.warden.test RSA SHA256:1/vjx4UoEhlTjU+i3NSDBA5pPtThnHtGs2E3BNK1Tdk agent
debug1: Will attempt key: /Users/adam_paterson/.warden/tunnel/ssh_key RSA SHA256:aPLEjn4a+CyyzsgWlJHytMKR347dRr/MoDqVnTzNJxw
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering public key: user@tunnel.warden.test RSA SHA256:1/vjx4UoEhlTjU+i3NSDBA5pPtThnHtGs2E3BNK1Tdk agent
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: Offering public key: /Users/adam_paterson/.warden/tunnel/ssh_key RSA SHA256:aPLEjn4a+CyyzsgWlJHytMKR347dRr/MoDqVnTzNJxw
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: No more authentication methods to try.
user@127.0.0.1: Permission denied (publickey,keyboard-interactive).

[sebastian-ehrling]

please run warden svc down and warden svc up and try again

[adam-paterson]

@sebastian-ehrling I've tried this at least 40 times. Even deleted the keys and regenerated them. However, for some reason this time it's worked.

Thank you.

[adam-paterson]

@sebastian-ehrling Same issue has occurred again warden svc down and warden svc up wasn't any help this time though.

[sebastian-ehrling]

@adam-paterson which version of docker do you use?
I have the same issue after upgrading to docker 2.5.0.0

[jschreck]

+1 - upgraded docker 2.5.0.0 and ssh stopped working.

[TommyChausson]

+1 Here, same issue after upgrading docker to 2.5.0.0

[Skullsneeze]

Same issue here. Just set-up a new mac, running the following:
Docker desktop: 2.5.0.0
Docker: 19.03.13
Docker-Compose: 1.27.4

When checking the tunnel service logs I see the following when trying to manually ssh'ing into the container:

Authentication refused: realpath /etc/authorized_keys/user failed: No such file or directory
Connection closed by authenticating user user 172.20.0.1 port 39850 [preauth]

However, when I warden svc exec tunnel /bin/bash into the tunnel container, and check if the file exists I see the following:

bash-5.0# ls -la /etc/authorized_keys/
total 16
drwxr-xr-x    1 root     root          4096 Nov  4 10:32 .
drwxr-xr-x    1 root     root          4096 Nov  4 10:32 ..
-rw-r--r--    1 user     user           405 Nov  4 09:45 user

What worked for me
After some digging and trying things out, I've set the following directive StrictModes no in /etc/ssh/sshd_config, in the tunnel container, and restarted the service (within docker desktop). After this is seems like I'm able to successfully connect again.

I believe this indicates that there is some issue with permissions for the ssh files. And setting this directive should only be used as a temporary solution.

[davidalger]

Just checked on the machine that I'd recently upgraded to 2.5.0.0 and the logs showed the same error @Skullsneeze logged above.

What resolves this for me (temporarily) is re-creating the container:

$ warden svc up --force-recreate tunnel
...
$ ssh tunnel.warden.test
Welcome to the Warden SSH tunnel container!

This tunnel container is used only for forwarding TCP
connections, generally to port 3306 of db containers
and is not typically used with an interactive shell.

b851372ef524:~$ 

As soon as the container has been restarted (either because Docker has restarted or when a manual warden svc restart tunnel has been run) the issue returns.

The file /etc/authorized_keys/user is mounted into the container:

    volumes:
    - /Users/davidalger/.warden/tunnel/ssh_key.pub:/etc/authorized_keys/user:rw
    - sshd_keys:/etc/ssh/keys:rw

I don't have time to dig further at the moment, but I have a sneaking suspicion this may be related to the use of the newer gRPC Fuse driver, possibly combined with something the startup script embedded in the panubo/sshd image that's in play here is doing, and switching back to OSXFS (i.e. turning off "Use gRPC FUSE for file sharing" in preferences) may workaround the issue.

[jschreck]

Thank you @davidalger - I can confirm that your workaround makes it work for me.

[adam-paterson]

Thank you, removing the tunnel (panubo/sshd) container, and recreating it does resolve the problem for now.

[sajidunnar]

I am having very strange issue, in ubuntu Table Plus works fine with tunnel but Navicate or other software does not connect, and somehow pub key is owned by root so if I start the app as root then once it got connected but again I am in loops and it dont work but table plus working fine

Sorry for bad english

[AnimNyan]

I'm getting an error from using windows using WSL2 with warden for magento 2.4.6 when trying to connect to the database with DBeaver. I think the error is probably because I'm using windows with WSL2 with an ubuntu VM and the extra step is causing the ssh connection to fail.

I've tried using these configuration settings to access my Database from Phpstorm and DBeaver: https://docs.warden.dev/configuration/database.html#phpstorm

The error I'm getting is in PHPStorm when I'm setting up the SSH configuration to connect to the database from warden. I'm getting a Connection to user@tunnel.warden.test:2222 Cannot connect to remote host: host not found

My settings in PHPStorm for the SSH Configuration is Host: tunnel.warden.test , Port: 2222, Username: user, Private key file: \wsl$\Ubuntu\home<myusername>.warden\tunnel\ssh_key and Parse config file ~/.ssh/config is turned on.

However, when I try the command within my ubuntu WSL2 command line:

ssh user@tunnel.warden.test
Welcome to the Warden SSH tunnel container!

This tunnel container is used only for forwarding TCP
connections, generally to port 3306 of db containers
and is not typically used with an interactive shell.

I've also checked in my /etc/ssh/ssh_config and see this entry which I see was mentioned above:

## WARDEN START ##
Host tunnel.warden.test
HostName 127.0.0.1
User user
Port 2222
IdentityFile ~/.warden/tunnel/ssh_key
## WARDEN END ##

It works. I looked at the solution here: #27 (comment) by @erikhansen but I don't see this file ~/.ssh/config in my Ubuntu WSL 2 machine.

I'm seeing this:

cat ~/.ssh/config
cat: /home/<myusername>/.ssh/config: No such file or directory

So I don't have anything to delete I'm just very confused because ssh works fine but the ssh configuration to my WSL 2 ubuntu vm running mysql does not work.

I've also tried using:
warden svc up --force-recreate tunnel

And I've also tried warden env down and warden env up to recreate all the containers, but no effect.

but it doesn't make a difference the host is unknown in php storm for the ssh connection and it refuses to connect.

Temporary workaround:
using warden db connect to run sql commands from the commandline instead which works.

[tonybenny2020]

Hi All, The solution for this issue is to update your JDBC drivers for MariaDB if you are using MariaDB .This can be achieved by Going to Dbeaver->Database->Driver Manager and then Double click on MariaDB in the list which will open a dialog window with tabs including Settings, Libraries etc, go to the Libraries tab and then click on Download/Update in the right-hand side. Then please download the update by checking the message 'MariaDB driver files are missing' in the dialog box. Don't forget to tick the Force download/overwrite checkbox, Click on the download botton and you will get the latest update of JDBC driver. Restart the Dbeaver and Just delete your DB connection in the Dbeaver. and re add the DB connection with the warden SSH details and connection setting by deleting the older connection settings. Click on the test connection it will work fine. Refer the screenshots for easier fix.

@AnimNyan @jschreck @davidalger