Fix tests hostkeys

[6543]

allow ssh-rsa as hostkey algorithm for integration tests

source: https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/26273

[zeripath]

Tests fail as the version of ssh we are using does not have this option - but instead has the older version instead:

pubkeyacceptedkeytypes

---

Edit:

I should note that the underlying problem requiring this is NOT due to HostKeyAlgorithm acceptance (that was fixed by #17376) but due to the PubkeyAcceptedAlgorithm/PubkeyAcceptedKeyTypes which is a deeper issue with the way in which x/crypto/ssh handshakes over SSH.

Fixing that will require handling an SSH extension. It is also worth noting that there is a design flaw in the way that signers are related to keys and there is no way to separately configure Signers.

[6543]

so should we update our test environment or let it just stay open until the ssh version is common?

[zeripath]

We should really fix the ssh problem - although that will mean forking X/crypto/ssh

[techknowlogick]

@zeripath Ive only been casually following the upstream issue, but my understanding is that there were two recent PRs that were merged to resolve this. Would updating our go.mod to use latest x/crypto be suitable enough

[zeripath]

I think you're referring to golang/crypto@b4de73f

This is handling the host key certificate problem in much the same way as my merged PR for Gitea does. It's unfortunately not going to handle the client public key problem that this PR is referring to.

It's worth noting that the above commit has not addressed the design flaw that keys and signers are related but are not the same. Neither Gitea nor the X/crypto/ssh can actually turn off ssh-rsa signature verification for hosts right now.

[techknowlogick]

Ah yes, that was it. Thanks for that explanation of the situation. It seems the current possible options are rather suboptimal in that case, the least preferred option (at least personally) would be to fork x/crypto.

[zeripath]

Ah yes, that was it. Thanks for that explanation of the situation. It seems the current possible options are rather suboptimal in that case, the least preferred option (at least personally) would be to fork x/crypto.

Yeah I don't particularly fancy the idea of having to maintain an SSH server... (Although to some extent our hope that we can abstract that problem away to someone else has now failed and I've had to learn (something) about the SSH protocol and KEY_EXT_INFO etc.)

I guess the question is how long do we think upstream will take to fix this issue?

Whilst this issue is still unsolved our SSH server is not a production ready item and cannot be used out of the box with up-to-date ssh. Upstream do not appear to be moving with any speed to resolve this, and from looking at the code there are several fundamental design issues including one that means we cannot in future disable host-key signature verification with the ssh-rsa signing algorithm. This is far from ideal.

For me to work out how to properly fix this and propose a PR would involve me gaining a much more complete understanding of the SSH protocol and the design of x/crypto/ssh. We would need to change the handshake.go file considerably and we're going to need to properly separate signers from keys as I proposed on the golang issue. Such a PR would not only take some time to write but no doubt some time to merge. In the meantime it would entail Gitea using and maintaining that fork until the PR or an equivalent was merged in. The trouble is that I don't have a complete understanding of the SSH protocol or how x/crypto/ssh is designed but do have enough knowledge to say that this is not a trivial task, I therefore expect that x/crypto/ssh might be slow to merge such a PR.

I wonder if gliderlabs' @belak has a better understanding of the design in x/crypto/ssh and whether they would be able to workaround the issue. (However, AFAICS there simply isn't a place we can hook in to to sort this out as it is fundamentally a problem with the way handshake.go is written.)

[zeripath]

We need to collect these issues in one place and open a proper Gitea issue for this.

---

For speed I'm going to collect things here:

• key exhange negotiation failed though client and server share some protocols #17175 reported that there was a key negotiation failure with newer versions of SSH and Gitea.
• Through investigation this was revealed to be a HostKey algorithm failure due to the key type being used to determine the signing algorithms available for the key.
• Fortunately the HostSigners are exposed on the underlying ssh server so although there is a fundamental flaw in the way signers (their type is not reportable separate from the key) Offer rsa-sha2-512 and rsa-sha2-256 algorithms in internal SSH #17281 was able to work around this by wrapping those keys around something to pretend to be signers.
• This then fixes the hostkey issue - however, I was unaware of the next issue...
• @andypost then commented on Configurable SSL cipher suite #9691 (comment) (Which is unfortunately an unrelated feature request issue relating to HTTPS not SSH.)
• My comment here Configurable SSL cipher suite #9691 (comment) describes that the issue is actually relating to the next step of authentication - the pubkeyalgorithm
• I then discovered that the problem is relating to the pubkey signing algorithms offered to the client Configurable SSL cipher suite #9691 (comment)
• There is an upstream issue: x/crypto/ssh: support for server-sig-algs extension (RFC8308) golang/go#49269 relating to supporting the required extension.
• And there is a PR here: ssh: add support for extension negotiation (rfc 8308) golang/crypto#197 that implements this but I'm not completely convinced its totally correct.
• In the meantime go have merged: golang/crypto@b4de73f
• This does a similar bodge to the one I did in Offer rsa-sha2-512 and rsa-sha2-256 algorithms in internal SSH #17281, however, whereas Offer rsa-sha2-512 and rsa-sha2-256 algorithms in internal SSH #17281 leaves the possibility of disabling certain host key signing algorithms in the future, if we support ssh-rsa we have to support rsa-sha2-256 and rsa-sha2-512, and no way is provided to support only rsa-sha2-256 and rsa-sha2-512. (The above commit does provide certificate support which is something that Offer rsa-sha2-512 and rsa-sha2-256 algorithms in internal SSH #17281 missed.)

---

What is actually needed:

• Signers need to be separately configurable and advertisable apart from the key types that they sign.
• The Handshake needs to support extension negotiation https://datatracker.ietf.org/doc/html/rfc8308 and in particular the SSH_MSG_EXT_INFO extension described therein.

[belak]

After digging through this for the past hour or so (which is probably a mistake for me since it's almost 4am here), I've come to pretty much the same conclusion - the only way I'm aware of to fix this is in the Go ssh internals, a hack in gliderlabs/ssh wouldn't be enough. I really have no desire at all to fork and maintain the core SSH code, especially when that isn't a part of my job, but I'd be happy to update or review PRs for gliderlabs/ssh to support upstream changes (if needed).

In terms of my knowledge, this was one part of the spec which I wasn't aware of, so I'm not sure how helpful I'll be. I'm a bit more familiar with the layer on top of this (channels, requests, etc).

On a related note, I think your comment here is relevant. It does a good job summarizing the changes which would need to happen to the Signer interface.

I'm also available in Discord for any quick questions, or if you just need someone to bounce ideas off of.

[wxiaoguang]

Could we get this fix in 1.16, or should we leave it to 1.17 (and then backport) ?

[techknowlogick]

@wxiaoguang if CI passes I'm happy to have it merged before 1.17

[zeripath]

As I say above - this PR currently relies on an option that our testing version of ssh doesn't have and that only affected versions of ssh have.

If we want to merge this we need to detect if ssh has this option before setting it. There's likely little point setting the old option as only unaffected versions should be affected.

However this is only a workaround bodge for the more substantial problem that the ssh package in x/crypto/ssh is broken and lacks the functionality to handle pubkey authentication with RSA except through the old deprecated ssh-rsa algorithm.

[lunny]

CI failed is related.

[6543]

-> #17798 :D

[6543]

I think it could be simpler if we should generate a new ed25519 ssh key and replace the old rsa key with it ...