sshd container image for ARM64

[jerrinot]

Hello,
I'm using withAccessToHost() and Testcontainers.exposeHostPorts() and I realized the sshd container this functionality depends on is built for Intel only. Would it be possible to create also ARM64 build? I see Ryuk now has ARM builds too. I could help with that, but I don't know how exactly is the sshd container image build. Thank you for this awesome library!

[kiview]

Hey @jerrinot, thanks for raising the issue. Have a look at this repo:
https://github.com/testcontainers/sshd-docker

Would be great if you can look into providing an ARM64 build 👍

[jerrinot]

hi @kiview,

thanks for a quick response! I actually checked the repo. There is just a Dockerfile definition, no script to actually build a container image out of it.

For multiplatform images, a Dockerfile is usually intact. You just build it differently. dockerx and friends. I don't know how this is built. Maybe @bsideup built it manually on his local? The last push was 2 years ago and I can imagine there wasn't much pressure to automate it.

I could try to contribute a GitHub Action file to automate it if you are interested in that.

[jerrinot]

Something like this should be sufficient if you are OK with just the latest tag in DockerHub: https://github.com/docker/build-push-action/blob/master/docs/advanced/multi-platform.md It's super simple, I guess a released version of TestContainers should not depend on latest.

This looks like a pretty good intro how to do it properly: https://dev.to/cloudx/multi-arch-docker-images-the-easy-way-with-github-actions-4k54

[kiview]

We are using GHA to build the multi-arch image in the case of Ryuk:
https://github.com/testcontainers/moby-ryuk/blob/master/.github/workflows/build-docker-image.yml

[jerrinot]

that looks good. let me know if I can help with that in any way.

[eddumelendez]

@jerrinot your contribution is very welcome. I think it make sense to add a GHA to https://github.com/testcontainers/sshd-docker

[jerrinot]

@eddumelendez: I sent a pull request
Let me know what you think. Cheers!

[eddumelendez]

new version should be updated here

[jerrinot]

it's also hardcoded in the docs:

testcontainers-java/docs/features/configuration.md

Line 56 in 737865f

> **sshd.container.image = testcontainers/sshd:1.0.0**

[bsideup]

The image is published now 🎉 @jerrinot please give it a try (with the mentioned configuration property)

https://hub.docker.com/layers/232132304/testcontainers/sshd/1.1.0/images/sha256-7879c6c99eeab01f1c6beb2c240d49a70430ef2d52f454765ec9707f547ef6f1?context=repo

[jerrinot]

@bsideup thanks! I did a quick test an there appears to be a problem:

org.testcontainers.containers.ContainerLaunchException: Container startup failed

	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:345)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:326)
	at org.testcontainers.containers.GenericContainer.starting(GenericContainer.java:1063)
	at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:338)
	... 22 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:537)
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:340)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
	... 23 more
Caused by: java.io.IOException: There was a problem while connecting to localhost:58384
	at org.testcontainers.shaded.com.trilead.ssh2.Connection.connect(Connection.java:755)
	at org.testcontainers.containers.PortForwardingContainer.createSSHSession(PortForwardingContainer.java:47)
	at org.testcontainers.containers.PortForwardingContainer.getSshConnection(PortForwardingContainer.java:27)
	at org.testcontainers.containers.PortForwardingContainer.start(PortForwardingContainer.java:73)
	at org.testcontainers.containers.GenericContainer.applyConfiguration(GenericContainer.java:836)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:374)
	... 25 more
Caused by: java.io.IOException: Key exchange was not finished, connection is closed.
	at org.testcontainers.shaded.com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:93)
	at org.testcontainers.shaded.com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:231)
	at org.testcontainers.shaded.com.trilead.ssh2.Connection.connect(Connection.java:711)
	... 30 more
Caused by: java.io.IOException: Cannot negotiate, proposals do not match.
	at org.testcontainers.shaded.com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:415)
	at org.testcontainers.shaded.com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:756)
	at org.testcontainers.shaded.com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:470)
	at java.base/java.lang.Thread.run(Thread.java:833)

Chances are I have something wrong in my test setup. I will investigate it later.

[jerrinot]

ok, even the ExposedHostTest is failing with the new sshd container. Chances are the new Alpine base image has a newer sshd with a stricter configuration. The ssh client in TestContainer is old and not really maintained. The latest release in Maven Central is from 2017

I will investigate it further, but as of now I see 2 options:
1: Customize the sshd container to make sshd more permissive.
2. Change ssh client impl. There are various forks of the trilead library. It seems e.g. Jenkins uses this: https://mvnrepository.com/artifact/org.jenkins-ci/trilead-ssh2/trilead-ssh2-build-217-jenkins-17

[jerrinot]

@bsideup I think you are using the Glassfish fork, arent you?
I tried to switch the client impl to JSch: 5ede16b

It's almost working. Except the test where it's binding to a port 80. I suspect it's due a newer ssh version not allowing to forward a privileged port (by default?) To be continued.

edit: apparently, I did something wrong before. now, the tests are failing.

[jerrinot]

@bsideup: this works for me: #5486

[kiview]

Thanks, @jerrinot. While it might make sense to switch to a better-maintained SSH client library, I am in favor of first getting it working through changing the sshd config as done in #5486.