Skip to content

CI: run .NET Framework Integration Tests on Windows #1615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 25, 2025

Conversation

mus65
Copy link
Contributor

@mus65 mus65 commented Mar 14, 2025

This runs the .NET Framework tests on Windows by setting up WSL2 in the Windows VM with Vampire/setup-wsl and starting the SSH server container with podman in the Ubuntu VM. This should catch more Windows specific issues than the current mono solution does.

To my surprise, this actually works. It's pretty slow though. A full test run takes about an hour. Also the already flaky test from #1253 has failed on every run so far. I assume the bad performance is somehow due to the nested virtualization.

We could either disable the flaky test and live with the long CI time or only test specific tests here, like before with mono. Or not do this at all since it still feels a little experimental...

Why not with docker? Docker Desktop is not supported on Windows Server. Setting up docker in the ubuntu VM manually is a lot more complex than podman since it must be run as a daemon. I tried this as well, but couldn't get it to work. It can't connect to the Ubuntu VM for some reason.

@Rob-Hague
Copy link
Collaborator

Awesome. I also tried to setup docker with WSL2 but decided I didn't know enough about it and gave up

@Rob-Hague
Copy link
Collaborator

Could it be separated into a third leg, such that the publish step does not depend on its success? And then we can see how it goes with reliability

Some of the slowness might be because a lot of the tests make several connections between test initialise and cleanup. For example, the tests in CipherTests each make 6 connections. But it doesn't seem to be that costly locally so perhaps not worth optimising

so publish doesn't depend on it
@Rob-Hague
Copy link
Collaborator

Have you normally only seen that particular test fail or others as well? If it's just that one, then I think we can disable it in CI (would like to keep it locally - I'm pretty sure it's a bug)

@mus65
Copy link
Contributor Author

mus65 commented Mar 20, 2025

I have only seen this particular test fail. I skip it now by checking for the CI env variable. There may be a better skip the test. NUnit has an [Explicit] attribute for this, but seems like MSTest doesn't have anything similar.

@Rob-Hague Rob-Hague merged commit 071a8d5 into sshnet:develop Mar 25, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants