Are the integration tests run anywhere?

[TBBle]

I just discovered, while poking at #901, that the AppVeyor CI script appears to just compile the integration tests, but does not run them.

Are they being run somewhere I can't see (or have overlooked)? Would it be useful to get them running on AppVeyor?

I was looking into using the AppVeyor Job Workflows to have the compile done once (as it is now), then then run the integration tests on both the VS2017 (Windows Server 2016) and VS2019 (Windows Server 2019) images, but I stopped because I couldn't see an easy way to pass the artifacts generated by the first job through to the fan-out jobs to run.

I suspect it's possible with sufficient script hackery, but I didn't want to implement that if it was going to end up being uninteresting, e.g., if I was replicating something that already existed somewhere.

[katiewasnothere]

As part of our internal package process, we do run the cri-containerd test suite, but that's it. Unfortunately, the tests in test/functional are pretty out of date and still rely on docker (for images). I've been working on updating the test/functional tests, (see related containerd/containerd#4848), but haven't finished up there. As for the test/runhcs tests, since we no longer use runhcs, we don't run these. Lastly, I imagine the tests under test/containerd-shim-runhcs-v1 are also out of date, but haven't looked recently.

The main reason we don't run them via appveyor is that LCOW tests require an LCOW kernel/initrd to run.

[TBBle]

Thank you, that makes sense. I got what I needed for my own functional test (worked on RS5), and will wait for your efforts on test/functional to bear fruit before worrying about it further. I think the platform where I've seen the problem (20H2) isn't available on AppVeyor anyway.

I am curious, if runhcs is being left untested and bit-rotting, have you considered deleting it and its tests?

[katiewasnothere]

@TBBle

I am curious, if runhcs is being left untested and bit-rotting, have you considered deleting it and its tests?

Yep! It's on our backlog, but we haven't looked at it in a while.