Look into adding support for GPU runners on CI

[cosmicBboy]

For #969, we need to add GPU-support for CI, since there's no way to be confident that pandera will run correctly on cudf without testing it. Relying on local testing won't be sustainable, so we need to do research on existing solutions like https://cirun.io/ .

[beckernick]

Hi! Came across this issue due to the cuDF mention. It would be wonderful for cuDF users to be able to use pandera.

Happy to try to help answer any questions that might come up!

[cosmicBboy]

@beckernick thanks! do you know how the cudf projects does unit tests on CI? I personally don't have an nvidia-enabled machine so can't test anything locally. Would love to support cudf but need CI infra in place to make sure it actually works with pandera... any tips would be much appreciated.

[beckernick]

RAPIDS projects like cuDF use a custom CI setup due to the significant GPU requirements that would likely be overkill for other projects.

At the moment, my understanding is that there are at least the following ways to do CI on GPUs with GH Actions (someone please feel free to correct me if I'm missing something):

• Manually create and use a self-hosted runner, which can point to a local GPU or a cloud GPU
• Use a tool like https://cirun.io/ that streamlines the above (and may have a free tier for open-source projects)

One thing that we've seen folks do when integrating GPU support/functionality into existing projects is to run the GPU-aware CI tests on an ad-hoc basis controlled by a specific phrase used in PR comments (rather than per commit or every PR). This can be done via the issue comment event workflow trigger. If the likelihood of a given PR's interaction with GPU-relevant code is very low, this reduces resource use.

We've seen projects use cirun to streamline this with excellent results. Perhaps @aktech (cirun creator/maintainer) has some additional insight.

(cc @quasiben @shwina)

[aktech]

Hey @beckernick thanks for considering cirun.io, yes you can surely use it for the above mentioned purposes and I am more than happy to help make it happen and yes it's free for open source without any restrictions. Feel free to let me know if you have any questions.