allow self-hosted runner to be restricted to container builds only

[igagis]

Describe the enhancement
Right now the security recommendation for self-hosted runners is not to host them on machines where there is any sensitive data. Because third party code can be injected via pull-requests.

It would be nice if self-hosted runner could be configured in a way to accept only jobs which have container: attribute, i.e. only jobs running inside of a docker container. This would allow to host such runners on machines without being afraid for sensitive data. I.e. if third party pull request modifies the workflow by removing the container: attribute then the job will be failed by the self-hosted runner.

[j3parker]

Disclaimer: I'm not a GitHub employee.

This sounds good but unfortunately wouldn't work, at least without very big redesigns. Docker containers, at least they way GitHub Actions uses them, don't provide that level of isolation. Here's a dumb example:

on: push
jobs:
  job:
    runs-on: ubuntu-latest
    container: alpine
    steps:
      - run: poweroff -f || true # will fail
      - run: apk add docker
      - run: docker run --privileged alpine sh -c 'poweroff -f'
      - run: echo hello # won't run

The job runs inside a container. The first poweroff -f will fail with this error:

poweroff: Operation not permitted

but to make uses: work (which often launches containers) GitHub Actions mounts /var/run/docker.sock into the jobs container. This socket lets your docker client communicate with the outside host e.g. to launch containers. So any containers used in uses: run adjacent to your job container, not inside it. Maybe this sounds weird, but Docker unfortunately wasn't built for nesting (even if some of the technologies like cgroups and namespaces are).

A big consequence of being able to run containers on the host like this is that you're effectively unsandboxed. Here we run a container with --privlieged and can actually power off the machine (the echo hello will never run.)

But you can do whatever, really. You could mount the root volume and read any file from the host, etc.

So how do you get sandboxing with GitHub Actions? The usual boring way: run one job per VM (tear down the VM after the job is done -- #510 is needed to make this good as far as I know) and don't make the VM's more privileged than you need (e.g. if you're running in AWS, pay mind to what VPC they're running in and what their instance role is.)

TL;DR: job containers aren't a security feature.

[igagis]

Thanks for info. Well, I suspected that it is not that easy.
Anyway, the purpose of this issue is to allow self-hosted runner to be configured to be sandboxed within a docker container. No matter how much redesign it will require. If it needs more time, then let it hang here and wait for it's hour.

It is often not needed to do anything privileged during builds so having self-hosted runners configured to be sandboxed would be very helpful, as it will significantly simplify the setup, i.e. no need for VMs.

[j3parker]

Yeah, just for a sense of what I mean by redesign... having docker-based steps where those containers run on the same host as the job container itself... you can't really avoid this (because of how Docker works currently)... so I wouldn't hold my breath for GitHub Actions turning containers into a security feature.

Again, though: I'm not a GitHub employee :)

[ghost]

@j3parker I tried to use rootless docker to avoid your poweroff scenario from above, but, alas, this won't work either because of #827

[j3parker]

Nice! That would definitely solve some problems.

I can imagine people still want to run a persistent runner(s) on a host and keep jobs isolated from each-other like this issue talks about. Rootless docker might help with that (e.g. maybe running per-repo runners as different users on your host + rootless docker could satisfy someones concerns) but might not. Restricting jobs to be container-based even with rootless docker wouldn't help though because you could still break out of the container.

Perhaps a significant change which would allow for sandboxing would be to use something like Firecracker to run either ephemeral runners or jobs themselves (i.e. just run them in a VM). This is how things like AWS Lambda isolate containers safely. The downside is that you need virtualization, which e.g. rules out a lot of scenarios for people that self-host runners (e.g. on AWS you'd need a bare-metal host).

If someone has a setup where firecracker would be appropriate they could of course just run the runner itself in Firecracker without waiting for changes from GitHub. To make a setup like that perfect for all use cases I think #510 needs to be fixed, though.

[bhack]

Do we have enough isolation running It in a Kubernetes pod?

https://medium.com/geekculture/github-actions-self-hosted-runner-on-kubernetes-55d077520a31

[JorgeSintes]

It's been a while since this thread was active, but it still comes up when searching for solutions on this topic. As of October 2024, GitHub now supports this by setting ACTIONS_RUNNER_REQUIRE_JOB_CONTAINER=true in the .env file of your runner. I've tested this on a self-hosted runner (version 2.319.1) on Linux x64, and it works.

This is documented, albeit briefly, on this page. However, it's worth noting GitHub's note on the page:

Note: This feature is currently in beta and is subject to change.

So, proceed with caution.