Autoscaling for repository runner

[int128]

What we want

We use several repository runners in our organization and put them into each namespace, for example,

• Namespace: foo-repository-runner, RunnerDeployment: foo-repository-runner for repository foo
• Namespace: bar-repository-runner, RunnerDeployment: bar-repository-runner for repository bar

We'd like to autoscale bar-repository-runner. We use both GitHub-hosted and self-hosted runner in the repository. Is any autoscaling strategy available for the repository runner?

What I did

First I tried TotalNumberOfQueuedAndInProgressWorkflowRuns but it scaled out to too many pods, because it counts the workflows for both GitHub-hosted and self-hosted runners.

I found PercentageRunnersBusy strategy is available as well. I added the following resource:

apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: bar-repository-runner
  namespace: bar-repository-runner
spec:
  scaleTargetRef:
    name: bar-repository-runner
  minReplicas: 1
  maxReplicas: 60
  metrics:
    - type: PercentageRunnersBusy
      repositoryNames:
        - ORG_NAME/bar

but I got the following error:

2021-01-18T09:43:00.531Z	ERROR	controllers.HorizontalRunnerAutoscaler	Could not compute replicas	{"horizontalrunnerautoscaler": "bar-repository-runner/bar-repository-runner", "error": "organization and repository are both empty"}

PercentageRunnersBusy seems to find the organization runners (not repository runners):

https://github.com/summerwind/actions-runner-controller/blob/48923fec562126e2ffe575c9d7694d18c0a56f3d/controllers/autoscaling.go#L206-L207

I think PercentageRunnersBusy is not applicable for the repository runner, right?

[ghost]

Hi @int128 - thanks for bringing this up. We can add repo scoped runners to the PercentageRunnersBusy scheme. Will send out a PR shortly.

[int128]

We investigated the autoscaling latency of TotalNumberOfQueuedAndInProgressWorkflowRuns and PercentageRunnersBusy. We put many applications into monorepo and currently up to approximately 70 of jobs are run in parallel when we changed some code. Finally it turned out PercentageRunnersBusy is too slow for us.

We will move all workflows to self-hosted runners and try TotalNumberOfQueuedAndInProgressWorkflowRuns. Thank you for your help.