multi-runner: dead letter queue fifo configuration missing

[mvanbaak]

If you set fifo = true on a runner, it will create an SQS fifo queue. The dead letter queue that is created if you set the redrive_build_queue setting is never created as an fifo queue.

This results in an error during terraform apply

At the moment it's not possible to use fifo and a dead letter queue.

[mvanbaak]

Example that goes wrong. If I set the fifo=false it works.

locals {
  aws_region = "eu-central-1"
  vpc_id = ""
  subnet_ids = [
  ]
}

resource "random_id" "random" {
  byte_length = 20
}

module "github-runners" {
  source  = "philips-labs/github-runner/aws//modules/multi-runner"
  version = "2.0.0-next.1"

  # Setup acts on webhook events from a github application
  github_app = {
    key_base64     = var.github_app_key_base64
    id             = var.github_app_id
    webhook_secret = random_id.random.hex
  }

  aws_region                        = local.aws_region
  vpc_id                            = local.vpc_id
  subnet_ids                        = local.subnet_ids
  runners_scale_up_lambda_timeout   = 60
  runners_scale_down_lambda_timeout = 60
  prefix                            = local.environment
  tags = {
    Environment = local.environment
  }
  webhook_lambda_zip                = "lambdas/webhook-2.0.0-next.1.zip"
  runner_binaries_syncer_lambda_zip = "lambdas/runner-binaries-syncer-2.0.0-next.1.zip"
  runners_lambda_zip                = "lambdas/runners-2.0.0-next.1.zip"

  # the actual runners using the information from above
  multi_runner_config = {
    "linux-arm64" = {
      matcherConfig : {
        labelMatchers     = ["self-hosted", "linux", "arm64"]
        exactMatch        = true
      }
      fifo                = true
      delay_webhook_event = 0
      redrive_build_queue = {
        enabled         = true
        maxReceiveCount = 10
      }
      enable_organization_runners       = true
      runner_config = {
        runner_os                       = "linux"
        runner_architecture             = "arm64"
        enable_ssm_on_runners           = true
        create_service_linked_role_spot = true
        instance_types                  = ["t4g.large", "c6g.large"]
        runners_maximum_count           = 5
        scale_down_schedule_expression  = "cron(* * * * ? *)"
      }
    },
    "linux-amd64" = {
      matcherConfig : {
        labelMatchers = ["self-hosted", "linux", "amd64"]
        exactMatch    = true
      }
      fifo                = true
      delay_webhook_event = 0
      redrive_build_queue = {
        enabled         = true
        maxReceiveCount = 10
      }
      enable_organization_runners       = true
      runner_config = {
        runner_os                       = "linux"
        runner_architecture             = "x64"
        enable_ssm_on_runners           = true
        create_service_linked_role_spot = true
        instance_types                  = ["t3a.large", "c6a.large"]
        runners_maximum_count           = 5
        scale_down_schedule_expression  = "cron(* * * * ? *)"
      }
    },
  }
}

[npalm]

@mvanbaak Thanks for taking the time to test the new multi-runner! @GuptaNavdeep1983 can you have a look at this issue?

[mvanbaak]

Sure thing!

We are in the process of moving from gitlab to github, and we need self-hosted runners.
This multi-runner module is exactly what we need. So far it's working out very good, for now this small issue.
I would have contributed a fix, but this is my very first time using terraform so it's already a lot of new things for the moment.

[mvanbaak]

This has been fixed with #2641

Tested in our setup, see screenshot.

Our setup:

module "github-runners" {
  source  = "philips-labs/github-runner/aws//modules/multi-runner"
  version = "2.0.0-next.2"

and setting fifo = true on the runner configs.

Thanks!