Not using the ODR image for the k8s static runner

[izaaklauer]

What does this fix?

Currently, if you run waypoint runner install -platform=kubernetes ... but you don't specify a specific -k8s-runner-image, you get the ODR image for the static runner, which appears to be incapable of launching ODRs.

Reverts this change: https://github.com/hashicorp/waypoint/pull/3800/files#diff-33ed847d2fd425bf13e47f27760372eccaa88ed101ae6f05d13780095f486ab1L263

Only the on-demand runner needs its own image, because it needs kaniko for builds. If you use the ODR image for the static runner, you get this error when attempting to launch tasks (in the static runner logs)

2022-09-15T22:23:54.331Z [INFO]  waypoint.runner.agent.runner: launching plugin: job_id=01GD1M0P0N0PP6K9Q4FSFZ58PC job_op=*gen.Job_StartTask type=TaskLauncher path=/kaniko/waypoint args=["/kaniko/waypoint", "plugin", "kubernetes"]
2022-09-15T22:23:54.413Z [ERROR] waypoint.runner.agent.runner.waypoint: plugin init error: job_id=01GD1M0P0N0PP6K9Q4FSFZ58PC job_op=*gen.Job_StartTask @module=plugin error="open /kaniko/tmp/plugin1917675357: permission denied" timestamp=2022-09-15T22:23:54.413Z
2022-09-15T22:23:54.418Z [ERROR] waypoint.runner.agent.runner: error creating plugin client: job_id=01GD1M0P0N0PP6K9Q4FSFZ58PC job_op=*gen.Job_StartTask
  err=
  | Unrecognized remote plugin message: 
  | 
  | This usually means that the plugin is either invalid or simply
  | needs to be recompiled to support the latest protocol.

This is what it looks like on the client side during a waypoint build:

$ waypoint build

» Building backend...
! Unrecognized remote plugin message: 
  
  This usually means that the plugin is either invalid or simply
  needs to be recompiled to support the latest protocol.

I haven't dug into why the kaniko image can't launch tasks - it seems like it should be able to - but going back to using the server image for the static runner fixes the problem.

Workaround

You can work around this by specifying a non-odr runner image during waypoint runner install. I.e.:

waypoint runner install -platform=kubernetes -server-addr=api.hashicorp.cloud:443 -k8s-runner-image=hashicorp/waypoint:latest

[evanphx]

Can we check that the runner profile that is created DOES use the odr image though?

[izaaklauer]

Getting this to use the ODR image in the runner profile turned out to be more complex! Fixed now though.

Tested:

• K8s runner install with all permutations of the -odr-image and -k8s-runner-image flag
• Docker install with no flags
• ECS install with no flags

And this works as you'd expect.

[izaaklauer]

This turned out to be unused.