Package Plugin in Docker Container

[marquesj2-ppb]

Describe the bug

We have a cluster that supports both ARM64 and AMD64 Nodes, right now to properly install the plugin we need to pass in the correct arch in the URL.
This means we need to add nodeSelectors for the correct arch or implement an init container that figures out the arch and downloads the right version and copies it to a volume.

If we had container packaged for each arch we would just need to copy it over, something like what contour plugin is doing. https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-contour?tab=readme-ov-file#install-rollouts-using-helm

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

[kostis-codefresh]

Hello

Just to understand your requirement.

The gateway api plugin already publishes binaries for both arm and x86

I looked at the contour plugin and I don't see any binary at their releases https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-contour/releases or packages

Are you asking for a single container that contains both architectures? Or two containers (one for each architecture) ?

[mgs255]

I ran into what I think the reporter is referring to a while ago and even went as far as trying a simple fix:

argoproj/argo-rollouts#3279

The problem is that if you run clusters on different architecture types you do not know which architecture the pod is running on. We need to be able to provide different builds of each plugin that are then loadable at runtime. The design I proposed falls short as I discovered when I tried to add documentation of the feature. The approach also needs to consider providing the SHA hash for each of the different architectures as well. Sorry for not completing that but I decided to park my exploration into this.

[kostis-codefresh]

Aren't Kubernetes taints/tolerations built for exactly this scenario?

[mgs255]

Yes, but, let's say you have two different node types, linux/amd64 and linux/arm64 and allow the argo (rollout) pods to run on either architecture. This is exactly how we run our workloads - everything runs on spot instances, and the argo services can run on both architectures.

We still have the issue that we don't know until run-time, the architecture of the rollout plugin that needs to be loaded. Hence the approach where we try to expand the template including the OS and ARCH env variables. Does this make sense?

[kostis-codefresh]

I see. So what is the expected solution here? Something on the Argo Rollouts side? Or the plugin side? Or simply tell people that this scenario is not supported (spot instances + different arch nodes) ?

[mgs255]

So my solution involved generating the correct architecture specific path to the plugin at runtime based on a template. See: https://github.com/argoproj/argo-rollouts/pull/3280/files

Such a capability would be required even if an OSI container was used to package different architecture binaries into a single bundle. The running process would need to select the correct version depending on the current architecture, right?

[marquesj2-ppb]

Was asking to have containers with the plugin binary built for each arch.

That way the install mechanism would be the same regardless of where it's running.
`cp /path/gateaway-api-plugin /plugins.

If there was an easy way to add support in Argo Rollouts, supporting go template or variable replacement based on arch.