ARM support

[tsrats]

Feature Request

I would like to request support for aarch64/arm support.

Describe alternatives you've considered

Currently, using TCE on x86_64, as well as vanilla kubernetes on both x86_64 and aarch64.

Additional context

Thanks for an already great release!

[github-actions]

Hey @tsrats! Thanks for opening your first issue. We appreciate your contribution and welcome you to our community! We are glad to have you here and to have your input on Tanzu Community Edition.

[lamw]

FYI - This project from @moondev might also be of interests/useful as part of ARM enablement for TCE

[davidvonthenen]

Yep, great ask. I have been working with @joefitzgerald about this. He actually did all the leg work. He did the leg work for getting us close to supporting arm64 for darwin and linux. The current blocker right now is we need to update certain packages/images (ie like the kind image) and also get Tanzu Framework to support it.

If you do a build of TCE (make release), you should see the darwin arm64 binaries being generated... however, without the kind image and TF support, you won't be able to deploy management or standalone clusters at the moment.

[moondev]

In addition to @dvonthenen 's items for bootstrapping aka tanzu management-cluster create from arm64, there is also the consideration of building machine images such as the OVA for use with ESXi arm edition. The machine image here is a POC ad-hoc build for upstream CAPV. The image is designed to be used with the multi-arch version of cluster-api-provider-vsphere-manager https://hub.docker.com/r/chadmoon/cluster-api-provider-vsphere-manager/tags - ideally arm64 build targets for https://github.com/kubernetes-sigs/image-builder can be upstreamed as well. Identifying all the needed TCE images for a bootstrapped cluster and potentially baking them in the image would be another possibility.

In any event I like @dvonthenen's suggestion to start with the kind image for local capd provider which would pave the way for other providers.

[andreiw]

Tanzu Arm support is critical for a comprehensive cloud and edge story that includes 64-bit Arm support.

[stmcginnis]

there is also the consideration of building machine images such as the OVA for use with ESXi arm edition

There are a few scenarios here, and we should probably make it clear which ones we want to support.

At a minimum, I see it useful having a tanzu CLI that can run on ARM machines. Since we need to use a bootstrap cluster to deploy to the chosen platform, we would need to have a kind or custom image that we can spin up locally in Docker to get things started. But after that point, it could just be normal x86 images that get deployed to vSphere, AWS, Azure, etc.

Then there's the case for actually deploying a management, standalone, or workload cluster to an ARM based machine. Docker deployments would obviously imply we need ARM images. But another scenario would be, for instance, running the tanzu CLI on an x86 machine to deploy to an ARM edition of ESXi.

So the client side of this is separate from the deployment provider side. You could have any combination of:

Client	Target
x86	x86
ARM	x86
x86	ARM
ARM	ARM

We wouldn't necessarily need to support all scenarios right away to start adding ARM support, but it might be useful to make it clear which scenarios we actually want to handle to prioritize the work.

[lamw]

@stmcginnis Makes sense

Perhaps, we should start off with simply enabling tanzu CLI to run on Arm, which will require an aarch64 binary + kind support on Arm for bootstrap cluster as Phase 1. This will eventually be needed to achieve Target: ARM which I believe is the desired outcome, at least that's what personally looking forward to

[stmcginnis]

Agree, that makes sense as a first step.

[joshrosso]

I've also opened a related issue in tanzu-framework since that's where most of this plumbing work would occur:

• Provide support for deploying on arm (linux) tanzu-framework#838

Regarding just ARM64 CLI binaries for linux, totally open to this and quite trivial. But if someone would like to suggest it, would you open a new issue? Thanks!

[tce-automation]

Please ignore this test...

cc: @vmware-tanzu/tce-owners

[dashaun]

I'm not convinced this use case is harder or easier to achieve.

I would like to add ARM64 node(s) to an existing x86 cluster.

[stmcginnis]

I would like to add ARM64 node(s) to an existing x86 cluster.

I have heard some talk of mixed clusters, but mostly for mixing Linux and Windows nodes. I'm not sure how difficult that would be for ARM, but I think management of workloads would still come down to label matching. So possible.

[tsrats]

I have been running a vanilla Kubernetes cluster with x86 and ARM since the release of the ESXi on ARM fling. With so many container images cross building for both platforms, it is not too difficult to manage. Anything that needs to run on one or the other just needs a label on the deployment.

[dashaun]

I also have existing mixed-arch clusters running: amd64 + ARM64 + armhf.