[PS-1197] Add pkg arm64 targets for windows, mac, and linux

[noahjahn]

Type of change

- [ ] Bug fix
- [ ] New feature development
- [ ] Tech debt (refactoring, code cleanup, dependency upgrades, etc)
- [x] Build/deploy pipeline (DevOps)
- [ ] Other

Objective

Fixes #5262

I wanted to install the Bitwarden CLI on my Raspberry Pi, so I added package scripts for arm64. I also added arm64 targets for mac and windows while I was at it.

Community post ref: https://community.bitwarden.com/t/arm-support-for-cli/42362

Code changes

• package.json: I simply added a new script to package the CLI with the arm64 architecture in each currently supported OS. I renamed the previous scripts that were already there and changed the dist file structure to include the architecture.

Before you submit

- [x] I have checked for **linting** errors (`npm run lint`) (required)
- [ ] I have added **unit tests** where it makes sense to do so (encouraged but not required)
- [x] This change requires a **documentation update** (notify the documentation team)
- [x] This change has particular **deployment requirements** (notify the DevOps team)

The way the CLI is deployed and the documentation for downloading the CLI will need to be updated. https://bitwarden.com/download/
https://bitwarden.com/help/cli/

The download link itself will need to have an added query parameter to download the specific CPU architecture. Example: https://vault.bitwarden.com/download/?app=cli&platform=linux&cpu=arm64

[MGibson1]

@bitwarden/dept-devops, this looks good from a build scripts perspective, but we'll likely need to update our release processes to actually use them.

I'm not familiar enough with our CLI release locations to know for sure.

@noahjahn are you hoping to make these available as downloadable packages or dev build options?

[mimartin12]

@MGibson1 agreed, at this moment, none of these artifacts would be published until we updated the workflows to pick them up.

[noahjahn]

@MGibson1 the goal for this PR is to make it easier for others looking to install the CLI on other platforms. So, rather than having to clone the repository and build themselves, they could be available as downloadable packages.

I've already used my forked repo to get a build of the CLI on my Raspberry Pi.

I figured this would require a change to how the artifacts are published. I'm happy to look at the current workflows to figure out what would be required to change there. I assume I would have to rely on your team for the actual future release of this to upload download links and documentation.

Thank you for reviewing and for the questions 😄

[joseph-flinn]

I've already used my forked repo to get a build of the CLI on my Raspberry Pi.

@noahjahn It seems like this change requires linux ARM architecture to build these new artifacts. Is my understanding correct?

[noahjahn]

@noahjahn It seems like this change requires linux ARM architecture to build these new artifacts. Is my understanding correct?

@joseph-flinn That would be one way to do it, but vercel/pkg allows generating executables for different architectures: https://github.com/vercel/pkg#targets

I tested this on Windows Subsystem for Linux device with x86_64 architecture after "configuring binfmt with QEMU." I just installed the packages mentioned apt install qemu binfmt-support qemu-user-static and was able to generate a Linux executable for arm64.

[joseph-flinn]

@noahjahn I've ran your changes through our CI pipelines. Could you take a look and see if you can get a successful CI pass? For security purposes, all CI run needs maintainer level approvals. For a faster feedback loop, I'd suggest enabling GitHub Actions on your forked repo, removing the last job that checks for failures in the build-cli.yaml workflow (since it requires a secret that you won't have).

[noahjahn]

@noahjahn I've ran your changes through our CI pipelines. Could you take a look and see if you can get a successful CI pass? For security purposes, all CI run needs maintainer level approvals. For a faster feedback loop, I'd suggest enabling GitHub Actions on your forked repo, removing the last job that checks for failures in the build-cli.yaml workflow (since it requires a secret that you won't have).

Yes, thanks for the info there. I'll be happy to test this out and will let you know what I find.

[yougotwill]

@noahjahn just want to say thank you for working on this! 🙏 Looking forward to a future pi release.

[yougotwill]

@joseph-flinn Any updates on this? 🙏

[noahjahn]

@joseph-flinn Any updates on this? 🙏

I have not finished the implementation yet. Turns out it was a little more involved with having to actually introduce a new build server that is ARM based.

I actually just put it on my radar to start looking at it again since I started this so long ago. I will plan to look at it this weekend and maybe have a solid solution for everyone to enjoy. 🙂

[cobalt2727]

If you need hardware, Oracle has a really generous free tier for their cloud VM service: https://blogs.oracle.com/developers/post/how-to-set-up-and-run-a-really-powerful-free-minecraft-server-in-the-cloud (just skip past the Minecraft stuff, of course - and it allows for more choices of OS than Oracle Linux)

[canine-afficionado]

Hi @noahjahn how are you going with this?

[noahjahn]

@noahjahn It seems like this change requires linux ARM architecture to build these new artifacts. Is my understanding correct?

@joseph-flinn That would be one way to do it, but vercel/pkg allows generating executables for different architectures: https://github.com/vercel/pkg#targets

I tested this on Windows Subsystem for Linux device with x86_64 architecture after "configuring binfmt with QEMU." I just installed the packages mentioned apt install qemu binfmt-support qemu-user-static and was able to generate a Linux executable for arm64.

@joseph-flinn I apologize, I definitely spoke too soon without enough research. That worked for Linux, but Windows and Mac builds for arm64 do require the builds to run on an ARM based architecture. This means that you would need some self-hosted GitHub action runners running on those two platforms.

I can look for some cheap hosting for Mac and Windows servers to setup self-hosted runners for testing, but would this be something Bitwarden would support if I go that route?

For now, I can take out the ARM builds for Mac and Windows and just add Linux ARM support without having to introduce any additional hosting costs from Bitwarden.

Let me know what your thoughts are here before I get too far.

[mannp]

Really looking forward to using Bitwarden cli for external secrets on my arm64 clusters, but it seems after nearly 1 year of the PR we don't have any arm64 builds.

Is it possible to split this out for cli only for a first pass, maybe? :)

Thanks in advance

[noahjahn]

Really looking forward to using Bitwarden cli for external secrets on my arm64 clusters, but it seems after nearly 1 year of the PR we don't have any arm64 builds.

Is it possible to split this out for cli only for a first pass, maybe? :)

Thanks in advance

No, this wouldn't be possible since we still need a dedicated ARM VM for building to be able to publish. I was able to build the app myself on my Raspberry Pi. If it's helpful, I can send you some steps on how to do this yourself?

[joseph-flinn]

@noahjahn I apologize for the extremely delayed response 😞

For now, I can take out the ARM builds for Mac and Windows and just add Linux ARM support without having to introduce any additional hosting costs from Bitwarden.

Let me know what your thoughts are here before I get too far.

At this current time, we cannot entertain taking on the maintenance cost of self-hosted Mac/Windows ARM runners. If we can achieve a Linux ARM version via emulation in GitHub's managed, I'd be happy to get that added into official Bitwarden support.

[noahjahn]

@noahjahn I apologize for the extremely delayed response 😞

For now, I can take out the ARM builds for Mac and Windows and just add Linux ARM support without having to introduce any additional hosting costs from Bitwarden.
Let me know what your thoughts are here before I get too far.

At this current time, we cannot entertain taking on the maintenance cost of self-hosted Mac/Windows ARM runners. If we can achieve a Linux ARM version via emulation in GitHub's managed, I'd be happy to get that added into official Bitwarden support.

No worries at all, let me look at this again with a fresh set of eyes and see if I can figure it out again

[theofficialgman]

hey all what is the holdup? I see that node/npm is used for building which works fine with cross compilation.
it is simply a matter of installing the cross compilers (on linux, get them from apt, since they aren't installed by default on linux github actions runners), setting the appropriate environment variables, and away you go

as an example see github desktop actions -> https://github.com/shiftkey/desktop/pull/897/files (we chose to build in docker for better dependency stability but that is not necessary from a cross compilation standpoint)

[noahjahn]

hey all what is the holdup? I see that node/npm is used for building which works fine with cross compilation. it is simply a matter of installing the cross compilers (on linux, get them from apt, since they aren't installed by default on linux github actions runners), setting the appropriate environment variables, and away you go

as an example see github desktop actions -> https://github.com/shiftkey/desktop/pull/897/files (we chose to build in docker for better dependency stability but that is not necessary from a cross compilation standpoint)

Cross compilation isn't the issue, I'm able to do that no problem. Running tests is where I keep getting blocked. I'm trying to emulate the execution of the CLI after it's built, I can't seem to get a working example just locally.

[theofficialgman]

Cross compilation isn't the issue, I'm able to do that no problem. Running tests is where I keep getting blocked. I'm trying to emulate the execution of the CLI after it's built, I can't seem to get a working example just locally.

only realistic option for now is to create an arm QEMU powered chroot environment (I personally use my own fork of https://github.com/pguyot/arm-runner-action to do so on my repos) and test the built binaries within there.

[mrdev023]

This issue concern also linux arm64 clients ? Or just CLI in arm64 ?

[heathcliff26]

Well one slightly sketchy way would be to run the tests inside a Dockerfile and use the builtin docker functionality for emulation. That way you can keep the same test setup for all arches.
The sketchy part is of course that you would be using docker built for a completely different purpose. But general concept would be:

FROM ubuntu:latest

RUN setup-test-enviroment.sh
COPY binaries /bitwarden-cli
RUN test.sh

And Workflows for multi-arch "container builds" are already well documented. You wouldn't even need the login part, as you don't need to push the result.

[mloiseleur]

Like all the others on this PR, I'm quite motivated to have a working binary on arm64.

So first, thanks @noahjahn for this PR. It helped me a lot to understand the build system of CLI.

Since it seems there is a consensus that a PR focused only on Linux arm64 can get merged, I managed to get all the steps working for this target and opened PR #7338.

[jomifepe]

Hey @noahjahn, thank you for these efforts, I look forward to having an arm64 binary released for macOS 🚀

GitHub now has an arm64 macOS runner, it was released in January, it's the macos14 one: https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available/

I really hope this can help speed up these features 🙏

[martonsz]

l, I can send you some steps on how to do this yourself?

@noahjahn I sent you an email asking you to please share the instructions for how to build the binary on raspberry 🙏

[martonsz]

l, I can send you some steps on how to do this yourself?

@noahjahn I sent you an email asking you to please share the instructions for how to build the binary on raspberry 🙏

@noahjahn I think that my email did not arrive... I tried with an address that I found in one of your commits.
How can I contact you? Keybase? 😄

[mfouesneau]

Is there any update on having arm support for the client?

[lostb1t]

its 2024, can we please have ARM builds?