Added ARM64 build support

[ava1ar]

Changes required to build master on arm64 machine (Samsung Chromebook+ with ArchLinux running inside crostini container was used). Builds and runs fine with the proposed changes.

[msftclas]

All CLA requirements met.

[jkol]

This is missing an arm64 entry in the object inside of getDebPackageArch in build\gulpfile.vscode.linux.js

[manfredbrandl]

Don‘t we need to update to Electron 2.0.x #52782 with this?

[ava1ar]

@manfredbrandl it was updated already when the PR was submitted, but got reverted later...

[manfredbrandl]

In #54873 Electron was updated to 2.0.5 .

[joaomoreno]

Guys, no need to keep this PR up-to-date. Supporting ARM64 is more than just accepting a PR. It requires us to update our release and test channels and commit to supporting it in the long run, which is not a development decision. The work here is awesome in which it contains the engineering effort to enable it, but there's a lot more to accepting such a change.

[ava1ar]

@joaomoreno I am making arm64 builds of vscode for myself, so just wanted to share the required changes with the community, so may be somebody else use it and start getting own builds with arm64. I perfectly understand, that new architecture unboarding requires lot of things to be done, so feel free to do whatever you want with this PR.

[magic-k]

Allowing an easy way to compile for arm and making arm builds official is not the same thing. I can build for arm64 already out of the box with the current master. This PR simply adds the components for arm64 to build packages. BTW the same lines for armhf are already there.
@joaomoreno i guess you confuse this PR with issues that want official arm builds.

[glebm]

@ava1ar The produced deb package is invalid. Adding an architecture mapping fixes it:

sed -i 's|arm: .armhf.|&, arm64: "arm64"|' build/gulpfile.vscode.linux.js

With the mapping added, the deb package seems to work just fine!

[glebm]

@roblourens Looks like ripgrep 0.9.0 release is missing the arm64 zip https://github.com/roblourens/ripgrep/releases/tag/0.9.0

[roblourens]

arm64 should be there now.

[glebm]

@roblourens Thanks!

[tingxueronghua]

So, is this problem solved?

[magic-k]

No. Only ripgrep for arm64. Microsoft has currently not the ressources to maintain arm builds. And obviously it's not very high on theit priority list. We have to wait fo arm64 windows notebooks :-)
I track vscode on my own git repo with the arm64 patches. So i can regulary build it on my own.

[headmelted]

I'd be really interested in finding out which doors need to be knocked on to get this supported officially.

I've been working to keep my cross-compiled ARM and ARM64 builds up-to-date, but as the build process deviates it gets harder to do so.

There's basically two routes to take at the moment:

1. Building natively on an ARM/ARM64 device (as in this thread) which I think should be supported in any case.
2. Cross-compiling from x64 to ARM/ARM64 so that it can be integrated into the existing CI pipelines (which seem to have moved to VSTS now?).

The work to support both of these avenues is now in place with the community so it would really just be a case of having Microsoft take this up.

Obviously committing to support any ARM variation officially is a big deal, but is it really that significant to just add a few lines of code here and there to support building with the CI pipeline and not releasing officially, if such changes don't impact on the official builds? (Some changes have been merged already, but not enough to build a fully native VS Code without a few further edits).

Speaking personally I have folks waiting on the ARM builds I've worked on up until now to use them in an educational setting (Chromebooks) and it's hard to communicate that it's getting more difficult to maintain them over time as the codebase is obviously growing, and due to the lack of being able to merge the compile-time support in.

Any help is much appreciated.

[joaomoreno]

Yeah, you are right, we can just merge this without actually releasing official builds. Would that be enough for you guys? Why wouldn't you simply have a fork of this repository with these changes on top, and build from there?

[headmelted]

Thanks! This would be a massive help, I'll get to work on a pull request. I have it in place now (more or less), it just needs cleaned up a little.

…

On Mon, Sep 10, 2018, 14:08 João Moreno ***@***.***> wrote: Yeah, you are right, we can just merge this in an not officially release builds. Would that be enough for you guys? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#52119 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKYVoPM5VBzZXNT9FvwKY9b4VLzhuuvvks5uZmRRgaJpZM4Uqu7z> .

[joaomoreno]

Wait. Please do not start any work. I was talking about this extremely minimal PR. What do you have in mind instead?

[glebm]

This PR has one small issue, #52119 (comment)

[ava1ar]

@glebm @joaomoreno Let me fix the mentioned issue and resolve the merge conflict, give me an hour or so.

[headmelted]

@joaomoreno sorry, replied to the email on my phone and got threads crossed.

There are a few other additions here and there needed for ARM64 that I've identified. In fairness, I'm exclusively cross-compiling from x64 so it may be the case that they only impact that scenario.

Basically, this surrounds issues about what "arm" means to the packaging system on the OS - generally it's taken as an alias for "armhf" or "armv7l", neither of which would work on an ARMV8/ARM64 device verbatim. I wouldn't expect this to affect someone compiling directly on ARM64 hardware, as this would be picked up from the OS in that instance.

I previously maintained a fork, the problem with that was that these are not significant changes (effectively just cleaning up a few "arm" references), so it was a bit of a waste for something that really belongs in the core codebase.

In any case, when I've ironed out the details I'll submit a PR that should clarify - as said it's very minimal changes so that you folks can decide if you want to include it or not.

Apologies if this hijacked the thread, just wanted to clarify.

[ava1ar]

@headmelted sounds like extra things you mention are for cross-compilation only, since I am able to successfully build the arm64 version of the vscode using only changes from this PR. I can include required changes to this PR if there are no objections from @joaomoreno and they are small. Otherwise let's split them into different PR to merge this first and cross-compilation related stuff later.

[headmelted]

Absolutely, sounds like the right way to go.

…

On Mon, Sep 10, 2018, 16:42 Aliaksandr Stelmachonak < ***@***.***> wrote: @headmelted <https://github.com/headmelted> sounds like extra things you mention are for cross-compilation only, since I am able to successfully build the arm64 version of the vscode using only changes from this PR. I can include required changes to this PR if there are no objections from @joaomoreno <https://github.com/joaomoreno> and they are small. Otherwise let's split them into different PR to merge this first and cross-compilation related stuff later. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#52119 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKYVoNPw-3CSBfhpRhxhkYt0gpMjFy_-ks5uZohrgaJpZM4Uqu7z> .

[joaomoreno]

@ava1ar Looks good, I'll check it asap.

ARM64 cross-compiling

@headmelted Sorry for being blunt, but since we do not produce any output in our continuous build, what use would be this for now?

[headmelted]

@joaomoreno It would really be for the benefit of anyone trying to compile the sources for their own use.

As an example, if I pull the source code on an ARM64 device I can build it (the @ava1ar changes in this PR help with that, and should definitely be merged). If I pull the source code on my laptop to cross-compile to ARM64, the build will fail (as is).

There are at least three reasons I can think of as to why you would want to support this.

1. If your ARM64 device is at the low-end it could take an age to compile (I can confirm this).
2. You may want to produce pre-compiled Linux packages for non-technical ARM users (I've been doing this for about a year and a half - specifically for classroom use).
3. Further to the point above, you may want to set up a continuous process (i.e. continuous delivery). Most providers only offer x86 hardware for this, which means you would need to either cross-compile or use QEMU (speaking from experience, this is even worse than compiling directly on low-end hardware as the emulation is extremely slow - and has timed out on any CI provider I've tried to use, preventing updateable releases).

I don't think this is the place to have this discussion, not least as it's now getting away from @ava1ar 's valid PR here, which was never my intent. If someone on the team is happy to discuss this seperately I'd love to work with you guys to get this working once and for all - but if not I can keep hacking around the scripts at my side to make it work for my own builds as I've been doing up until now.

I'd just add that I know of a lot of people who are relying on the builds I've been doing up to now in exciting, non-traditional scenarios (i.e. ARM and Chromebook devices in the classroom) that would really benefit from having this in place, even if it isn't going to be supported officially (which I understand is a lot more complex bureaucratically than just having builds working).

Anyway, as said this is not the place to discuss, and I didn't mean to hijack. @ava1ar , my apologies. Hope you guys can put me in touch with someone who can help me get this in place. 👍

[joaomoreno]

@headmelted No worries, it was a great explanation, thanks!

Hope you guys can put me in touch with someone who can help me get this in place. 👍

I've brought it up for discussion. 👍

[joaomoreno]

LGTM

[joaomoreno]

@jrieken Asking you to review the version bump of electron-mksnapshot.

[jrieken]

👍

[joaomoreno]

In it goes! Thanks! 🍻

[ava1ar]

Thanks a lot for merging this! ARM64 users just became a bit happier :)

[headmelted]

Looking at this again I think if I adapt my patches to fit in with this PR (I was using different arm aliases - doesn't really matter) I can get the needed additions down to two lines of JS in the same file. Will add when this is merged in.

[headmelted]

PR raised as #58436.

I can re-work my own scripts to fit in with the scheme used here (as it should be). This should be all that's required to allow me (or anyone else) to cross-compile for ARM and ARM64.

[joaomoreno]

@headmelted That's great news, merged it. We can then close #24943 ?

[headmelted]

Sure. 🙂 It's two different things really - that PR was more involved in that it would actually get the CI/CD builds of Code cross-compiling to ARM directly. This one is a much lower-impact addition that will allow someone to cross-compile to ARM using their own build scripts. I think given the difficulties you mentioned in officially supporting ARM this is the best way forward - it'll support cross-compiling now if someone wants to do that. 🙂 Thanks for the quick turnaround by the way!

…

On Tue, Sep 11, 2018, 16:44 João Moreno ***@***.***> wrote: @headmelted <https://github.com/headmelted> That's great news. We can then close #24943 <#24943> ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#52119 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKYVoG_-A4TKO4vhy8bRSS0WkIL4bSNuks5uZ9p4gaJpZM4Uqu7z> .

[roblourens]

I updated yarn.lock for electron-mksnapshot

[Tyriar]

LGTM 👍

[joaomoreno]

I updated yarn.lock for electron-mksnapshot

D'oh

[DVLP]

How to build for Dex?