Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support aarch64 Linux with page sizes >= 16k #5967

Closed
maximbaz opened this issue Sep 27, 2022 · 5 comments · Fixed by #6541
Closed

Support aarch64 Linux with page sizes >= 16k #5967

maximbaz opened this issue Sep 27, 2022 · 5 comments · Fixed by #6541
Assignees
@kdy1
Labels
C-bug
Milestone
v1.3.21

Comments

@maximbaz
Copy link

maximbaz commented Sep 27, 2022 •
edited
Loading

Describe the bug

swc uses jemalloc, which in turn uses the build system's page size as internal page size, and fails if the target's page size is larger [1]. Some Linux distributions patch jemalloc itself (e.g. [2]), but Github actions uses a "standard" jemalloc, which has this problem.

Instead of using a patched jemalloc, it is possible to configure jemalloc in runtime to use largest possible size, thus making it compatible with all aarch64 systems.

The solution is simple: add export JEMALLOC_SYS_WITH_LG_PAGE=16 before running cargo build for aarch64 system.

This is a very common case with rust projects, as you can imagine 🙂 Here's just one example [3].

Let me know if it makes sense!

1: jemalloc/jemalloc#467
2: https://github.com/archlinuxarm/PKGBUILDs/blob/master/extra/jemalloc/PKGBUILD
3: archlinuxarm/PKGBUILDs#1940

Input code

No response

Config

No response

Playground link

No response

Expected behavior

No crash on Linux with 16k page size.

Actual behavior

No response

Version

1.3.3

Additional context

No response
@maximbaz maximbaz added the C-bug label Sep 27, 2022
@kdy1
Copy link
Member

kdy1 commented Sep 28, 2022

@Brooooooklyn Thoughts?
Do you think this worths the effort?

This was referenced Oct 6, 2022
Closed
Merged
kodiakhq bot pushed a commit to vercel/next.js that referenced this issue Oct 12, 2022
@maximbaz
Support aarch64 Linux with page size >= 16k (#41229)
de8ae73 
Fixes #39311

Backport of upstream patch swc-project/swc#6075

More details are described in swc-project/swc#5967

I confirm that the swc plugin, when compiled with this patch, works on my Linux aarch64 machine.

This will work on all aarch64 Linux systems, the patch is configuring jemalloc to use largest possible page size, instead of taking the value from the machine where you are compiling it. 

And here's the compiled binary, if you want to give it a go!

[next-swc.linux-arm64-gnu.node.zip](https://github.com/vercel/next.js/files/9728262/next-swc.linux-arm64-gnu.node.zip)


Let me know if that makes sense! 🙂
@kdy1 kdy1 added this to the Planned milestone Oct 17, 2022
Kikobeats pushed a commit to Kikobeats/next.js that referenced this issue Oct 24, 2022
@maximbaz @Kikobeats
Support aarch64 Linux with page size >= 16k (vercel#41229)
33fbe7e 
Fixes vercel#39311

Backport of upstream patch swc-project/swc#6075

More details are described in swc-project/swc#5967

I confirm that the swc plugin, when compiled with this patch, works on my Linux aarch64 machine.

This will work on all aarch64 Linux systems, the patch is configuring jemalloc to use largest possible page size, instead of taking the value from the machine where you are compiling it. 

And here's the compiled binary, if you want to give it a go!

[next-swc.linux-arm64-gnu.node.zip](https://github.com/vercel/next.js/files/9728262/next-swc.linux-arm64-gnu.node.zip)


Let me know if that makes sense! 🙂
@mstephenson6
Copy link

Ran into this on Asahi (Arch) Linux on Apple M1 today trying to npm install @swc/core. In a fresh working directory with no node_modules, the install seems to finish successfully. But when installing @swc/core in a long list of package.json dependencies, the postinstall step fails with:

<jemalloc>: Unsupported system page size
memory allocation of 160 bytes failed

I traced this to binding.js, when it imports @swc/core-linux-arm64-gnu/swc.linux-arm64-gnu.node along the same lines as this issue discusses. The swc executable in that directory will run on my M1 Linux system, however, which I cannot explain.

@Brooooooklyn
Copy link
Member

@kdy1 I think we can use system allocator on Linux aarch64 GNU platform

@kdy1
Copy link
Member

kdy1 commented Nov 29, 2022

@Brooooooklyn Yeah it will fix the issue

@kdy1 kdy1 self-assigned this Nov 29, 2022
@kdy1 kdy1 mentioned this issue Nov 29, 2022
Merged
kdy1 added a commit that referenced this issue Nov 29, 2022
@kdy1
chore(bindings): Fix jemalloc on aarch64 (#6541)
d56b5a8 
**Related issue:**

 - Closes #5967.
@kdy1 kdy1 modified the milestones: Planned, v1.3.21 Nov 30, 2022
@swc-bot
Copy link
Collaborator

swc-bot commented Dec 30, 2022

This closed issue has been automatically locked because it had no new activity for a month. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.

@swc-project swc-project locked as resolved and limited conversation to collaborators Dec 30, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@kdy1 kdy1

Labels
C-bug
Milestone
v1.3.21
5 participants
@maximbaz @mstephenson6 @Brooooooklyn @kdy1 @swc-bot