Test building arm bindings #53

dapplion · 2022-02-09T06:24:08Z

WIP attempts at building arm bindings.
Closes #51

Ideas

Github actions does not supports runners in arm actions/runner-images#2552 (comment)

So we must roll our own runner somehow. The idea I got is to run buildx with arm and then somehow extract the resulting bindings from it.

Current status

Stuck with a failed build, see below

dapplion · 2022-02-09T06:24:35Z

The build fails without any indication of what's wrong

#8 [4/6] RUN PYTHON="/usr/bin/python" node dist/scripts/install.js
#8 1.611 Retrieving BLST native bindings...
#8 4.059 Error: Error importing BLST native bindings: 404 Not Found
#8 4.059     at Object.downloadReleaseAsset (/dist/scripts/downloadReleaseAsset.js:19:15)
#8 4.059     at processTicksAndRejections (node:internal/process/task_queues:96:5)
#8 4.059     at async Object.downloadBindings (/dist/scripts/downloadBindings.js:9:5)
#8 4.059     at async install (/dist/scripts/install.js:33:9)
#8 4.062 Building BLST native bindings from source...
#8 4.065 BLST_WRAP_CPP_PREBUILD /prebuild/blst_wrap.cpp exists, SWIG will be skipped
#8 4.092 Launching node-gyp {
#8 4.092   nodeJsExec: '/usr/local/bin/node',
#8 4.092   nodeGypExec: '/node_modules/node-gyp/bin/node-gyp.js',
#8 4.092   cwd: '/blst/bindings/node.js',
#8 4.092   BLST_WRAP_CPP_PREBUILD: '/prebuild/blst_wrap.cpp'
#8 4.092 }
#8 6.506 gyp info it worked if it ends with ok
#8 6.531 gyp info using node-gyp@7.1.2
#8 6.541 gyp info using node@17.4.0 | linux | arm64
#8 8.036 gyp info find Python using Python version 3.9.7 found at "/usr/bin/python"
#8 9.184 gyp http GET https://nodejs.org/download/release/v17.4.0/node-v17.4.0-headers.tar.gz
#8 11.14 gyp http 200 https://nodejs.org/download/release/v17.4.0/node-v17.4.0-headers.tar.gz
#8 24.85 gyp http GET https://nodejs.org/download/release/v17.4.0/SHASUMS256.txt
#8 25.66 gyp http 200 https://nodejs.org/download/release/v17.4.0/SHASUMS256.txt
#8 25.80 (node:20) [DEP0150] DeprecationWarning: Setting process.config is deprecated. In the future the property will be read-only.
#8 25.80 (Use `node --trace-deprecation ...` to show where the warning was created)
#8 25.84 gyp info spawn /usr/bin/python
#8 25.87 gyp info spawn args [
#8 25.87 gyp info spawn args   '/node_modules/node-gyp/gyp/gyp_main.py',
#8 25.87 gyp info spawn args   'binding.gyp',
#8 25.87 gyp info spawn args   '-f',
#8 25.87 gyp info spawn args   'make',
#8 25.87 gyp info spawn args   '-I',
#8 25.87 gyp info spawn args   '/blst/bindings/node.js/build/config.gypi',
#8 25.87 gyp info spawn args   '-I',
#8 25.87 gyp info spawn args   '/node_modules/node-gyp/addon.gypi',
#8 25.87 gyp info spawn args   '-I',
#8 25.87 gyp info spawn args   '/root/.cache/node-gyp/17.4.0/include/node/common.gypi',
#8 25.87 gyp info spawn args   '-Dlibrary=shared_library',
#8 25.87 gyp info spawn args   '-Dvisibility=default',
#8 25.87 gyp info spawn args   '-Dnode_root_dir=/root/.cache/node-gyp/17.4.0',
#8 25.87 gyp info spawn args   '-Dnode_gyp_dir=/node_modules/node-gyp',
#8 25.87 gyp info spawn args   '-Dnode_lib_file=/root/.cache/node-gyp/17.4.0/<(target_arch)/node.lib',
#8 25.87 gyp info spawn args   '-Dmodule_root_dir=/blst/bindings/node.js',
#8 25.87 gyp info spawn args   '-Dnode_engine=v8',
#8 25.87 gyp info spawn args   '--depth=.',
#8 25.87 gyp info spawn args   '--no-parallel',
#8 25.87 gyp info spawn args   '--generator-output',
#8 25.87 gyp info spawn args   'build',
#8 25.87 gyp info spawn args   '-Goutput_dir=.'
#8 25.87 gyp info spawn args ]
#8 27.97 gyp info spawn make
#8 27.97 gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
#8 27.98 gyp ERR! build error 
#8 27.99 gyp ERR! stack Error: `make` failed with exit code: 1
#8 27.99 gyp ERR! stack     at ChildProcess.onExit (/node_modules/node-gyp/lib/build.js:194:23)
#8 27.99 gyp ERR! stack     at ChildProcess.emit (node:events:520:28)
#8 27.99 gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
#8 28.01 gyp ERR! System Linux 5.13.0-27-generic
#8 28.01 gyp ERR! command "/usr/local/bin/node" "/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
#8 28.01 gyp ERR! cwd /blst/bindings/node.js
#8 28.01 gyp ERR! node -v v17.4.0
#8 28.01 gyp ERR! node-gyp -v v7.1.2
#8 28.01 gyp ERR! not ok 
#8 28.05 Error: Error building BLST native bindings: Command failed: /usr/local/bin/node /node_modules/node-gyp/bin/node-gyp.js rebuild
#8 28.05 gyp info it worked if it ends with ok
#8 28.05 gyp info using node-gyp@7.1.2
#8 28.05 gyp info using node@17.4.0 | linux | arm64
#8 28.05 gyp info find Python using Python version 3.9.7 found at "/usr/bin/python"
#8 28.05 gyp http GET https://nodejs.org/download/release/v17.4.0/node-v17.4.0-headers.tar.gz
#8 28.05 gyp http 200 https://nodejs.org/download/release/v17.4.0/node-v17.4.0-headers.tar.gz
#8 28.05 gyp http GET https://nodejs.org/download/release/v17.4.0/SHASUMS256.txt
#8 28.05 gyp http 200 https://nodejs.org/download/release/v17.4.0/SHASUMS256.txt
#8 28.05 (node:20) [DEP0150] DeprecationWarning: Setting process.config is deprecated. In the future the property will be read-only.
#8 28.05 (Use `node --trace-deprecation ...` to show where the warning was created)
#8 28.05 gyp info spawn /usr/bin/python
#8 28.05 gyp info spawn args [
#8 28.05 gyp info spawn args   '/node_modules/node-gyp/gyp/gyp_main.py',
#8 28.05 gyp info spawn args   'binding.gyp',
#8 28.05 gyp info spawn args   '-f',
#8 28.05 gyp info spawn args   'make',
#8 28.05 gyp info spawn args   '-I',
#8 28.05 gyp info spawn args   '/blst/bindings/node.js/build/config.gypi',
#8 28.05 gyp info spawn args   '-I',
#8 28.05 gyp info spawn args   '/node_modules/node-gyp/addon.gypi',
#8 28.05 gyp info spawn args   '-I',
#8 28.05 gyp info spawn args   '/root/.cache/node-gyp/17.4.0/include/node/common.gypi',
#8 28.05 gyp info spawn args   '-Dlibrary=shared_library',
#8 28.05 gyp info spawn args   '-Dvisibility=default',
#8 28.05 gyp info spawn args   '-Dnode_root_dir=/root/.cache/node-gyp/17.4.0',
#8 28.05 gyp info spawn args   '-Dnode_gyp_dir=/node_modules/node-gyp',
#8 28.05 gyp info spawn args   '-Dnode_lib_file=/root/.cache/node-gyp/17.4.0/<(target_arch)/node.lib',
#8 28.05 gyp info spawn args   '-Dmodule_root_dir=/blst/bindings/node.js',
#8 28.05 gyp info spawn args   '-Dnode_engine=v8',
#8 28.05 gyp info spawn args   '--depth=.',
#8 28.05 gyp info spawn args   '--no-parallel',
#8 28.05 gyp info spawn args   '--generator-output',
#8 28.05 gyp info spawn args   'build',
#8 28.05 gyp info spawn args   '-Goutput_dir=.'
#8 28.05 gyp info spawn args ]
#8 28.05 gyp info spawn make
#8 28.05 gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
#8 28.05 gyp ERR! build error 
#8 28.05 gyp ERR! stack Error: `make` failed with exit code: 1
#8 28.05 gyp ERR! stack     at ChildProcess.onExit (/node_modules/node-gyp/lib/build.js:194:23)
#8 28.05 gyp ERR! stack     at ChildProcess.emit (node:events:520:28)
#8 28.05 gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
#8 28.05 gyp ERR! System Linux 5.13.0-27-generic
#8 28.05 gyp ERR! command "/usr/local/bin/node" "/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
#8 28.05 gyp ERR! cwd /blst/bindings/node.js
#8 28.05 gyp ERR! node -v v17.4.0
#8 28.05 gyp ERR! node-gyp -v v7.1.2
#8 28.05 gyp ERR! not ok 
#8 28.05 
#8 28.05     at ChildProcess.exithandler (node:child_process:399:12)
#8 28.05     at ChildProcess.emit (node:events:520:28)
#8 28.05     at maybeClose (node:internal/child_process:1090:16)
#8 28.05     at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5) {
#8 28.05   killed: false,
#8 28.05   code: 1,
#8 28.05   signal: null,
#8 28.05   cmd: '/usr/local/bin/node /node_modules/node-gyp/bin/node-gyp.js rebuild'
#8 28.05 }
#8 28.05 Error: Error downloading and building BLST native bindings. No fallback
#8 28.05     at install (/dist/scripts/install.js:60:11)
#8 28.05     at processTicksAndRejections (node:internal/process/task_queues:96:5)
#8 ERROR: process "/dev/.buildkit_qemu_emulator /bin/sh -c PYTHON=\"/usr/bin/python\" node dist/scripts/install.js" did not complete successfully: exit code: 1
------
 > [4/6] RUN PYTHON="/usr/bin/python" node dist/scripts/install.js:
#8 28.05     at maybeClose (node:internal/child_process:1090:16)
#8 28.05     at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5) {
#8 28.05   killed: false,
#8 28.05   code: 1,
#8 28.05   signal: null,
#8 28.05   cmd: '/usr/local/bin/node /node_modules/node-gyp/bin/node-gyp.js rebuild'
#8 28.05 }
#8 28.05 Error: Error downloading and building BLST native bindings. No fallback
#8 28.05     at install (/dist/scripts/install.js:60:11)
#8 28.05     at processTicksAndRejections (node:internal/process/task_queues:96:5)
------
Dockerfile:7
--------------------
   5 |     
   6 |     # Required for `--platform linux/arm64` for some reason it can't find pythong
   7 | >>> RUN PYTHON="/usr/bin/python" node dist/scripts/install.js
   8 |     RUN ls prebuild
   9 |     
--------------------
error: failed to solve: process "/dev/.buildkit_qemu_emulator /bin/sh -c PYTHON=\"/usr/bin/python\" node dist/scripts/install.js" did not complete successfully: exit code: 1

dapplion

Let's go!

dapplion · 2022-02-10T04:36:26Z

Merging as all arm builds succeeded, but Main / build (macos-latest, 13) (pull_request) is the only that failed

Test building arm bindings

6ed9885

dapplion requested review from mpetrunic and wemeetagain as code owners February 9, 2022 06:24

dapplion and others added 4 commits February 9, 2022 12:02

Fix OS

c4da245

Build project inside arm64 image

1de8fd4

Combine arm64 build & copy steps

f1e9b99

Load arm64 image into docker

be6deca

dapplion commented Feb 10, 2022

View reviewed changes

Build arm for all docker versions

8d41d3d

dapplion force-pushed the dapplion/arm branch from 70a6e51 to 8d41d3d Compare February 10, 2022 03:18

dapplion added 3 commits February 10, 2022 08:55

Disable engine checks

f1a5d0a

Test also for Node 17

c9ca0c0

Skip Node v17 for arm

1c072d4

dapplion merged commit 7c669b2 into master Feb 10, 2022

dapplion deleted the dapplion/arm branch February 10, 2022 04:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test building arm bindings #53

Test building arm bindings #53

dapplion commented Feb 9, 2022

dapplion commented Feb 9, 2022

dapplion left a comment

dapplion commented Feb 10, 2022

Test building arm bindings #53

Test building arm bindings #53

Conversation

dapplion commented Feb 9, 2022

dapplion commented Feb 9, 2022

dapplion left a comment

Choose a reason for hiding this comment

dapplion commented Feb 10, 2022