Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support building for externally shared js builtins #2643

Merged
merged 1 commit into from
Jan 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
* [Test](#test)
* [Coverage](#coverage)
* [Update `WPTs`](#update-wpts)
* [Building for externally shared node builtins](#external-builds)
* [Developer's Certificate of Origin 1.1](#developers-certificate-of-origin)
* [Moderation Policy](#moderation-policy)

Expand Down Expand Up @@ -165,6 +166,15 @@ npm run test
npm run coverage
```

<a id="external-builds"></a>
### Building for externally shared node builtins

If you are packaging `undici` for a distro, this might help if you would like to use
an unbundled version instead of bundling one in `libnode.so`.

To enable this, pass `EXTERNAL_PATH=/path/to/global/node_modules/undici` to `build/wasm.js`.
You shall also pass this path to `--shared-builtin-undici/undici-path` in Node.js's `configure.py`.

<a id="developers-certificate-of-origin"></a>
## Developer's Certificate of Origin 1.1

Expand Down
41 changes: 30 additions & 11 deletions build/wasm.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

const { execSync } = require('child_process')
const { writeFileSync, readFileSync } = require('fs')
const { join, resolve } = require('path')
const { join, resolve, basename } = require('path')

const ROOT = resolve(__dirname, '../')
const WASM_SRC = resolve(__dirname, '../deps/llhttp')
Expand All @@ -15,6 +15,8 @@ let WASM_CFLAGS = process.env.WASM_CFLAGS || '--sysroot=/usr/share/wasi-sysroot
let WASM_LDFLAGS = process.env.WASM_LDFLAGS || ''
const WASM_LDLIBS = process.env.WASM_LDLIBS || ''

const EXTERNAL_PATH = process.env.EXTERNAL_PATH

// These are relevant for undici and should not be overridden
WASM_CFLAGS += ' -Ofast -fno-exceptions -fvisibility=hidden -mexec-model=reactor'
WASM_LDFLAGS += ' -Wl,-error-limit=0 -Wl,-O3 -Wl,--lto-O3 -Wl,--strip-all'
Expand Down Expand Up @@ -60,18 +62,31 @@ if (hasApk) {
writeFileSync(join(WASM_OUT, 'wasm_build_env.txt'), buildInfo)
}

const writeWasmChunk = EXTERNAL_PATH
? (path, dest) => {
const base64 = readFileSync(join(WASM_OUT, path)).toString('base64')
writeFileSync(join(WASM_OUT, dest), `
const { Buffer } = require('node:buffer')

module.exports = Buffer.from('${base64}', 'base64')
`)
}
: (path, dest) => {
writeFileSync(join(WASM_OUT, dest), `
const { fs } = require('node:fs')

module.exports = fs.readFileSync(require.resolve('./${basename(path)}'))
`)
}

// Build wasm binary
execSync(`${WASM_CC} ${WASM_CFLAGS} ${WASM_LDFLAGS} \
${join(WASM_SRC, 'src')}/*.c \
-I${join(WASM_SRC, 'include')} \
-o ${join(WASM_OUT, 'llhttp.wasm')} \
${WASM_LDLIBS}`, { stdio: 'inherit' })

const base64Wasm = readFileSync(join(WASM_OUT, 'llhttp.wasm')).toString('base64')
writeFileSync(
join(WASM_OUT, 'llhttp-wasm.js'),
`module.exports = '${base64Wasm}'\n`
)
writeWasmChunk('llhttp.wasm', 'llhttp-wasm.js')

// Build wasm simd binary
execSync(`${WASM_CC} ${WASM_CFLAGS} -msimd128 ${WASM_LDFLAGS} \
Expand All @@ -80,8 +95,12 @@ execSync(`${WASM_CC} ${WASM_CFLAGS} -msimd128 ${WASM_LDFLAGS} \
-o ${join(WASM_OUT, 'llhttp_simd.wasm')} \
${WASM_LDLIBS}`, { stdio: 'inherit' })

const base64WasmSimd = readFileSync(join(WASM_OUT, 'llhttp_simd.wasm')).toString('base64')
writeFileSync(
join(WASM_OUT, 'llhttp_simd-wasm.js'),
`module.exports = '${base64WasmSimd}'\n`
)
writeWasmChunk('llhttp_simd.wasm', 'llhttp_simd-wasm.js')

if (EXTERNAL_PATH) {
writeFileSync(join(ROOT, 'loader.js'), `
'use strict'

module.exports = require('node:module').createRequire('${EXTERNAL_PATH}/loader.js')('./index-fetch.js')
`)
}
4 changes: 2 additions & 2 deletions lib/client.js
Original file line number Diff line number Diff line change
Expand Up @@ -479,15 +479,15 @@ async function lazyllhttp () {

let mod
try {
mod = await WebAssembly.compile(Buffer.from(require('./llhttp/llhttp_simd-wasm.js'), 'base64'))
mod = await WebAssembly.compile(require('./llhttp/llhttp_simd-wasm.js'))
} catch (e) {
/* istanbul ignore next */

// We could check if the error was caused by the simd option not
// being enabled, but the occurring of this other error
// * https://github.com/emscripten-core/emscripten/issues/11495
// got me to remove that check to avoid breaking Node 12.
mod = await WebAssembly.compile(Buffer.from(llhttpWasmData || require('./llhttp/llhttp-wasm.js'), 'base64'))
mod = await WebAssembly.compile(llhttpWasmData || require('./llhttp/llhttp-wasm.js'))
}

return await WebAssembly.instantiate(mod, {
Expand Down
4 changes: 3 additions & 1 deletion lib/llhttp/llhttp-wasm.js

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion lib/llhttp/llhttp_simd-wasm.js

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
"*.d.ts",
"index.js",
"index-fetch.js",
"loader.js",
"lib",
"types",
"docs"
Expand Down
Loading