Support V8 C++ APIs for "nan" addons and other packages to work

[xhyrom]

Affected packages

• better-sqlite3
• node-canvas@v2 (node-canvas v3 is supported)
• midi
• libxmljs
• bignum
• zeromq
• node-libcurl
• bson-ext
• heapdump
• @datadog/native-metrics
• tiny-secp256k1
• tree-sitter-c-sharp
• chacha-native
• @sentry/profiling-node
• pdfjs-dist
• gl
• leveldown / pouchdb
• sleep
• cpu-features
• odbc
• msgpackr
• re2
• node-pty
• i2c

Functions implemented

• v8::Isolate::GetCurrent()
• v8::Isolate::TryGetCurrent()
• v8::Isolate::GetCurrentContext()
• v8::Number::New(v8::Isolate*, double)
• v8::Number::Value() const
• v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int)
• v8::String::WriteUtf8(v8::Isolate*, char*, int, int*, int) const
• v8::api_internal::ToLocalEmpty()
• v8::String::Length() const
• v8::External::New(v8::Isolate*, void*)
• v8::External::Value() const
• v8::Object::New(v8::Isolate*)
• v8::Object::Set(v8::Local<v8::Context>, v8::Local<v8::Value>, v8::Local<v8::Value>)
• v8::Object::SetInternalField(int, v8::Local<v8::Data>)
• v8::Object::SlowGetInternalField(int)
• v8::HandleScope::HandleScope(v8::Isolate*)
• v8::HandleScope::~HandleScope()
• v8::FunctionTemplate::GetFunction(v8::Local<v8::Context>)
• v8::FunctionTemplate::New(v8::Isolate*, void (*)(v8::FunctionCallbackInfo<v8::Value> const&), v8::Local<v8::Value>, v8::Local<v8::Signature>, int, v8::ConstructorBehavior, v8::SideEffectType, v8::CFunction const*, unsigned short, unsigned short, unsigned short) (only when all parameters past data are their default values)
• v8::ObjectTemplate::NewInstance(v8::Local<v8::Context>)
• v8::ObjectTemplate::SetInternalFieldCount(int)
• v8::ObjectTemplate::InternalFieldCount() const
• v8::ObjectTemplate::New(v8::Isolate*, v8::Local<v8::FunctionTemplate>) (only when no FunctionTemplate is provided)
• v8::EscapableHandleScopeBase::EscapeSlot(unsigned long*)
• v8::EscapableHandleScopeBase::EscapableHandleScopeBase(v8::Isolate*)
• node_module_register
• v8::Function::SetName(v8::Local<v8::String>)
• v8::Value::IsBoolean() const
• v8::Boolean::Value() const
• v8::Value::FullIsTrue() const
• v8::Value::FullIsFalse() const
• v8::EscapableHandleScope::EscapableHandleScope(v8::Isolate*)
• v8::EscapableHandleScope::~EscapableHandleScope()
• v8::Value::IsObject() const
• v8::Value::IsNumber() const
• v8::Value::IsUint32() const
• v8::Value::Uint32Value(v8::Local<v8::Context>) const
• v8::Value::IsUndefined() const
• v8::Value::IsNull() const
• v8::Value::IsNullOrUndefined() const
• v8::Value::IsTrue() const
• v8::Value::IsFalse() const
• v8::Value::IsString() const
• v8::Boolean::New(v8::Isolate*, bool)
• v8::Object::GetInternalField(int)
• node::AddEnvironmentCleanupHook(v8::Isolate*, void (*)(void*), void*)
• node::RemoveEnvironmentCleanupHook(v8::Isolate*, void (*)(void*), void*)
• v8::HandleScope::CreateHandle(v8::internal::Isolate*, unsigned long)
• v8::internal::IsolateFromNeverReadOnlySpaceObject(unsigned long)
• v8::Array::New(v8::Isolate*, v8::Local<v8::Value>*, unsigned long)
• v8::Object::Get(v8::Local<v8::Context>, v8::Local<v8::Value>)

Major areas of unimplemented functionality

• Array
• ObjectTemplate with a constructor function
• Less-simple cases of FunctionTemplate
• ArrayBuffer
• bun install fixes (need to pass node-gyp the right version)
• Doubles
• UTF-16 strings
• Persistent handles

What version of Bun is running?

0.8.0+ad326b77342dd3d8585a30b7da803d32f9c11fe2

What platform is your computer?

Linux 6.4.10-artix1-1 x86_64 unknown

What steps can reproduce the bug?

import Database from 'better-sqlite3';
const db = new Database('foobar.db', {});

console.log(db.prepare("CREATE TABLE TEST").run());

What is the expected behavior?

Run as normally

What do you see instead?

/usr/bin/bun: symbol lookup error: /home/hyro/Workspace/booo/node_modules/better-sqlite3/build/Release/better_sqlite3.node: undefined symbol: node_module_register

Additional information

Related thread: 1144181146809270332
Message: https://discord.com/channels/876711213126520882/1144181146809270332/1144184224006557747

All versions are affected

[floydjones1]

This also happens for node-libcurl

[issadarkthing]

I also encounter this error

[Zykatious]

this also happens for bson-ext

[autoscatto]

also for heapdump

[vroudge]

same for @datadog/native-metrics

[uditdc]

Likewise for tiny-secp256k1

[ozyman42]

Also for cdktf-cli #5306

[obedm503]

Also happens with tree-sitter-c-sharp

[Scitz0]

Same error for chacha-native

[metawrap-dev]

same for node-canvas

[4i8]

bun: symbol lookup error: /root/rut/node_modules/canvas/build/Release/canvas.node: undefined symbol: node_module_register
same for canvas

[edi9999]

Same for libxmljs

[floydjones1]

How can we get a little more attention to this. This is holding me back from using bun in production. 😢
node-libcurl

bun: symbol lookup error: /home/.../node_modules/node-libcurl/lib/binding/node_libcurl.node: undefined symbol: node_module_register

[M-Gonzalo]

bun: symbol lookup error: /root/rut/node_modules/canvas/build/Release/canvas.node: undefined symbol: node_module_register same for canvas

me too

[frani]

similar error:

bun: symbol lookup error: /usr/src/app/node_modules/@sentry/profiling-node/lib/sentry_cpu_profiler-linux-x64-glibc-115.node: undefined symbol: _ZN2v87Isolate10GetCurrentEv

[xbura1han]

Also happening for Sentry & bcrypt
Sentry code:
Code:

const Sentry = require('@sentry/bun');
const { ProfilingIntegration } = require("@sentry/profiling-node")

Error:

bun: symbol lookup error: /root/projects/userBackend/node_modules/@sentry/profiling-node/lib/sentry_cpu_profiler-linux-x64-glibc-115.node: undefined symbol: _ZN2v87Isolate10GetCurrentEv

[dayblox]

same for cap

[190n]

Now that #12821 is merged, the next version of Bun will include some of the first working V8 functions. See the top of this issue for more detailed status. Packages more complex than cpu-features or sleep are likely to still run into cases I have not covered yet, and there may be functions where we're never able to bridge the gap between how V8 and JavaScriptCore represent data, but we're excited to have the first version of this functionality landing soon for testing.

[chriskrogh]

@190n thank you for the hard work! based on #5835 (comment) this gets us closer to supporting node-canvas which unblocks jsdom! we can't wait to use bun at @Faire

[Jarred-Sumner]

@190n thank you for the hard work! based on #5835 (comment) this gets us closer to supporting node-canvas which unblocks jsdom! we can't wait to use bun at @Faire

@chriskrogh you can use canvas@next instead of the latest version and it should work, as that uses napi instead of v8 APIs

[chriskrogh]

@Jarred-Sumner it works! 🥳 no more canvas issues on bun install!!!
we only have 2 more offending packages:

• llamaindex
  • tries to link @datastax/astra-db-ts
  • we don't have that many usages. might try to switch to ai by vercel
• @grpc/grpc-js
  • this one is actually used by most of our app servers, so we might need to wait for support here

but after removing those two, I was able to get a clean install which was lightning fast! thanks again for all the progress here! hopefully grpc doesn't take that much more? 🤞

[KilianB]

Is grpc related to v8 incompatibilities? I thought it was blocked by https2 server support

[rgillan]

@Jarred-Sumner et al, please note I've deleted a comment re ssh2 server issues as it is not related to the V8 APIs, will raise a separate ticket shortly (have identified a potential issue)

[jerrywoo96]

Is there a list or is anyone able to make a list of packages that are not yet resolved or implemented? This issue thread is becoming too large to keep track of which packages are still outstanding.

[190n]

That exists in the top comment on this issue, but it probably deserves to be the most prominent part instead of the list of functions. I'll move it to the top.

[rgillan]

@190n if you could add msgpackr to the list as well please, although will be ok once Array::New is done I expect. Really appreciate all your efforts on this.

[190n]

Right, thanks for the reminder.

[etcd]

@190n thoughts on also adding re2 to the list (given that issue #12560 was marked as a duplicate of this one)?

[karlhorky]

@190n also node-pty, following these comments:

• node-pty unable to be run from bun #7362 (comment)
• Bun support microsoft/node-pty#632 (comment)

[nachat-ayoub]

how much time could it take to support the "gl" module?

[adworacz]

@190n if you could add msgpackr to the list as well please, although will be ok once Array::New is done I expect. Really appreciate all your efforts on this.

For anyone else who's confused by this - technically msgpackr does support Bun, it's just the "native acceleration" feature that msgpackr has that won't work.

[rgillan]

@190n if you could add msgpackr to the list as well please, although will be ok once Array::New is done I expect. Really appreciate all your efforts on this.

For anyone else who's confused by this - technically msgpackr does support Bun, it's just the "native acceleration" feature that msgpackr has that won't work.

so there's been investigation by @190n back in July when we first found this, and you can find more in #12754 with their findings. One of the great things with msgpackr is the native acceleration, and that needs v8::Array::New

[190n]

One of the great things with msgpackr is the native acceleration, and that needs v8::Array::New

As an update on that, I'm not sure if we'll be able to support their specific use of v8::Array::New, because they're using it by casting an array of NAPI values into an array of V8 values. That only works if NAPI values are V8 values, which is true in Node but not Bun. So while we can get v8::Array::New working for packages using the V8 API normally, for msgpackr specifically I think the solution would be more along the lines of finding out why they detect us as Node and stopping that from happening. So ideally their native acceleration can still work in Bun but it should be using exclusively NAPI functions instead of mixing NAPI and V8.

[Frtrillo]

Same issue for @newrelic/native-metrics and symbol lookup