Electron 13 build failure #90

jinu-jang · 2021-07-20T00:01:55Z

Anyone tried to build for Electron 13?

Trying it with prebuild -r electron -t 13.0.1 leads to errors such as
tree-sitter\src\node.cc(32,129): error C2661: 'v8::ArrayBuffer::New': no overloaded function takes 3 arguments
tree-sitter\src\node.cc(36,42): error C3536: 'js_transfer_buffer': cannot be used before it is initialized
tree-sitter\src\node.cc(36,69): error C2665: 'v8::Uint32Array::New': none of the 2 overloads could convert all the argument types

I'm aware of #83, and the problems described above happen after changing binding.gyp to following

      "cflags": [
        "-std=c++17",
      ],
      'xcode_settings': {
        'CLANG_CXX_LANGUAGE_STANDARD': 'c++17',
      },

This does not happen when building for other versions of electron or node.

The text was updated successfully, but these errors were encountered:

Symbolk · 2021-08-12T03:00:43Z

Yeah, I also have this problem, which is really exhausting:

I am developing a VSCode extension that depends on this native module: https://github.com/Symbolk/somanyconflicts. Previously, I have managed to conquer this problem for Node 14.16.0, Electron 12.0.4, electron-rebuild 2.3.5,node-gyp 8.1.0. However, recently VSCode upgrades its Electron to 13.1.7 (https://www.npmjs.com/package/electron-releases), and my users reported that the extension cannot run so I have to rebuild it. Then the annoying error happens again, just like the previous one about std::remove_cv_t.

I have given up it, and looking for a WebAssembly solution.

sergei-dyshel · 2021-09-03T18:15:47Z

ArrayBuffer API changed in newer V8 version, I have a fix in sergei-dyshel@961870e.

gpetrov · 2021-10-28T18:15:57Z

@maxbrunsfeld - could you incorporate the fix?

FoamScience · 2021-12-17T19:06:35Z

Any update on this?
@sergei-dyshel Could you please PR the fix to this repo?
I also need to use Tree-Sitter in a VSCode extension, and limiting users to vscode-1.56.x is not wise.

Symbolk · 2021-12-18T09:57:18Z

Any update on this? @sergei-dyshel Could you please PR the fix to this repo? I also need to use Tree-Sitter in a VSCode extension, and limiting users to vscode-1.56.x is not wise.

Hi, I suggest to use WASM version, like what I have done: https://github.com/Symbolk/somanyconflicts

gpetrov · 2021-12-18T11:05:18Z

The WASM version is slower, a real native version like the purpose of this repo is always preferred.

The repository just needs more maintenance but unfortunately @maxbrunsfeld seems to not have enough time for it, which is a real shame for such great project.

FoamScience · 2021-12-18T11:32:49Z

Exactly, I chose this repo over the WASM version specifically because I've read it's slower; but then I read somewhere that this is no longer the case. However a native solution is always the best.
From @Symbolk's commit, it doesn't look much work to switch back and forth between WASM and native versions, so I guess
I'll continue using the WASM version for now and wait for this repo to get updated.
Thanks!

FoamScience · 2021-12-18T19:58:35Z

Any update on this? @sergei-dyshel Could you please PR the fix to this repo? I also need to use Tree-Sitter in a VSCode extension, and limiting users to vscode-1.56.x is not wise.

Hi, I suggest to use WASM version, like what I have done: https://github.com/Symbolk/somanyconflicts

So, I tried to use the WASM version myself; but LSP logs say:

Assertion failed: bad export type for `_ZNSt3__24cerrE`: undefined

Any idea on what that might mean?

dfgordon · 2021-12-19T17:54:23Z

Can anyone give a hint about prospects for an official fix?
As a related comment it looks like #52 could help with this sort of issue in a more general way, but from the conversation there I can't tell if it is still progressing.

sergei-dyshel · 2021-12-20T19:43:39Z

@sergei-dyshel Could you please PR the fix to this repo?
I also need to use Tree-Sitter in a VSCode extension, and limiting users to vscode-1.56.x is not wise.

I don't see any point in opening CR. Maintainer seems to not be checking this repo otherwise he would fix this long ago.
Meanwhile you can create a fixed branch in your own fork and use it as a dependency in package.json. For example I have:

"dependencies": {
    # ...
	"tree-sitter": "github:sergei-dyshel/node-tree-sitter#master-next",
    # ...
}

FoamScience · 2021-12-20T20:43:27Z

Yes, that's what I'm doing now, still getting some undefined references but nothing a good debugging session wouldn't fix. Thanks.

…

On Mon, Dec 20, 2021, 8:43 PM Sergei Dyshel ***@***.***> wrote: @sergei-dyshel <https://github.com/sergei-dyshel> Could you please PR the fix to this repo? I also need to use Tree-Sitter in a VSCode extension, and limiting users to vscode-1.56.x is not wise. I don't see any point in opening CR. Maintainer seems to not be checking this repo otherwise he would fix this long ago. Meanwhile you can create a fixed branch in your own fork and use it as a dependency in package.json. For example I have: "dependencies": { # ... "tree-sitter": "github:sergei-dyshel/node-tree-sitter#master-next", # ... } — Reply to this email directly, view it on GitHub <#90 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIHAT2DOEQZUZQG5B5ZVVZTUR6BPNANCNFSM5AUV5WCA> . You are receiving this because you commented.Message ID: ***@***.***>

dfgordon · 2021-12-22T16:40:55Z

So, I tried to use the WASM version myself; but LSP logs say:
Assertion failed: bad export type for `_ZNSt3__24cerrE`: undefined
Any idea on what that might mean?

I was getting this also. The error went away after rebuilding the WASM using docker instead of emsdk.

FoamScience · 2021-12-23T18:31:38Z

@sergei-dyshel Could you please PR the fix to this repo?
I also need to use Tree-Sitter in a VSCode extension, and limiting users to vscode-1.56.x is not wise.

I don't see any point in opening CR. Maintainer seems to not be checking this repo otherwise he would fix this long ago. Meanwhile you can create a fixed branch in your own fork and use it as a dependency in package.json. For example I have:
"dependencies": {
    # ...
	"tree-sitter": "github:sergei-dyshel/node-tree-sitter#master-next",
    # ...
}

Even with this approach (which works) I still get a:

tree_sitter_runtime_binding.node: undefined symbol: _ZN2v811ArrayBuffer3NewEPNS_7IsolateESt10shared_ptrINS_12BackingStoreEE

referring to a missing symbol definition for v8::ArrayBuffer::New(v8::Isolate*, std::shared_ptr<v8::BackingStore>)
which should be there
Shouldn't v8 be linked automatically?

FoamScience · 2022-01-03T18:40:20Z

Happy new year guys
@sergei-dyshel When I use your repo for my vscode extension, I can rebuild modules for electron 13
But at runtime, VSCode says there is a symbol lookup error concerning v8::ArrayBuffer::New(v8::Isolate*, std::shared_ptr<v8::BackingStore>) (undefined).

nm says it's not defined too, but I guess that's OK as it seems VSCode loads libraries at runtime (there are a lot of undefined symbols from v8 at this point). Also, note that nm does not know a candidate library to look for the symbol in:

nm -C ./node_modules/tree-sitter/build/Release/obj.target/tree_sitter_runtime_binding.node | grep ArrayBuffer::New
                 U v8::ArrayBuffer::NewBackingStore(void*, unsigned long, void (*)(void*, unsigned long, void*), void*)
                 U v8::ArrayBuffer::New(v8::Isolate*, std::shared_ptr<v8::BackingStore>)

ldd does not mention anything about libv8 or libnode.a:

ldd ./node_modules/tree-sitter/build/Release/obj.target/tree_sitter_runtime_binding.node
	linux-vdso.so.1 (0x00007ffc2c1ad000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f59d9b3a000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f59d9b17000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f59d9925000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f59d97d6000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f59d9da5000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f59d97b9000)

Anyone has experienced this before? Any help is much appreciated as it's driving me crazy 😄

dfgordon · 2022-01-07T11:27:23Z

I tried this and had a similar experience, but didn't pursue and switched to WASM, which seems to work fine. Since rust and WASM bindings were moved into tree-sitter/lib it is probable that these are the bindings that will get the most consistent support from the main developers. So I join @Symbolk and suggest switch to WASM. For me the two issues with the switch were getting emscripten to produce a compatible WASM (solution is use Docker), and that I didn't know much about asynchronous functions in typescript (it's not hard to learn).

FoamScience · 2022-01-07T12:56:40Z

Yes, exactly, plus I confirm there is no performance overhead when using the WASM version; at least for my parser.

gpetrov · 2022-01-07T13:46:20Z

Well it is all good with the WASM build but the real native build provides 5x better performance.

A quick benchmark with parsing a html file yields:

On MacBook Pro M1 MAX:

File Size 25KB:

Parser	Time
Native Tree Sitter	1.71 ms
WASM Tree Sitter	8.20 ms
Native DOM Parser	2.69 ms
Parse5	16.71 ms

File Size 200KB:

Parser	Time
Native Tree Sitter	13.27 ms
WASM Tree Sitter	46 ms
Native DOM Parser	9.38 ms
Parse5	51.41 ms

On Windows 10 Pro with AMD Ryzen 7 3800X 8 Core 3.90 GHz:

File Size 25KB:

Parser	Time
Native Tree Sitter	4.55 ms
WASM Tree Sitter	12.03 ms
Native DOM Parser	3.71 ms
Parse5	25.17 ms

File Size 200KB:

Parser	Time
Native Tree Sitter	30.18 ms
WASM Tree Sitter	64.26 ms
Native DOM Parser	15.32 ms
Parse5	77.87 ms

So as try can see the native tree Sitter is much faster and on smaller files even faster than the native DOMParser in from the browser.

So we would really like to have a native version of the node-tree-sitter as well, next to the WASM version

However without a good rewrite to support NAPI as started in #52
Even with the current patches from #95 the current version will only work till Electron 13.

Starting from Electron 14 all non context aware modules that are not NAPI compatible, cannot be loaded any more as stated in: electron/electron#18397

So please @maxbrunsfeld have an urgent look on this.

jasonwilliams · 2022-04-18T18:19:15Z

tree-sitter doesn't build alongside the latest version of VSCode there error i get is:

error User\workspace\vscode\node_modules\tree-sitter: Command failed.
Exit code: 1
Command: prebuild-install || node-gyp rebuild
Arguments:
Directory: User\workspace\vscode\node_modules\tree-sitter
Output:
User\workspace\vscode\node_modules\node-abi\index.js:36
  throw new Error('Could not detect abi for version ' + target + ' and runtime ' + runtime + '.  Updating "node-abi" might help solve this issue if it is a new release of ' + runtime)
  ^

Error: Could not detect abi for version 17.4.0 and runtime electron.  Updating "node-abi" might help solve this issue if it is a new release of electron
    at getAbi (User\workspace\vscode\node_modules\node-abi\index.js:36:9)
    at module.exports (User\workspace\vscode\node_modules\prebuild-install\rc.js:52:57)
    at Object.<anonymous> (User\workspace\vscode\node_modules\prebuild-install\bin.js:8:25)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
    at node:internal/main/run_main_module:17:47

User\workspace\vscode\node_modules\tree-sitter>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "" rebuild )
gyp info it worked if it ends with ok
gyp info using node-gyp@8.4.1
gyp info using node@16.14.0 | win32 | x64
gyp info find Python using Python version 3.10.4 found at "User\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\python.exe"
gyp info find VS using VS2019 (16.11.32228.343) found at:
gyp info find VS "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools"
gyp info find VS run with --verbose for detailed information
gyp info spawn User\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\python.exe
gyp info spawn args [
gyp info spawn args   'user\\AppData\\Roaming\\nvm\\v16.14.0\\node_modules\\npm\\node_modules\\node-gyp\\gyp\\gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'msvs',
gyp info spawn args   '-I',
gyp info spawn args   'user\\workspace\\vscode\\node_modules\\tree-sitter\\build\\config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'user\\AppData\\Roaming\\nvm\\v16.14.0\\node_modules\\npm\\node_modules\\node-gyp\\addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'user\\AppData\\Local\\node-gyp\\Cache\\17.4.0\\include\\node\\common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=user\\AppData\\Local\\node-gyp\\Cache\\17.4.0',
gyp info spawn args   '-Dnode_gyp_dir=user\\AppData\\Roaming\\nvm\\v16.14.0\\node_modules\\npm\\node_modules\\node-gyp',
gyp info spawn args   '-Dnode_lib_file=user\\\AppData\\\\Local\\\\node-gyp\\\\Cache\\\\17.4.0\\\\<(target_arch)\\\\node.lib',
gyp info spawn args   '-Dmodule_root_dir=user\\workspace\\vscode\\node_modules\\tree-sitter',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'user\\workspace\\vscode\\node_modules\\tree-sitter\\build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin\MSBuild.exe
gyp info spawn args [
User\workspace\vscode\node_modules\tree-sitter\src\conversions.cc(32,118): error C2661: 'Nan::Set': no overloaded function takes 2 arguments [User\workspace\vscode\node_modules\tree-sitter\build\tree_sitter_runtime_binding.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin\MSBuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (User\AppData\Roaming\nvm\v16.14.0\node_modules\npm\node_modules\node-gyp\lib\build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:520:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Windows_NT 10.0.22000gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd User\workspace\vscode\node_modules\tree-sitter

Sarrus1 · 2022-08-11T10:37:34Z

Hi, any update on this?

bencwallace · 2023-01-08T18:47:44Z

@FoamScience

nm -C ./node_modules/tree-sitter/build/Release/obj.target/tree_sitter_runtime_binding.node | grep ArrayBuffer::New
                 U v8::ArrayBuffer::NewBackingStore(void*, unsigned long, void (*)(void*, unsigned long, void*), void*)
                 U v8::ArrayBuffer::New(v8::Isolate*, std::shared_ptr<v8::BackingStore>)

I think I've found the symbol for New in the electron (13.6.9) binary:

nm -C node_modules/electron/dist/electron | grep 'v8::ArrayBuffer::New'
0000000002caa680 T v8::ArrayBuffer::NewBackingStore(v8::Isolate*, unsigned long)
0000000002caa790 T v8::ArrayBuffer::NewBackingStore(void*, unsigned long, void (*)(void*, unsigned long, void*), void*)
0000000002caa3c0 T v8::ArrayBuffer::New(v8::Isolate*, unsigned long)
0000000002caa4b0 T v8::ArrayBuffer::New(v8::Isolate*, std::__1::shared_ptr<v8::BackingStore>)

The last of these seems to be what is needed. Unfortunately, it seems to be using the LLVM runtime.

I guess if you can run prebuild against libc++, it might solve this problem.

FoamScience · 2023-01-08T19:09:13Z

Oh, that it is actually good news; Thanks! I have switched to WASM builds since, and they work - I can't remember if this was the only error but if I find time, I'll probably go back and try the native version

@FoamScience

nm -C ./node_modules/tree-sitter/build/Release/obj.target/tree_sitter_runtime_binding.node | grep ArrayBuffer::New
                 U v8::ArrayBuffer::NewBackingStore(void*, unsigned long, void (*)(void*, unsigned long, void*), void*)
                 U v8::ArrayBuffer::New(v8::Isolate*, std::shared_ptr<v8::BackingStore>)

I think I've found the symbol for New in the electron (13.6.9) binary:

nm -C node_modules/electron/dist/electron | grep 'v8::ArrayBuffer::New'
0000000002caa680 T v8::ArrayBuffer::NewBackingStore(v8::Isolate*, unsigned long)
0000000002caa790 T v8::ArrayBuffer::NewBackingStore(void*, unsigned long, void (*)(void*, unsigned long, void*), void*)
0000000002caa3c0 T v8::ArrayBuffer::New(v8::Isolate*, unsigned long)
0000000002caa4b0 T v8::ArrayBuffer::New(v8::Isolate*, std::__1::shared_ptr<v8::BackingStore>)

The last of these seems to be what is needed. Unfortunately, it seems to be using the LLVM runtime.

I guess if you can run prebuild against libc++, it might solve this problem.

bencwallace · 2023-01-08T19:25:32Z

Please let me know if you end up with a reasonably clean solution.

This blog post contains an interesting deep dive into a similar issue that the author encountered when moving to chromium 90.0.4415.0. It seems Electron 13 was the first version to use a version of chromium greater than or equal to that. Their ultimate suggestion is similar to @gpetrov's (use NAPI).

verhovsky · 2023-06-24T21:52:56Z

This should be fixed with node-tree-sitter v0.20.4. If someone could confirm or re-open the issue if it's still not fixed, that would be great.

segevfiner · 2024-07-11T11:18:44Z

node-tree-sitter is now Node-API and should be compatible with Electron natively. This can be closed.

dfgordon mentioned this issue Dec 8, 2021

electron 13 compatibility tree-sitter/tree-sitter#1522

Closed

gpetrov mentioned this issue Jan 7, 2022

Can't be installed for Node v16 nodegit/nodegit#1840

Open

jasonwilliams mentioned this issue Apr 18, 2022

[Draft] - Tree Sitter Service (POC) microsoft/vscode#147648

Closed

This comment was marked as off-topic.

Sign in to view

verhovsky mentioned this issue Mar 9, 2023

Support Node 19 and 20 #127

Closed

This comment was marked as outdated.

Sign in to view

verhovsky mentioned this issue Jun 24, 2023

Support Electron 13+ on Windows #151

Merged

verhovsky closed this as completed in #151 Jun 24, 2023

verhovsky reopened this Jul 22, 2023

amaanq closed this as completed Jul 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Electron 13 build failure #90

Electron 13 build failure #90

jinu-jang commented Jul 20, 2021 •

edited

Loading

Symbolk commented Aug 12, 2021

sergei-dyshel commented Sep 3, 2021

gpetrov commented Oct 28, 2021

FoamScience commented Dec 17, 2021

Symbolk commented Dec 18, 2021

gpetrov commented Dec 18, 2021

FoamScience commented Dec 18, 2021 •

edited

Loading

FoamScience commented Dec 18, 2021 •

edited

Loading

dfgordon commented Dec 19, 2021

sergei-dyshel commented Dec 20, 2021

FoamScience commented Dec 20, 2021 via email

dfgordon commented Dec 22, 2021

FoamScience commented Dec 23, 2021

FoamScience commented Jan 3, 2022

dfgordon commented Jan 7, 2022

FoamScience commented Jan 7, 2022

gpetrov commented Jan 7, 2022 •

edited

Loading

jasonwilliams commented Apr 18, 2022 •

edited

Loading

Sarrus1 commented Aug 11, 2022 •

edited

Loading

This comment was marked as off-topic.

bencwallace commented Jan 8, 2023

FoamScience commented Jan 8, 2023 •

edited

Loading

bencwallace commented Jan 8, 2023

This comment was marked as outdated.

verhovsky commented Jun 24, 2023

segevfiner commented Jul 11, 2024

Electron 13 build failure #90

Electron 13 build failure #90

Comments

jinu-jang commented Jul 20, 2021 • edited Loading

Symbolk commented Aug 12, 2021

sergei-dyshel commented Sep 3, 2021

gpetrov commented Oct 28, 2021

FoamScience commented Dec 17, 2021

Symbolk commented Dec 18, 2021

gpetrov commented Dec 18, 2021

FoamScience commented Dec 18, 2021 • edited Loading

FoamScience commented Dec 18, 2021 • edited Loading

dfgordon commented Dec 19, 2021

sergei-dyshel commented Dec 20, 2021

FoamScience commented Dec 20, 2021 via email

dfgordon commented Dec 22, 2021

FoamScience commented Dec 23, 2021

FoamScience commented Jan 3, 2022

dfgordon commented Jan 7, 2022

FoamScience commented Jan 7, 2022

gpetrov commented Jan 7, 2022 • edited Loading

On MacBook Pro M1 MAX:

File Size 25KB:

File Size 200KB:

On Windows 10 Pro with AMD Ryzen 7 3800X 8 Core 3.90 GHz:

File Size 25KB:

File Size 200KB:

jasonwilliams commented Apr 18, 2022 • edited Loading

Sarrus1 commented Aug 11, 2022 • edited Loading

This comment was marked as off-topic.

bencwallace commented Jan 8, 2023

FoamScience commented Jan 8, 2023 • edited Loading

bencwallace commented Jan 8, 2023

This comment was marked as outdated.

verhovsky commented Jun 24, 2023

segevfiner commented Jul 11, 2024

jinu-jang commented Jul 20, 2021 •

edited

Loading

FoamScience commented Dec 18, 2021 •

edited

Loading

FoamScience commented Dec 18, 2021 •

edited

Loading

gpetrov commented Jan 7, 2022 •

edited

Loading

jasonwilliams commented Apr 18, 2022 •

edited

Loading

Sarrus1 commented Aug 11, 2022 •

edited

Loading

FoamScience commented Jan 8, 2023 •

edited

Loading