Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error during npm install #76

Closed
fsa317 opened this issue Jun 12, 2024 · 11 comments
Closed

Error during npm install #76

fsa317 opened this issue Jun 12, 2024 · 11 comments
Assignees
@adhityan
Labels
bug Something isn't working

Comments

@fsa317
Copy link

fsa317 commented Jun 12, 2024

Trying to install the package on my mac and get the following error:

RagTest % npm i @llm-tools/embedjs

npm WARN deprecated @aws-sdk/signature-v4@3.374.0: This package has moved to @smithy/signature-v4
npm WARN deprecated @aws-sdk/protocol-http@3.374.0: This package has moved to @smithy/protocol-http
npm WARN deprecated text-encoding@0.7.0: no longer maintained
npm ERR! code 1
npm ERR! path /Users/apapf/MyCode/RagTest/node_modules/mmmagic
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! CC(target) Release/obj.target/libmagic/deps/libmagic/src/apprentice.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/apptype.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/ascmagic.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/cdf.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/cdf_time.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/compress.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/der.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/encoding.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/fsmagic.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/funcs.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/is_tar.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/magic.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/print.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/readcdf.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/readelf.o
npm ERR!   CC(target) Release/obj.target/libmagic/deps/libmagic/src/softmagic.o
npm ERR!   LIBTOOL-STATIC Release/magic.a
npm ERR!   CXX(target) Release/obj.target/magic/src/binding.o
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.3.1
npm ERR! gyp info using node@20.3.1 | darwin | arm64
npm ERR! gyp info find Python using Python version 3.11.4 found at "/opt/homebrew/opt/python@3.11/bin/python3.11"
npm ERR! gyp info spawn /opt/homebrew/opt/python@3.11/bin/python3.11
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/apapf/MyCode/RagTest/node_modules/mmmagic/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/Users/apapf/Library/Caches/node-gyp/20.3.1',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/Users/apapf/Library/Caches/node-gyp/20.3.1/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/Users/apapf/MyCode/RagTest/node_modules/mmmagic',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp info spawn make
npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm ERR! In file included from ../src/binding.cc:1:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/node.h:73:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8.h:24:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-array-buffer.h:12:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-local-handle.h:12:
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:465:30: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! SHARED_EXTERNAL_POINTER_TAGS(CHECK_SHARED_EXTERNAL_POINTER_TAGS)
npm ERR!                              ^
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:465:30: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:465:30: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:465:30: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:465:30: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! PER_ISOLATE_EXTERNAL_POINTER_TAGS(CHECK_NON_SHARED_EXTERNAL_POINTER_TAGS)
npm ERR!                                   ^
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:466:35: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:693:61: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR!     static_assert(kJSObjectType + 1 == kFirstJSApiObjectType);
npm ERR!                                                             ^
npm ERR!                                                             , ""
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:694:55: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR!     static_assert(kJSObjectType < kLastJSApiObjectType);
npm ERR!                                                       ^
npm ERR!                                                       , ""
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-internal.h:695:63: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR!     static_assert(kFirstJSApiObjectType < kLastJSApiObjectType);
npm ERR!                                                               ^
npm ERR!                                                               , ""
npm ERR! In file included from ../src/binding.cc:1:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/node.h:73:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8.h:24:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-array-buffer.h:13:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-object.h:9:
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-maybe.h:106:45: error: no template named 'is_lvalue_reference_v' in namespace 'std'; did you mean 'is_lvalue_reference'?
npm ERR!   template <class U, std::enable_if_t<!std::is_lvalue_reference_v<U>>*>
npm ERR!                                        ~~~~~^~~~~~~~~~~~~~~~~~~~~
npm ERR!                                             is_lvalue_reference
npm ERR! /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__type_traits/is_reference.h:24:29: note: 'is_lvalue_reference' declared here
npm ERR! struct _LIBCPP_TEMPLATE_VIS is_lvalue_reference : _BoolConstant<__is_lvalue_reference(_Tp)> {};
npm ERR!                             ^
npm ERR! In file included from ../src/binding.cc:1:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/node.h:73:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8.h:24:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-array-buffer.h:13:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-object.h:9:
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-maybe.h:106:69: error: expected '(' for function-style cast or type construction
npm ERR!   template <class U, std::enable_if_t<!std::is_lvalue_reference_v<U>>*>
npm ERR!                                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-maybe.h:123:43: error: no template named 'is_lvalue_reference_v' in namespace 'std'; did you mean 'is_lvalue_reference'?
npm ERR! template <class T, std::enable_if_t<!std::is_lvalue_reference_v<T>>* = nullptr>
npm ERR!                                      ~~~~~^~~~~~~~~~~~~~~~~~~~~
npm ERR!                                           is_lvalue_reference
npm ERR! /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__type_traits/is_reference.h:24:29: note: 'is_lvalue_reference' declared here
npm ERR! struct _LIBCPP_TEMPLATE_VIS is_lvalue_reference : _BoolConstant<__is_lvalue_reference(_Tp)> {};
npm ERR!                             ^
npm ERR! In file included from ../src/binding.cc:1:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/node.h:73:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8.h:24:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-array-buffer.h:13:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-object.h:9:
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-maybe.h:123:67: error: expected '(' for function-style cast or type construction
npm ERR! template <class T, std::enable_if_t<!std::is_lvalue_reference_v<T>>* = nullptr>
npm ERR!                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
npm ERR! In file included from ../src/binding.cc:1:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/node.h:73:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8.h:33:
npm ERR! In file included from /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-function.h:11:
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-function-callback.h:151:66: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR!                 kReturnValueDefaultValueIndex - kReturnValueIndex);
npm ERR!                                                                  ^
npm ERR!                                                                  , ""
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-function-callback.h:153:50: warning: 'static_assert' with no message is a C++17 extension [-Wc++17-extensions]
npm ERR!                 kIsolateIndex - kReturnValueIndex);
npm ERR!                                                  ^
npm ERR!                                                  , ""
npm ERR! In file included from ../src/binding.cc:3:
npm ERR! ../../nan/nan.h:700:39: warning: 'IdleNotificationDeadline' is deprecated: Use MemoryPressureNotification() to influence the GC schedule. [-Wdeprecated-declarations]
npm ERR!     return v8::Isolate::GetCurrent()->IdleNotificationDeadline(
npm ERR!                                       ^
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8-isolate.h:1291:3: note: 'IdleNotificationDeadline' has been explicitly marked deprecated here
npm ERR!   V8_DEPRECATE_SOON(
npm ERR!   ^
npm ERR! /Users/apapf/Library/Caches/node-gyp/20.3.1/include/node/v8config.h:550:39: note: expanded from macro 'V8_DEPRECATE_SOON'
npm ERR! # define V8_DEPRECATE_SOON(message) [[deprecated(message)]]
npm ERR!                                       ^
npm ERR! 23 warnings and 4 errors generated.
npm ERR! make: *** [Release/obj.target/magic/src/binding.o] Error 1
npm ERR! gyp ERR! build error 
npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
npm ERR! gyp ERR! stack     at ChildProcess.onExit (/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:203:23)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:511:28)
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:293:12)
npm ERR! gyp ERR! System Darwin 23.5.0
npm ERR! gyp ERR! command "/opt/homebrew/Cellar/node/20.3.1_1/bin/node" "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /Users/apapf/MyCode/RagTest/node_modules/mmmagic
npm ERR! gyp ERR! node -v v20.3.1
npm ERR! gyp ERR! node-gyp -v v9.3.1
npm ERR! gyp ERR! not ok
@adhityan adhityan added the bug Something isn't working label Jun 12, 2024
@adhityan
Copy link
Collaborator

Let me take a look.

@fsa317
Copy link
Author

fsa317 commented Jun 12, 2024 via email

@adhityan
Copy link
Collaborator

This looks like an issue with one of the dependencies. They have a PR addressing this but it has not been merged for months (mscdex/mmmagic#171). I will try and get the upstream patched; if not, I will publish a version using alternative package or a forked version.

@fsa317
Copy link
Author

fsa317 commented Jun 12, 2024

The install of mmmagic works for me but then when I run the app I created I get:

node:internal/modules/cjs/loader:1072
  const err = new Error(message);
              ^

Error: Cannot find module '../build/Release/magic'
Require stack:
- /Users/apapf/MyCode/RagTest/node_modules/mmmagic/lib/index.js
- /Users/apapf/MyCode/RagTest/node_modules/stream-mmmagic/lib/stream-mmmagic.js
-

I notice there is no Release directory inside node_modules/mmmagic/build directory. Is that expected?

@adhityan
Copy link
Collaborator

adhityan commented Jun 13, 2024 •
edited
Loading

Published a new version (0.0.89) with a patched version of mmagic. NPM Install is working fine in my mac and tested the library in linux. Could you confirm if issue is resolved for you?

@fsa317
Copy link
Author

fsa317 commented Jun 13, 2024

Published a new version (0.0.89) with a patched version of mmagic. NPM Install is working fine in my mac and tested the library in linux. Could you confirm if issue is resolved for you?

So the install worked, but still having issues using things. I am trying to install this in a nextjs app by the way.

So now the npm install works. Then I did npm install hnswlib-node. However when I try to import { HNSWDb } from '@llm-tools/embedjs/vectorDb/hnswlib-db'; it does not work, basically says it can't find it. I see the file inside a similar directory in node_modules but with dist in the path.

@fsa317
Copy link
Author

fsa317 commented Jun 13, 2024

What's interesting is that if I change the import to import { HNSWDb } from '@llm-tools/embedjs/vectorDb/hnswlib'; (remove the -db) I get an error in my IDE but then at runtime I get a different error:

Module not found: Can't resolve '../build/Release/magic'

https://nextjs.org/docs/messages/module-not-found

Import trace for requested module:
./node_modules/stream-mmmagic-patched/lib/stream-mmmagic.js

@adhityan
Copy link
Collaborator

adhityan commented Jun 14, 2024 •
edited
Loading

Yes, the import path is import { HNSWDb } from '@llm-tools/embedjs/vectorDb/hnswlib'. You can refer to all the import paths in the documentation. This is done so that people need to install only the minimum dependencies they need. The end goal here is to move to a monorepo setup.

Actually, NextJs has some issues with node native packages. I am currently actually testing this out and building a sample app here. It's still a WIP but I hope to get support added soon.

@fsa317
Copy link
Author

fsa317 commented Jun 14, 2024 via email

@adhityan
Copy link
Collaborator

adhityan commented Jul 6, 2024

Will post an update when NextJs is supported. Closing the issue for now.

@adhityan adhityan closed this as completed Jul 6, 2024
@adhityan
Copy link
Collaborator

adhityan commented Jul 7, 2024

So finally have this working with NextJs. Check the example here with pinecone - https://github.com/llm-tools/chat-bot-nextjs-template.

Point to note - not all vector databases work with NextJs. It may be possible to get each of them wotking with specific config changes to the bundler but by default some of them will not work. The base library itself will work fine out of the box. Beyond that, several API based vector databases work out of the box.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@adhityan adhityan

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@adhityan @fsa317