google-cloud/profiler failing to install on MacOS with Node v13.8.0

[drveresh]

Hi.

Unable to install "@google-cloud/profiler" for cloud functions. I tried to reinstall, clean cache and other trials, but nothing seems to be working. Initially, it was failing with " node-pre-gyp", but later got it installed with the version "0.14.0", but the issue remain same.

I really appreciate your help to move forward.

Environment details

• OS: Mac Catalina, 10.15.3 (19D76)
• Node.js version: v13.8.0
• npm version: 6.13.6
• @google-cloud/profiler version: 4.0.0

ERRORS

HOST:functions XYZ$ npm install @google-cloud/profiler

> pprof@2.0.0 install /Users/XYZ/PATH/functions/node_modules/pprof
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download 
node-pre-gyp WARN Tried to download(404): https://storage.googleapis.com/cloud-profiler/pprof-nodejs/release/v2.0.0/node-v79-darwin-x64-unknown.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for pprof@2.0.0 and node@13.8.0 (node-v79 ABI, unknown) (falling back to source compile with node-gyp) 
No receipt for 'com.apple.pkg.CLTools_Executables' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLILeo' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLI' found at '/'.

gyp: No Xcode or CLT version detected!
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:351:16)
gyp ERR! stack     at ChildProcess.emit (events.js:321:20)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)
gyp ERR! System Darwin 19.3.0
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/Users/XYZ/PATH/functions/node_modules/pprof/build/node-v79-darwin-x64-unknown/pprof.node" "--module_name=pprof" "--module_path=/Users/XYZ/PATH/functions/node_modules/pprof/build/node-v79-darwin-x64-unknown" "--napi_version=5" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v79"
gyp ERR! cwd /Users/XYZ/PATH/functions/node_modules/pprof
gyp ERR! node -v v13.8.0
gyp ERR! node-gyp -v v5.0.5
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/Users/XYZ/PATH/functions/node_modules/pprof/build/node-v79-darwin-x64-unknown/pprof.node --module_name=pprof --module_path=/Users/XYZ/PATH/functions/node_modules/pprof/build/node-v79-darwin-x64-unknown --napi_version=5 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v79' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/XYZ/PATH/functions/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:321:20)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:1026:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)
node-pre-gyp ERR! System Darwin 19.3.0
node-pre-gyp ERR! command "/usr/local/bin/node" "/Users/XYZ/PATH/functions/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /Users/XYZ/PATH/functions/node_modules/pprof
node-pre-gyp ERR! node -v v13.8.0
node-pre-gyp ERR! node-pre-gyp -v v0.14.0
node-pre-gyp ERR! not ok 
Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/Users/XYZ/PATH/functions/node_modules/pprof/build/node-v79-darwin-x64-unknown/pprof.node --module_name=pprof --module_path=/Users/XYZ/PATH/functions/node_modules/pprof/build/node-v79-darwin-x64-unknown --napi_version=5 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v79' (1)
npm WARN acorn-import-meta@1.0.0 requires a peer of acorn@^6.0.0 but none is installed. You must install peer dependencies yourself.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! pprof@2.0.0 install: `node-pre-gyp install --fallback-to-build`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the pprof@2.0.0 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/XYZ/.npm/_logs/2020-04-22T01_51_21_535Z-debug.log

Steps to reproduce

$npm install @google-cloud/profiler

Thanks!

[nolanmar511]

Breaking this answer up a bit, because there are a couple disparate things to note.

Using the module on MacOS

This module depends on a module with a native component (pprof). While this native module has pre-built binaries available for Linux and Linux Alpine, we do not have pre-built binaries available for Windows or MacOS, so additional dependencies are required on those operating systems.

We use node-gyp to build the native portion of the module. Instructions for using node-gyp on MacOS are available here. Following those instructions to ensure the necessary dependencies are available would be a first step for install on MacOS, but we don't have any unit or integration testing on MacOS right now, so there might be more issues after that -- please do post here if there are.

I think Cloud Functions is run on Linux VMs -- so adding profiler as a dependency there shouldn't be an issue.

Should this module be used with Cloud Functions? (Maybe, but with some caveats)

But, since you're using cloud functions, there are a few other things I should note. First, Cloud Profiler collects one profile (of each type) per minute per deployment (defined here). These leads to a few problems when using it with Cloud profiler: unless the function is pretty long-lived (a few minutes), profiles may not be collected at all; and the profiler will need to be re-initialized on each call to the function, so profiling will be higher overhead than it would be otherwise (the shorter the function, the higher the overhead). Because of this, Profiler is not supported for Cloud Functions.

Still, there may be value in using Profiler with Cloud Functions (if an application's performance has not been measured in production, there's often a lot of low-hanging fruit to be found for performance optimizations) -- a shorter experiment, rather than enabling profiler indefinitely is probably better in this case.

[drveresh]

Thanks for the details, and I hope this OS dependency will be removed sooner for the better build experience.

Also, I was able to install everything and set up profiling for cloud functions, but it didn't track anything useful, and 98% of the events belong to internal NodeJs APIs and some system related. Ideally, I was expecting that it would track app-specific functions against CPU, Memory, I/O, Network, etc., but it didn't and it is hardly useful.

I hope the Firebase team considers this request and build an easy tool to evaluate cloud functions.

[nolanmar511]

Thanks for the details, and I hope this OS dependency will be removed sooner for the better build experience.

Unfortunately, the OS-dependency cannot be removed at this time. There is an open feature request for pre-built binaries for Mac OS (google/pprof-nodejs#33); but because profiling requires depending on a module with a native component, the OS dependency cannot be removed completely.

Also, I was able to install everything and set up profiling for cloud functions, but it didn't track anything useful, and 98% of the events belong to internal NodeJs APIs and some system related. Ideally, I was expecting that it would track app-specific functions against CPU, Memory, I/O, Network, etc., but it didn't and it is hardly useful.

Our team is continuing to work on improving this produce, and so we hope to add support for new platforms soon.

This documentation page introduces Cloud Profiler, including what profile types are collected for each language and what platforms are supported. It might details that would allow one to decide whether or not Cloud Profiler could be helpful for a specific use case. In some cases filtering can be used to view only sections of the profile currently of interest, though, depending on the use case, filtering may or may not be helpful.

I hope the Firebase team considers this request and build an easy tool to evaluate cloud functions.

I'd recommend you reach out to the Firebase team for any Firebase feature requests.

[nolanmar511]

I'm going to mark this issue as closed.

Please feel free to re-open this issue or file another issue if additional follow-up is necessary.