Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: improve build performance #27924

Closed
bnoordhuis opened this issue May 27, 2019 · 4 comments
Closed

build: improve build performance #27924

bnoordhuis opened this issue May 27, 2019 · 4 comments
Labels
build Issues and PRs related to build files or the CI. performance Issues and PRs related to the performance of Node.js.

Comments

@bnoordhuis
Copy link
Member

A no-op build (no files changed) currently takes about 25 seconds. It should be < 1 second.

$ time make V=
/Applications/Xcode.app/Contents/Developer/usr/bin/make -C out BUILDTYPE=Release V=
  TOUCH 164b55c604d79564ccc044dad7a9a348370196f7.intermediate
  ACTION _Users_bnoordhuis_src_v1_x_tools_v8_gypfiles_v8_gyp_v8_base_target_run_torque_action 164b55c604d79564ccc044dad7a9a348370196f7.intermediate
  TOUCH 01e7f2a1286937e70d628711040382e58d32c150.intermediate
  ACTION Generating inspector protocol sources from protocol json 01e7f2a1286937e70d628711040382e58d32c150.intermediate
  TOUCH 964af9128e18298610dbd37c8e7efa19e756f179.intermediate
  ACTION Generating node protocol sources from protocol json 964af9128e18298610dbd37c8e7efa19e756f179.intermediate
  TOUCH ed07fabe6d2426fb5f658fe6fc03c566e72b1ad9.intermediate
  ACTION generating: "/Users/bnoordhuis/src/v1.x/out/Release/obj.target/v8_snapshot/geni/embedded.S" "/Users/bnoordhuis/src/v1.x/out/Release/obj.target/v8_snapshot/geni/snapshot.cc" ed07fabe6d2426fb5f658fe6fc03c566e72b1ad9.intermediate
  CC(target) /Users/bnoordhuis/src/v1.x/out/Release/obj.target/generate_snapshot/geni/embedded.o
  CXX(target) /Users/bnoordhuis/src/v1.x/out/Release/obj.target/generate_snapshot/geni/snapshot.o
  LIBTOOL-STATIC /Users/bnoordhuis/src/v1.x/out/Release/libgenerate_snapshot.a
  LINK(target) /Users/bnoordhuis/src/v1.x/out/Release/cctest
  LINK(target) /Users/bnoordhuis/src/v1.x/out/Release/mkcodecache
  ACTION _Users_bnoordhuis_src_v1_x_node_gyp_node_target_run_mkcodecache /Users/bnoordhuis/src/v1.x/out/Release/obj/gen/node_code_cache.cc
  CXX(target) /Users/bnoordhuis/src/v1.x/out/Release/obj.target/node/gen/node_code_cache.o
  LINK(target) /Users/bnoordhuis/src/v1.x/out/Release/node
  LINK(target) /Users/bnoordhuis/src/v1.x/out/Release/node_mksnapshot
rm ed07fabe6d2426fb5f658fe6fc03c566e72b1ad9.intermediate 964af9128e18298610dbd37c8e7efa19e756f179.intermediate 01e7f2a1286937e70d628711040382e58d32c150.intermediate 164b55c604d79564ccc044dad7a9a348370196f7.intermediate
if [ ! -r node -o ! -L node ]; then ln -fs out/Release/node node; fi

real    0m25.788s
user    0m22.921s
sys     0m2.474s

(This is with warmed up disk caches. A cold build is a little slower still.)

It does a lot of things it doesn't need to do, like:

  1. Run torque for some reason.
  2. Regenerate the inspector protocol files.
  3. Recreate the snapshot.
  4. Recreate the code cache.
  5. Relink cctest.

The first four should only run if the relevant files changed. cctest should only be built/linked when executing a test target (e.g. make test.)

@bnoordhuis bnoordhuis added build Issues and PRs related to build files or the CI. performance Issues and PRs related to the performance of Node.js. labels May 27, 2019
@targos
Copy link
Member

targos commented May 27, 2019

/cc @refack

@addaleax
Copy link
Member

I think this is a duplicate of #25135?

@refack
Copy link
Contributor

refack commented May 27, 2019

Yeah, I'm painfully aware of this :( It's also the root cause of #22006.

FWIW building with ninja does not manifest this bug, now (#27504) integrated with the main Makefile by simply configuring with ./configure --ninja.

For make the WIP is at refack/GYP3#32
For MSBuild the tracking is in my head.

FTR: #23982 would have brute-forced this bug out.

@bnoordhuis
Copy link
Member Author

I'll close this for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI. performance Issues and PRs related to the performance of Node.js.
Projects
None yet
Development

No branches or pull requests

4 participants