Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow running NPM tools from execroot #2297

Merged
merged 1 commit into from
Dec 18, 2020
Merged

feat: allow running NPM tools from execroot #2297

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 33 additions & 26 deletions internal/node/launcher.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,7 @@ RUN_LINKER=true
NODE_PATCHES=true
# TODO(alex): change the default to false
PATCH_REQUIRE=true
FROM_EXECROOT=false
for ARG in ${ALL_ARGS[@]+"${ALL_ARGS[@]}"}; do
case "$ARG" in
# Supply custom linker arguments for first-party dependencies
Expand All @@ -198,6 +199,8 @@ for ARG in ${ALL_ARGS[@]+"${ALL_ARGS[@]}"}; do
--nobazel_node_patches) NODE_PATCHES=false ;;
# Disable the linker pre-process (undocumented and unused; only here as an escape value)
--nobazel_run_linker) RUN_LINKER=false ;;
# If running an NPM package, run it from execroot instead of from external
--bazel_run_from_execroot) FROM_EXECROOT=true ;;
# Let users pass through arguments to node itself
--node_options=*) USER_NODE_OPTIONS+=( "${ARG#--node_options=}" ) ;;
# Remaining argv is collected to pass to the program
Expand Down Expand Up @@ -231,29 +234,6 @@ if [ "$NODE_PATCHES" = true ]; then
LAUNCHER_NODE_OPTIONS+=( "--require" "$node_patches_script" )
fi

if [ "$PATCH_REQUIRE" = true ]; then
require_patch_script=${BAZEL_NODE_PATCH_REQUIRE}
# See comment above
case "${require_patch_script}" in
# Absolute path on unix
/* ) ;;
# Absolute path on Windows, e.g. C:/path/to/thing
[a-zA-Z]:/* ) ;;
# Otherwise it needs to be made relative
* ) require_patch_script="./${require_patch_script}" ;;
esac
LAUNCHER_NODE_OPTIONS+=( "--require" "$require_patch_script" )
# Change the entry point to be the loader.js script so we run code before node
MAIN=$(rlocation "TEMPLATED_loader_script")
else
# Entry point is the user-supplied script
MAIN=TEMPLATED_entry_point_execroot_path
# TODO: after we link-all-bins we should not need this extra lookup
if [[ ! -f "$MAIN" ]]; then
MAIN=TEMPLATED_entry_point_manifest_path
fi
fi

# Tell the node_patches_script that programs should not escape the execroot
# Bazel always sets the PWD to execroot/my_wksp so we go up one directory.
export BAZEL_PATCH_ROOT=$(dirname $PWD)
Expand All @@ -274,14 +254,14 @@ if [[ "$PWD" == *"/bazel-out/"* ]]; then
echo "No 'bazel-out' folder found in path '${PWD}'!"
exit 1
fi
readonly execroot=${PWD:0:${index}}
export BAZEL_PATCH_GUARDS="${execroot}/node_modules"
EXECROOT=${PWD:0:${index}}
else
# We are in execroot or in some other context all together such as a nodejs_image or a manually
# run nodejs_binary. If this is execroot then linker node_modules is in the PWD. If this another
# context then it is safe to assume the node_modules are there and guard that directory if it exists.
export BAZEL_PATCH_GUARDS="${PWD}/node_modules"
EXECROOT=${PWD}
fi
export BAZEL_PATCH_GUARDS="${EXECROOT}/node_modules"
if [[ -n "${BAZEL_NODE_MODULES_ROOT:-}" ]]; then
if [[ "${BAZEL_NODE_MODULES_ROOT}" != "${BAZEL_WORKSPACE}/node_modules" ]]; then
# If BAZEL_NODE_MODULES_ROOT is set and it is not , add it to the list of bazel patch guards
Expand All @@ -291,6 +271,33 @@ if [[ -n "${BAZEL_NODE_MODULES_ROOT:-}" ]]; then
fi
fi

if [ "$PATCH_REQUIRE" = true ]; then
require_patch_script=${BAZEL_NODE_PATCH_REQUIRE}
# See comment above
case "${require_patch_script}" in
# Absolute path on unix
/* ) ;;
# Absolute path on Windows, e.g. C:/path/to/thing
[a-zA-Z]:/* ) ;;
# Otherwise it needs to be made relative
* ) require_patch_script="./${require_patch_script}" ;;
esac
LAUNCHER_NODE_OPTIONS+=( "--require" "$require_patch_script" )
# Change the entry point to be the loader.js script so we run code before node
MAIN=$(rlocation "TEMPLATED_loader_script")
else
# Entry point is the user-supplied script
MAIN=TEMPLATED_entry_point_execroot_path
# TODO: after we link-all-bins we should not need this extra lookup
if [[ ! -f "$MAIN" ]]; then
if [ "$FROM_EXECROOT" = true ]; then
MAIN="$EXECROOT/$MAIN"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does something break if we always use FROM_EXECROOT? do you understand what if anything still relies on the old path?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests seem to pass except for //internal/node/test:main_test when executed with bazel test; it works fine with bazel run. Maybe this is because paths change when running in the sandbox? The code could probably be adapted to also deal with that case

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay I think we should immediately attempt to get the default flipped in a PR following this one.

reasoning:

  • will have to wait 6mo otherwise
  • long flag flip rollouts are a bummer
  • in the meantime if other users hit the same bug you did it will be really hard for them to discover this opt-in flag that fixes it

else
MAIN=TEMPLATED_entry_point_manifest_path
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to make the same logical fix in the .bzl file that calls the launcher, rather than change the shell script? If it is, then that's the better layer to make the change

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be _nodejs_binary_impl. I wanted to reuse the logic to detect the execroot path, which is in the launcher. I'm not sure if there's another way to calculate it, or if it's even possible to move that logic to the rule implementation, since it seems to be based around detecting whether some paths exist.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexeagle any further comment here?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I think that makes sense.

fi
fi
fi

# The EXPECTED_EXIT_CODE lets us write bazel tests which assert that
# a binary fails to run. Otherwise any failure would make such a test
# fail before we could assert that we expected that failure.
Expand Down