chore(NA): @kbn/pm new commands to support development on Bazel packages

[mistic]

One step forward on #69706

That PR should close the gap about the problem raised by @spalger at #96227 (comment)

It introduces two new commands (yarn kbn build-bazel and yarn kbn watch-bazel) that can be used to build and watch packages already migrated into Bazel. After discussing with @spalger and @tylersmalley we've decided to go that simple way for now and evaluate how well it work for us. Additionally I've also added warnings both on yarn kbn run and yarn kbn watch.

It already adds documentation on the getting started development guide about this matter.

[elasticmachine]

Pinging @elastic/kibana-operations (Team:Operations)

[jbudz]

I tested process watching and building on Windows and didn't run into any issues. I know tylersmalley and spalger were more involved architecturally so leaving this as a comment.

[spalger]

I'm on board with this approach to smart bazel commands that just build what's necessary, but I find the log output to be pretty sub-par. Can we spend a little time on the developer experience here before we start transitioning everything over to this?

yarn kbn build-bazel

Success case:

I'd really like to know more about what was built, ideally broken down by package, and less about the inner workings about Bazel.

Error case:

This error output is much worse than the (admittedly not great) error output currently produced by kbn/pm:

I really think we should be doing better than we are currently, or at least have a plan for how we're going to do better.

yarn kbn watch-bazel

Success case:

I'm a little concerned that it takes 20 seconds to initialize the environment so that we can run the build just for @elastic/datemath. I'm sure some of the problem is that we're calling yarn install in order to load the packages, but is that 20 seconds going to turn into 100 seconds when we have everything in the repo running in bazel?

Also, running yarn install twice is very strange, and again, the output is almost completely irrelevant to the package author and is instead all focused on the Bazel internals, which I don't really like.

This is all in comparison to this on master, which takes about 2 seconds:

Error case:

This suffers from the same problems as the standard build, the error is totally buried by irrelevant logs

[mistic]

@spalger I feel like I've address all of your feedback except the error case logging clutter. I'll take a better look on Monday about what our options are for the error logs when building the packages but I feel like for now that is probably the best we can do without spending too much time on replacing the pkg_npm rule.

[mistic]

@elasticmachine merge upstream

[alexeagle]

Hey, just dropping by to say it's great you're adopting Bazel + rules_nodejs and I'm interested to improve the developer experience to make the logging work the way you need :)

[mistic]

@alexeagle thanks for stopping by ❤️ ! I'll work with you offline in order to give you an idea around where we think logging could be improved 👍

[mistic]

@elasticmachine merge upstream

[spalger]

The latest changes reduce the time watch-bazel takes to initialize and include a summary of the packages that were built in build-bazel. If we can also tell ts_project to use tsc --pretty and remove the double logging happening on errors in build-bazel then I think we would have implemented all that we can to improve the logging output from our end.

I still think there's a lot of noise in these logs coming from Bazel but I'm on board with iterating on that with the Bazel team, especially in these areas:

1. I think that green highlights are best reserved for success messages, rather than used for every INFO: message
   

2. Parts of this error are truncated to avoid being too verbose, but I think the truncation is more clever than useful. I'd argue that Bazel should just strip the CLI info from this and focus on the most informative bits, then leave the other info to more verbose logging levels

    info [bazel] ERROR: /Users/spalger/kbn-dev/master/kibana/packages/elastic-datemath/BUILD.bazel:41:11: Compiling TypeScript project //packages/elastic-datemath:tsc [tsc -p packages/elastic-datemath/tsconfig.json] failed: (Exit 2): tsc.sh failed: error executing command bazel-out/host/bin/external/npm/typescript/bin/tsc.sh --pretty --project packages/elastic-datemath/tsconfig.json --outDir bazel-out/darwin-fastbuild/bin/packages/elastic-datemath/target --rootDir ... (remaining 6 argument(s) skipped)
    info [bazel]
    info [bazel] Use --sandbox_debug to see verbose messages from the sandbox tsc.sh failed: error executing command bazel-out/host/bin/external/npm/typescript/bin/tsc.sh --pretty --project packages/elastic-datemath/tsconfig.json --outDir bazel-out/darwin-fastbuild/bin/packages/elastic-datemath/target --rootDir ... (remaining 6 argument(s) skipped)
    info [bazel]
    info [bazel] Use --sandbox_debug to see verbose messages from the sandbox
   

   vs

   info [bazel] ERROR: Compiling TypeScript project //packages/elastic-datemath:tsc failed: (Exit 2)
   info [bazel] Use --sandbox_debug to see verbose messages from the sandbox
   

3. I'm probably wrong here but my understanding is that --sandbox_debug impacts the way things are handled on the filesystem which seems like a lot more than a flag enabling "verbose messages". Additionally, it doesn't work with iBazel, and is highlighted with empty lines which make it stand out a lot, resulting in more noise.

[mistic]

@spalger @tylersmalley I think I've mainly walked across all the left feedback and I really think we are in a much better place there we were a couple days before. I will raise the rest of the logging concerns within the Bazel team as discussed and I hope we can continue to improve on this field. Thank you very much for all your feedback and inputs and thanks for helping this project getting even better @spalger

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 1d83240
• Storybooks Preview
• Documentation Changes

Metrics [docs]

✅ unchanged

History

• 💚 Build #118892 succeeded 7950d47
• 💚 Build #118883 succeeded 8623196
• 💚 Build #118862 succeeded b610ca7
• 💚 Build #118840 succeeded 03d5744
• 💚 Build #118668 succeeded 13d3394

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[kibanamachine]

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.