Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add compile cache support to wd_?s_bundle, enable for node #3044

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

mikea
Copy link
Collaborator

@mikea mikea commented Nov 1, 2024

Extracted from #3004, not used on load yet.

downstream pr 9121

@mikea mikea requested review from jasnell and anonrig November 1, 2024 21:56
@mikea mikea requested review from a team as code owners November 1, 2024 21:56
@mikea
Copy link
Collaborator Author

mikea commented Nov 1, 2024

$ cat bazel-bin/src/node/node.capnp
@0xbcc8f57c63814005;

# generated by @workerd//build/wd_js_bundle.bzl

using Modules = import "/workerd/jsg/modules.capnp";

const nodeBundle :Modules.Bundle = (
  modules = [
    (name = "node:_stream_duplex", src = embed "node__stream_duplex", type = builtin, compileCache = embed "node__stream_duplex_cache", ),
    (name = "node:_stream_passthrough", src = embed "node__stream_passthrough", type = builtin, compileCache = embed "node__stream_passthrough_cache", ),
    (name = "node:_stream_readable", src = embed "node__stream_readable", type = builtin, compileCache = embed "node__stream_readable_cache", ),

.....

@mikea
Copy link
Collaborator Author

mikea commented Nov 1, 2024

$ du -b bazel-out/k8-fastbuild/bin/src/node/*_cache
552     bazel-out/k8-fastbuild/bin/src/node/node_assert_cache
560     bazel-out/k8-fastbuild/bin/src/node/node_assert_strict_cache
744     bazel-out/k8-fastbuild/bin/src/node/node_async_hooks_cache
1824    bazel-out/k8-fastbuild/bin/src/node/node_buffer_cache
6424    bazel-out/k8-fastbuild/bin/src/node/node_crypto_cache
3712    bazel-out/k8-fastbuild/bin/src/node/node_diagnostics_channel_cache
456     bazel-out/k8-fastbuild/bin/src/node/node_events_cache
3248    bazel-out/k8-fastbuild/bin/src/node/node-internal_constants_cache
3264    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_dh_cache
3328    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_hash_cache
1960    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_hkdf_cache
5456    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_keys_cache
1288    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_pbkdf2_cache
3304    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_random_cache
1584    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_scrypt_cache
1312    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_spkac_cache
1872    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_util_cache
4896    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_x509_cache
872     bazel-out/k8-fastbuild/bin/src/node/node-internal_debuglog_cache
8608    bazel-out/k8-fastbuild/bin/src/node/node-internal_events_cache
4528    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_assert_cache
2616    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_assertionerror_cache
18016   bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_buffer_cache
3568    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_comparisons_cache
1408    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_diffs_cache
13704   bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_errors_cache
18512   bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_inspect_cache
3936    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_path_cache
4936    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_querystring_cache
2400    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_stringdecoder_cache
5072    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_types_cache
3768    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_url_cache
1856    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_utils_cache
8440    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_zlib_base_cache
7640    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_zlib_cache
1904    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_zlib_constants_cache
5872    bazel-out/k8-fastbuild/bin/src/node/node-internal_legacy_url_cache
4168    bazel-out/k8-fastbuild/bin/src/node/node-internal_mock_cache
1528    bazel-out/k8-fastbuild/bin/src/node/node-internal_process_cache
3456    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_adapters_cache
1656    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_compose_cache
4280    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_duplex_cache
1360    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_legacy_cache
2920    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_pipeline_cache
1016    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_promises_cache
13928   bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_readable_cache
1896    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_transform_cache
8768    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_util_cache
8016    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_writable_cache
3320    bazel-out/k8-fastbuild/bin/src/node/node-internal_validators_cache
2552    bazel-out/k8-fastbuild/bin/src/node/node_module_cache
1440    bazel-out/k8-fastbuild/bin/src/node/node_path_cache
1456    bazel-out/k8-fastbuild/bin/src/node/node_path_posix_cache
1456    bazel-out/k8-fastbuild/bin/src/node/node_path_win32_cache
456     bazel-out/k8-fastbuild/bin/src/node/node_process_cache
1152    bazel-out/k8-fastbuild/bin/src/node/node_querystring_cache
3064    bazel-out/k8-fastbuild/bin/src/node/node_stream_cache
1176    bazel-out/k8-fastbuild/bin/src/node/node_stream_consumers_cache
784     bazel-out/k8-fastbuild/bin/src/node/node__stream_duplex_cache
592     bazel-out/k8-fastbuild/bin/src/node/node__stream_passthrough_cache
576     bazel-out/k8-fastbuild/bin/src/node/node_stream_promises_cache
960     bazel-out/k8-fastbuild/bin/src/node/node__stream_readable_cache
592     bazel-out/k8-fastbuild/bin/src/node/node__stream_transform_cache
2000    bazel-out/k8-fastbuild/bin/src/node/node_stream_web_cache
824     bazel-out/k8-fastbuild/bin/src/node/node__stream_writable_cache
640     bazel-out/k8-fastbuild/bin/src/node/node_string_decoder_cache
784     bazel-out/k8-fastbuild/bin/src/node/node_test_cache
2048    bazel-out/k8-fastbuild/bin/src/node/node_url_cache
4952    bazel-out/k8-fastbuild/bin/src/node/node_util_cache
2688    bazel-out/k8-fastbuild/bin/src/node/node_util_types_cache
13184   bazel-out/k8-fastbuild/bin/src/node/node_zlib_cache

@mikea mikea force-pushed the maizatskyi/2024-11-01-node-compile-caches branch from 06279dd to 5296fcb Compare November 1, 2024 22:12
Copy link
Member

@anonrig anonrig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't also enable it for consumption on load right? I think we should do it on the same PR.

@mikea
Copy link
Collaborator Author

mikea commented Nov 4, 2024

This doesn't also enable it for consumption on load right? I think we should do it on the same PR.

why? This thing is brittle so I'd like to land it together with EW side. Also as of now this is a no-op and has very low (0?) risk in production. Load is different and has to be treated differently. Makes all sense to me to break this down in separete PRs.

@mikea mikea force-pushed the maizatskyi/2024-11-01-node-compile-caches branch from 5296fcb to c3095e4 Compare November 4, 2024 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants