Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cherry-pick proto_lang_toolchain Starlarkfication and proto_common module #15854

Merged
merged 26 commits into from
Jul 14, 2022
Merged

Cherry-pick proto_lang_toolchain Starlarkfication and proto_common module #15854

merged 26 commits into from
Jul 14, 2022

Commits on Jul 14, 2022

  1. Put protoc label to a constant. 

    Renamed StrictProtoDepsViolationMessage to ProtoConstants and added protoc label there.

PiperOrigin-RevId: 409142099
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    d9b403e View commit details
    Browse the repository at this point in the history

  2. Remove proto_lang_toolchain rule's $(PLUGIN_OUT) placeholder and simp… 

    …lify ProtoCompileActionBuilder.

`$(OUT)` placeholder is replaced with `%s` so it can be directly used in addFormatted. This simplifies construction of proto compile action and makes it possible for Starlark version to have the same performance.

github shows no uses of the placeholder: https://github.com/search?q=%22%24%28PLUGIN_OUT%29%22&type=Repositories

PiperOrigin-RevId: 412286743
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    16b62f5 View commit details
    Browse the repository at this point in the history

  3. Add plugin_format_flag attribute to ProtoLangToolchainRule 

    This makes it possible to pair it with command_line attribute which could contain dependent data,
for example:

```
proto_lang_toolchain(
    name = "j2objc_proto_toolchain",
    blacklisted_protos = [],
    command_line = "--PLUGIN_j2objc_out=file_dir_mapping,generate_class_mappings:$(OUT)",
+   plugin_format_flag = "--plugin=protoc-gen-PLUGIN_j2objc=%s",
    plugin = "//third_party/java/j2objc:proto_plugin",
    runtime = "//third_party/java/j2objc:proto_runtime",
    visibility = ["//visibility:public"],
)
```

It also retains reference to the flag on proto_lang_toolchain rule and so doesn't cause a memory regression when proto_lang_libraries are Starlarkyfied.

PiperOrigin-RevId: 412287195
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    4158e52 View commit details
    Browse the repository at this point in the history

  4. Proxy proto compiler and proto opts over proto_lang_toolchain rule. 

    This is needed for the proto_common.generate_code function.

This change doesn't modify any of proto_lang_toolchain public attributes.

PiperOrigin-RevId: 437713619
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    4099a43 View commit details
    Browse the repository at this point in the history

  5. Extend proto_lang_toolchain rule with progress_message and mnemonic a… 

    …ttributes.

This is needed for proto_common.generate_sources function.

Two public attributes are added to proto_lang_toolchain rule. Both have defaults, so no immediate migration of targets is needed.

PiperOrigin-RevId: 437714094
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    d53efdd View commit details
    Browse the repository at this point in the history

  6. Starlarkify proto_lang_toolchain and ProtoLangToolchainInfo provider 

    Added StarlarkProtoLangToolchainTest which uses starlarkified rule for verification.

I've deleted blacklisted_protos and forbidden_protos since they are not needed anymore.

PiperOrigin-RevId: 444223497
    @kotlaja @comius
    kotlaja authored and comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    b0181e7 View commit details
    Browse the repository at this point in the history

  7. Roll forward of bazelbuild@ae349e9: Export ProtoLangToolchainInfo pro… 

    …vider and flip proto_lang_toolchain rule

NEW: I've moved ProtoLangToolchainProvider from providers.bzl file to proto_common.bzl file since proto_common doesn't allow loading other components. I've also deleted providers.bzl file since we don't need it anymore.

Automated rollback of commit 5a6b1a8.

*** Reason for rollback ***

Good to submit since the blocking error has been resolved.

*** Original change description ***

Automated rollback of commit ae349e9.

*** Reason for rollback ***

This CL breaks proto_common.bzl file which seems that can't load providers.bzl file.

*** Original change description ***

Export ProtoLangToolchainInfo provider and flip proto_lang_toolchain rule

I’ve exported ProtoLangToolchainInfo provider from it’s native class by adding two new functions: one that’s creating starlark provider (create function), and the other that’s wrapping the starlark provider as a nat

***

PiperOrigin-RevId: 446401388
    @kotlaja @comius
    kotlaja authored and comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    92b9518 View commit details
    Browse the repository at this point in the history

  8. Use data from transitive_proto_sources instead of transitive_sources … 

    …in proto_lang_toolchain.

The problem with latter is, that transitive_source can contain "renamed" files (in _virtual_includes subdirectory), which doesn't work for the detection that needs original files (ProtoSource.original_source_file).

PiperOrigin-RevId: 446924924
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    1682183 View commit details
    Browse the repository at this point in the history

  9. Remove allow_files from proto_lang_toolchain's attributes 

    It's the same behaviour in native code.

PiperOrigin-RevId: 446988764
    @kotlaja @comius
    kotlaja authored and comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    fa217b1 View commit details
    Browse the repository at this point in the history

  10. Remove native implementation of proto_lang_toolchain rule 

    PiperOrigin-RevId: 446995789
    @kotlaja @comius
    kotlaja authored and comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    4c0640b View commit details
    Browse the repository at this point in the history

  11. Fix name in proto_lang_toolchain rule 

    Added a wrapper for proto_lang_toolchain in order to have two implementations - one with public proto_compiler attribute, and the other one with the private one.
Restructured proto_lang_toolchain rules' implementation - merged two rule's definitions into one.

PiperOrigin-RevId: 447016335
    @kotlaja @comius
    kotlaja authored and comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    a14e613 View commit details
    Browse the repository at this point in the history

  12. Cherry-pick missing things.

    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    cc6c523 View commit details
    Browse the repository at this point in the history

  13. Separate ExecException.java from main actions target. 

    PiperOrigin-RevId: 413105213
    @comius
    wilwell authored and comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    d2ded6c View commit details
    Browse the repository at this point in the history

  14. ResourceSet in StarlarkAction API 

    Added optional `resource_set` parameter to `run` and `run_shell` in StarlarkActionApi. `resource_set` is `StarlarkCallable` object that returns dict with resource set (cpu, memory, local_test).

PiperOrigin-RevId: 415224490
    @comius
    wilwell authored and comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    d887fe4 View commit details
    Browse the repository at this point in the history

  15. Implement and expose proto_common.compile call. 

    Design doc: https://docs.google.com/document/d/1dY_jfRvnH8SjRXGIfg8av-vquyWsvIZydXJOywvaR1A/edit

PiperOrigin-RevId: 440098122
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    193399d View commit details
    Browse the repository at this point in the history

  16. Fix ProtoCommon tests.

    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    0049701 View commit details
    Browse the repository at this point in the history

  17. Add experimental_progress_message parameter to proto_common.compile. 

    This will make migration to the new call easier (because we don't have progress_message set yet on the proto_lang_toolchain rules).

Design doc: https://docs.google.com/document/d/1dY_jfRvnH8SjRXGIfg8av-vquyWsvIZydXJOywvaR1A/edit

PiperOrigin-RevId: 440098602
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    46f0658 View commit details
    Browse the repository at this point in the history

  18. Implement proto_common.experimental_should_generate_code. 

    Design doc: https://docs.google.com/document/d/1dY_jfRvnH8SjRXGIfg8av-vquyWsvIZydXJOywvaR1A/edit

PiperOrigin-RevId: 440109298
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    c4b8b95 View commit details
    Browse the repository at this point in the history

  19. Implement proto_common.declare_generated_files. 

    Design doc: https://docs.google.com/document/d/1dY_jfRvnH8SjRXGIfg8av-vquyWsvIZydXJOywvaR1A/edit

PiperOrigin-RevId: 441097041
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    600bd5e View commit details
    Browse the repository at this point in the history

  20. Fix docstring in proto_common 

    PiperOrigin-RevId: 445149402
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    8b8f4e8 View commit details
    Browse the repository at this point in the history

  21. Add proto_info parameter to proto_common.compile 

    Adding it will make it possible to migrate the uses from proto_library_target to proto_info. When the migration is done proto_library_target will be removed.

The cost of changing this now is still low, because it's not yet released/used in Bazel.

PiperOrigin-RevId: 455616355
Change-Id: Ieb0f03b0600e1f90b72a61f90420675075c79a9e
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    80d9ae2 View commit details
    Browse the repository at this point in the history

  22. Migrate proto_library_target to proto_info in proto_common.declare_ge… 

    …nerated_files.

PiperOrigin-RevId: 456475071
Change-Id: I2882d80cd4f7fdd9b8dcba3347930eaf1f194d0a
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    bc452c1 View commit details
    Browse the repository at this point in the history

  23. Migrate proto_library_target to proto_info in proto_common.experiment… 

    …al_should_generate_code

PiperOrigin-RevId: 460162832
Change-Id: I57a6fa4c6e6c9618cf9edb8518e17b46fc90be9f
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    0b381ac View commit details
    Browse the repository at this point in the history

  24. Remove proto_library_target from proto_common 

    PiperOrigin-RevId: 460406536
Change-Id: I10021f32fb40e163ded02ebab8297902b63760fa
    @comius
    comius committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    3b78ca9 View commit details
    Browse the repository at this point in the history

  25. Merge branch 'release-5.3.0' into cherry-pick-proto_lang_toolchain

    @ckolli5
    ckolli5 authored Jul 14, 2022
    Configuration menu
    Copy the full SHA
    18ae08c View commit details
    Browse the repository at this point in the history

  26. Merge branch 'release-5.3.0' into cherry-pick-proto_lang_toolchain

    @ckolli5
    ckolli5 authored Jul 14, 2022
    Configuration menu
    Copy the full SHA
    e5427b2 View commit details
    Browse the repository at this point in the history