consider ways to configure (conflicting) codegen implementations

[tdyas]

In #14030, the point came up that certain codegen implementation might conflict with one another if activated at the same time. In the case of that PR, the conflicting implementations would be the Apache Thrift and Scrooge Thrift code generators for Java.

Ideas for how to configure:

1. Have specific backends be aware of it each other and error if the other is configured. This might lead to circular dependency issues.
2. Add skip_GEN_NAME fields to the applicable target types (e.g., thrift_sources) and have the codegen backends not run if their specific skip field is set. This is brittle to new codegen backends being enabled since it might require also updating lots of targets to have a skip field for the new backend.
3. Add a generators field to the applicable target types (e.g., thrift_sources) that takes a list of strings that are generator names. Only those specific generators will be enabled for that target. This is probably more scalable than (2) since its semantics would not change if an additional codegen backend was enabled. Plus it will fail if a configured generator has its codegen backend disabled.
4. Use labelled dependencies to allow the usage sites to configure which generator to use via a dependency label. For example, the dependency could be dependencies=["src/thrift/foo!generator=scrooge_java"]. This can be duplicative if every usage site had to declare the generator label versus just putting fields on the thrift_sources. Moreover, it would not work with dependency inference unless there were a way to specify which generator to use which leads back to (2) and (3). It could be useful as an override syntax though.
5. (Re-?)introduce language-specific codegen target types. So there would still be protobuf_sources to configure Protobuf-specific options. But there would also be python_protobuf_sources which would depend on a specific protobuf_sources target via a source_target attribute (which would be inferred). This is where the python_source_root field would live. That could have a generator attribute to select which codegen backend to use where that makes sense (i.e., a java_protobuf_sources could select from Apache Thrift or Scrooge). This would allow having multiple LANG_protobuf_sources targets if there were a need to vary options. Consuming targets could then override dependency inference if there were any ambiguity in the usual way.

[tdyas]

Related point: We currently inject dependencies for codegen regardless of whether the generator is used or not. That behavior will need to match whatever is chosen for codegen generally. Alternatively, consider whether GeneratedSources should carry the runtime dependencies as an additional field so there is no need to use InjectDependenciesRequest as it is currently being used.

[tdyas]

Added an additional point (5) regarding language-specific codegen target types.

[tdyas]

Multiple generators enabled is causing an error for me in practice:

18:44:40.46 [ERROR] 1 Exception encountered:

  AmbiguousCodegenImplementationsException: Multiple of the registered code generators can generate PythonSourceField from ProtobufSourceField. It is ambiguous which implementation to use.

Possible implementations:

  * GenerateJavaFromProtobufRequest
  * GeneratePythonFromProtobufRequest

I have both java and python protobuf codgen backends enabled in my test repo, but there is no Java code written yet. This failure occurs when just testing the Python code with ./pants_from_sources test src/python/protobuf_example::

[Eric-Arellano]

Hmmm, that sounds like a bug. GenerateJavaFromProtobufRequest shouldn't be claiming to generate PythonSource. I'll investigate right now.

[tdyas]

class GenerateJavaFromProtobufRequest(GenerateSourcesRequest):
    input = ProtobufSourceField
    output = PythonSourceField

my bug ... ugh

[tdyas]

This issue impacts #14258 which describes supporting compiled languages in codegen.

[stuhood]

#14694 is a dupe of this one, but was opened after consensus was reached on adding a generator(s) field. Closing in favor of that one.