Allowing cc_library to depend on android_library, like with objc_library

[cpsauer]

Hello, awesome Bazel folks,

This one's a feature request: It would be awesome if a cc_library could depend on an android_library, just as a cc_library can depend on an objc_library.

The underlying problem I'm trying to solve (with Bazel in general, really) is creating a cross-platform library that's mostly shared native code. That native code needs to call some platform-specific code. This works quite well on, e.g., Apple platforms, since you can just select() the right objc_library to depend on for a given platform, and makes me very happy that I'm using Bazel.

Using Android, however, is a pain point for these cross-platform libraries. You often need to call into Java using the JNI, but, at least to the best of my knowledge, you can't really model those C++ -> Android (aar/java) dependencies in Bazel, because cc_libraries can't depend on android_libraries. So you're stuck having two separate dependency trees going down to the root that can get out of sync: one for java and one for C++.

I think it's going to be possible hide the ugliness with a macro that constructs the two separate graphs in parallel, but that's more of summoning up a horrible hack than an elegant Bazel solution. Plus, I bet having the C++ depend on the .so and .h in the aar is going to be more of an adventure. I'm envisioning having to extract them and suppress linking, like in https://github.com/bazelbuild/bazel/blob/master/tools/android/aar_resources_extractor.py. If others want these hacks, happy to share em once I finish, just lmk.

Some links that shed more light on what's needed:

• Android Studio/Gradle seem to be working on solving this problem with prefabs.
  • That illuminates some of the structure of the problem: Specifying dependencies in the graph, exposing headers, and doing the right linking checking and bundling with .so's. Obviously they aren't going for the cross-platform use case Bazel is.
• You can see fbjni grappling with similar issues here.
• As are tensorflow users...where the recommended fix is to ditch bazel for make...: How to enable C++ API usage from AAR builds (and integrate into gradle project)? tensorflow/tensorflow#39148
• The golang folks are grappling with a similar issue, which they (delightfully) call the C++ sandwich :) #2176

It'd be great to solve this problem in Bazel, and model cross-platform build graphs elegantly!

Thanks so much for your consideration,
Chris

[aiuto]

Note: I assigned this to team-Android, but team-Rules-CPP or configurability might find it interesting.

@cpsauer I never found this a particular problem. In my context, I built a C++ library portable to iOS, android, linux, macos, and windows. It needed to download URLs so it required a platform specific implementation for each. It was much like you said. Most platforms would depend on a C++ API, and we would select() the correct implementation by platform. For the Android one though, I didn't see a need to depend on the android library. I depended on a cc_library shim that impemented the download API and upcalled to JNI.

When the application started it would call an initializer in the shim library to pass in a context that the library could reuse to construct the JNI upcall in the future. It was always the Java code in control of the relationship. so the android_library could always depend down to the cc_ilbrary, just like on the other platforms.

I have not fully read all your pointers. It seems like you are hinting that you want .h files exported from the android_library to be available to the cc_library. But I don't see how or why.

[cpsauer]

Thanks, @aiuto. Any chance that library is public? Would love to see/learn from how you did things. Keen observations all around.

I hear you on your skepticism. I do think there's a need here, though, when you have C++ code calling into Java instead of the other way around. When the dependency graph gets deep, it's really nice to be able to model dependencies as they actually are, with C++ depending on the platform-specific language (Java in this case, but objc in the iOS case). Totally agree you can pretend the dependency goes the other way when it doesn't (or just have both Java and C++ libraries hanging separately off the root android target) but this gets uglier the deeper the dependencies get; it's nice to have Bazel directly model the dependences that exist in the codebase.

You're right on the pointers entailing a need to depend on the native interfaces of aars. After hacking and thinking some more here, there are actually two sub-needs:
(1) The first we've discussed the most, is being able to have cc_libraries depend on android_libraries in the Bazel graph, and having everything get linked/bundled together correctly in the output when you're compiling from source. Since the android_library might in turn depend on cc_libraries, it needs to expose their headers.
(2) The second thing is being able to to have cc_libraries depend on prebuilt android libraries (aar_imports) that export a native interface. These aars are basically modeling a C++ -> Java dependency internally, hence a native interface bundled with a Java implementation. You're totally right that this need is in each of those links: the Android Studio folks are working hard on this with Prefabs; the C++ -> Java aars, like fbjni, are using it to model their dependencies; and the tensorflow user (pre-prefab) wishes he could depend on the native interface out of the aar. There are three sub-sub-parts of supporting prefabs/native interfaces from aar's: being able to have cc_library -> aar_import, exposing the aar's prefab (or other) headers to the cc_library, and linking the aar's .so's into the cc_library's output in the end.

[aiuto]

I can't make what I have done public. It's part of a few products. The way I think about it is the `deps` of a cc_library should be to things you need for C++ linkage. When C++ calls to Java through JNI you end up looking up the things dynamically. I think of it as a data dependency. But that simple case aside, I have not had a need for android_librarys that export a native interface. So maybe I just don't fully appreciate your need.

…

On Tue, Feb 23, 2021 at 9:19 PM Christopher Sauer ***@***.***> wrote: Thanks, @aiuto <https://github.com/aiuto>. Any chance that library is public? Would love to see/learn from how you did things. Keen observations all around. I hear you on your skepticism. I do think there's a need here, though, when you have C++ code calling into Java instead of the other way around. When the dependency graph gets deep, it's really nice to be able to model dependencies as they actually are, with C++ depending on the platform-specific language (Java in this case, but objc in the iOS case). Totally agree you can pretend the dependency goes the other way when it doesn't (or just have both Java and C++ libraries hanging separately off the root android target) but this gets uglier the deeper the dependencies get; it's nice to have Bazel directly model the dependences that exist in the codebase. You're right on the pointers entailing a need to depend on the native interfaces of aars. After hacking and thinking some more here, there are actually two sub-needs: (1) The first we've discussed the most, is being able to have cc_libraries depend on android_libraries in the Bazel graph, and having everything get linked/bundled together correctly in the output when you're compiling from source. Since the android_library might in turn depend on cc_libraries, it needs to expose their headers. (2) The second thing is being able to to have cc_libraries depend on prebuilt android libraries (aar_imports) that export a native interface. These aars are basically modeling a C++ -> Java dependency internally, hence a native interface bundled with a Java implementation. You're totally right that this need is in each of those links: the Android Studio folks are working hard on this with Prefabs; the C++ -> Java aars, like fbjni, are using it to model their dependencies; and the tensorflow user (pre-prefab) wishes he could depend on the native interface out of the aar. There are three sub-sub-parts of supporting prefabs/native interfaces from aar's: being able to have cc_library -> aar_import, exposing the aar's prefab (or other) headers to the cc_library, and linking the aar's .so's into the cc_library's output in the end. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13092 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXHHHB3CSIJ62GLQCTLXPTTARO3ZANCNFSM4YBTJWQA> .

[ahumesky]

I'm a little confused on what you're trying to do exactly.

You often need to call into Java using the JNI, but, at least to the best of my knowledge, you can't really model those C++ -> Android (aar/java) dependencies in Bazel, because cc_libraries can't depend on android_libraries. So you're stuck having two separate dependency trees going down to the root that can get out of sync: one for java and one for C++.

So I get the "Java / android_library depending on cc library" part, because in the Java code you declare native methods and the native implementations need to be loaded. That's why in Bazel you do
android_binary -> android_library -> cc_library.

And I know that JNI code can call Java methods. But what does it mean for a cc_library to depend on Java code? I know for example that there's javah / javac -h, so are you looking to generate native code, or validate the Java calls that native code does, based on the Java code? (and aars contain jars, so validation would need to read bytecode too)

But to reply to the request itself, the problem with a cc_library depending on an android_library or an aar_import is that all the other "Android stuff" gets dropped -- cc_library won't know what to do with the Java providers and Android resource providers and proguard providers, etc, and any of the other things that might get added one day [1]. There isn't any sort of generic mechanism for automatically propagating providers, so cc_library would have to explicitly know about all these Android-specific things, which is not very desirable. So you end up having to create parallel dependency trees anyway.

[1] For some back story, internally we (not really intentionally) allowed java_library to depend on android_library, and the same problem occurred: java_library only saw the Java providers from the android_library and dropped everything else, and it was confusing for users and a pain to clean up.

[cpsauer]

Hey all, thanks for reading. I think I've explained badly enough that we're talking past each other a bit. To fix that, I propose we take a more concrete journey together through the use case.

Say you've got a sizable chunk of C++ code. Maybe its C++ dependency graph looks like

cc_library("A") -> cc_library("B") -> cc_library("C")
              |
               -> cc_library("B2")
              |
               -> ...and a whole lot more, with deep branching.

And your code depends on calls into the operating system, for something like the app name, or resources, or the screen resolution or whatever. If there's a C/C++ API for those platform-specific needs (Linux, Windows, etc.), great, you're all set, you just select in some more cc_libraries for those platforms. If there isn't, you need your C++ to be able to call into whatever platform language there is. On Apple platforms, this is easy peasy. Your graph (after selects) becomes:

Depended upon by an ios_framework, somewhere several layers up:

cc_library("A") -> cc_library("B") -> cc_library("C") -> objc_library("D")
              |
               -> cc_library("B2") -> objc_library("C2") -> objc_library("D2")
              |
               -> ...and a whole lot more, with deep branches depending on objc

And yay! Everything works. Things get all linked together by, say, the ios_framework that consumes the cc_library, and things like Apple-specific resources from the objc_libraries get propagated all the way down. You're super happy and delighted at how well Bazel has modeled the structure of the multi-language codebase.

And then you try to add Android support. Now, your cc_libraries need to call Java code from C++ through the JNI. You need to do this when there's only a Java interface, since whatever you need to do isn't supported by the NDK. (To clarify: No native code generation involved, not Java calling C++; instead, C++ calling Java through the JNI. No validating, just proper bundling based on dependencies.)

Most of this platform specific work is calling system libraries from Java, but some is calling other Google AARs, like ARCore say, that may have a native interface. The graph of what code (logically) depends on what--and the call graph--looks like the following, exactly equivalent to the structure for other platforms:

Depended upon by an android_binary, somewhere several layers up:

cc_library("A") -> cc_library("B") -> cc_library("C") -> android_library("D")
              |
               -> cc_library("B2") -> aar_import("ARCore")
              |
               -> ...and a whole lot more, with deep branches depending on Android code

It'd be great if you could express that dependency graph in Bazel! I promise this isn't just a "me currently" use case: As examples, Google products I've worked on internally in the past, FB, Spotify all have had dependency graphs like this that cause lots of pain. And Bazel's polyglot/cross-platform amazingness is (I think) uniquely suited to handle them well--if there's a way we could possibly propagate the right attributes up to be bundled together the way Apple rules do.

(The options for hacking around really aren't great, especially for the aar_import of an AAR that's trying to export a C-interface, like Prefabs or ARCore. Happy to expand more on workarounds/hacks if you want, but I've already written too much, so I'll stop for now.)

[ahumesky]

I think I understand now: you have native code that calls Java code, and that Java code needs to make it into the app somehow (i.e., no other code in the app or targets in your build is depending on / referencing that Java code). Right?

If I have it right, the issue is basically the same as I mentioned above:

the problem with a cc_library depending on an android_library or an aar_import is that all the other "Android stuff" gets dropped -- cc_library won't know what to do with the Java providers and Android resource providers and proguard providers, etc, and any of the other things that might get added one day [1]. There isn't any sort of generic mechanism for automatically propagating providers, so cc_library would have to explicitly know about all these Android-specific things, which is not very desirable.

There are ways around the problem of cc_library not propagating the Java+Android things, such as android_binary using an aspect to collect all the providers it wants from its dependencies, instead of the usual provider propagation patterns, but this would be a significant refactoring of the rules. So it's not clear how we can easily support this.

I think you allude to it above ("you can pretend the dependency goes the other way"), but to name it explicitly as a workaround: because the native code and Java code don't actually get compiled together, you should be able to flip it around so that the android_library depends on the cc_library, and it should just work.

That might not be too satisfactory, but there's a way to structure this slightly differently that might make it a little more clear what the intent is:

android_library(
    name = "foo",
    exports = [
        ":foo-java",
        ":foo-native",
    ],
)

android_library(
    name = "foo-java",
    srcs = [...],
)

cc_library(
    name = "foo-native",
    srcs = [...],
)

[cpsauer]

Yes! Exactly. I really appreciate the time you've spent thinking about this with me, Alex.

For the android_library case you mention above, I'm knee-deep writing a macro that automates that flipping. The cc->android cases are 3-5 dependency layers deep, so I think the only way to not break the cc_library for other platforms is to flip the dependencies all the way down to the root node. My approach is to write a drop-in replacement for cc_library that automatically constructs an android_library graph parallel to the cc_library graph. I think it'll work if I replace all my calls to cc_library with that macro. Not elegant, but I think it'll serve--I'm hoping at least until there's a clean official fix. You (obviously) know way more about Bazel than I, so please lmk if you see a cleaner way to do things.

On the subject of provider propagation, could there be something to learn from how it's done with objc_library propagating apple-specific files it might transitively depend on? We're at the limits of my Bazel understanding here, but maybe they're able to just pipe everything through the standard data routes. Could something like that work?

The macro workaround should also solve the case of C++ depending on the Java interfaces of prebuilt AARs.

What it doesn't solve is the case of depending on AARs that instead export a native interface. That one is way nastier. (Also knee-deep in a workaround--which is what spawned the PR). Since all my prebuilt AARs are external, I'm working on a repository rule that unwraps them to be able to exposes their Prefab headers/.so's for inclusion. It's an adventure. Definitely open to your advice if you know of a better way!

Thanks so much,
Chris

[ahumesky]

Regarding provider propagation and the objc and cc rules, my understanding is that the objc rules are much closer to the cc rules, in particular objc uses the same native compilation infrastructure as the cc rules and passes the same CcInfo provider that cc_library expects.

From my quick reading of the code though, it does look like cc_library drops some objc stuff:
objc_library provides an ObjcProvider and some J2Objc providers:
https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/objc/ObjcLibrary.java;l=99-102;drc=2d28044316bf1ea9935f85e9a6876062abce169f

but cc_library doesn't propagate those:
https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/cpp/CcLibrary.java;l=480-494;drc=2d28044316bf1ea9935f85e9a6876062abce169f

It's possible though that something else like an aspect somewhere gathers up the left-behind providers (I think there's a J2Objc aspect somewhere).

Regarding aars, creating a repository rule to unpack the aars seems fine, though you may have some trouble using that with rules_jvm_external if you're using that to get your aars. Alternatively there may be some way to create a rule that takes the aar_import targets created from rules_jvm_external / maven_install, and reads the AndroidNativeLibsInfo provider and then wraps them in a CcInfo that cc_library will accept. That would require some investigation to get right.

[cpsauer]

Thanks, Alex. I hashed out the structure of the hacks in transit this morning and have something...expectedly ugly but working. AARs from maven were indeed an adventure. I'll see how many of the worst hacks I can get rid of, using your good advice.

[ahumesky]

Happy to be of help!

[cpsauer]

An update: This has turned out to be super useful, and the workaround has evolved to something that works quite elegantly. If enough other folks need this, I'd be happy to consider releasing or contributing, like with our autocomplete tooling! Just lmk with a comment or a 👍🏻 on the original comment.