gradle fetchDeps is platform-dependent

[FliegendeWurst]

Describe the bug

As observed in #353165, fetchDeps only resolves the dependencies needed for the build platform.
This leads to build failures later on if the same deps are used on other platforms.

Steps To Reproduce

Use openjfx in a gradle project. This may apply to other dependencies too.

Expected behavior

Just Works ™

Screenshots

       >    > Could not find javafx-controls-17-linux-aarch64.jar (org.openjfx:javafx-controls:17).
       >      Searched in the following locations:
       >          https://repo.maven.apache.org/maven2/org/openjfx/javafx-controls/17/javafx-controls-17-linux-aarch64.jar
       >    > Could not find javafx-graphics-17-linux-aarch64.jar (org.openjfx:javafx-graphics:17).
       >      Searched in the following locations:
       >          https://repo.maven.apache.org/maven2/org/openjfx/javafx-graphics/17/javafx-graphics-17-linux-aarch64.jar
       >    > Could not find javafx-base-17-linux-aarch64.jar (org.openjfx:javafx-base:17).
       >      Searched in the following locations:
       >          https://repo.maven.apache.org/maven2/org/openjfx/javafx-base/17/javafx-base-17-linux-aarch64.jar

Additional context

For this particular package it might be possible to use nixpkgs' openjfx package.

Metadata

Notify maintainers

gradle: @lorenzleutgeb @liff @NixOS/java
mitm-cache: @chayleaf

---

Note for maintainers: Please tag this issue in your PR.

---

Add a 👍 reaction to issues you find important.

[liff]

If I understand this correctly: given the way mitm-cache works, the deps.json cannot be populated with platform-dependent dependencies without actually running the build on all supported platforms?

So, in the case of keyboard-layout-editor, we’d need to collect deps.json on all platforms and somehow merge it?

[FliegendeWurst]

So, in the case of keyboard-layout-editor, we’d need to collect deps.json on all platforms and somehow merge it?

Indeed, that would be the solution. Unless Gradle has some flag to specify the platform to build for.

[chayleaf]

Gradle builds scripts are Turing-complete, and dependency resolution is Turing-complete as well, the only way to remotely achieve anything like automatically fetching dependencies for all platforms would be static analysis of Gradle build scripts, which is definitely out of scope, the current integration took a lot of effort already. Please look at some of the existing Gradle packages - they run Gradle multiple times in the update script to fetch dependencies for all platforms.

Which is to say, fetchDeps can't not be platform-dependent, no matter how you implement it, because of how Gradle works.

I did have the idea of automatically detecting some common architecture names in filenames and automatically fetching jar files by substituting the architecture in the URL - it doesn't sound like too good of an idea, but it could be enabled with a whitelist or behind a flag as an experimental feature.

[liff]

Taking an example from jabref, this should work as for keyboard-layout-editor to resolve dependencies for all architectures:

  gradleUpdateScript = ''
    runHook preBuild

    gradle nixDownloadDeps -Dos.family=linux -Dos.arch=amd64
    gradle nixDownloadDeps -Dos.family=linux -Dos.arch=aarch64
    gradle nixDownloadDeps -Dos.name='mac os x' -Dos.arch=amd64
    gradle nixDownloadDeps -Dos.name='mac os x' -Dos.arch=aarch64
  '';

[tomodachi94]

It looks like this is specific to OpenJFX. I cannot find any reference to these properties in the gradle manual: https://docs.gradle.org/current/userguide/build_environment.html#gradle_properties_reference

[chayleaf]

I cannot find any reference to these properties in the gradle manual

Gradle itself is a Java application that mostly doesn't have any platform-dependent behavior. Whenever there is platform-dependent behavior, it's the build scripts checking env vars and etc to figure out what dependencies to pick, what code to build, etc.

os.name and friends are system properties, see Java docs. However, it doesn't mean this is how os detection is always done in Gradle. See, for example, SIGNALD_TARGET env var.

In case of OpenJFX, the official way is to use the OpenJFX plugin (which uses this), but it's often done manually (don't have the time to dig for examples so here's one from OpenJFX issues).

Gradle itself uses uname behind the scenes (and signald above uses the Gradle implementation unless you override the env var), falling back to system properties.

In essence, os.name-like properties is the closest we have to overriding the OS, but it's not necessary unless the Gradle program uses native libs, and it's not sufficient unless you've looked at the build script and confirmed it.