F-Droid can't build

[licaon-kter]

Describe your suggested feature

ref: https://gitlab.com/fdroid/fdroiddata/-/jobs/7760790503#L379

Other details

No response

Acknowledgements

• I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
• I have written a short but informative title.
• I will fill out all of the requested information in this form.

/LE: fyi https://gitlab.com/fdroid/fdroiddata/-/commit/f3e76bdabb388c9a7a197388af5222bb01d8a91c

[Bnyro]

According to TeamNewPipe/NewPipeExtractor#1066, the dependency might need to be lowercased to ...teamnewpipe:NewPipeExtractor... instead of ...TeamNewpipe:NewPipeExtractor..., although it worked fine for me the other way yesterday, probably due to dependency caching.

In CI (gh actions) of this repository, it sometimes passes and sometimes fails with the error you mentioned, I love deterministic build failures...

I hope I can find time for looking at it tomorrow.

[Bnyro]

Can you please try building from the latest commit (1d4304c)?

This works for me now after clearing the IDEA caches.

[licaon-kter]

@Bnyro 1d4304c builds fine locally

[licaon-kter]

hmmm https://gitlab.com/fdroid/checkupdates-bot-fdroiddata/-/jobs/8396839770#L229

[Bnyro]

LibreTube has used that dependency for 2 years now, I don't see the issue?

[licaon-kter]

New scanner code picked it up.

Just exclude does not make that dep FOSS, when it was built from non-FOSS code.

[IzzySoft]

@Bnyro for what it's worth, I try to build the versions since 1.25.1 and fail with the same error. With v0.26.0 it is

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:mergeReleaseNativeLibs'.
> Could not resolve all files for configuration ':app:releaseRuntimeClasspath'.
   > Could not find com.github.teamnewpipe:NewPipeExtractor:6e3a4a6d9de61eafb73e7eb1b714847b5077856d.
     Searched in the following locations:
       - https://dl.google.com/dl/android/maven2/com/github/teamnewpipe/NewPipeExtractor/6e3a4a6d9de61eafb73e7eb1b714847b5077856d/NewPipeExtractor-6e3a4a6d9de61eafb73e7eb1b714847b5077856d.pom
       - https://repo.maven.apache.org/maven2/com/github/teamnewpipe/NewPipeExtractor/6e3a4a6d9de61eafb73e7eb1b714847b5077856d/NewPipeExtractor-6e3a4a6d9de61eafb73e7eb1b714847b5077856d.pom
       - https://jitpack.io/com/github/teamnewpipe/NewPipeExtractor/6e3a4a6d9de61eafb73e7eb1b714847b5077856d/NewPipeExtractor-6e3a4a6d9de61eafb73e7eb1b714847b5077856d.pom
     Required by:
         project :app

Not sure if maybe switching to the latest release including that commit (v0.24.3 was release 3 days ago) might solve that issue?

[Bnyro]

Not sure if maybe switching to the latest release including that commit (v0.24.3 was release 3 days ago) might solve that issue?

Indeed it might be a good idea to switch to their latest stable release now, we previously required an unreleased version due to breaking changes from YouTube.

[Bnyro]

New scanner code picked it up.

Just exclude does not make that dep FOSS, when it was built from non-FOSS code.

Thus we either need to drop that dependency or get the Upstream Non-Free anti-feature, right?

[IzzySoft]

switch to their latest stable release

If you have a commit with that, and an APK built from that commit, feel free to give me a ping to check for RB again (that was why I tried to build, see Reproducible Builds, special client support and more at IzzyOnDroid – currently 354 apps (29.1%) are RB here already).

[licaon-kter]

@Bnyro drop ye

UpstreamNonFree means that upstream has non free but F-Droid has all free because it cleans up the non-free before building

There's never a non-free to build from.

[Bnyro]

@IzzySoft 2d66a6a

[IzzySoft]

Thanks @Bnyro! If you have an APK built from that commit with your workflow, I can check if it's RB if you want (so we're prepared for the next release and can have the "green shield up").

[Bnyro]

Sure, I'll send you a build this evening 👍

[Bnyro]

The universal apk: app-universal-release.zip, built from 2d66a6a.

[IzzySoft]

That was not build from the commit you specified, @Bnyro 😉

  -rw-r--r--  0.0 unx       56 b-       51 defN 1981-01-01 01:01:02 a18242e2 META-INF/com/android/build/gradle/app-metadata.properties
- -rw-r--r--  0.0 unx      120 b-      118 defN 1981-01-01 01:01:02 fd28c097 META-INF/version-control-info.textproto
- -rw-r--r--  0.0 unx    11663 b-    11663 stor 1981-01-01 01:01:02 eaa2c9cd assets/dexopt/baseline.prof
+ -rw-r--r--  0.0 unx      120 b-      118 defN 1981-01-01 01:01:02 d47bfabf META-INF/version-control-info.textproto
+ -rw-r--r--  0.0 unx    11656 b-    11656 stor 1981-01-01 01:01:02 ee9c069c assets/dexopt/baseline.prof
  -rw-r--r--  0.0 unx      292 b-      292 stor 1981-01-01 01:01:02 227b7cba assets/dexopt/baseline.profm
  -rw-r--r--  0.0 unx  2422680 b-  1141281 defN 1981-01-01 01:01:02 57021890 classes.dex
  -rw-r--r--  0.0 unx   673128 b-   286620 defN 1981-01-01 01:01:02 58f32d66 classes2.dex
- -rw-r--r--  0.0 unx  5822264 b-  2621671 defN 1981-01-01 01:01:02 4f352f35 classes3.dex
+ -rw-r--r--  0.0 unx  5822272 b-  2621689 defN 1981-01-01 01:01:02 230e9716 classes3.dex
  -rw-r--r--  0.0 unx  5019616 b-  2382937 defN 1981-01-01 01:01:02 c9aacb69 lib/arm64-v8a/libcronet.119.0.6045.31.so

it was built from 9458b55 according to META-INF/version-control-info.textproto. But I'm flexible, so let me try with that commit again…

  -rw-r--r--  0.0 unx      120 b-      118 defN 1981-01-01 01:01:02 fd28c097 META-INF/version-control-info.textproto
- -rw-r--r--  0.0 unx    11663 b-    11663 stor 1981-01-01 01:01:02 eaa2c9cd assets/dexopt/baseline.prof
+ -rw-r--r--  0.0 unx    11656 b-    11656 stor 1981-01-01 01:01:02 ee9c069c assets/dexopt/baseline.prof
  -rw-r--r--  0.0 unx      292 b-      292 stor 1981-01-01 01:01:02 227b7cba assets/dexopt/baseline.profm
  -rw-r--r--  0.0 unx  2422680 b-  1141281 defN 1981-01-01 01:01:02 57021890 classes.dex
  -rw-r--r--  0.0 unx   673128 b-   286620 defN 1981-01-01 01:01:02 58f32d66 classes2.dex
- -rw-r--r--  0.0 unx  5822264 b-  2621671 defN 1981-01-01 01:01:02 4f352f35 classes3.dex
+ -rw-r--r--  0.0 unx  5822272 b-  2621689 defN 1981-01-01 01:01:02 230e9716 classes3.dex
  -rw-r--r--  0.0 unx  5019616 b-  2382937 defN 1981-01-01 01:01:02 c9aacb69 lib/arm64-v8a/libcronet.119.0.6045.31.so

Marginally better. Let's check the Dex… Hm, strange. OK, according to your Github workflows you build on Ubuntu 24 using JDK-17. I've been building on Debian bookworm with JDK-17. Should be close enough, but wouldn't be the first time this made a difference – so let me make another run on Ubuntu. We have 22.04 here (ubuntu:jammy), giving the same results as the Debian Bookworm run. Let me try ubuntu:noble (same results), then I'm out asking if you maybe use something other than your workflow says, e.g. JDK-21? OK, no dice either, gives the same hash.

Looking closer at the dex diff, it doesn't really look like something related to a specific JDK or OS mismatch. I'm attaching it here for you, maybe it gives you a better idea than it gives me: dex.diff.gz

[IzzySoft]

PS: a guess would be a "dirty tree" (local changes) or left-over artifacts from previous build. Can we exclude those here?

[Bnyro]

That was not build from the commit you specified, @Bnyro 😉

Oh, I didn't think a merge commit would matter 😌

I'm building releases on my machine locally, not via workflows, but still I'm using OpenJDK 17.

Possibly, as you said, there might be some leftovers from previous builds, I'll try my luck again when I get to have time (sorry for possible delays).

[IzzySoft]

No prob. When building locally, it's a good idea to insert clean for "release builds" (and those I shall test for RB), e.g. ./gradlew clean assembleRelease – or hitting "clean" before "build" in Studio if you're a cat chasing your mouse 😜 As it sometimes matters, maybe also mention which JDK (OpenJDK preferred) you're using and on which OS you're building. Plus of course any other "specific" things there might be (e.g. using a different NDK or whatever).

[Bnyro]

@IzzySoft same commit as before (9458b55), built on Void Linux using OpenJDK 17 and a "cleaned" build environment.
app-universal-release.zip

[IzzySoft]

    "upstream_signed_apk_sha256": "0312167fd80c653e4f5cb16f53cfbdd9fdc07b4223a2b6487055e1ef827858ac",
    "built_unsigned_apk_sha256": "69ff7344aacd6654d2d6728c5b62e95efeb786eda45b78fef27de36cd0e976df",
    "signature_copied_apk_sha256": "0312167fd80c653e4f5cb16f53cfbdd9fdc07b4223a2b6487055e1ef827858ac"

That means RB 🥳 Congrats!

PS: Please let me know when it's available as release, so I establish it with my builder here. Thanks!

[Bnyro]

That means RB 🥳 Congrats!

Awesome, thanks for your guidance!

[FireMasterK]

New scanner code picked it up.

Just exclude does not make that dep FOSS, when it was built from non-FOSS code.

There are alternative cronet implementations that are ABI compatible and FOSS. This is the cronet-embedded dependency. Why does this matter when the resulting binary doesn't use any non-free code? The non-free code is used at compile-time and never at runtime.

https://github.com/androidx/media/blob/1a7da45dd9c6419a43fab95132ed0aa0999300aa/libraries/datasource_cronet/build.gradle#L27
I'm sure this line patched to use the cronet-embedded dependency and the resulting binary would still be the same, so why do we have this limitation?

[licaon-kter]

Ask yourself this: can you build that lib without that non-FOSS code? If not... it's not FOSS

[FireMasterK]

Ask yourself this: can you build that lib without that non-FOSS code? If not... it's not FOSS

Yes you can, I just did this in my fork to test this: https://github.com/FireMasterK/LibreTube/tree/media3-cronet

[licaon-kter]

@FireMasterK that's not what I've asked 🤷

but thanks for providing a solution maybe 👍