Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gradle: Run the clean task before anything else to make sure nothing is cached. #2705

Merged
merged 2 commits into from
Nov 21, 2022

Conversation

misl6
Copy link
Member

@misl6 misl6 commented Nov 19, 2022

Headache time 🤕⏱️!

As clearly described by @DexerBR in #2704 :

The app size grows unexpectedly from 18.5MB to 26.4MB in the second build. From the second to next builds, the sudden size changes seem to stabilize.

For a still unknown(likely a Gradle bug) reason, the produced APK is bigger on non-first builds.
After some analysis (unpacked the APK, checked and compared the content), I can say that only the size of the APK itself is different, not the content, as when it gets unzipped, the size of the unzipped content is the same between the first build and subsequent ones.

The files md5 hashes are the same, except for the assets/private.tar file (and its content), as I'm not building with SOURCE_DATE_EPOCH set, which means that compileall uses the timestamp as the invalidation mode, making the build non-reproducible.

That's where the 💡 toggled on.

What's the size of my assets/private.tar? 17MB
What's the APK size difference between the first and subsequent builds? 17MB
Ouch! 😓

For some reason, Gradle creates a bigger APK, which is able to contain all the assets and code + 2x private.tar (the old one, and the new one), but then only packages into it the newer one.

I will try to investigate it on Gradle (and Android Gradle Plugin) codebase, but meanwhile, cleaning the build folder (via ./gradlew clean *****) avoids any caching and ensures a clean build, every time, and doesn't seem to take any considerable amount of time.

EDIT:
Found a discussion on Google Issue Tracker about something similar: https://issuetracker.google.com/issues/172762362

If an entry has been deleted in the middle of the archive, Zipflinger will not leave a “hole” there. This is done in order to be compatible with top-down parsers such as jarsigner or the JDK zip classes. To this effect, Zipflinger fills empty space with virtual entries (a.k.a a Local File Header with no name, up to 64KiB extra and no Central Directory entry). Alignment is also done via “extra field”.

AndreMiras
AndreMiras previously approved these changes Nov 19, 2022
Copy link
Member

@AndreMiras AndreMiras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for racking your brain on it, that's a very weird bug indeed

Comment on lines 1112 to 1114
# WARNING: We should make sure to clean the build directory before building.
# Looks like our private.tar is causing issues to gradle.
output = shprint(gradlew, "clean", gradle_task, _tail=20,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice findings and fix 💪
Do you think we should reference this pull request or initial bug report with a refs #2705 in the comment since you explained it all well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was waiting to split the original issue, into multiple chunks (as actually contains multiple issues, and some of them are still being triaged) and then totally forgot to add a reference.

So, I guess that now that we have a PR to reference, I will reference the PR. 👍

@misl6 misl6 merged commit 63e6fb6 into kivy:develop Nov 21, 2022
shyamnathp pushed a commit to shyamnathp/python-for-android that referenced this pull request Feb 17, 2023
…is "cached". (kivy#2705)

* Gradle: Run the clean task before anything else to make sure nothing is cached.

* Reference the PR for the future
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants