-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bazel] Improve Mobile-Install Incremental Manifest Generating by applying multi-thread #12085
[Bazel] Improve Mobile-Install Incremental Manifest Generating by applying multi-thread #12085
Conversation
All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the ℹ️ Googlers: Go here for more info. |
@googlebot I consent |
b0910fb
to
7da6509
Compare
CLAs look good, thanks! ℹ️ Googlers: Go here for more info. |
@ahumesky mind taking a look when available? |
ping @ahumesky? |
ping @ahumesky again can you take a look? |
Seems like @ahumesky is not available? @laurentlb @gregestren is there anyone else who can help review this PR? |
I'll poke around to see... |
Thank you for the thorough analysis, and sorry for the delay here! |
Background
While using
mobile-Install
, we noticed that it constantly takes more to run on incremental build. Take our app for example, the incremental build metrics for a single line kotlin code change looks like this:After digging into it, I found that the bottleneck is the "Incremental Manifest Generating" action which takes a lot of time (35+ sec) for multidex build. The time is spent on the checksum calculation for all dex files. The SHA256 checksum for each dex file takes around 1-2 sec. Currently in our app we have 10 dex shards and each dex zip contains 2-3 dex files, processing them sequentially takes more than 30 seconds.
Change
In this PR, I added multithread support for this script so that the checksum calculation can be done concurrently and it improved the "Incremental Manifest Generating" to be done in 6 second (80%+ improvement).
Result & Test
After applying this change, the total incremental build time has been reduced to 43 seconds, with a 30s+ improvement from
Incremental Manifest Generating
step.Before:
After:
You can also easily verify this from command line: