Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Runs build in macos to generate a cache for androidTest runs #1019

Merged
merged 13 commits into from
Nov 10, 2023

Conversation

JoseAlcerreca
Copy link
Contributor

@JoseAlcerreca JoseAlcerreca commented Nov 6, 2023

Experiments to fix #1011 and #1018

Relying on gradle-build-action for cache sharing. Followed some guidance in:

https://github.com/gradle/gradle-build-action#deduplication-of-gradle-user-home-cache-entries

@JoseAlcerreca
Copy link
Contributor Author

So far it seems like the linux job takes 9-11 minutes (with cache hits), and will fails screenshot tests first, then jvm tests, spotless. Then it builds and uploads APKs. We probably want to fail spotless first.

If the build is broken, it can be caught by either the linux job or the androidTest-build job, whichever finds it first because they run in parallel.

@JoseAlcerreca
Copy link
Contributor Author

Looking at the low-hanging fruit for the androidTest-run job:

  1. Runners wait for the emulator to boot for 100 seconds.

After that, the connectedCheck task is run and then gradle takes a while to assemble everything.

This seems like it could be done in parallel? Not sure how much CPU the emulator boot takes.

  1. Caching the emulator image
    How this action be compitable with gradle/gradle-build-action ReactiveCircus/android-emulator-runner#229

This looks good but wouldn't it take most of the 10Gb cache?

  1. Configuration cache is disabled. See [Bug]: Configuration cache disabled due to *OssLicensesTask  #1022

  2. The reactivecircus action seems to be downloading and installing a lot of stuff that should already be present in macos-latest. Warning - Platform tools already installed ReactiveCircus/android-emulator-runner#265

For example, the command [command]/bin/sh -c \sdkmanager --install system-images;android-26;default;x86_64' --channel=0 > /dev/null

takes 53 seconds to run.

@JoseAlcerreca
Copy link
Contributor Author

On 2: I'm running a test in https://github.com/JoseAlcerreca/nowinandroid/actions/runs/6788492716/job/18453744948 to see how much cache it takes. After this run, I'll try with an ATD image.

@JoseAlcerreca
Copy link
Contributor Author

On 2: I'm running a test in https://github.com/JoseAlcerreca/nowinandroid/actions/runs/6788492716/job/18453744948 to see how much cache it takes. After this run, I'll try with an ATD image.

avd-26 Cache Size: ~4551 MB (4771592759 B)
avd-30 Cache Size: ~3835 MB (4021327173 B)

We'd probably run out of cache for other things that might impact CI times more.

@yschimke
Copy link
Contributor

yschimke commented Nov 8, 2023

This looks good but wouldn't it take most of the 10Gb cache?

Should we consider a separate job that runs, produces the AVD snapshot, then upload as build artifacts instead of cache?

It might be worthwhile having another form of cache for this?

@JoseAlcerreca
Copy link
Contributor Author

Should we consider a separate job that runs, produces the AVD snapshot, then upload as build artifacts instead of cache?

I don't think there's an API to download the build artifacts from another workflow, is there?
From a quick search:
"Downloading files: You can only download artifacts that were uploaded during the same workflow run. When you download a file, you can reference it by name."

Change-Id: I4c94aadaa9f6b7fafce17ce949c922973bd522d0
Change-Id: I85b5b3c52b8b1c03980147d9e45ec6b64ada8e8e
@JoseAlcerreca
Copy link
Contributor Author

I think we can start reviewing this PR:

(1) Moved some of the build to a previous step so that gradle has a bit more resources to run. Probably irrelevant.

(2) Can't do this because of the size of the AVD images

(3) Pinged the authors of the plugin and will look into it separately. We'll probably skip the problematic tasks on CI.

(4) Filed #1026 for tracking

@JoseAlcerreca JoseAlcerreca marked this pull request as ready for review November 8, 2023 15:39
@JoseAlcerreca
Copy link
Contributor Author

"Configuration cache entry stored."

Now the next run should be faster.

Change-Id: Ic2e6fe27597b29097b33e74601d32f3c70f10683
@JoseAlcerreca
Copy link
Contributor Author

JoseAlcerreca commented Nov 10, 2023

Took 21 minutes in API 26 and 30 in API 30 without configuration cache. Triggering another build now.

Nevermind, the cache won't be saved until this is merged into main. Ready for review.

@JoseAlcerreca JoseAlcerreca merged commit 3f22248 into main Nov 10, 2023
4 checks passed
@JoseAlcerreca JoseAlcerreca deleted the ja/ci-cache branch November 10, 2023 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

In CI the test workflow shouldn't run if the build workflow fails
4 participants