Skip to content

Commit

Permalink
feat: implement cache-dependency-path option to control caching depen…
Browse files Browse the repository at this point in the history
…dency (#499)
  • Loading branch information
itchyny authored Nov 22, 2023
1 parent 78078da commit 9eda6b5
Show file tree
Hide file tree
Showing 14 changed files with 277 additions and 58 deletions.
93 changes: 93 additions & 0 deletions .github/workflows/e2e-cache-dependency-path.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
name: Validate cache with cache-dependency-path option

on:
push:
branches:
- main
- releases/*
paths-ignore:
- '**.md'
pull_request:
paths-ignore:
- '**.md'

defaults:
run:
shell: bash

jobs:
gradle1-save:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [macos-latest, windows-latest, ubuntu-latest]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Run setup-java with the cache for gradle
uses: ./
id: setup-java
with:
distribution: 'adopt'
java-version: '11'
cache: gradle
cache-dependency-path: __tests__/cache/gradle1/*.gradle*
- name: Create files to cache
# Need to avoid using Gradle daemon to stabilize the save process on Windows
# https://github.com/actions/cache/issues/454#issuecomment-840493935
run: |
gradle downloadDependencies --no-daemon -p __tests__/cache/gradle1
if [ ! -d ~/.gradle/caches ]; then
echo "::error::The ~/.gradle/caches directory does not exist unexpectedly"
exit 1
fi
gradle1-restore:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [macos-latest, windows-latest, ubuntu-latest]
needs: gradle1-save
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Run setup-java with the cache for gradle
uses: ./
id: setup-java
with:
distribution: 'adopt'
java-version: '11'
cache: gradle
cache-dependency-path: __tests__/cache/gradle1/*.gradle*
- name: Confirm that ~/.gradle/caches directory has been made
run: |
if [ ! -d ~/.gradle/caches ]; then
echo "::error::The ~/.gradle/caches directory does not exist unexpectedly"
exit 1
fi
ls ~/.gradle/caches/
gradle2-restore:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [macos-latest, windows-latest, ubuntu-latest]
needs: gradle1-save
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Run setup-java with the cache for gradle
uses: ./
id: setup-java
with:
distribution: 'adopt'
java-version: '11'
cache: gradle
cache-dependency-path: __tests__/cache/gradle2/*.gradle*
- name: Confirm that ~/.gradle/caches directory has not been made
run: |
if [ -d ~/.gradle/caches ]; then
echo "::error::The ~/.gradle/caches directory exists unexpectedly"
exit 1
fi
2 changes: 1 addition & 1 deletion .github/workflows/e2e-cache.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
# Need to avoid using Gradle daemon to stabilize the save process on Windows
# https://github.com/actions/cache/issues/454#issuecomment-840493935
run: |
gradle downloadDependencies --no-daemon -p __tests__/cache/gradle
gradle downloadDependencies --no-daemon -p __tests__/cache/gradle1
if [ ! -d ~/.gradle/caches ]; then
echo "::error::The ~/.gradle/caches directory does not exist unexpectedly"
exit 1
Expand Down
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ This action allows you to work with Java and Scala projects.

- `cache`: Quick [setup caching](#caching-packages-dependencies) for the dependencies managed through one of the predefined package managers. It can be one of "maven", "gradle" or "sbt".

- `cache-dependency-path`: The path to a dependency file: pom.xml, build.gradle, build.sbt, etc. This option can be used with the `cache` option. If this option is omitted, the action searches for the dependency file in the entire repository. This option supports wildcards and a list of file names for caching multiple dependencies.

#### Maven options
The action has a bunch of inputs to generate maven's [settings.xml](https://maven.apache.org/settings.html) on the fly and pass the values to Apache Maven GPG Plugin as well as Apache Maven Toolchains. See [advanced usage](docs/advanced-usage.md) for more.

Expand Down Expand Up @@ -115,10 +117,13 @@ Currently, the following distributions are supported:

### Caching packages dependencies
The action has a built-in functionality for caching and restoring dependencies. It uses [toolkit/cache](https://github.com/actions/toolkit/tree/main/packages/cache) under hood for caching dependencies but requires less configuration settings. Supported package managers are gradle, maven and sbt. The format of the used cache key is `setup-java-${{ platform }}-${{ packageManager }}-${{ fileHash }}`, where the hash is based on the following files:

- gradle: `**/*.gradle*`, `**/gradle-wrapper.properties`, `buildSrc/**/Versions.kt`, `buildSrc/**/Dependencies.kt`, `gradle/*.versions.toml`, and `**/versions.properties`
- maven: `**/pom.xml`
- sbt: all sbt build definition files `**/*.sbt`, `**/project/build.properties`, `**/project/**.scala`, `**/project/**.sbt`

When the option `cache-dependency-path` is specified, the hash is based on the matching file. This option supports wildcards and a list of file names, and is especially useful for monorepos.

The workflow output `cache-hit` is set to indicate if an exact match was found for the key [as actions/cache does](https://github.com/actions/cache/tree/main#outputs).

The cache input is optional, and caching is turned off by default.
Expand All @@ -132,6 +137,9 @@ steps:
distribution: 'temurin'
java-version: '17'
cache: 'gradle'
cache-dependency-path: | # optional
sub-project/*.gradle*
sub-project/**/gradle-wrapper.properties
- run: ./gradlew build --no-daemon
```

Expand All @@ -144,6 +152,7 @@ steps:
distribution: 'temurin'
java-version: '17'
cache: 'maven'
cache-dependency-path: 'sub-project/pom.xml' # optional
- name: Build with Maven
run: mvn -B package --file pom.xml
```
Expand All @@ -157,6 +166,9 @@ steps:
distribution: 'temurin'
java-version: '17'
cache: 'sbt'
cache-dependency-path: | # optional
sub-project/build.sbt
sub-project/project/build.properties
- name: Build with SBT
run: sbt package
```
Expand Down
108 changes: 87 additions & 21 deletions __tests__/cache.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import * as fs from 'fs';
import * as os from 'os';
import * as core from '@actions/core';
import * as cache from '@actions/cache';
import * as glob from '@actions/glob';

describe('dependency cache', () => {
const ORIGINAL_RUNNER_OS = process.env['RUNNER_OS'];
Expand Down Expand Up @@ -64,25 +65,30 @@ describe('dependency cache', () => {
ReturnType<typeof cache.restoreCache>,
Parameters<typeof cache.restoreCache>
>;
let spyGlobHashFiles: jest.SpyInstance<
ReturnType<typeof glob.hashFiles>,
Parameters<typeof glob.hashFiles>
>;

beforeEach(() => {
spyCacheRestore = jest
.spyOn(cache, 'restoreCache')
.mockImplementation((paths: string[], primaryKey: string) =>
Promise.resolve(undefined)
);
spyGlobHashFiles = jest.spyOn(glob, 'hashFiles');
spyWarning.mockImplementation(() => null);
});

it('throws error if unsupported package manager specified', () => {
return expect(restore('ant')).rejects.toThrow(
return expect(restore('ant', '')).rejects.toThrow(
'unknown package manager specified: ant'
);
});

describe('for maven', () => {
it('throws error if no pom.xml found', async () => {
await expect(restore('maven')).rejects.toThrow(
await expect(restore('maven', '')).rejects.toThrow(
`No file in ${projectRoot(
workspace
)} matched to [**/pom.xml], make sure you have checked out the target repository`
Expand All @@ -91,15 +97,16 @@ describe('dependency cache', () => {
it('downloads cache', async () => {
createFile(join(workspace, 'pom.xml'));

await restore('maven');
await restore('maven', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('**/pom.xml');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('maven cache is not found');
});
});
describe('for gradle', () => {
it('throws error if no build.gradle found', async () => {
await expect(restore('gradle')).rejects.toThrow(
await expect(restore('gradle', '')).rejects.toThrow(
`No file in ${projectRoot(
workspace
)} matched to [**/*.gradle*,**/gradle-wrapper.properties,buildSrc/**/Versions.kt,buildSrc/**/Dependencies.kt,gradle/*.versions.toml,**/versions.properties], make sure you have checked out the target repository`
Expand All @@ -108,41 +115,53 @@ describe('dependency cache', () => {
it('downloads cache based on build.gradle', async () => {
createFile(join(workspace, 'build.gradle'));

await restore('gradle');
await restore('gradle', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml\n**/versions.properties'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
it('downloads cache based on build.gradle.kts', async () => {
createFile(join(workspace, 'build.gradle.kts'));

await restore('gradle');
await restore('gradle', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml\n**/versions.properties'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
it('downloads cache based on libs.versions.toml', async () => {
createDirectory(join(workspace, 'gradle'));
createFile(join(workspace, 'gradle', 'libs.versions.toml'));

await restore('gradle');
await restore('gradle', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml\n**/versions.properties'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
});
it('downloads cache based on buildSrc/Versions.kt', async () => {
createDirectory(join(workspace, 'buildSrc'));
createFile(join(workspace, 'buildSrc', 'Versions.kt'));
it('downloads cache based on buildSrc/Versions.kt', async () => {
createDirectory(join(workspace, 'buildSrc'));
createFile(join(workspace, 'buildSrc', 'Versions.kt'));

await restore('gradle');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
await restore('gradle', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml\n**/versions.properties'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
});
describe('for sbt', () => {
it('throws error if no build.sbt found', async () => {
await expect(restore('sbt')).rejects.toThrow(
await expect(restore('sbt', '')).rejects.toThrow(
`No file in ${projectRoot(
workspace
)} matched to [**/*.sbt,**/project/build.properties,**/project/**.scala,**/project/**.sbt], make sure you have checked out the target repository`
Expand All @@ -151,8 +170,11 @@ describe('dependency cache', () => {
it('downloads cache', async () => {
createFile(join(workspace, 'build.sbt'));

await restore('sbt');
await restore('sbt', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.sbt\n**/project/build.properties\n**/project/**.scala\n**/project/**.sbt'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('sbt cache is not found');
});
Expand All @@ -161,19 +183,19 @@ describe('dependency cache', () => {
createDirectory(join(workspace, 'project'));
createFile(join(workspace, 'project/DependenciesV1.scala'));

await restore('sbt');
await restore('sbt', '');
const firstCall = spySaveState.mock.calls.toString();

spySaveState.mockClear();
await restore('sbt');
await restore('sbt', '');
const secondCall = spySaveState.mock.calls.toString();

// Make sure multiple restores produce the same cache
expect(firstCall).toBe(secondCall);

spySaveState.mockClear();
createFile(join(workspace, 'project/DependenciesV2.scala'));
await restore('sbt');
await restore('sbt', '');
const thirdCall = spySaveState.mock.calls.toString();

expect(firstCall).not.toBe(thirdCall);
Expand All @@ -182,11 +204,55 @@ describe('dependency cache', () => {
it('downloads cache based on versions.properties', async () => {
createFile(join(workspace, 'versions.properties'));

await restore('gradle');
await restore('gradle', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml\n**/versions.properties'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
describe('cache-dependency-path', () => {
it('throws error if no matching dependency file found', async () => {
createFile(join(workspace, 'build.gradle.kts'));
await expect(
restore('gradle', 'sub-project/**/build.gradle.kts')
).rejects.toThrow(
`No file in ${projectRoot(
workspace
)} matched to [sub-project/**/build.gradle.kts], make sure you have checked out the target repository`
);
});
it('downloads cache based on the specified pattern', async () => {
createFile(join(workspace, 'build.gradle.kts'));
createDirectory(join(workspace, 'sub-project1'));
createFile(join(workspace, 'sub-project1', 'build.gradle.kts'));
createDirectory(join(workspace, 'sub-project2'));
createFile(join(workspace, 'sub-project2', 'build.gradle.kts'));

await restore('gradle', 'build.gradle.kts');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('build.gradle.kts');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');

await restore('gradle', 'sub-project1/**/*.gradle*\n');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'sub-project1/**/*.gradle*'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');

await restore('gradle', '*.gradle*\nsub-project2/**/*.gradle*\n');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'*.gradle*\nsub-project2/**/*.gradle*'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
});
});
describe('save', () => {
let spyCacheSave: jest.SpyInstance<
Expand Down
File renamed without changes.
File renamed without changes.
12 changes: 12 additions & 0 deletions __tests__/cache/gradle2/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
.gradle
**/build/
!src/**/build/

# Ignore Gradle GUI config
gradle-app.setting

# Avoid ignoring Gradle wrapper jar file (.jar files are usually ignored)
!gradle-wrapper.jar

# Cache of project
.gradletasknamecache
Loading

0 comments on commit 9eda6b5

Please sign in to comment.