Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify caching by reducing launch+cache outcomes #35

Merged
merged 2 commits into from
Feb 20, 2019
Merged

Simplify caching by reducing launch+cache outcomes #35

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0e1e5a9
Simplify caching by reducing launch+cache outcomes
sclevine Jan 21, 2019
f94c3b7
Fix description of analysis to match caching simplication
sclevine Jan 21, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 26 additions & 22 deletions buildpack.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,11 +149,11 @@ Executable: `/bin/develop <layers[EC]> <platform[AR]> <plan[E]>`, Working Dir: `

Using the [Layer Content Metadata](#layer-content-metadata-(toml)) provided by a buildpack in a `<layers>/<layer>.toml` file, the lifecycle MUST determine:

- Whether the layer directory in `<layers>/<layer>/` should be available to the app.
- Whether the layer directory in `<layers>/<layer>/` should be available to subsequent buildpacks.
- Whether and how the layer directory in `<layers>/<layer>/` should be persisted to subsequent builds of the same OCI image.
- Whether the layer directory in `<layers>/<layer>/` should be available to the app (via the `launch` boolean).
- Whether the layer directory in `<layers>/<layer>/` should be available to subsequent buildpacks (via the `build` boolean).
- Whether and how the layer directory in `<layers>/<layer>/` should be persisted to subsequent builds of the same OCI image (via the `cache` boolean).

This section does not apply to the Development Setup phase, which does not generate an OCI image.
This section does not apply to the Development Setup phase, which does not generate an OCI image.

#### Launch Layers

Expand All @@ -164,15 +164,22 @@ The lifecycle MUST make all launch layers accessible to the app as defined in th
The lifecycle MUST include each launch layer in the built OCI image.
The lifecycle MUST also store the Layer Content Metadata associated with each layer so that it can be recovered using the layer Diff ID.

Before the next re-build:
- The lifecycle MUST restore each `<layers>/<layer>.toml` file containing Layer Content Metadata from the previous build to the same location.
- If a layer is cached, the lifecycle MAY also restore the corresponding `<layers>/<layer>/` directory from the previous build to the same location.
- If a layer is not cached, the lifecycle MUST NOT restore the corresponding layer contents from any previous build.

After the next re-build:
- If a buildpack keeps `launch = true` in `<layers>/<layer>.toml` and leaves no `<layers>/<layer>/` directory, then the lifecycle MUST keep the corresponding layer from the previous build in the built OCI image.
- If a buildpack keeps `launch = true` in `<layers>/<layer>.toml` and leaves a `<layers>/<layer>/` directory, then the lifecycle MUST replace the corresponding layer from the previous build with the directory contents.
- If a buildpack removes `launch = true` from `<layers>/<layer>.toml` or deletes `<layers>/<layer>.toml`, then the lifecycle MUST NOT include any corresponding layer in the built OCI image.
Before a given re-build:
- If a launch layer is marked `cache = false`, the lifecycle:
- MUST restore the entire `<layers>/<layer>.toml` file from the previous build to the same path and
- MUST NOT restore the corresponding `<layers>/<layer>/` directory from any previous build.
- If a launch layer is marked `cache = true`, the lifecycle:
- MUST either restore the entire `<layers>/<layer>.toml` file and corresponding `<layers>/<layer>/` directory from the previous build to the same paths or
- MUST restore neither the `<layers>/<layer>.toml` file nor corresponding `<layers>/<layer>/` directory.

After a given re-build:
- If a buildpack keeps `launch = true` in `<layers>/<layer>.toml` and leaves no `<layers>/<layer>/` directory, the lifecycle:
- MUST keep the corresponding layer from the previous build in the OCI image and
- MUST replace the `<layers>/<layer>.toml` in the OCI image with the version present after the re-build.
- If a buildpack keeps `launch = true` in `<layers>/<layer>.toml` and leaves a `<layers>/<layer>/` directory, the lifecycle:
- MUST replace the corresponding layer in the OCI image with the directory contents present after the re-build and
- MUST replace the `<layers>/<layer>.toml` in the OCI image with the version present after the re-build.
- If a buildpack removes `launch = true` from `<layers>/<layer>.toml` or deletes `<layers>/<layer>.toml`, then the lifecycle MUST NOT include any corresponding layer in the OCI image.

#### Build Layers

Expand All @@ -181,8 +188,8 @@ A buildpack MAY specify that a `<layers>/<layer>/` directory is a build layer by
The lifecycle MUST make all build layers accessible to subsequent buildpacks as defined in the [Environment](#environment) section.

Before the next re-build:
- If the layer is cached, the lifecycle MAY restore the `<layers>/<layer>/` directory and Layer Content Metadata from any previous build to the same location.
- If the layer is not cached, the lifecycle MUST NOT restore either the `<layers>/<layer>/` directory or the Layer Content Metadata from any previous build.
- If the layer is marked `cache = true`, the lifecycle MAY restore the `<layers>/<layer>/` directory and Layer Content Metadata from any previous build to the same path.
- If the layer is marked `cache = false`, the lifecycle MUST NOT restore the `<layers>/<layer>/` directory or the Layer Content Metadata from any previous build.

#### Other Layers

Expand Down Expand Up @@ -261,9 +268,7 @@ The lifecycle MUST fail detection if any of those buildpacks specifies a mixin a

### Purpose

The purpose of analysis is:
- To restore `<layers>/<layer>.toml` files that buildpacks may use to optimize the build and export phases.
- To restore `<layers>/<layer>/` directories that are specified as cached launch layers.
The purpose of analysis is to restore `<layers>/<layer>.toml` files that buildpacks may use to optimize the build and export phases.

### Process

Expand All @@ -282,10 +287,9 @@ The lifecycle MUST skip analysis and proceed to the build phase if no such image

For each buildpack in the group,

1. Any `<layers>/<layer>` directories that are cached locally from any previous build are restored to their same filesystem locations.
2. Each `<layers>/<layer>.toml` file that was present at the end of the build of the previously created OCI image is retrieved.
3. Each `<layers>/<layer>.toml` file is placed on the filesystem so that it appears in the corresponding buildpack's `<layers>/` directory during the build phase.
4. Each `<layers>/<layer>.toml` file containing `launch = true` and `cache = true` has the contents of the corresponding `<layers>/<layer>` directory replaced with its contents in the previously created OCI image.
1. All `<layers>/<layer>.toml` files with `cache = true` and corresponding `<layers>/<layer>` directories from any previous build are restored to their same filesystem locations.
2. Each `<layers>/<layer>.toml` file with `launch = true` and `cache = false` that was present at the end of the build of the previously created OCI image is retrieved.
3. A given `<layers>/<layer>.toml` file with `launch = true` and `cache = true` and corresponding `<layers>/<layer>` directory are both removed if either do not match their contents in the previously created OCI image.

After analysis, the lifecycle MUST proceed to the build phase.

Expand Down