From 0e1e5a9e0eb8fba7e76fa833ffa84faf74cc62cb Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Sun, 20 Jan 2019 19:15:25 -0500 Subject: [PATCH 1/2] Simplify caching by reducing launch+cache outcomes Signed-off-by: Stephen Levine --- buildpack.md | 37 ++++++++++++++++++++++--------------- 1 file changed, 22 insertions(+), 15 deletions(-) diff --git a/buildpack.md b/buildpack.md index bee07dd3..66df29e6 100644 --- a/buildpack.md +++ b/buildpack.md @@ -149,11 +149,11 @@ Executable: `/bin/develop `, Working Dir: ` Using the [Layer Content Metadata](#layer-content-metadata-(toml)) provided by a buildpack in a `/.toml` file, the lifecycle MUST determine: -- Whether the layer directory in `//` should be available to the app. -- Whether the layer directory in `//` should be available to subsequent buildpacks. -- Whether and how the layer directory in `//` should be persisted to subsequent builds of the same OCI image. +- Whether the layer directory in `//` should be available to the app (via the `launch` boolean). +- Whether the layer directory in `//` should be available to subsequent buildpacks (via the `build` boolean). +- Whether and how the layer directory in `//` should be persisted to subsequent builds of the same OCI image (via the `cache` boolean). -This section does not apply to the Development Setup phase, which does not generate an OCI image. +This section does not apply to the Development Setup phase, which does not generate an OCI image. #### Launch Layers @@ -164,15 +164,22 @@ The lifecycle MUST make all launch layers accessible to the app as defined in th The lifecycle MUST include each launch layer in the built OCI image. The lifecycle MUST also store the Layer Content Metadata associated with each layer so that it can be recovered using the layer Diff ID. -Before the next re-build: -- The lifecycle MUST restore each `/.toml` file containing Layer Content Metadata from the previous build to the same location. -- If a layer is cached, the lifecycle MAY also restore the corresponding `//` directory from the previous build to the same location. -- If a layer is not cached, the lifecycle MUST NOT restore the corresponding layer contents from any previous build. - -After the next re-build: -- If a buildpack keeps `launch = true` in `/.toml` and leaves no `//` directory, then the lifecycle MUST keep the corresponding layer from the previous build in the built OCI image. -- If a buildpack keeps `launch = true` in `/.toml` and leaves a `//` directory, then the lifecycle MUST replace the corresponding layer from the previous build with the directory contents. -- If a buildpack removes `launch = true` from `/.toml` or deletes `/.toml`, then the lifecycle MUST NOT include any corresponding layer in the built OCI image. +Before a given re-build: +- If a launch layer is marked `cache = false`, the lifecycle: + - MUST restore the entire `/.toml` file from the previous build to the same path and + - MUST NOT restore the corresponding `//` directory from any previous build. +- If a launch layer is marked `cache = true`, the lifecycle: + - MUST either restore the entire `/.toml` file and corresponding `//` directory from the previous build to the same paths or + - MUST restore neither the `/.toml` file nor corresponding `//` directory. + +After a given re-build: +- If a buildpack keeps `launch = true` in `/.toml` and leaves no `//` directory, the lifecycle: + - MUST keep the corresponding layer from the previous build in the OCI image and + - MUST replace the `/.toml` in the OCI image with the version present after the re-build. +- If a buildpack keeps `launch = true` in `/.toml` and leaves a `//` directory, the lifecycle: + - MUST replace the corresponding layer in the OCI image with the directory contents present after the re-build and + - MUST replace the `/.toml` in the OCI image with the version present after the re-build. +- If a buildpack removes `launch = true` from `/.toml` or deletes `/.toml`, then the lifecycle MUST NOT include any corresponding layer in the OCI image. #### Build Layers @@ -181,8 +188,8 @@ A buildpack MAY specify that a `//` directory is a build layer by The lifecycle MUST make all build layers accessible to subsequent buildpacks as defined in the [Environment](#environment) section. Before the next re-build: -- If the layer is cached, the lifecycle MAY restore the `//` directory and Layer Content Metadata from any previous build to the same location. -- If the layer is not cached, the lifecycle MUST NOT restore either the `//` directory or the Layer Content Metadata from any previous build. +- If the layer is marked `cache = true`, the lifecycle MAY restore the `//` directory and Layer Content Metadata from any previous build to the same path. +- If the layer is marked `cache = false`, the lifecycle MUST NOT restore the `//` directory or the Layer Content Metadata from any previous build. #### Other Layers From f94c3b741dda6b053472289825fab716e4111bf3 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Sun, 20 Jan 2019 23:16:36 -0500 Subject: [PATCH 2/2] Fix description of analysis to match caching simplication Signed-off-by: Stephen Levine --- buildpack.md | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/buildpack.md b/buildpack.md index 66df29e6..6ad37c23 100644 --- a/buildpack.md +++ b/buildpack.md @@ -268,9 +268,7 @@ The lifecycle MUST fail detection if any of those buildpacks specifies a mixin a ### Purpose -The purpose of analysis is: -- To restore `/.toml` files that buildpacks may use to optimize the build and export phases. -- To restore `//` directories that are specified as cached launch layers. +The purpose of analysis is to restore `/.toml` files that buildpacks may use to optimize the build and export phases. ### Process @@ -289,10 +287,9 @@ The lifecycle MUST skip analysis and proceed to the build phase if no such image For each buildpack in the group, -1. Any `/` directories that are cached locally from any previous build are restored to their same filesystem locations. -2. Each `/.toml` file that was present at the end of the build of the previously created OCI image is retrieved. -3. Each `/.toml` file is placed on the filesystem so that it appears in the corresponding buildpack's `/` directory during the build phase. -4. Each `/.toml` file containing `launch = true` and `cache = true` has the contents of the corresponding `/` directory replaced with its contents in the previously created OCI image. +1. All `/.toml` files with `cache = true` and corresponding `/` directories from any previous build are restored to their same filesystem locations. +2. Each `/.toml` file with `launch = true` and `cache = false` that was present at the end of the build of the previously created OCI image is retrieved. +3. A given `/.toml` file with `launch = true` and `cache = true` and corresponding `/` directory are both removed if either do not match their contents in the previously created OCI image. After analysis, the lifecycle MUST proceed to the build phase.