Added spec changes for Image Extension

[jkutner]

WIP specification of buildpacks/rfcs#173

[jkutner]

@sclevine @natalieparellano PTAL and let me know if this is on the right track

[jkutner]

Per @sclevine, we should not introduce a new API version, but instead tie this to one of the existing APIs. We also need to address this in the extension.toml

[jabrown85]

Suggested change

	- MUST NOT write to the `<app>` directory.
	- MUST NOT write to the `<app>` directory.

Is this one of those places where we hope they won't but can't do anything if they do?

[jkutner]

@jabrown85 i think it depends on the platform. Like, if it's running the extension in it's own container, it might be able to exclude <app> dir changes from it's snapshot. But if it's all running on the same filesystem/volume it won't be able to.

This isn't actually a suggested change right?

[jabrown85]

Yeah, not a suggested change. I must have clicked the wrong button :)

[jabrown85]

This is the first time run.Dockerfile is mentioned, should we omit this specific file name here in favor of keeping that in the image-extensions.md? Just refer to Dockerfile here in a generic sense maybe...

[jabrown85]

I would love to see a section on how a platform should use this $build_id. I kind of forgot how/why it is done this way and when a platform would ever be able to safely use the same UUID.

[jabrown85]

I think we may need /workspace | CNB_APP_DIR as well, since they are allowed/expected to read from that directory.

[natalieparellano]

https://github.com/buildpacks/samples/pull/113/files#r726206827 talks about creating a separate [[order-ext]] in builder.toml - would that translate into a separate file?

[natalieparellano]

I'm not quite sure what to make of these bullets. I think they could be removed (when talking about <layers> they make sense but we don't really care about the output directory for extensions, only the files contained there).

[natalieparellano]

What is this for?

[natalieparellano]

Suggested change

	The `base_image` arg allows the Dockerfile to reference the original base image.
	The `base_image` arg MUST be the original base image.

[natalieparellano]

Suggested change

	The `build_id` arg allows the Dockerfile to invalidate the cache after a certain layer and must be defaulted to `0`. The executor of the Dockerfile will provide the `build_id` as a UUID (this eliminates the need to track this variable).
	The `build_id` arg MUST be a UUID. This allows the Dockerfile to invalidate the cache after a certain layer.

[natalieparellano]

We should also mention (somewhere) that meta-buildpacks may not contain extensions.

[natalieparellano]

Just making a quick note for posterity on what we'll probably still want to add:

• maybe make more clear the distinction between extending the build image vs. the run image (something like "image-extender MAY run within the context of the build container to extend the build image directly prior to buildpack execution"); we could also clarify that extending the build image and extending the run image are likely different phases (or phase executions) that could happen in parallel
• probably some updates to the distribution spec for builder changes
• updates to the platform spec around rebase (e.g., use of the --force flag)
• resolution of current questions around the presence of /cnb/image/genpkgs in the runtime base image
  (EDIT: added)
• what metadata (if any) should go on the image so that data from previous build extensions can be restored?

[natalieparellano]

I started working on a POC lifecycle that can:

• Run /bin/detect for extensions as part of regular detect
• Run /bin/build for extensions as a new invocation of build
  Together with the "extender" POC that @cmoulliard is working on, this could help uncover the remaining uncertainties that we'll need to resolve in the spec. Some questions and comments so far are noted here and listed below:

• Questions around how extensions should be identified in order.toml and group.toml (see RFC comment)
• What should be the format of the file that we’re using to pass the Dockerfile paths & args to the extender? And what should be the path (currently using /layers/config/metadata.toml, but we probably want a separate file for extensions and buildpacks)
• optional=false is the default value for buildpacks, but optional=true is the default value for extensions - how to handle this cleanly?
• We need the build phase for extensions to output plan.toml so that we can filter the plan for buildpacks
• What should be the default parent dir for extensions output - maybe /layers/config/ext?

[hone]

@jkutner I'm going to closes this in lieu of #298. Feel free to re-open if that's not the case.