RFC: Support for pack.toml

[jkutner]

Readable

[buildpack-bot]

Maintainers,

As you review this RFC please queue up issues to be created using the following commands:

/queue-issue <repo> "<title>" [labels]...
/unqueue-issue <uid>

Issues

(none)

[loewenstein]

I understand that e.g. builder might be difficult to impossible to support. On the other hand side, introducing this special handling for the platform maintained as part of CNB core feels a bit like opening the box of Pandora.

[hone]

@jkutner do you mind if I move this to "blocked" on #191?

[loewenstein]

Is #191 the same thing that @jromero is currently drafting in https://hackmd.io/4LGBHOZAQ6GMOTeDf2U3_w?view#What-it-is? If yes, the existence of project.toml namespaces like io.buildpacks.pack.* leads me to the question if that would supersede the proposal for a pack specific configuration file...?

If I am right in this, then section about merging pack specific config seems to build a circle with this rfc that might be worth resolving ;)

Thoughts? @jromero @jkutner

[jkutner]

@loewenstein I think this RFC started the conversation that led to @jromero's hackmd draft RFC. I'm supportive of that one, and will likely close this once we turn his into a real RFC

[loewenstein]

@loewenstein I think this RFC started the conversation that led to @jromero's hackmd draft RFC. I'm supportive of that one, and will likely close this once we turn his into a real RFC

I was asking because the proposal of @jromero seems to try to accommodate this one ;)

[jromero]

@jkutner, I submitted the RFC for a Prepare Operation and excluded the section referring to the pack.toml specifically in order to keep the RFC as concise as possible. I'd like to propose that we continue to make changes to this RFC separately with the other RFC in mind.

[jromero]

@jkutner have you had a chance to review my latest comments?

@dfreilich your feedback would also be appreciated here since this seems rather impacting.

[loewenstein]

Has there been any decision with regards to additional file vs. some flexibility and additional tables in the existing project descriptor file?

[jkutner]

@loewenstein no decision, but I'm reopening this PR for discussion. My hope is that this will allow us to introduce pack specific attributes in the pack.toml (like for multiple images) that would not necessarily be honored by all platforms. Is that something that might work for you?

[jkutner]

Suggested change

	* If one or more descriptors are provided on the command line they will all be merged and used.
	* If one or more descriptors are provided on the command line, all top-level keys will all be merged and the resulting descriptor will be used.

[loewenstein]

@loewenstein no decision, but I'm reopening this PR for discussion. My hope is that this will allow us to introduce pack specific attributes in the pack.toml (like for multiple images) that would not necessarily be honored by all platforms. Is that something that might work for you?

I guess I'd still prefer to have "project" related configuration in a single file, i.e. the project.yaml file. With the sole exception of maybe entirely only pack concerning config, but I think pack has config for e.g. default builders already in the user home.

[jkutner]

@loewenstein IIUC that matches the intent of this proposal. Any general project configuration can go in project.toml (including pack config if you want it there). Pack-only config can go in pack.toml. What's the gap from what you described?

[joshwlewis]

io.buildpacks.builder already seems to be supported in project.toml here: https://github.com/buildpacks/spec/blob/main/extensions/project-descriptor.md#iobuildpacksbuilder-optional. Is this different somehow?

[jkutner]

I think that may have been added since I first wrote this. I'll update with a different example that shows something that's specific to pack (like tag or maybe env-file)

[hone]

Moving this into FCP for next Thurs Dec 8.

[hone]

@jkutner how does this play with $HOME/.pack/config.toml? For example here is what mine looks like:

default-builder-image = "heroku/buildpacks:20"
experimental = true

[[trusted-builders]]
  name = "ops0-artifactrepo1-0-prd.data.sfdc.net/docker-sam/buildpacks/sfdc_centos7_cnb:builder"

Would these be supported in pack.toml as a way to have app/project level overrides? I'm thinking something along the lines of how git-config does this.

[loewenstein]

@loewenstein IIUC that matches the intent of this proposal. Any general project configuration can go in project.toml (including pack config if you want it there). Pack-only config can go in pack.toml. What's the gap from what you described?

I guess my main concern is with things that are maybe pack only today, but will be picked up by other platforms in the future. From my point of view, it would make sense for most of the configuration you would possible want to support to go through the project descriptor spec and potentially align support across platforms.

[jjbustamante]

I really like the motivation: advertises that the repo containing the file can be built with pack

Now, I have the following doubt: Is there an idea to accomplish that? I mean, for example, if we think of developers that use vscode and they want some extension for Docker, then they have a lot of options. Are we considering creating or maintaining our own IDEs extensions for CNB tooling?

[schneems]

I would like some outputs from pack to indicate the config being used (even better if it could trace it back to the origin and say why). That way users can know at a glance if their changes have been picked up or if they're not being picked up, get a hint that maybe another file is taking precedence.

I'm unsure whether that desire can be encoded in the RFC in a sensible way.

[hone]

Suggested change

	* An app MAY have both a `pack.toml` and a `project.toml`, in which case the `project.toml` will be ignored.
	* An app MAY have both a `pack.toml` and a `project.toml` where they will be merged and used.

[jkutner]

@hone we've gone back and forth on this. Merging could actually be very complicated, and if we support merging we should only do it at the top level shared key (i.e. we're not going to try and merge lists of things for example)

[loewenstein]

@jkutner this is precisely why I have a problem with a platform specific descriptor covering platform agnostic configuration.

I would want to have as much as possible in an agnostic way, at least from the perspective of all platforms other than pack. Why? Because for 99% of the non pack platforms I can imagine, pack will be the platform for local execution. See kpack and our integration into project Piper as two examples and Tekton most likely as a third.

Also, as I pointed out in #189 (comment) the “project.toml will be ignored” statement contradicts other parts of this proposal.

[sambhav]

I still see multiple project descriptor related RFCs open. Can we consolidate them and figure out how each fits in the larger picture? I don't want there to be multiple ways of configuration as that just leads to complicated code and lots of corner cases. If we do decide to implement multiple related RFCs, I would also like to see how they interplay with each other and what precedence would look like amongst them.

[jkutner]

I'm going to close this RFC again. There is enough about merging pack.toml and project.toml that isn't clear, and there isn't enough interest in using pack.toml (even from me) to make me move it forward.