RFC to introduce Cloud Native Buildpacks lifecycle #796
Conversation
beyhan
requested review from
a team,
rkoster,
beyhan,
stephanme,
ameowlia and
ChrisMcGowan
and removed request for
a team
|
|
||
| #### Add Paketo Stack(s) | ||
|
|
||
| This RFC does not include the addition of a new stack to Cloud Foundry, rather the resulting droplet would run on top of the existing `cflinuxfs4`. This should work for most, if not all apps, as the stack is much bigger (in terms of native packages installed) than the stacks used by Paketo. |
There was a problem hiding this comment.
The mechanism that currently exists rebases the base layer(stack) on every application restart onto the droplet.
The behaviour CNBs have is to add the base image (stack equivalent) at build time to a resulting OCI Image.
The way this mechanism works is different when building internally or externally with the same CNB.
The way CNBs add the base layer while building can be favorable for the stack depreciation problem where currently its not possible to remove a stack with application still depending on it. When the stack gets added at build time this changes as then missing stacks just prevent restaging. Already build droplets include the stack in the droplet and keep running.
However this also comes with a some downsides:
- Stack update means restaging all apps(resource costly, complex)
- The stack is saved multiple times in the blobstore(storage and traffic)
Although using the build workflow that is common for CNBs in CF would help plus make the systems behave similar it comes with some costs im not sure if they are worth it yet but maybe something to consider.
There was a problem hiding this comment.
I'd say that the thought of using CNBs to produce full OCI images is independent from this "potential future work". Also for a droplet based approach, the smaller Paketo base images in Static, Tiny and Base are an improvement for application security imho.
There was a problem hiding this comment.
@FloThinksPi I would say you are calling for a separate RFC that's largely independent from the Cloud Native Buildpack topic. Are we good to resolve this question?
There was a problem hiding this comment.
Not to de-rail - but is this a point worth considering further:
The stack is saved multiple times in the blobstore(storage and traffic)
This is a core benefit of an OCI registry that it should see the layers and thus save a tremendous amount of space. Re-using the blobstore certainly is simplest, but there are cons on the other end to my eye.
Maybe that concern belongs in the follow on RFC?
There was a problem hiding this comment.
I am not sure if moving CF integrated CNBs from creating droplets to creating container images is something we want. If we don't, then there is no problem with a stack saved multiple times, because the roots is saved separately from the droplets.
|
|
||
| ## Problem | ||
|
|
||
| The v2 buildpacks are effectively in maintenance mode and do not receive substantial new features. By not integrating with v3 buildpacks, Cloud Foundry is missing out on new buildpacks (e.g. Java Native and Web Servers CNBs) as well as any new features that are added to the still-actively-developed v3 buildpacks. |
There was a problem hiding this comment.
The folks on our side are doing development on both V2 and CNBs in parallel, I'm seeing new capabilities come online for V2 frequently as well. What's giving you folks this impression? This will be significant work in quite a few areas to support CNBs in CF and I want to make it's worth it.
There was a problem hiding this comment.
We've taken this pretty literally from RFC 0017.
Regarding the amount of work...
This will be significant work in quite a few areas to support CNBs in CF and I want to make it's worth it.
... our plan was not only to propose this RFC, but also to work on contributing the implementation.
There was a problem hiding this comment.
@dsboulder would you want to propose a different wording or are we good to resolve this question?
| - No OCI registry necessary | ||
| - Reuse cflinuxfs4 as rootfs during build and run | ||
|
|
||
| ### High Level Implementation |
There was a problem hiding this comment.
Ideally a heading would be added discussion what to do with service bindings. I know some prior work was done in: buildpacks/libcnb#228, but is this library used by all Paketo buildpacks? and what about non Paketo cnbs?
There was a problem hiding this comment.
I never saw a great solution to the VCAP services => CN bindings problem? Is there a way to make this work that isn't a ton of brittle translation tables?
There was a problem hiding this comment.
@rkoster @dsboulder @beyhan Should we add a little subsection referring to buildpacks/libcnb#228, the possibility to do the same for packit based Paketo buildpacks and referring to #804 for solving it once and for all?
In short, I would probably write this as
Buildpacks that need credentials from services should be adapted to read from
VCAP_SERVICESuntil #804 removes this requirement.
Note: Buildpacks based onlibpakorlibcnbalready understand credentials fromVCAP_SERVICESsince libcnb#228 has been merged and released.
There was a problem hiding this comment.
Ideally a heading would be added discussion what to do with service bindings.
Should it really? It feels a bit out of place because...
a) This is not necessarily cf related, but rather buildpack related
b) This feels like an implementation detail as already established in the predecessor rfc (here)
What should we mention here? That cf does not have to change?
There was a problem hiding this comment.
Ok, then can we mention that it is out of scope for this RFC to change anything in how binding information is handled by the CF runtime. That means CNB based apps will get VCAP_SERVICES as env var and apps or buildpacks have to deal with this. Future RFCs like #804 will deal with this.
There was a problem hiding this comment.
There was a problem hiding this comment.
Should it really? It feels a bit out of place because...
Sure now that we have an RFC addressing the problem of servicebinding spec compatibility there is no need to discuss it in this RFC. A link to the service-binding RFC would be nice so that future readers understand how they relate.
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
|
@beyhan we added a section regarding consuming buildpacks from private registries. We did not yet decide on an implementation. Does this require additional reviewers? |
@modulo11 When the discussions are resolved I will require review from the involved people anyway. Regarding the change could you please comment why the current support for private registries described in https://docs.cloudfoundry.org/devguide/deploy-apps/push-docker.html#registry isn't helpful or can't be reused. I think this comment/explanation could be added also to the RFC. |
Co-authored-by: Jan von Löwenstein <jan.von.loewenstein@sap.com>
I updated the section. It's basically the lack of support for multiple registries. |
|
@cloudfoundry/toc if @rkoster and @beyhan are ok with #796 (comment), i.e. relying on the Service Binding section in the older CNB RFC, there are no pending discussions left I believe. How do we bring this forward for a vote? I guess it would be good to have explicit buy-in to review the necessary changes in @cloudfoundry/wg-app-runtime-platform (new app lifecycle in the Diego Bosh release) and @cloudfoundry/wg-app-runtime-interfaces (CAPI and CLI changes). Once the vote is done and positive and this RFC merged, we'd need to request the creation of one new repository (for the new app lifecycle) - is there a process for this? |
|
@loewenstein I added your last comment as topic for the next TOC meeting. |
beyhan
requested review from
rkoster,
loewenstein,
stephanme,
FloThinksPi,
c0d1ngm0nk3y,
pbusko and
PlamenDoychev
Co-authored-by: Jan von Löwenstein <jan.von.loewenstein@sap.com>
|
We discussed the last comment during the TOC meeting today 23.04.24 and we decided to start the FCP with the goal to approve this RFC next Tuesday 30.04.24. |
| - Increase adoption of Cloud Native Buildpacks | ||
| - Open the door for eventual deprecation of the v2 buildpacks, reducing maintenance costs (v2 buildpack deprecation is NOT included in this RFC) | ||
| - No fundamental changes to the architecture of CF | ||
| - Result of the staging process will be a droplet |
There was a problem hiding this comment.
The CNB lifecycle supports exporting to OCI layout on disk (the resulting image can even be "sparse" in that run image layers will be missing). This should facilitate exporting to a CF droplet without having to re-implement large parts of the exporter.
|
@loewenstein Additionally, we discussed that @ameowlia, @rkoster and @stephanme will support the contributions to get approved. You can also open a pr like #730 to add the new lifecycle repository and @ameowlia will help with finding the right place for it. |
Co-authored-by: Jan von Löwenstein <jan.von.loewenstein@sap.com>
This RFC tries to gently introduce Cloud Native Buildpacks to Cloud Foundry. It builds upon RFC 0017 but introduces CNBs directly in the platform instead of shimming them.
Preview
@cloudfoundry/toc
CC @loewenstein @c0d1ngm0nk3y, @pbusko, @nicolasbender, @beyhan, @stephanme, @stefanlay, @PlamenDoychev