Fixups for flatten feature on pack 0.30.0

[jjbustamante]

Summary

We are planning to release a new experimental feature to flatten builder layers and buildpack package. This feature was requested by the following issue #1595. Since pack 0.30.0-pre2 release a version of the feature was released.
This issue is meant to capture all the bugs to be fixed before we can ship the next pack.

TODO:

• --flatten-exclude should exclude from flattening (vs separately flatten)
• --flatten-include

---

• Flatten buildpack package is move to be experimental #1783 - pack buildpack package --flatten must be released as experimental
• Avoid writing duplicate tar files for flattened buildpacks #1781 - disk issue bug
• Explode flattened buildpacks when pack build #1787 - pack build --buildpack <flattened> bug

[anthonydahanne]

Hello @jjbustamante !
With the addition of --flatten-include, could we add another option that could allow the user to flatten by groups?

Let's take this example, my-composite has 4 different component buildpacks defined in it:

my-composite
├─componentA
├─componentB
├─componentC
├─componentD

I want to flatten A & B together, then C& D together, to obtain:

my-composite
├─componentA&B
├─componentC&D

With pack buildpack package --flatten --flatten-exclude componentC,componentD, I would obtain:

my-composite
├─componentA&B
├─componentC
├─componentD

With pack buildpack package --flatten --flatten-include componentC,componentD, I would obtain:

my-composite
├─componentA
├─componentB
├─componentC&D

As you can see, there's no way to obtain the desired outcome with neither --flatten-include nor --flatten-exclude.

What if we also added a new argument, named --flatten-groups that would allow the user to specify something like:

pack buildpack package --flatten --flatten-groups componentA,componentB|componentC,componentD

to obtain the desired outcome.

Does that make sense? What do you think?

Thank you!

cc @pivotal-david-osullivan , @dmikusa , @loewenstein , @natalieparellano

[jjbustamante]

Hi @anthonydahanne Thanks for this feedback!!!

I haven't defined the flatten-include flag yet but, maybe also like this could work?

pack buildpack package --flatten --flatten-include componentA,componentB --flatten-include componentC,componentD

[anthonydahanne]

Hello!
Yes, that could work as well! Thanks!

[natalieparellano]

@anthonydahanne could what you describe not be obtained with

my-composite
├─compositAB
  ├─componentA
  ├─componentB
├─componentC
├─componentD

And pack buildpack package --flatten --depth=1?

[anthonydahanne]

hello @natalieparellano , as discussed this PM, we (Java Paketo buildpacks) are not using depth, but have been using exclude, even if moving to include based selections, with several inclusions possible, would be preferred.