2021 Roadmap

[ryanmoran]

What should Paketo Buildpacks focus on in 2021?

[ryanmoran]

The Ruby buildpack should have support for alternative interpreters (e.g. JRuby, Rubinius, TruffleRuby).

[ryanmoran]

Similarly, we could add support for Jython and PyPy for the Python Buildpack.

[ForestEckhardt]

Community outreach to spread knowledge and increase adoption of Paketo buildpacks.

[evankanderson]

In particular, it would be great to have some documentation/examples on how to extend a buildpack for your organization, rather than starting over from scratch.

For example, if I need to make sure that my images have a certain set of certificates, or are based off a certain image, what's the "gentle" way of doing this on top of the existing Paketo work, with a minimum amount of forking and carrying patches? (Obviously, there will need to be some patches, but right now I might decide to simply rewrite rather than try to carry large patches. The drawback is that I never pick up the future improvements to paketo from after my copy.)

[cldfzn]

Also as part of the documentation, perhaps a guide to transitioning away from Dockerfiles to buildpacks. For example, one of the things that I'm routinely asked about is volumes and permissions. With Dockerfiles you just create a volume and configure the permissions in the Dockerfile itself. Documenting the current recommended approaches to local configuration/work directories, and other common issues would help with moving to buildpacks.

[ryanmoran]

The buildpacks currently use a mix of buildpack.yml and environment variables to allow for configuration of the build process. We should complete a transition to environment variables across all of the Paketo buildpacks and work to align their format, log output, and documentation such that buildpack users are better aware of what configuration options are set.

[fg-j]

The project should should make inroads with developers in the various language ecosystems to develop a stronger feedback loop between buildpack features and the dev user experience. This could include: blog posts, getting on mailing lists for various langauge ecosystems, survey on the Paketo website asking for more information from (prospective) users.

[ryanmoran]

There is an emerging use case for buildpacks on "function platforms" (e.g. AWS Lambda, Azure Functions, Google Cloud Functions, knative). We should develop buildpacks that enable this use case for the language ecosystems we support.

[ForestEckhardt]

Make the overall user experience and interface (i.e. buildpack configuration, logging, bill of materials) more uniform across all of the Paketo buildpacks.

[ryanmoran]

We should get some input from designers about these interfaces. I'd certainly like to see log output that is clear and terse by default, but can be configured to provide verbose output when needed. Making that a common configuration across all of the buildpacks would be awesome.

[ForestEckhardt]

Grow the number of language ecosystems that Paketo supports to that we can provide a solution for the largest number of use cases possible.

[kvedurmu]

Perhaps we could look into buildpacks for static content frameworks such as Hugo, Jekyll, etc.

[kvedurmu]

We should also work with @dmikusa-pivotal to move his Rust buildpack into Paketo Community org. as specified in https://github.com/paketo-buildpacks/rfcs/blob/main/accepted/0014-rust.md and work on re-architecting it.

[ryanmoran]

There's also this Deno buildpack: https://github.com/andymoe/deno-buildpack

[dmikusa]

+1 for adding some higher-level buildpacks, like the static content frameworks @kvedurmu mentioned. WordPress is also huge & could be a nice add-on in the PHP area.

[ekcasey]

+1 for R (lots of data science applications), Hugo, and Rust

Stack content frameworks are interesting because a wider variety of folks that aren't full time engineers develop static content and a "just works" experience could be very appealing to them.

Wordpress would be hugely valuable but it might be tricky given that it is inherently stateful.

[kvedurmu]

We've made some changes to the website this past year, but I think there's a lot more we can do to improve the homepage nav and docs experience.

[kvedurmu]

We should also look into creating better tutorials on the website. Some ideas:

• Build a simple buildpack
• Optimize your simple buildpack
• Customize your stack

[sophiewigmore]

This definitely ties into this one: #2 (comment)

[kvedurmu]

Make it easier for developers to use Paketo buildpacks in non-production environments.

[fg-j]

e.g. making it easier to run tests within the build process

[fg-j]

e.g. simple configuration that produces images that handshake well with debuggers (e.g. the Debug Buildpack)

[fg-j]

e.g. simple configuration that produces images that handshake well with popular CI/CD tooling

[kvedurmu]

Part of this is ultimately dependent on some sort of "development" API within the CNB project that enables hot reloading of code (e.g making a code change in IDE is propagated to running app image without re-building).

[kvedurmu]

Ties into a couple other comments on this thread, but providing consistent configurations across all buildpacks (via environment variables) also feels important.

[ryanmoran]

We should complete a robust implementation of the Python buildpack and graduate it to the paketo-buildpacks org.

[ryanmoran]

We should complete a re-architecture of the PHP buildpack following paketo-buildpacks/php#267.

[kvedurmu]

We should create a language family web server buildpack as specified in https://github.com/paketo-buildpacks/rfcs/blob/main/accepted/0006-web-servers.md

[kvedurmu]

We should think about how to better track frequency of buildpack releases after dependency updates/new version lines

[ryanmoran]

We should build automation that tracks dependency updates from the dep-server and keeps our buildpack dependencies up-to-date.

[thitch97]

We should more actively seek out new, interesting & unorthodox use cases for buildpacks. This could help to create a more compelling argument for adoption of the technology among new and varied demographics.

[thitch97]

This is a great example: #2 (comment)

[dmikusa]

I would like to see a nice, easy way for users to browse through buildpacks, view dependencies, configuration options, provides/requires integration info, etc... Most of this seems like it could be automatically scraped from metadata files.

Similar to what was available for v2 buildpacks here -> https://buildpacks.cloudfoundry.org/#/buildpacks

[dmikusa]

More transparency about how our stacks are built and what exactly goes into them. This request would be partly education/awareness, but part also making sure that configuration files and information used for this process are publicly visible.

In addition, for the paranoid, the build process for the stacks should be documented so I could follow our docs and build the stack myself (ideally getting the same stack, i.e. reproducible build).

[dmikusa]

Adopt and promote StackOverflow as a preferred location for asking questions related to Paketo buildpacks. StackOverflow is the clear choice of where developers go to ask questions. To facilitate engagement, we should meet folks where they already are going. It also provides a better permanent/searchable record of information than Slack.

[ekcasey]

Reference docs for every buildpack including:

1. Configuration via environment variables
2. Configuration via bindings
3. Build plan entries
4. Contributed labels
5. Contributed environment variables
6. Contributed Slices
7. Contributed profiles or exec.d binaries
8. Contributed Bill of Materials entries (including schema)
9. Contributed asset info (dependency name/version/licenses)
10. Contributed Process types
11. Supported stacks

Ideally we should be able to generate reference docs from buildpack.toml.

[ekcasey]

Include APM buildpacks such as paketo-buildpacks/google-stackdriver and paketo-buildpacks/azure-application-insights in the nodejs language family.

[kvedurmu]

issues for this - paketo-buildpacks/nodejs#311 and paketo-buildpacks/nodejs#310

[ekcasey]

Upgrade all buildpacks to the latest buildpack API.

[ekcasey]

Add utility buildpacks to every language family for a consistent experience

• https://github.com/paketo-buildpacks/ca-certificates
• https://github.com/paketo-buildpacks/environment-variables
• https://github.com/paketo-buildpacks/image-labels
• https://github.com/paketo-buildpacks/procfile

[ekcasey]

Publish all buildpacks to the buildpack registry (https://github.com/buildpacks/registry-index) for easy use with pack

[ekcasey]

Document how to use a different JVM distro (e.g. OpenJDK, Azul Zulu, Amazon Corretto).

[ekcasey]

Java buildpacks on tiny stack.

[ryanmoran]

We'd like to do the same for .Net Core: paketo-buildpacks/dotnet-core#323

[fg-j]

"It just works" support in buildpacks for code generators running before other build steps. go generate is the most obvious example, but as far as I can tell, automatically generating boilerplate code is desirable in lots of language ecosystems. I think it'd be good to:

a) investigate common workflows for code generation in currently supported language ecosystems
b) if some already "just work", document how to use them in a buildpacks context
c) implement new buildpacks or extend existing ones to add even more support for generating code during build

[ryanmoran]

I've gone through the comments above and synthesized the suggestions into a few top-line themes. Here is what we think is going to be important to focus on in this next year:

Maturation of the existing buildpack featureset

• Uniform buildpack interface
  • Common configuration input
  • Common logging output
  • Common Bill of Materials
  • Utility buildpacks
    • ca-certificates
    • environment-variables
    • image-labels
    • procfile
• New internals, runtimes, and integrations
  • PHP rearchitecture
  • New runtimes for the Ruby buildpack
  • New runtimes for the Python buildpack
  • APM integrations for the Node.js buildpack
  • Code generation (e.g. go generate)
• New stack support
  • Go on Windows
  • Java on Tiny
  • .Net Core on Tiny

Expansion of the buildpack ecosystem

• Community outreach
  • Blog posts
  • Stack Overflow
  • Mailing list posts
  • Survey
• Documentation and Reference Materials
  • Tutorials
    • Build a simple buildpack
    • Optimize your simple buildpack
    • Extend buildpack behavior
    • Build a stack
    • Customize your stack
    • How do we build our stacks?
  • Auto-generated Reference Materials
    • Configuration via environment variables
    • Configuration via bindings
    • Build plan entries
    • Contributed labels
    • Contributed environment variables
    • Contributed slices
    • Contributed profile.d scripts or exec.d binaries
    • Contributed bill of materials entries (including schema)
    • Contributed asset info (dependency name/version/licenses)
    • Contributed process types
    • Supported stacks
• Discoverability
  • Publish buildpacks on the CNB registry
  • Dashboard with buildpack details (like Existing Cloud Foundry Dashboard)
    • Dependencies
    • Configuration options
    • Provides/Requires integration APIs
    • Stack contents
  • Track release frequency/latency
  • Track dependency updates
• New buildpacks
  • Python
  • Functions
  • Rust
  • TypeScript
  • Swift
  • Erlang
  • Elixr
  • Dart
  • R
  • LLVM (Clang)
  • Perl
  • Lua
  • Ada
  • Julia
  • Haskell
  • Crystal
  • Nim
  • Pony
  • Deno
  • Web Server language family
  • Hugo
  • Jekyll
  • WordPress

Non-production use cases

• IDE integrations
• Local-development framework integrations
• Running tests
• Integrations with debuggers
• CI/CD integrations

[zhangtbj]

Cool and thanks for the sharing!

A quick question about function platforms.

Will normal application buildpacks and function platform buildpacks be ine ONE buildpacks/builder image or they will be two separate buildpacks/builder images?

Thanks!

[ryanmoran]

I think that ideally we would have them in a single buildpack, but we are still working out what the best experience will look like.

[zhangtbj]

Yes, It is better to have an all-in-one buildpack to cover and auto-detect all. But I think Paketo needs to take care:

• The builder image size, add more buildpacks means the builder image size will be increase more and more, it is better to control or reduce the size
• How to detect the normal app and function, like Node.js application and Node.js function. What I can think is: introduce a new config file, like paketo.yaml in source code then mark it is a normal app or function, etc...

We are studying the Riff Function buildpacks: https://github.com/projectriff/builder

It is a good news if Paketo also support that in future. Thanks! :)

[warlockdn]

Support for M1 devices.

[JuanM04]

Support for Yarn Berry (both v2 and v3) (ref)