Establishing Comprehensive Guidelines for Component Maturity in Kubeflow

[StefanoFioravanzo]

Historically, we never had an extensive and clearly defined maturity model for our components, covering the entire lifecycle from “new alpha” all the way to “stable with long term support”. We need a well-defined framework to gauge component maturity, now more than ever, to set expectations right, to allow Kubeflow release teams to have a decision making playbook and work together with WG leads to ensure the quality and stability of components.

Current State and Challenges:

Despite having a versioning scheme indicating alpha, beta, and stable stages, as documented in our support policies, Kubeflow currently lacks detailed guidelines for component maturity. This absence makes it challenging for component owners to understand what is required to progress through these stages. Moreover, the original guidelines on what constitutes a "stable" component, available here, are outdated and were crafted by contributors who are no longer active in our community.

The critical missing parts are definitions of requirements to qualify a component as “alpha” but ready to be included in a release, and the graduation criteria to beta and then stable. Tying together this maturity model with the support model is also required.

Why Better Guidelines are Important:

• Short-term: New components are continually being proposed and developed. Clear guidelines will aid in evaluating these contributions consistently and integrating them smoothly into Kubeflow releases.
• Long-term: Well-defined and organized guidelines are foundational for fostering community participation, growth, and collaboration. They ensure that all contributors are aligned on quality standards and expectations, facilitating a more robust and reliable ecosystem.

Guideline Development:

My proposal is to rework our outdated “Guidelines for Applications Requirements”, focusing first on “alpha” requirements for new components to be accepted in a release. Some high level topics that these guidelines should focus on (some of these are already addressed in existing doc) are: code stability, testing infrastructure, documentation, upgrades and multi-version support, community and user adoption, feature completeness, working group maturity.

Additional Considerations:

• Incorporation of security requirements?
• Consolidation and updating of existing guidelines and support policies.
• We should learn from CNCF and other projects, such as the Kubernetes enhancements process and Production Readiness Review (PRR), detailed in this blog post.
• With "component" I not only mean an entire project/repo but also significant features within those projects (think about the new Fine Tune APIs for LLM).
• We should take inspiration from what other communities are doing, but think critically about what we need as the Kubeflow community, especially considering our current evolution state. This will not be a one time thing, but a leaving and breathing document/guideline that will evolve and mature over time.

Please share your thoughts @jbottum @andreyvelich @terrytangyuan @johnugeorge @james-jwu @kubeflow/release-managers

[StefanoFioravanzo]

@akgraner maybe you can add some perspective based on your experience from other communities.

[jbottum]

@StefanoFioravanzo thanks for this work. My initial view is that the original guidelines are a good starting place. They should be updated and they may need to be adapted to incorporate the CNCF graduation requirements. I like initially focusing on the stable, 1.0, graduated guidelines because of our outstanding CNCF workload and requirements. I think many components are considered stable and having a checklist will be a best practice to help deliver consistent quality and user experience. My concern with starting with alpha is that some components might want to build functionality rather than admin functionality (i.e. full multi-user), and I kinda would like to leave the alpha, beta definitions somewhat flexible, but I am open other opinions.