Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add operational criteria to Enhancement template #552

Merged
merged 2 commits into from
Dec 9, 2020
+11 −2
Merged

Add operational criteria to Enhancement template #552

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
51832d6
Add operational criteria to Enhancement template
jeremyeder Nov 30, 2020
7715e5b
Address comments
jeremyeder Dec 2, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions guidelines/enhancement_template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ around the enhancement process.
- [ ] Enhancement is `implementable`
- [ ] Design details are appropriately documented from clear requirements
- [ ] Test plan is defined
- [ ] Operational readiness criteria is defined
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is vague to me. I like the idea, but I think there are probably 200 different ideas of what this means.

Specifically, I'm not sure if this is 'As an managed OpenShift SRE' or 'As a customer cluster operator'. I know it's partly both, but focusing on one or the other might help engineers understand what they actually need to be thinking about.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to decide how much goes here vs another doc. I can link out, if you think that's the right approach.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summarizing our offline discussion; the enhancement template doesn't include details or links to other implementation guides, so we will continue that pattern here. I intend to find teams (maybe review some existing enhancements) to work with directly to gather some more detail.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should change that pattern, but ok for now given there's an effort to define these somewhere.

- [ ] Graduation criteria for dev preview, tech preview, GA
- [ ] User-facing documentation is created in [openshift-docs](https://github.com/openshift/openshift-docs/)

Expand Down Expand Up @@ -102,13 +103,15 @@ and make progress.

This is where we get down to the nitty gritty of what the proposal actually is.

### User Stories [optional]
### User Stories

Detail the things that people will be able to do if this is implemented.
Include as much detail as possible so that people can understand the "how" of
the system. The goal here is to make this feel real for users without getting
bogged down.

Include a story on how this proposal will be operationalized: lifecycled, monitored and remediated at scale.

#### Story 1

#### Story 2
Expand Down Expand Up @@ -145,6 +148,7 @@ to implement the design. For instance,
Consider the following in developing a test plan for this enhancement:
- Will there be e2e and integration tests, in addition to unit tests?
- How will it be tested in isolation vs with other components?
- What additional testing is necessary to support managed OpenShift service-based offerings?

No need to outline all of the test cases, just the general strategy. Anything
that would count as tricky in the implementation and anything particularly
Expand Down Expand Up @@ -189,12 +193,17 @@ These are generalized examples to consider, in addition to the aforementioned [m
- End user documentation, relative API stability
- Sufficient test coverage
- Gather feedback from users rather than just developers
- Enumerate service level indicators (SLIs), expose SLIs as metrics
- Write symptoms-based alerts for the component(s)

##### Tech Preview -> GA

- More testing (upgrade, downgrade, scale)
- Sufficient time for feedback
- Available by default
- Backhaul SLI telemetry
- Document SLOs for the component
- Conduct load testing

**For non-optional features moving to GA, the graduation criteria must include
end to end tests.**
Expand All @@ -218,7 +227,7 @@ enhancement:

Upgrade expectations:
- Each component should remain available for user requests and
workloads during upgrades. Any exception to this should be
workloads during upgrades. Ensure the components leverage best practices in handling [voluntary disruption](https://kubernetes.io/docs/concepts/workloads/pods/disruptions/). Any exception to this should be
identified and discussed here.
- Micro version upgrades - users should be able to skip forward versions within a
minor release stream without being required to pass through intermediate
Expand Down