Added proposal process. #15
Conversation
There was a problem hiding this comment.
I'm a big fan of this approach - making big design docs public will only increase the visibility and accountability of our descisions.
In the spirit of this doc - it needs to be reviwed and approved by the leads so @squat & @simonpasquier
Also we should mention what make a confidential proposal? We talked yesterday about what makes a confidential doc: reference to specific customers, specific configurations etc. Is a confidential proposal any different? It could propose something that needs to be communicated with a partner perhaps? Worth being explicit.
bwplotka
force-pushed
the
proposal-proposal
branch
from
b239387 to
ea0d956
Compare
|
Thanks for review @ianbillett Addressed comments, PTAL |
|
LGTM - I see this move as a positive thing. We tend to live in Github for the most part and I think this makes collaboration easier/better. Also benefits by making decisions open and adding visibility. |
There was a problem hiding this comment.
I'm comfortable with these changes - but let's get @squat and @simonpasquier signoff too as team leads 👍
|
|
||
| Let's assume the idea sounds interesting to you; what to do next, where to propose it? How to review it? Follow the algorithm below: | ||
|
|
||
|  |
There was a problem hiding this comment.
I am a bit concerned with having only leads and architects as reviewers. That amounts to a pool of 3 people out of ~20 who can add feedback and approve changes. It also inhibits knowledge sharing and collaboration within the team since other members will be left out.
There was a problem hiding this comment.
I agree, was thinking about this too. Everyone should review. But question is, who is making final decision? What is the final vote? I would be totally for equal vote priority, but this would mean you have to have 11 out of 20 ppl to review which is unrealistic too. So at then end lead and architects has to know about it no?
This is BTW rare: What it will come next is that I will propose same to CMO, Observatorium, Prometheus Operator, Prometheus, Thanos etc, and then normal procedure applies as to any bigger change (consensus in maintainers team)
There was a problem hiding this comment.
Any proposed flow? Just get it reviewed by interested team members you choose? Consensus in the team?
There was a problem hiding this comment.
I think it makes complete sense to have the architect and lead from the respective team in the list of reviewers.
I would, however, also encourage involving team members with the subject-specific domain knowledge in the review process. We have a list of contributors in this doc that we can use to select the appropriate team members. I think anywhere from 2 to 4 reviewers is manageable without creating too much overhead.
In addition to that, anyone should feel free to comment on the proposal, regardless of whether they're selected as reviewers or not. In my view, a proposal should be treated as a tool for collaboration, innovation and dissipating knowledge within the team.
There was a problem hiding this comment.
Review should be open to everyone, even outside of the team: this is open-source after all and anyone can comment, which is free input for us.
We do want some accountability when it comes to approval to ensure that things do not hang around in limbo indefinitely and ensure that proposals are either accepted or rejected. I originally proposed a few weeks ago at a team sync that we follow a structured process for our design proposals to avoid them getting stuck in this design-purgatory that many of our docs have ended up in. This limbo is really sad because it means that engineers who write designs never get feedback on their work, recognition for their good ideas (especially if the idea is implemented but the doc was not "accepted"), and appreciation for the hard effort they made. I had suggested we follow a process inspired by Hashicorp's RFC process [0]:
Once you’ve written the first draft of an RFC, share it with your team. They’re likely to have the most context on your proposal and its potential impacts, so most of your feedback will probably come at this stage. Any team member can comment on and approve an RFC, but you need explicit approval only from the appropriate team leads in order to move forward. Once the RFC is approved and shared with stakeholders, you can start implementing the solution. For major projects, also share the RFC to the company-wide email list. While most members of the mailing list will just read the email rather than the full RFC, sending it to the list gives visibility into major decisions being made across the company.
In our case, the architect would be ultimately responsible for signing off on something, but ANYONE can give feedback. We can even say: X people are chosen to review and give expert input on given document and present recommendations to the architect who ultimately signs off (or not). Alternatively, these X people could approve/reject the document by consensus.
[0]
There was a problem hiding this comment.
Added mention to that to our process
|
|
||
| * Even if our Google Design docs organized in our team drive, those Google documents are not easily discoverable. | ||
| * Existing Google doc-based documents are hard to consume. The formatting is widely different. Naming is inconsistent. | ||
| * Document creation is rarely actionable. There is no review process, so the effort of creating a relevant document might be wasted, as the document is lost. This also leads to docs being in the half-completed state, demotivating readers to look at it. |
There was a problem hiding this comment.
How will the new process ensure that people drive their proposals to completion?
There was a problem hiding this comment.
Very solid point. It's always based on authors motivation. Usually, the problem is not author motivation but lack of process, lack of visibility or decision making time. I think this proposal fixes those things.
It does not fix the motivation part. If author does not want to finish the proposal, maybe it's not that important and it will get just stale PR, closed after some time of inactivitiy
There was a problem hiding this comment.
In my opinion, the guarantee of a process that says "your doc will be reviewed by a human in X time and you will be given feedback" is a good motivation. This, coupled with "your work is recognized and you can add it to your list of achievements" is much more than we currently have and should motivate the completion of designs more than we already do. The goals is not to ensure 100% completion of a design doc; it's impossible and not part of anyone's job description. But a structures process should create a positive difference IMO
|
|
||
| Let's assume the idea sounds interesting to you; what to do next, where to propose it? How to review it? Follow the algorithm below: | ||
|
|
||
|  |
There was a problem hiding this comment.
Is there an SLO on how long we want proposals to:
- Stay in Review state
- Stay in Approved state before starting implementation.
There was a problem hiding this comment.
I don't think so. It's similar to PR (code change) process now? Depends on sprint priority. Do you think we need SLO?
There was a problem hiding this comment.
I think there should be an SLO for how long things stay in review state but not necessarily how long before they are implemented. Our problem to-date has been that docs hang around in review-limbo for eternity, which is super demotivating for writers and means they don't get credit for their good ideas. Whether or not a design is implemented is a different problem: the design may be a great idea but the feature may not be prioritized by the PM so it's not on our roadmap for the coming X quarters. Architects and leads should be responsible for circulating designs to the right ppl, e.g. managers and PMs, and to advocate for prioritizing them as needed, but it's not only up to the team to decide what we work on. Otherwise, cynically, this SLO would be a backdoor to override the needs of Red Hat as a business and get engineer hours dedicated to pet projects that have architect approval.
There was a problem hiding this comment.
What specific SLO we should set?
Each iteration between change and review should not last longer than 3 days? 🤔
There was a problem hiding this comment.
I think it makes sense to set something that is simple to start with. We can always adjust it if we need to. E.g. we can say each proposal needs to be approved or rejected within 1 month. We can also easily measure this SLO to see whether we are breaching it.
I guess the more important question is, what is the SLA for breaching the SLO :)
There was a problem hiding this comment.
@squat yeah I get your point, makes total sense and at the end of the day someone or some relativity small group need to take the lead and "steer the ship" so I am all for it in that respect.
There was a problem hiding this comment.
The problem IMO has not been that no one reads proposals but rather that they are read and they are never officially taken out of limbo.
I agree that this is a problem, I am just not sure if approving a proposal and not acting on it makes things better. With the current team processes, sprint and roadmap planning is done by a handful of people. This makes it hard for proposal authors to get visibility into if and when their proposal would be planned for implementation. Approving a proposal and not implementing it for a year or two still feels like a limbo state.
It was mentioned above that approving and implementing a proposal should be treated as different problems. I think this could be a good way to look at things. However, not having a feedback loop for the implementation phase leaves a big gap in the process.
There was a problem hiding this comment.
I totally see this problem, but I think it's better to NOT couple the proposal process with the implementation process.
I am just not sure if approving a proposal and not acting on it makes things better.
I have to disagree. (: Let's say we don't have time to implement a new idea which someone has in mind. Do we think it will help to NOT do the proposal at least? I feel it's extremely helpful to at least cover that idea while it's hot, so anyone can take it up anytime later. Especially when we are working open-source it helps the community to get it up for us too! Not mentioning we would have a ready answer if someone asks about new idea that was discussed before. We can point how we would discuss this and that person can either update proposal if needed with new info or see what was decided before.
Also we do the same for bugs no? We acknowledge them but sometimes we can't fix everything. Still it's good to report them instead of ignoring and move the fix process as much as further as possible.
However, not having a feedback loop for the implementation phase leaves a big gap in the process.
In some way there is a process for planning things for our teams: This is part of backlog grooming, sprint planning, syncs with team leads. Those are not properly documented, and probably there should be another piece of documentation for that WDYT? (:
There was a problem hiding this comment.
Ok, this seems like a good direction. To summarize:
- Keep approval and implementation phases independent
- Potentially iterate on adding visibility into planning and grooming processes of the teams. I would not rush this work to much before the team stabilizes. Constantly updating documentation can be a bit wasteful :)
There was a problem hiding this comment.
Proposal - follow KEP process.
- Proposal itself should state who is the approver/reviewer (creator can assign on first draft and can be changed in further iterations)
- Proposal should state if it is implementable right now or if it isn't
- Proposals without any activity are labeled as
staleafter 60d of inactivity - Stale proposals are closed after an additional 30d of inactivity
More in KEP-0000 - https://github.com/kubernetes/enhancements/blob/master/keps/sig-architecture/0000-kep-process/README.md
|
|
||
| * Even if our Google Design docs organized in our team drive, those Google documents are not easily discoverable. | ||
| * Existing Google doc-based documents are hard to consume. The formatting is widely different. Naming is inconsistent. | ||
| * Document creation is rarely actionable. There is no review process, so the effort of creating a relevant document might be wasted, as the document is lost. This also leads to docs being in the half-completed state, demotivating readers to look at it. |
There was a problem hiding this comment.
Very solid point. It's always based on authors motivation. Usually, the problem is not author motivation but lack of process, lack of visibility or decision making time. I think this proposal fixes those things.
It does not fix the motivation part. If author does not want to finish the proposal, maybe it's not that important and it will get just stale PR, closed after some time of inactivitiy
|
|
||
| Let's assume the idea sounds interesting to you; what to do next, where to propose it? How to review it? Follow the algorithm below: | ||
|
|
||
|  |
There was a problem hiding this comment.
I agree, was thinking about this too. Everyone should review. But question is, who is making final decision? What is the final vote? I would be totally for equal vote priority, but this would mean you have to have 11 out of 20 ppl to review which is unrealistic too. So at then end lead and architects has to know about it no?
This is BTW rare: What it will come next is that I will propose same to CMO, Observatorium, Prometheus Operator, Prometheus, Thanos etc, and then normal procedure applies as to any bigger change (consensus in maintainers team)
|
|
||
| Let's assume the idea sounds interesting to you; what to do next, where to propose it? How to review it? Follow the algorithm below: | ||
|
|
||
|  |
There was a problem hiding this comment.
I don't think so. It's similar to PR (code change) process now? Depends on sprint priority. Do you think we need SLO?
bwplotka
force-pushed
the
proposal-proposal
branch
2 times, most recently
from
23d5e64 to
ed8ada5
Compare
bwplotka
force-pushed
the
proposal-proposal
branch
2 times, most recently
from
9851044 to
19dfa8f
Compare
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
bwplotka
force-pushed
the
proposal-proposal
branch
from
19dfa8f to
38ee631
Compare
Co-authored-by: Filip Petkovski <filip.petkovsky@gmail.com>
* Added proposal process. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Update content/Proposals/Done/202106-proposals-process.md Co-authored-by: Filip Petkovski <filip.petkovsky@gmail.com> Co-authored-by: Filip Petkovski <filip.petkovsky@gmail.com>
Signed-off-by: Bartlomiej Plotka bwplotka@gmail.com