Expand refinement capacity and increase velocity by coaching other eng team members in task refinement

[sgibson91]

The engineering team have hit a very good problem in that we are regularly running out of work mid-iteration, and there is not enough already refined tasks either to pull from. Currently, engineering tasks are refined by @yuvipanda. However, if we are going to increase the velocity of the refinement process to match the velocity of the engineering team in completing task, he cannot continue to refine tasks on his own. Hence, Yuvi should begin coaching other members of the team to learn the refining skill in order to increase refinement capacity.

[yuvipanda]

I'm going to approach this by trying to separate prioritization from refinement. As part of my role I'm still responsible for prioritization but refinement needs to be spread out a lot more.

[yuvipanda]

#4640 and #4453 (comment) are examples of the approach I am going to take.

[yuvipanda]

2i2c-org/frx-challenges#17 is another example.

[yuvipanda]

Goals

Goals are:

1. Retain the massively improved focus we have had a team since Bucharest
2. Improve processes incrementally, so we avoid problems of what's on paper being different from what we actually follow

Scope of this issue

Purely for the purposes of this issue, here's a hand-drawn diagram of how I see our process.

5 and 4 are currently done by the entire team.

Since Bucharest, and until this sprint, I was primarily doing 2 and 3.

Responsibility for 1 is split out based on allocation - product board drives product-dev, the eng calendar and internal eng roadmap drives internal-eng, support drives tech-support, #sync-partnerships-support-requests drive (and other places) partnership-support, and the process being built in https://github.com/2i2c-org/meta/issues/1289 drives upstream-citizen work. This will be further clarified and worked on, but is out of scope for this issue.

Keeping Goal 2 in mind, the scope of this particular issue is to separate 2 and 3. I am going to continue doing (2) (along with product and partnerships), but not 3 anymore. Just like 4 and 5, 3 should also be done by the entire team. This incremental improvement is the scope of this particular issue.

Overall prioritization continues to be my responsibility for now.

"Up next" column

We have had a column called "Backlog" in the Engineering Board for a while. But because we aren't a pure product team (where backlogs are determined by product) nor a pure services team (where backlogs are determined by client contracts), it's kinda sat unused. The column where things go to die, as things consistently stay there without ever moving elsewhere.

So 'Backlog' is gone. Instead, there is now an "Up Next" column. It contains a prioritized list of issues that are ready to be worked on pending further refinement. These have already been prioritized (through the various mechanisms specific to each allocation), and when refined, can be moved into Refined column so they may be picked up during a future sprint. This is different from a traditional Backlog, which is basically everything that could be worked on. This is a column of tasks that should be worked on.

How is this column to be used?

Anyone can pick up any tasks currently unassigned in the "Up Next" column and refine them. However, to avoid diffusion of responsibility, I'm also going to start pinging people and asking them to refine specific tasks from this column. Similar to how each engineer has 30min a day (max) reserved for doing support tasks, I now ask they have upto 30min a day set aside for doing refinement work. As we practice doing more refinement, we can do more within that time period. There'll be a short term velocity hit, but it'll be very well worth it.

How do things get into this column?

Currently, there are two ways things get into this column:

1. There's already something else in this column, and as part of refinement, you create new issues that need further refinement.
2. You create a new issue elsewhere, and ping Yuvi (on slack or via email). He will triage it appropriately, prioritize it and redirect it where needed. This is not a sustainable system, and is designed to be retired by something else before end of October 2024.

Measurement

A useful way to measure how this is working is to see how many tasks we commit to as a team where I (Yuvi) am the person who refined it. Eventually it should probably be zero.

[haroldcampbell]

@yuvipanda to help with prioritisation we can adopt a RICE scoring mechanism.