Automated backlog grooming

[BillWagner]

The dotnet/docs repo has more than 1500 open issues in our backlog. That's not manageable for us to track work. We're building a plan to bulk close stale issues based on five variables. That will trim the active backlog to a set of issues that we intend to address.

This plan is not intended to avoid the tasks identified in the issues being closed. Rather, it is to organize our backlog so we can effectively manage it. Many of the issues are stale. Many reflect plans that have changed. Others reflect work that has become unnecessary because of other completed tasks. Our intent is to run this process once, with a commitment to stay current on our backlog grooming.

Process

We are building a query script to do the following:

• Find all issues matching any of the criteria listed below.
• Add a "won't fix" label to each of those issues. That will help us find any of these later to understand why they were closed.
• Add a comment that points to this issue clarifying that the issue author may re-open the issue if they feel it should not have been closed.
• Close the issue.

Variables

There are five variables we will use to close issues:

• "P?" priority labels (P0, P1, P2, P3): These have been added by people examining the issue in the past. When none have been set, the script will behave as though "P2" was set.
• "Pri?" Automated labels (Pri1, Pri2, Pri3) set by our label bot: These are added based on the traffic to an article. When none have been set, the script will behave as though "Pri2" was set for any article that has the "document details" codeblock. (Any issue without that codeblock should have "Pri3" assigned, unless assigned a different priority manually):
• Author: We create different criteria for those issues opened by our team and partners, and customers.
• Age: Different criteria are applied to issues based on when it was opened.

Criteria

The following table has our first pass at the criteria. Headers are:

• P: "P?" priority label
• Pri: "Pri?" priority label
• CodeBlock: The presence or absence of a "Document details" block. Its presence indicates an issue filed from the live article.
• Author: Indicates if the author is a team member, or a partner.
• Age: This column indicates how old an issue must be for it to be automatically closed.

The rows with a strikethrough should never occur. The labeling bot should always assign "Pri3" to any issue that does not contain the document details block when no other priority label has been set. The script will process any rows that do occur.

The ? in the priority column indicates an issue where no priority label has been applied.

To make suggested changes to the criteria, copy the table below, and make a comment changing the values in the Age column. Add Bold to any you change to help us spot your edits.

P	Pri	Author	CodeBlock	Age
0	1	External	Yes	No
0	1	External	No	No
0	1	Internal	Yes	No
0	1	Internal	No	No
0	2	External	Yes	No
0	2	External	No	No
0	2	Internal	Yes	No
0	2	Internal	No	No
0	3	External	Yes	No
0	3	External	No	No
0	3	Internal	Yes	No
0	3	Internal	No	No
0	?	External	Yes	No
0	?	External	No	No
0	?	Internal	Yes	No
0	?	Internal	No	No
1	1	External	Yes	24m
1	1	External	No	No
1	1	Internal	Yes	No
1	1	Internal	No	24m
1	2	External	Yes	18m
1	2	External	No	12m
1	2	Internal	Yes	12m
1	2	Internal	No	18m
1	3	External	Yes	12m
1	3	External	No	12m
1	3	Internal	Yes	12m
1	3	Internal	No	12m
1	?	External	Yes	18m
1	?	External	No	12m
1	?	Internal	Yes	12m
1	?	Internal	No	18m
2	1	External	Yes	18m
2	1	External	No	12m
2	1	Internal	Yes	12m
2	1	Internal	No	18m
2	2	External	Yes	12m
2	2	External	No	9m
2	2	Internal	Yes	9m
2	2	Internal	No	12m
2	3	External	Yes	9m
2	3	External	No	9m
2	3	Internal	Yes	9m
2	3	Internal	No	9m
2	?	External	Yes	12m
2	?	External	No	9m
2	?	Internal	Yes	9m
2	?	Internal	No	12m
3	1	External	Yes	9m
3	1	External	No	9m
3	1	Internal	Yes	9m
3	1	Internal	No	12m
3	2	External	Yes	9m
3	2	External	No	6m
3	2	Internal	Yes	6m
3	2	Internal	No	9m
3	3	External	Yes	6m
3	3	External	No	6m
3	3	Internal	Yes	3m
3	3	Internal	No	6m
3	?	External	Yes	9m
3	?	External	No	6m
3	?	Internal	Yes	6m
3	?	Internal	No	9m
2	1	External	Yes	18m
?	1	External	No	12m
?	1	Internal	Yes	12m
?	1	Internal	No	18m
?	2	External	Yes	12m
?	2	External	No	9m
2	2	Internal	Yes	9m
?	2	Internal	No	12m
?	3	External	Yes	9m
?	3	External	No	9m
?	3	Internal	Yes	6m
?	3	Internal	No	9m
?	?	External	Yes	12m
?	?	External	No	9m
?	?	Internal	Yes	9m
?	?	Internal	No	12m

[BillWagner]

The first run of the tool that queries for issues found the following results:

Closing 884 of 1539

• CloseCriteria { PLabel = P1, PriLabel = Missing, HasDocIssue = True, IsInternal = True }:
  • Total issues: 22, Closing 0 older than -1 months
• CloseCriteria { PLabel = P1, PriLabel = Missing, HasDocIssue = True, IsInternal = False }:
  • Total issues: 96, Closing 50 older than 18 months
• CloseCriteria { PLabel = P1, PriLabel = Pri2, HasDocIssue = True, IsInternal = True }:
  • Total issues: 4, Closing 0 older than 12 months
• CloseCriteria { PLabel = P1, PriLabel = Pri2, HasDocIssue = True, IsInternal = False }:
  • Total issues: 19, Closing 0 older than 12 months
• CloseCriteria { PLabel = P1, PriLabel = Pri3, HasDocIssue = False, IsInternal = True }:
  • Total issues: 71, Closing 50 older than 12 months
• CloseCriteria { PLabel = P1, PriLabel = Pri3, HasDocIssue = False, IsInternal = False }:
  • Total issues: 18, Closing 11 older than 18 months
• CloseCriteria { PLabel = P2, PriLabel = Missing, HasDocIssue = True, IsInternal = True }:
  • Total issues: 66, Closing 38 older than 18 months
• CloseCriteria { PLabel = P2, PriLabel = Missing, HasDocIssue = True, IsInternal = False }:
  • Total issues: 376, Closing 305 older than 12 months
• CloseCriteria { PLabel = P2, PriLabel = Missing, HasDocIssue = False, IsInternal = True }:
  • Total issues: 1, Closing 1 older than 12 months
• CloseCriteria { PLabel = P2, PriLabel = Pri1, HasDocIssue = True, IsInternal = True }:
  • Total issues: 1, Closing 0 older than 12 months
• CloseCriteria { PLabel = P2, PriLabel = Pri1, HasDocIssue = True, IsInternal = False }:
  • Total issues: 1, Closing 0 older than 9 months
• CloseCriteria { PLabel = P2, PriLabel = Pri2, HasDocIssue = True, IsInternal = True }:
  • Total issues: 8, Closing 0 older than 9 months
• CloseCriteria { PLabel = P2, PriLabel = Pri2, HasDocIssue = True, IsInternal = False }:
  • Total issues: 26, Closing 6 older than 9 months
• CloseCriteria { PLabel = P2, PriLabel = Pri3, HasDocIssue = False, IsInternal = True }:
  • Total issues: 90, Closing 85 older than 9 months
• CloseCriteria { PLabel = P2, PriLabel = Pri3, HasDocIssue = False, IsInternal = False }:
  • Total issues: 45, Closing 38 older than 12 months
• CloseCriteria { PLabel = P3, PriLabel = Missing, HasDocIssue = True, IsInternal = True }:
  • Total issues: 1, Closing 1 older than 9 months
• CloseCriteria { PLabel = P3, PriLabel = Missing, HasDocIssue = True, IsInternal = False }:
  • Total issues: 30, Closing 8 older than 9 months
• CloseCriteria { PLabel = P3, PriLabel = Pri2, HasDocIssue = True, IsInternal = False }:
  • Total issues: 3, Closing 0 older than 6 months
• CloseCriteria { PLabel = P3, PriLabel = Pri3, HasDocIssue = False, IsInternal = True }:
  • Total issues: 11, Closing 11 older than 6 months
• CloseCriteria { PLabel = P3, PriLabel = Pri3, HasDocIssue = False, IsInternal = False }:
  • Total issues: 9, Closing 4 older than 9 months
• CloseCriteria { PLabel = Missing, PriLabel = Missing, HasDocIssue = True, IsInternal = True }:
  • Total issues: 60, Closing 12 older than 18 months
• CloseCriteria { PLabel = Missing, PriLabel = Missing, HasDocIssue = True, IsInternal = False }:
  • Total issues: 296, Closing 199 older than 12 months
• CloseCriteria { PLabel = Missing, PriLabel = Missing, HasDocIssue = False, IsInternal = False }:
  • Total issues: 1, Closing 0 older than 18 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri1, HasDocIssue = True, IsInternal = True }:
  • Total issues: 7, Closing 0 older than 12 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri1, HasDocIssue = True, IsInternal = False }:
  • Total issues: 10, Closing 0 older than 9 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri1, HasDocIssue = False, IsInternal = True }:
  • Total issues: 1, Closing 0 older than 9 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri1, HasDocIssue = False, IsInternal = False }:
  • Total issues: 1, Closing 1 older than 12 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri2, HasDocIssue = True, IsInternal = True }:
  • Total issues: 20, Closing 4 older than 9 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri2, HasDocIssue = True, IsInternal = False }:
  • Total issues: 105, Closing 7 older than 9 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri2, HasDocIssue = False, IsInternal = True }:
  • Total issues: 3, Closing 0 older than 6 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri2, HasDocIssue = False, IsInternal = False }:
  • Total issues: 1, Closing 0 older than 9 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri3, HasDocIssue = True, IsInternal = True }:
  • Total issues: 2, Closing 0 older than 12 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri3, HasDocIssue = True, IsInternal = False }:
  • Total issues: 8, Closing 0 older than 9 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri3, HasDocIssue = False, IsInternal = True }:
  • Total issues: 73, Closing 31 older than 9 months
• CloseCriteria { PLabel = Missing, PriLabel = Pri3, HasDocIssue = False, IsInternal = False }:
  • Total issues: 53, Closing 22 older than 12 months

[akoeplinger]

Cross-posting from #6773 (comment): since only repo collaborators can reopen a closed issue I assume you really mean "open a new one"?

[BillWagner]

@akoeplinger

Cross-posting from #6773 (comment): since only repo collaborators can reopen a closed issue I assume you really mean "open a new one"?

Thanks for noting this. I'm watching notifications and I'll reopen issues based on author comments. (I think the original author can also reopen issues, but I'll watch to make sure).

[Youssef1313]

@BillWagner I wasn't able to re-open issues for which I was the original author. GitHub allows me to reopen only if I'm who closed it.

Also tagging @dotnet/docs doesn't work for external contributors.

I've tagged you on a few issues that I think needs to be re-opened.

[ocdtrekkie]

Project maintainers can reopen closed issues regardless of who closed them. External users can only reopen an issue if they themselves closed it.

[binki]

@BillWagner

@akoeplinger

Cross-posting from #6773 (comment): since only repo collaborators can reopen a closed issue I assume you really mean "open a new one"?

Thanks for noting this. I'm watching notifications and I'll reopen issues based on author comments. (I think the original author can also reopen issues, but I'll watch to make sure).

Original author cannot reopen issues.

I still am awaiting instructions on how to “tag @dotnet/docs teams”. How does one do this with GitHub?

[BillWagner]

@ocdtrekkie

We understand your frustration. We discussed this internally for some time before coming to this conclusion. The reality is that we just couldn't manage the backlog by hand. Also, note that we tailored the age of issues based on several variables. While we know that an automated system will never be perfect, we tried to minimize the number of valid issues to close while maximizing closing those that no longer applied. I'm watching comments to see which we will reopen.

Our commitment going forward is to stay on top of new issues better and not have this happen again. Furthermore, as we update or rework articles, we can find issues that were closed with the "won't fix" label and make sure the root cause is fixed.

[sharwell]

I still am awaiting instructions on how to “tag @dotnet/docs teams”. How does one do this with GitHub?

Only members of the dotnet org can mention @dotnet/docs. It's either automatic or not possible, depending on the user. However, even without the group mention we one of the triage members should see the reply.

[akoeplinger]

One alternative would've been to use a system like https://probot.github.io/apps/stale/ which first posts a comment on stale issues and only if nobody replies within some time frame closes the issue.

[BillWagner]

@akoeplinger We looked at that. We wanted to have better control over which old issues were closed. (We looked at page views, whether it was targeting an article, or a general concern, whether it was an internal work item, or a customer concern, and its age).

[BillWagner]

Tagging the @dotnet/docs team only works for members of the dotnet org.

You can add a comment, and any of us on the triage team are watching for them.

Thanks for your patience.

[StingyJack]

@akoeplinger - pls no stale bot. That thing and its ilk are community deadly as they dont inspect for a contributor, member, owner, etc. to have done anything on an issue before slamming it closed. There are very few customer (or even generally personal) interactions that create more negative feelings (spanning disappoint, upset, hostility, rage, etc) than crassly ignoring someone asking you for help. Do that to your userbase enough and it becomes a reputation that is very hard to reverse.

@BillWagner - This kind of system/response is similar to the developer community's "Sorry we know the thing you bought is having a problem thats crippling your productivity, but we are too busy with something else for the last three months to even read what you wrote, so it will now be closed (and locked so you cant even respond to this in protest)". Making someone a victim of policy cant feel good to do anymore than it feels to be the victim.

You should probably update the message in that to not say "Tag @dotnet/docs" since it wont work for most of us.

[rcollette]

Cannot re-open
#7489

If you reviewed it and tagged in the backlog with a priority, it would seem like the need was acknowledged. I'm not sure why you would auto close it.

[BillWagner]

Adding a new on #7489: We agreed to keep that closed based on another article that addresses the questions.

[zspitz]

@StingyJack I don't think it's quite the same. The team here has a large backlog and is putting to the side those issues that aren't burningly important to someone. If you indicate that your issue is important (by tagging @dotnet/docs) the team is willing to consider reopening (that's been my experience so far).

On the flip side, if the author can't even be bothered to respond with said tag, I would think it shows a certain level of uninterest.

(It's not like Developer Community, where filing issues often/usually feels like a directed path to the circular file.)

[StingyJack]

@zspitz - the action taken here is fine so long as nobody starts showing symptoms of the MCLA fever ("must close/lock/archive"). User abandonment is one thing, using the stalebot is another.

( The ~600 issues I've reported on devcomm unquestionably tell the story that the circular file is both real and very likely and not just a feeling. )

[BillWagner]

Closing, as all issues we planned to close have been closed. (We'll still respond to requests to reopen issues).

Thanks for the comments, discussion, and request where our tool caught issues that should be reopened.