Investigate automation of closing issues that are labeled as "enhancement" but have not gained sufficient community interest

[darrelmiller]

• We need to determine what heuristic to use to identify "not sufficient interest" e.g. How long since the last comment? How many thumbs up vs thumbs down? By who? etc.

[baywet]

What do you think of that?

flowchart TD
  openIssue["author open"]
  labelIssue["a triager labels it as feature request"]
  addMessage["explanatory message the author needs to gather support by people upvoting the issue"]
  startCounter["counter for community support starts"]
  isSupportGathered{"has the community support been achieved?"}
  closeIssue["issue gets closed"]
  prMerge["merge associated PR"]
  addMessageForSchedule["worklow adds a message to prompt scheduling"]
  addMilestone["triager adds a milestone, removes feature request, and adds enhancement"]
  openIssue -- "triage" --> labelIssue
  labelIssue -- "worklow posts a message" --> addMessage
  addMessage -- "start the counter" --> startCounter
  startCounter -- "counter runs out" --> isSupportGathered
  isSupportGathered -- "no" --> closeIssue
  isSupportGathered -- "yes" --> addMessageForSchedule
  addMessageForSchedule ----> addMilestone
  addMilestone ---> prMerge
  prMerge -- "pr closes issue" --->closeIssue

Loading

The "community support" could be defined as:

• Quorum from a select group of people
• Open to anyone, needs to pass an arbitrary number, and needs to have a ratio thumbs up/thumbs down > 1

[lornajane]

I'd like to see some stats on how our currently-open and recently-closed feature requests score by the proposed heuristics.

[baywet]

Couple of notes from the meeting:

• the backlog management is not as formalized as I had thought originally, the use of milestone/label for the "accepted feature requests" is not useful at that point.
• we should focus more on the "clearing out ideas that do not get enough support" for the time being, which is where most value lies. It'll give more clarity to the repository.
• the TDC didn't seem to have a strong preference between populate vote and elective college, but there are a lot of issues that have been upvoted to date.
• people use more than just the thumbs up reaction, so maybe we should bucketize positive vs negative reactions.
• delays have been around 3 years at the time being, this is probably too long to immediate benefits.
• maybe the activity on the issue (i.e. last comment) should be taken into consideration to extend the clock.

(effectively we should eliminate the "yes" branch of the diagram here)

[baywet]

@lornajane what kind of stats are you looking for? How long have issues been open on average? the upvotes? something else?

[lornajane]

I'd like to see some proposed criteria, and which existing issues would be affected by the automation

[baywet]

I built a little script on my end to try and get some example.
If we query for issues opened for more than 3 months and labeled as OpenAPI.Next Proposal, and if we apply the following rule to them:

• Any issue with fewer than 100 reactions gets closed.
• Any issue with a ratio of positive reactions (thumbs up, heart, rocket, hooray) over negative reactions (confused, thumbs down) lower than 2 gets closed.

Here is what the script returns

Closing 46 issues with less than 100 reactions and created more than 90 days ago
Closing issue #2143 because of low engagement
Closing issue #2096 because of low engagement
Closing issue #690 because of low engagement
Closing issue #685 because of low engagement
Closing issue #684 because of low engagement
Closing issue #683 because of low engagement
Closing issue #680 because of low engagement
Closing issue #645 because of low engagement
Closing issue #625 because of low engagement
Closing issue #622 because of low engagement
Closing issue #620 because of low engagement
Closing issue #589 because of low engagement
Closing issue #586 because of low engagement
Closing issue #585 because of low engagement
Closing issue #579 because of low engagement
Closing issue #574 because of low engagement
Closing issue #569 because of low engagement
Closing issue #567 because of low engagement
Closing issue #566 because of low engagement
Closing issue #563 because of low engagement
Closing issue #560 because of low engagement
Closing issue #555 because of low engagement
Closing issue #550 because of low engagement
Closing issue #541 because of low engagement
Closing issue #522 because of low engagement
Closing issue #519 because of low engagement
Closing issue #508 because of low engagement
Closing issue #480 because of low engagement
Closing issue #466 because of low engagement
Closing issue #398 because of low engagement
Closing issue #370 because of low engagement
Closing issue #369 because of low engagement
Closing issue #342 because of low engagement
Closing issue #298 because of low engagement
Closing issue #272 because of low engagement
Closing issue #239 because of low engagement
Closing issue #236 because of low engagement
Closing issue #230 because of low engagement
Closing issue #215 because of low engagement
Closing issue #213 because of low engagement
Closing issue #164 because of low engagement
Closing issue #136 because of low engagement
Closing issue #123 because of low engagement
Closing issue #122 because of low engagement
Closing issue #110 because of low engagement
Closing issue #93 because of low engagement
Keeping issue #521 with positive reaction ratio of 193
Keeping issue #445 with positive reaction ratio of 271
Keeping issue #433 with positive reaction ratio of 104
Keeping issue #348 with positive reaction ratio of 188
Keeping issue #256 with positive reaction ratio of 639
Keeping issue #182 with positive reaction ratio of 118

What do you think? Of course we could dig deeper, export all issues as CSV, an try to cluster them in excel or so.

[karenetheridge]

Instead of closing these "dead" feature request issues, may I suggest they be moved to Github Discussions? Then if there is enough traction on the discussion, a new issue/PR can be created for it.

[handrews]

I dislike discussions because they just sit there forever, and the UX once they get complicated is horribly confusing... but other folks seem to like them so ¯\(ツ)/¯

I do think that if we are using discussions (and again, I'd really rather not but I'm pretty sure I lost that argument long ago), then putting big feature requests there is better. So I would support a policy of moving active big feature request issues to discussions.

But for the more-or-less dead backlog... To me, it's more valuable to outright reject things if we're not doing them. Or to consolidate them under Moonwalk if they could find a home there. It's precisely the "things just sit open forever" that we got multiple complaints about near the end of last year.

I think people want to know what's active and has a chance of happening. Creating 100 or so new discussions around ideas that never gained real traction feels like the opposite of that, to me.

Let's bring some clarity and say "no" to things. If they're really that needed, someone will open it again and we can consider it in the context of 2024 and not 2017 or whatever.

[lornajane]

I think I'm in agreement with everyone here, on one thing or another! I don't like the idea of automatically closing issues, it is a very blunt instrument. That said, with 500 open issues, a tiny percentage of which can be actioned for OpenAPI 3.x, we do need to identify the ones that need our attention and which cannot proceed. It feels honest to me to close those that we are certain are not in the picture. There seems to be general agreement that we have things merged already that would merit a 3.2 - but anything that feels like more than a minor change should be redirected to 4.0 at this point.

Some should become discussions for Moonwalk - there are things we can implement there that weren't possible or even just didn't have enough support to get into 3.x.

Discussions: I can kind of see how they might be useful for questions rather than todo items, but I'm not sure how much engagement/activity we have there, or whether it's useful to spread things around without having clear guidelines of what goes where (apparently we can move things from one place to another). If we do want to use discussions, let's propose what we use them for and how they're managed first.

[lornajane]

@handrews How do you feel about closing this issue since I don't see much enthusiasm for automatically closing issues? I did find the discussion helpful (thanks @baywet ) though

[handrews]

@lornajane I agree, let's keep going with the manual closing (and thanks for your work there!)

[handrews]

@karenetheridge I filed #3518 to define (among other things) criteria for closing vs migrating issues to discussions