EPIC: Plan for next steps

[xmnlab]

from @jaimergp, in the conda-forge element channel:

• Unattended issue where the last participant is also the opener, which means that the problem is not resolved and they were left without an answer. #3
• PRs with green checks and no comments #4
• Issues or PRs where a core member has requested help from other core member or team but didn't obtain a reply #5
• Feedstocks where the autotickbot has accummulated more than 3 open PRs with none or very few interactions #6
• Also a panel specific for core members: conda-forge-repodata-patches, admin-requests, conda-forge-pinning. These three repos usually involve a core member making a decision and sometimes notifications are not loud enough 😬 If there is a signal there we can collect/amplify in a panel, it would be nice! #7
• Also not sure if the current categories are very informative. "Critical" for example doesn't necessarily means that those feedstocks are unattended (conda-forge-pinning is very well alive). They are just noisy repos. #8
• "Time since last reply" seems to be a good complementary metric, but it's also not perfect #9

#2 (comment)
Some more suggestions by @beckermr:

• repos that have tons of downstream children have only a few maintainers #10
• repos whose build times are close to 6 hours #11
• repos with failed builds on main #12

[jaimergp]

Some more suggestions by @beckermr:

• repos that have tons of downstream children have only a few maintainers
• repos whose build times are close to 6 hours
• repos with failed builds on main

[xmnlab]

@jaimergp and @beckermr thank you for your suggestions! Appreciate that!

[h-vetinari]

Feedstocks where the autotickbot has accummulated more than 3 open PRs

It might also make sense to differentiate between version updates and migrations. If there's no open version update PRs, that might be just because it's a package with very low release cadence, or because the bot failed (see below). Alternatively, if many failed migration PRs pile up, then the feedstock is almost always dormant/dead.

Speaking about the version bot failures, there's a section on the conda-forge status page dealing with feedstocks where the autotickbot failed to open a PR in the first place - i.e. one step before you can reasonably rely on that metric for most feedstocks.

We discussed a long time ago to make automated issues for these, but that hasn't materialized yet. If it's easier to put that information into a dashboard, that might already be a substantial improvement.

Other than that:

• This effort probably also needs a whitelist. Things like conda-forge-{pinning,repodata-patches}-feedstock will never go out of date, and pytorch/tensorflow are under constant attention as well (only very hard to build, hence the pent up PRs).
• It should ideally pick up feedstocks from abandoned feedstocks? conda-forge/conda-forge.github.io#1724 / Maintenance of k*-ecosystem conda-forge/conda-forge.github.io#1861, and avoid high-volume (not least due to many abi_migration_branches) feedstocks like arrow
  • for that, something like "last-non-bot-commit" or "last-non-bot-comment" would probably be helpful metrics
• discount open PRs that are in draft mode - sometimes larger refactors on a feedstock are open for a long time, but that's not a sign of bad health
• don't weight issues too heavily; often there are questions that are left open for reasons of visibility, or problems that aren't fixable in the foreseeable future, or a list of todos for further incremental improvements etc.
• Some sensitivity about nomenclature. Issuing a "warning" about feedstocks with 1 open issue and one open PR is not going to be helpful IMO, especially in an age where automated tools are liable to start screaming at their users about these things (c.f. the situation with CVEs).
  • I consider it very important that this does not become a blunt hammer used against still-active maintainers, especially where it's a false positive.
  • The simplest is probably either a cut-off below which nothing is displayed, or a category below "Warning" that just lists the scraped status of these feedstocks under a more neutral heading.

[xmnlab]

Thanks @h-vetinari for your inputs!
I will take a time to process all the inputs here.