Prevent notification storms on corrupted/improper PRs

[dagwieers]

So almost daily we get some broken PRs with lots of existing commits in a new PR. Often the owner of the branch did something wrong and has no clue to resolve it and eventually just closes the ticket. But the real damage is already done at the time, as everybody involved with any of these commits (directly or indirectly) are getting mailed by ansibot.

Examples:

• Lookup plugin update: Update AWS SSM lookup to support getting parameters by path ansible#31505
• module for OCI (Oracle cloud infrastructure) ansible#31654
• Stable 2.4 ansible#31665

Either we should be able to identify these corrupt PRs, or at least prevent to send out notifications to more than an arbitrary number of people e.g. 25 ? Maybe check existing cases for a good treshold ?

[pilou-]

Is more than 30 commits and more than 30 files modified a good filter for these PR ?

[cben]

Proposal: a simple "more than N people at once" limit for automatic notification.

If more than that, ansibot instead of mentioning all the people would explain that it suspects it's a mistake, and will list the usernames it would have cc'd (without @, or within backticks literal), so humans can (A) understand what's happening (B) manually cc all or some people if really intended (eg. large cross-cutting refactoring).

[jctanner]

first implementation could be to disable all notifications if a merge commit is detected (something we already check for).

[dagwieers]

Another common case recently we also have to consider:
ansible/ansible#33448 (comment)

[dagwieers]

We are now closing one specific case, but I'd like to also consider a more general solution for other causes (like the one in my previous comment). I want to prevent any more of these cases because they make people want to drop out of any notifications.

[gundalow]

Seems to be back. opened #837

[dagwieers]

ansible/ansible#34011