Remove max delivery interval threshold

[minori-fh]

Problem Statement

Currently there is a minimum and maximum delivery interval for alert emails.

This configuration seems to create three scenarios:

• below min: delayed
• min - max: email can be sent but this is not guaranteed. Attempt is made to batch alert emails by collecting them over time to reduce email noise.
• max: whatever is in the digest queue must be sent

Solution Brainstorm

The current configuration causes confusion for the second scenario min-max. Request for keeping one threshold (min) which will definitively and predictably control batching.

Product Area

Alerts

[linear]

ENG-5640

[minori-fh]

Relevant internal ticket: https://sentry.zendesk.com/agent/tickets/161720

[getsantry]

Auto-routing to @getsentry/product-owners-alerts for triage ⏲️

[alxwrk]

@minori-fh Title should be "Remove max ..."

[malwilley]

There might be a misunderstanding here with how the min/max delivery thresholds work. Here is my understanding of how it is used under the hood:

Example 1 with minimum of 5mins, maximum of 30min:

• An event comes in at 00:00:00. The digest is scheduled to go out at 00:05:00 (5 mins later).
• Another events comes in at 00:01:00. This pushes the digest delivery to 00:05:30 due to a hidden increment_delay of 30 seconds.
• No other events come in, and the digest is delivered for both events 00:05:30.

Example 2 with minimum of 5mins, maximum of 30min

• An event comes in at 00:00:00. The digest is scheduled to go out at 00:05:00 (5 mins later).
• An event comes in every second. Each event delays scheduling another 30 seconds, until the maximum of 30 mins.
• When 30 mins is reached, a digest is sent for all events received in the last 30 mins.

In practice, it works as the settings page describes. Emails are sent at minimum 5 mins after the first alert and at most 30 mins after the first alert (if alerts are coming in more frequently). They can be set to the same value if that's not your desired behavior.

[alxwrk]

There might be a misunderstanding here with how the min/max delivery thresholds work.

Considering that how max works:

1. can't be understood from the product, inline
2. can't be understood from the sentry help docs
3. can't be understood from two weeks with sentry support

There's definitely a problem here.

So first of all is @malwilley's explanation definitely correct? Since it differs from support (@minori-fh) and is qualified with "my understanding" I have to ask for a definitive confirmation as I already wasted time discussing multiple possibly incorrect explanations.

An event comes in at 00:00:00. The digest is scheduled to go out at 00:05:00 (5 mins later).

For this scenario are we assuming the an alert email has just been sent out (00:00:00)? Because the min interval only makes sense between the previous alert email and a possible current one.

Another events comes in at 00:01:00. This pushes the digest delivery to 00:05:30 due to a hidden increment_delay of 30 seconds.

This does not make sense. If the previous email was at 00:00:00 then any new alerts between 00:00:00 and 00:05:00 should be batched and after the min interval passes and the digest is sent at 00:05:00, the batching timer should be reset. This would be simple, logical and easy to understand, without two weeks with support and a consultation with the devs. It's entirely unclear why the digest delivery should be continually pushed out, beyond the min interval. If your own support doesn't understand this, perhaps you should consider simplifying.