Making conversations with failed replies easier to discover

[DavidAnderson684]

In this issue (and those created from it) - #3327 - some good steps were made in making issues with non-delivery of mails easier to discover and process.

Following that, we've encountered another way in which the experience can leave undelivered messages difficult to detect, which could be improved. If an email is not delivered then it remains in the queue, is retried, and this information is displayed in the conversation itself. However, the only way that the problem will be discovered is if someone manually checks the queue (since the conversation itself is likely to be in a closed state when the support agent replied), or if the original creator of the conversation gets in touch again to ask why he has not received a reply.

In our case we happened to check the queue, and saw:

3. App\Jobs\SendReplyToCustomer 	
Queue 	emails
Message 	#12345
Attempts 	123  (View log)
Created At 	Oct 24, 2023 15:25
Next Attempt 	Oct 24, 2023 16:25

So, we went to the message, and saw that it had been failing to send for 6 days, receiving an SMTP 550 each time, due to an over-sized attachment which the support agent had added (SMTP response shown in the conversation was: "550 Message too large").

Upon discovering it, the support agent was advised to instead send a new message, using an external file storage solution to link to the file, instead of attaching it. But this only happened because of the manual check on the queue.

I suggest three improvements that would lead to a better experience:

1. If a message has been on the queue for more than a certain number of hours (48? 72?) it is reasonable to expect that some intervention is needed - the conversation should automatically be changed to an "Active" state so that a support agent sees it.

2. Add a setting to Freescout allowing the maximum total email size to be configured. If Freescout detects that the mail will be bigger than this, it should refuse to allow sending, so that the support agent can take action immediately instead of waiting for later notification of failure.

3. The SMTP response 550 Message too large can be detected and handled similarly to 2). (This is the response from the exim mailer daemon - https://exim.org/ - which is the most popular, accounting for about half of all SMTP daemons - so it's reasonable to specifically detect the response).

[freescout-helpdesk]

Normally delivery issues should be monitored by admin who receives alerts when there are new records in "Send Errors" log. And admin lets support agent know that this or that conversation has problems with delivery by adding a Note or by contacting support agent directly.

https://github.com/freescout-helpdesk/freescout/wiki/FAQ#how-can-support-team-know-that-there-are-problems-with-sending-emails-and-emails-are-not-being-sent

[freescout-helpdesk]

Max message size option: #3482

[DavidAnderson684]

Thank you for pointing me to https://github.com/freescout-helpdesk/freescout/wiki/FAQ#how-can-support-team-know-that-there-are-problems-with-sending-emails-and-emails-are-not-being-sent . However, the screenshot of "Logs Monitoring" there does not match what I see (FreeScout 1.8.105, PHP 8.1.24). It shows:

But I see:

i.e. There are no "Logs to monitor" checkboxes.

I'd still suggest that if a mail hasn't been sent after a certain amount of time (e.g. 24 or 48 hours), that the conversation is automatically put it in an "Active" state; this removes the dependency on an admin's vigilance (and protects against other failures, e.g. admin didn't see the email (e.g. he's forgetting to monitor/on holiday/message went to spam etc.). And in cases where the support agent can resolve the issue themselves (e.g. in the example of a "too large" email it's actually the support agent who needs to take action, not the admin), then that can lead to a quicker resolution.

[DavidAnderson684]

Also, from that link - "If there are any problems with background mail queue processing the following alert will be shown in the mailbox to all support agents:" - in what circumstances should that be shown? In this case a mail was in the queue for 6 days, but nothing was visible except in the queue itself.

[freescout-helpdesk]

Click "Save".

"problems with background mail queue processing" are different from mail delivery problems.

[DavidAnderson684]

Thanks.

One more thing - I'd buy the Slack module if it could alert of failed send failures (i.e. the things that get logged in the current log). I've turned on admin alerting for those log events - but of course, those alerts are going to come by email. If complete email server failure occurs, then we'll detect that eventually, but Slack is nicer since it's instant. So, if those log messages also got logged via Slack, that would add more robustness to our setup. (I've not got the module currently because we have no use for its current features).

[freescout-helpdesk]

We don't plan to activate conversations on delivery issues. If needed it can be implemented via a custom module (https://github.com/freescout-helpdesk/freescout/wiki/Modules-Development).