Failure to connect to mail server causes a timeout, eventual Glassfish crash

[bricas]

FYI: Running Dataverse 4.11

I experienced a hard-to-diagnose issue where an action (e.g. publishing a dataset) would fail, but with no visible indication as to why.

I was able to figure out that our local SMTP relay wasn't online. Once that service was restored, the operations completed normally.

A number of tasks send out email notifications. Even if the SMTP server was offline, I would expect these operations to complete successfully -- but show a warning about not being able to send out the notification.

[donsizemore]

@bricas at @pdurbin's suggestion I tried to recreate this on a test server but was able to publish successfully.

[bricas]

@donsizemore interesting. It was definitely the factor that corrected my issue. I had initially tried restarting glassfish/postgresql but still got stuck with just a blue spinner.

To explain our situation further, we have a postfix relay on localhost that simply ships messages to our institution's smtp server.

[pdurbin]

@bricas thanks for opening this issue. @donsizemore tried stopping postfix ( http://irclog.iq.harvard.edu/dataverse/2019-03-29#i_89642 ) but maybe you can help us dig deeper into this relay scenario. We need to know how to reproduce the issue.

Thanks also for your take on what you think the user experience should be when email is down or unavailable. From http://irclog.iq.harvard.edu/dataverse/2019-03-29#i_89625 it sounds like @pameyer 's take was more that publishing could be prevented until email is back up (and explain this to the user who is trying to publish).

[mheppler]

Related to Do not require email server for development environments #5328, as well as No provider for smtp #5723.

[pdurbin]

@bricas hi, have you seen this since? Are you still interested in this issue? Any more details (beyond what you've already provided, thanks) that would help us reproduce this? Thanks.

[bricas]

We've not seen this issue since my original report. We also added monitoring so we would be notified if postfix was every offline.

IMO, It would still be ideal to not fail when SMTP is offline -- but I doubt we'll be affected going forward.

[pdurbin]

Resolved by this pull request:

• Enable mail MTA/SMTP configuration using MicroProfile Config #9939

[poikilotherm]

Actually there is a possibility this might still affect us. Today when trying out a few more things I had one occasion where adding a dataset hang while the mailserver was trying to be reached.

If this happens again, we shall think about using a jakarta.concurrency.Executor to make a timeout happen. Potentially adding to a queue for later retry of the notification. Might be a pattern for other operations like DataCite, too.