Unable to configure the send mail functionality

[arm2arm]

According to the documentation https://github.com/reanahub/reana/tree/master/helm/reana we have configured the values.yaml to enable email notifications and validations. Unfortuneatly our internal mailserver is not SSL encrypted. So in order to test the mail functions we moved to DEBUG mode according this line in the https://github.com/reanahub/reana-commons/blob/1b9aa72a7090870f60335e75e592e6f3737debc2/reana_commons/email.py#L48 we have tryed to avoid SSL and password for the mail server, so we have added in values.yaml following:

# REANA components configuration
components:
  reana_server:
    environment:
      REANA_EMAIL_SMTP_SERVER: "XXX.XXX.XX.XXX"
      REANA_EMAIL_SMTP_PORT: 25
      REANA_EMAIL_SENDER: "test@aip.de" 
      REANA_EMAIL_RECEIVER: "noreply@aip.de"
      REANA_EMAIL_LOGIN: ""
      REANA_EMAIL_PASSWORD: ""

      FLASK_ENV: "development"

Looks like our mail server getting requests but email is rejected by following error:

postfix/smtp[21505]: XXXXXXX: to=<XXXX@XXXX>, relay=127.0.0.1[127.0.0.1]:10024, delay=0.07, delays=0.01/0.01/0/0.06, dsn=2.6.0, status=sent (250 2.6.0 Ok, discarded, id=14729-02 - BAD HEADER)

Any idea how to fix this?
Would be great that REANA can use directly mail server without SSL if no password is set and the mailserver port is 25

thank you beforehand,
Arman.

[tiborsimko]

Hi @arm2arm, the best would be to amend reana-commons source code so that TLS would be optional.

If your mail server does not necessitate any TLS or any login, would it be sufficient to comment out the server.starttls() and the server.login() calls in the reana-commons/email.py main business logic part below?

    with smtplib.SMTP(REANA_EMAIL_SMTP_SERVER, REANA_EMAIL_SMTP_PORT) as server:
        if os.getenv("FLASK_ENV") != "development":
            server.starttls(context=context)
            server.login(login_email, REANA_EMAIL_PASSWORD)
        server.send_message(message)
        logging.info(
            "Email sent, login: {}, sender: {}, receiver: {}".format(
                login_email, sender_email, receiver_email
            )
        )
        logging.info("Body:\n{}".format(message))

For example, you can make a test such as connecting into the running container:

$ kubectl exec -i -t deployment/reana-server -c rest-api -- ipython

and, in the iPython shell, try to send email following the same commands but without these two lines.

Would it work? Or would your email server necessitate some other options to be set?

Once we find out, we'll be able to make a new release of reana-commons where the TLS would be optional and governed by some Helm values.

[arm2arm]

I have setup a TLS mailserver connection.
Now the email server works but not with reana. I have tested on the machines now with the python script:

cat send_email.py 
import smtplib, ssl
from email.mime.text import MIMEText

# Email configuration
REANA_EMAIL_SMTP_SERVER = "xxxx" 
REANA_EMAIL_SMTP_PORT = 465
REANA_EMAIL_LOGIN =  "reana-service"
REANA_EMAIL_SENDER = "reana-service@xxx"
REANA_EMAIL_PASSWORD = "xxxx"

sender = REANA_EMAIL_SENDER
receivers = ['xxxx']

port = REANA_EMAIL_SMTP_PORT
user = REANA_EMAIL_LOGIN
password = REANA_EMAIL_PASSWORD

msg = MIMEText('This is test mail')

msg['Subject'] = 'Test mail'
msg['From'] = 'reana-service@xxx'
msg['To'] = 'xxxx'

context = ssl.create_default_context()

with smtplib.SMTP_SSL(REANA_EMAIL_SMTP_SERVER, port, context=context) as server:

    server.login(user, password)
    server.sendmail(sender, receivers, msg.as_string())
    print('mail successfully sent')

It works inside the container without problems.
Also I checked in the environment variables, they are ok REANA_EMAIL*

reana gives strange error after long waiting:

root@reana:~# kubectl exec -i -t deployment/reana-server -- flask reana-admin token-grant --email  XXXXX  --admin-access-token $REANA_ACCESS_TOKEN
Defaulted container "rest-api" out of: rest-api, scheduler
User XXX (XXX) access token status is revoked, do you want to proceed? [y/N]: y
Token for user XXX (XXXX) granted.

Token: XXXX
Something went wrong while granting token:
(-1, b'\x02\x02')

Any hints how to solve this?

thank you beforehand

[mdonadoni]

Hi @arm2arm, the handling of emails was improved recently (see reanahub/reana-commons@1b9aa72). Would it be possible to try this new version out to check if the issue persists?

These changes are not present in any release yet, so one possibility would be to deploy REANA from master. Otherwise, we can build some custom docker images that include these improvements for the specific version of REANA you have deployed. One other possibility would be to run the code that sends emails manually from inside the cluster, for example by using ipython, as @tiborsimko proposed in his previous reply.

Once we identify the issue, we can proceed with making it possible to disable TLS with a Helm value, if it is still needed.

[arm2arm]

Dear @mdonadoni, sure I can try them on our environment, could you please provide some instructions how to deploy the dev version? currently I am using steps with KIND described here: https://docs.reana.io/administration/deployment/deploying-locally/

[mdonadoni]

If you only want to deploy the dev version locally to test things out, you can follow the steps in the developers wiki. However, please keep in mind that by following this guide you will set up a full development environment, which might require quite some time.

We are currently in the middle of releasing a new alpha version (0.9.0-alpha.6, see #672), so you can also consider waiting for it to be ready, given that it will be much easier to deploy. Would this work for you?

[mdonadoni]

We have released version 0.9.0-alpha.6 of REANA, which you can now easily deploy. You can follow the same guide Deploying locally, with just a few changes. In particular, you should fetch these files instead of the ones present in the documentation page:

• https://raw.githubusercontent.com/reanahub/reana/0.9.0-alpha.6/etc/kind-localhost-30443.yaml
• https://raw.githubusercontent.com/reanahub/reana/0.9.0-alpha.6/scripts/prefetch-images.sh
• https://raw.githubusercontent.com/reanahub/reana/0.9.0-alpha.6/scripts/create-admin-user.sh

You will also need to add --version 0.9.0-alpha.6 to the helm install ... command.

[mdonadoni]

The issues seem to be related to how REANA connects to SMTP servers.

This is currently how REANA sends emails (src):

    with smtplib.SMTP(REANA_EMAIL_SMTP_SERVER, REANA_EMAIL_SMTP_PORT) as server:
        if os.getenv("FLASK_ENV") != "development":
            server.starttls(context=context)
            server.login(login_email, REANA_EMAIL_PASSWORD)
        server.send_message(message)

A plain-text connection to the SMTP server is opened, after which a STARTTLS command is issued to upgrade to an encrypted connection. However, in this case it's necessary to connect to the server directly with SSL.

Things to do:

• add possibility to choose SMTP-over-SSL instead of SMTP + STARTTLS (see smptlib.SMTP_SSL)
• add support for plaint-text SMTP without any kind of encryption (e.g. what is being used for the development email server)
• make login optional
• update the docs