Unable to connect with TLS encryption using STARTTLS 587 #17134

JLueke · 2019-09-13T15:06:35Z

Steps to reproduce

Enter settings (in config or via settings) for remote SMTP server
with STARTTLS 587 and required login credentials for authentication.
Try to send test email via the settings page.

Expected behaviour

I receive a test mail.

Actual behaviour

With mail_smtpdebug enabled in the config I get following error message (domains are replaced like "mailprovider.de" as the server of the mail provider and "nextcloud.on.mydomain.com" for the nextcloud instance):

Beim Senden der E-Mail ist ein Problem aufgetreten. Bitte überprüfe Deine Einstellungen. (Fehler: Unable to connect with TLS encryption Log data: ++ Starting Swift_SmtpTransport << 220 smtp1.mailprovider.de ESMTP >> EHLO nextcloud.on.mydomain.com << 250-smtp1.mailprovider.de 250-PIPELINING 250-SIZE 51200000 250-ETRN 250-STARTTLS 250-AUTH PLAIN LOGIN 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN >> STARTTLS << 220 2.0.0 Ready to start TLS !! Unable to connect with TLS encryption (code: 0))

I only get the test mail if I use no encryption.

Server configuration detail

Operating system: Linux 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x86_64

Webserver: Apache/2.4.38 (Debian) (apache2handler)

Database: mysql 5.7.24

PHP version:

7.3.9
Modules loaded: Core, date, libxml, openssl, pcre, sqlite3, zlib, ctype, curl, dom, fileinfo, filter, ftp, hash, iconv, json, mbstring, SPL, PDO, session, posix, Reflection, standard, SimpleXML, pdo_sqlite, Phar, tokenizer, xml, xmlreader, xmlwriter, mysqlnd, apache2handler, apcu, exif, gd, imagick, intl, ldap, memcached, pcntl, pdo_mysql, pdo_pgsql, redis, sodium, zip, Zend OPcache

Nextcloud version: 16.0.4 - 16.0.4.1

Updated from an older Nextcloud/ownCloud or fresh install:

Where did you install Nextcloud from: Official Docker Image

Signing status

List of activated apps

Enabled:
 - accessibility: 1.2.0
 - activity: 2.9.1
 - calendar: 1.7.0
 - cloud_federation_api: 0.2.0
 - comments: 1.6.0
 - dav: 1.9.2
 - federatedfilesharing: 1.6.0
 - federation: 1.6.0
 - files: 1.11.0
 - files_pdfviewer: 1.5.0
 - files_rightclick: 0.15.1
 - files_sharing: 1.8.0
 - files_texteditor: 2.8.0
 - files_trashbin: 1.6.0
 - files_versions: 1.9.0
 - files_videoplayer: 1.5.0
 - gallery: 18.3.0
 - issuetemplate: 0.5.0
 - logreader: 2.1.0
 - lookup_server_connector: 1.4.0
 - mail: 0.17.0
 - nextcloud_announcements: 1.5.0
 - notifications: 2.4.1
 - oauth2: 1.4.2
 - password_policy: 1.6.0
 - privacy: 1.0.0
 - provisioning_api: 1.6.0
 - recommendations: 0.4.0
 - serverinfo: 1.6.0
 - sharebymail: 1.6.0
 - sociallogin: 1.16.7
 - spreed: 6.0.4
 - support: 1.0.0
 - survey_client: 1.4.0
 - systemtags: 1.6.0
 - theming: 1.7.0
 - theming_customcss: 1.3.0
 - twofactor_backupcodes: 1.5.0
 - updatenotification: 1.6.0
 - viewer: 1.1.0
 - workflowengine: 1.6.0
Disabled:
 - admin_audit
 - encryption
 - files_external
 - firstrunwizard
 - user_ldap

Configuration (config/config.php)

{
    "htaccess.RewriteBase": "\/",
    "memcache.local": "\\OC\\Memcache\\APCu",
    "apps_paths": [
        {
            "path": "\/var\/www\/html\/apps",
            "url": "\/apps",
            "writable": false
        },
        {
            "path": "\/var\/www\/html\/custom_apps",
            "url": "\/custom_apps",
            "writable": true
        }
    ],
    "default_language": "de",
    "default_locale": "de_DE",
    "logtimezone": "Europe\/Berlin",
    "skeletondirectory": "",
    "lost_password_link": "disabled",
    "login_form_autocomplete": false,
    "sort_groups_by_name": true,
    "social_login_auto_redirect": true,
    "allow_user_to_change_display_name": false,
    "share_folder": "ungeordnete Shares",
    "remember_login_cookie_lifetime": 0,
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "secret": "***REMOVED SENSITIVE VALUE***",
    "trusted_domains": [
        "nextcloud.on.mydomain.com",
        "on.mydomain.com"
    ],
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "dbtype": "mysql",
    "version": "16.0.4.1",
    "overwrite.cli.url": "https:\/\/nextcloud.on.mydomain.com",
    "overwriteprotocol": "https",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbport": "",
    "dbtableprefix": "oc_",
    "mysql.utf8mb4": true,
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "installed": true,
    "mail_smtpdebug": true,
    "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpsecure": "tls",
    "mail_smtpauth": true,
    "mail_smtpauthtype": "LOGIN",
    "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
    "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
    "mail_smtptimeout": 30,
    "mail_smtpport": 587
}

Are you using external storage, if yes which one:

Are you using encryption:

Are you using an external user-backend, if yes which one:

Client configuration

Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3338.0 Safari/537.36

Operating system: MAC OS

Logs

Web server error log

/var/log/apache2/error.log

is empty

Nextcloud log

is empty

Browser log

POST https://nextcloud.on.mydomain.com/index.php/settings/admin/mailtest 400 (Bad request)
XHR failed loading: POST "https://nextcloud.on.mydomain.com/index.php/settings/admin/mailtest".

The text was updated successfully, but these errors were encountered:

solracsf · 2019-09-13T15:37:48Z

Unable to reproduce, i can connect to external SMTP using STARTTLS and port 587.

JLueke · 2019-09-13T16:23:16Z

Do you maybe know what can cause the error? I tried the settings with thunderbird and it had no problem sending the email.

j-ed · 2019-09-13T16:29:05Z

Due to the fact that the STARTTLS command is sent but it couldn't initiate a TLS session, I would recommend to check if the certificate chain of the mail server can be verified by Nextcloud.

JLueke · 2019-09-13T16:40:07Z

Due to the fact that the STARTTLS command is sent but it couldn't initiate a TLS session, I would recommend to check if the certificate chain of the mail server can be verified by Nextcloud.

@j-ed is there a console command i can use for that?

JLueke · 2019-09-13T16:50:53Z

is that it?

openssl s_client -connect smtp.mailprovider.de:587 -starttls smtp

outputs this:

CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1
verify return:1
depth=0 CN = *.goneo.de
verify return:1
140051321431168:error:141A318A:SSL routines:tls_process_ske_dhe:dh key too small:../ssl/statem/statem_clnt.c:2156:

Certificate chain
0 s:CN = *.mailprovider.de
i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1
1 s:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1
i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2

Server certificate
-----BEGIN CERTIFICATE-----
// the certificate
-----END CERTIFICATE-----
subject=CN = *.mailprovider.de

issuer=C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1

No client certificate CA names sent

SSL handshake has read 3599 bytes and written 345 bytes
Verification: OK

New, (NONE), Cipher is (NONE)
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1568393235
Timeout : 7200 (sec)
Verify return code: 0 (ok)
Extended master secret: no

j-ed · 2019-09-13T16:50:55Z

Good that someone else had the same problem in the past 😉

JLueke · 2019-09-13T17:23:44Z

@j-ed would you elaborate how there is a match with the other ticket? I saw that ticket before but didn't saw the connection or a solution for my case. I don't want to disabled the certificate verification, if thats the proposed solution. Using port 465 with STARTTLS didn't worked. As far as I know the solution with PHP or sendmail as the sendmode doesn't fit, since the mail server is not on the same machine as my Nextcloud instance.

j-ed · 2019-09-13T17:40:42Z

The mentioned ticked covered a connection problem on port 587 with STARTTLS and the status is closed, therefore my assumption was that it leads you into the right direction.
Port 465 only supports direct encrypted connection so that doesn't work.

What about the Nextcloud log file, could you fina any messages related to your send problem in it?

Have you searched the Nextcloud help forum for a solution of your problem? I found e.g. the following positing covering a TLS problem. BTW, that is usually the right location to ask questions, this is only a bug tracker 😉

https://help.nextcloud.com/t/server-cant-send-email-notification-error-tls-connection-streambuffer-php-94/27621/17

JLueke · 2019-09-13T18:00:50Z

The Nextcloud log file is empty.

And yes, I did search the forum before and the there mentioned tickets, where in the end most cases refer to a fix by Nextcloud 16 shipping Swiftmailer 6.1.3.

If I remember correctly most of the tickets weren't claimed to be support ticket. That's why I used the issue ticket instead of the forum. I guess I'll try my luck there, if this issue doesn't qualify as one.

JLueke · 2019-09-30T16:42:19Z

I tested older Nextcloud (Docker) versions and my SMTP settings are working until Nextcloud 14.0.13-apache.

13.0.4 works
14.0.0-apache, 14.0.7-apache, 14.0.12-apache works

14.0.13-apache fails with:

A problem occurred while sending the email. Please revise your settings. (Error: Unable to connect with TLS encryption Log data: ++ Starting Swift_SmtpTransport << 220 smtp1.goneo.de ESMTP >> EHLO localhost << 250-smtp1.goneo.de 250-PIPELINING 250-SIZE 51200000 250-ETRN 250-STARTTLS 250-AUTH PLAIN LOGIN 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN >> STARTTLS << 220 2.0.0 Ready to start TLS !! Unable to connect with TLS encryption (code: 0))

JLueke · 2019-09-30T17:14:05Z

it works for 14.0.13RC1-apache, too

kesselb · 2019-09-30T17:16:19Z

Connecting with Nextcloud 17 to smtp.goneo.de:587 works for me.

JLueke · 2019-09-30T17:22:53Z

That's weird, it doesn't work for me with Nextcloud Docker 17-apache, even though it's working with the same settings before 14.0.13-apache.

JLueke · 2019-09-30T17:26:31Z

@kesselb did you use encryption STARTTLS, too? Without encryption it works just fine, but I don't want to drop the encryption.

kesselb · 2019-09-30T17:28:12Z

Cannot send any emails because I don't have a account there but connected via StartTLS.

JLueke · 2019-09-30T17:36:50Z

@kesselb how did you setup Nextcloud? Also with the 17-apache Docker image?

I still get the same error, even if I try fake credentials and clear the "From address" fields.

kesselb · 2019-09-30T17:45:13Z

Also with the 17-apache Docker image?

Yes

JLueke · 2019-09-30T20:50:58Z

@kesselb would you share the (mail settings) of your config.php, which let to this error?
And do you have any custom apps enabled?

My colleague tested the Docker 17-apache version and got the same error like me -.-

kesselb · 2019-09-30T21:11:51Z

  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'tls',
  'mail_sendmailmode' => 'smtp',
  'mail_from_address' => 'mail',
  'mail_domain' => 'nextcloud-test.com',
  'mail_smtphost' => 'smtp.goneo.de',
  'mail_smtpport' => '587',
  'mail_smtpauth' => 1,
  'mail_smtpname' => 'aaa',
  'mail_smtppassword' => 'bbb',

JLueke · 2019-09-30T21:51:01Z

Unfortunately I still get the my old error.

guillaumv · 2019-10-05T09:39:07Z

Same here. Using Nexctloud 17.0.0 and the same SMTP settings work with other email clients.

Only on nextcloud I get the :

Unable to connect with TLS encryption Log data: ++ Starting Swift_SmtpTransport << 220 smtp7.infomaniak.ch ESMTP Infomaniak Network Relay Mail Servers; Sat, 5 Oct 2019 11:00:54 +0200 >> EHLO mycloud-integration.nostraterra.ch << 250-smtp7.infomaniak.ch Hello None.236.80.80.in-addr.arpa [80.80.236.11] (may be forged), pleased to meet you 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-8BITMIME 250-SIZE 250-DSN 250-AUTH LOGIN PLAIN 250-STARTTLS 250-DELIVERBY 250 HELP >> STARTTLS << 220 2.0.0 Ready to start TLS !! Unable to connect with TLS encryption (code: 0))

very puzzling..

JLueke · 2019-10-07T18:00:27Z

@guillaumv did you check, if it works on earlier Versions?
Especially the docker versions 14.0.12-apache and 14.0.13-apache?

kesselb · 2019-10-08T18:55:54Z

Nextcloud 14 => Swiftmailer v6.0.2
Nextcloud 15 => Swiftmailer v6.1.3

https://github.com/swiftmailer/swiftmailer/blob/master/CHANGES maybe one of these changes broke it. Still need to figure out if this is related to nextcloud or swiftmailer.

dimm0 · 2019-10-19T01:24:16Z

No luck still?
Also hitting this on 17-apache...

guillaumv · 2019-10-20T13:54:09Z

@JLueke No I havent tried with earlier version than 16.3.5 and 17 (same error on both but I havent got the time to test much so far)

I got this solved with mail mail provider who simply told me to disable STARTTLS. I did and I can now send email with their SMTP. However I have to compromise on security for now I guess...

p0lyg0ne · 2019-12-16T12:55:25Z

I also have exactly the same problem however it only happens with smtp.goneo.de.
Sending emails via Thunderbird is possible but with the same credentials and smtp settings nextcloud is not able to send an email.

I've tried the same with an address from web.de, nextcloud is able to send emails via their smtp servers. My humble opinion would be, some server side configuration on smtp.goneo.de upsets nextcloud...

dimm0 · 2019-12-16T17:41:35Z

I'm still stuck with the only university mail server I can use...

kojo1984 · 2020-02-17T10:21:16Z

Problem is on the server side, certificates that are used for SSL/TLS are not trusted or cert chain is bad (I've added all my certs into appropriate stores, openssl test is OK, but this doesn't help).

Not the happiest solution, but you should add $transport->setStreamOptions(array('ssl' => array('allow_self_signed' => true, 'verify_peer' => false))); into /var/www/html/lib/private/Mail/Mailer.php (or where ever your Mailer.php is) and replace the existing line $transport->setEncryption($smtpSecurity); ... or comment it out with //.

That line should be between lines 260 and 270 in Mailer.php (mine was on 262).

Tested on NextCloud17.0.3.1, Apache 2.4.38, Debian 10.

vindic · 2020-02-28T06:40:40Z

For me this solution worked only adding this parameter in the config.php file

'mail_smtpstreamoptions' => array ( 'ssl' => array ( 'allow_self_signed' => true, 'verify_peer' => false, 'verify_peer_name' => false ) )

For Nextcloud 18 on Ubuntu 18 connected to Exchange 2016 email server.

wagnerfl85 · 2020-05-03T16:39:08Z

I have the same problem and the solution of vindic is working for me to. But: i have a working mail server with trusted certificates (letsencrypt). The postfix server is configured to only use TLSv1.2.

So, why do I have to use this workaround?

kesselb · 2020-05-03T17:57:56Z

So, why do I have to use this workaround?

tl;dr: Because the server you installed Nextcloud on does not trust the certificate that is used by your mail server.

Nextcloud uses Swiftmailer (a library) to send emails. Swiftmailer (or PHP probably) uses the systems certificate storage for certificate validation.

You (Client) -> Mail server: Client (e.g. Thunderbird / whatever) will validate the certificate. That's you using your mail server.
Nextcloud (Client) -> Mail server: Client (Swiftmailer and/or PHP) connects to the mail server and validate the certificate. That's Nextcloud sending an email.

If 1 works but 2 fails usually the server fails to validate the certificate. Why does it fail? The list of certificates contains only the big certificate companies. A reseller pays another certificate company for a intermediate certificate. With this intermediate certificate a reseller is able to sign certificates without being on the list (but the big companies are also using intermediates). Let's encrypt is also using a intermediate certificate.

How to fix that: Make sure the server (Nextcloud is installed on) is able to establish a secure connection to the mail server.

Please visit https://help.nextcloud.com/ or Stack Overflow for such questions. It's not really a issue with Nextcloud but the server configuration. Establishing connections to other services and validating certificates is something the operating system is responsible for. Nextcloud just logs the response. There is no way to fix that.

wagnerfl85 · 2020-05-04T07:49:15Z

Sorry for this miss placed question. I thought it would fit to the problem and it could be related with Nextcloud directly.

Thanks, especially because of my wrong placement, for you detailed answer. Now i understand whats the problem. :-)

ghost · 2020-06-03T07:52:56Z

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

pwepwe973 · 2020-12-02T14:30:50Z

same problem and For me this solution worked to

only adding this parameter in the config.php file

'mail_smtpstreamoptions' => array ( 'ssl' => array ( 'allow_self_signed' => true, 'verify_peer' => false, 'verify_peer_name' => false ) )

For Nextcloud 20 on docker and synology server mail

inverse · 2021-10-05T19:08:04Z

@pwepwe973 your solution is disabling the certificate validation though.

See: https://www.php.net/manual/en/context.ssl.php for conext of each of those params.

A better solution would be to update the cacerts on your distribution.

On Debian based setups it would be something like

sudo update-ca-certificates --fresh

pwepwe973 · 2021-10-07T22:13:04Z

@pwepwe973 your solution is disabling the certificate validation though.

See: https://www.php.net/manual/en/context.ssl.php for conext of each of those params.

A better solution would be to update the cacerts on your distribution.

On Debian based setups it would be something like
sudo update-ca-certificates --fresh

hello
this is not my solution it was suggested earlier in the conversation I was just saying that the solution worked for me

hugalafutro · 2021-12-27T15:40:37Z

I've read through the thread and through the good explanation in post #17134 (comment), but as a non-technical hobbyist user I have to ask why Nextcloud is the only app that has this problem for me?

Everywhere else I use my email server be it in webmail app or in android gmail app as ms exchange account or as smtp mailserver for all my local machines/vm it works without a hitch. Trying to use it in same way in NC or setup my mail server in NC mail app fails.

I think understand why it happens and I can see the reason if I run openssl s_client -starttls smtp -crlf -connect [my mail server]:587, but how is Microsoft Exchange and Gmail deem it safe enough to "just work" and Nextcloud's mail implementation doesn't?

I apologize if I misunderstand the problematics, but this is how I see it from what I can understand.

JLueke added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Sep 13, 2019

kesselb added feature: emails needs info labels May 3, 2020

ghost added the stale Ticket or PR with no recent activity label Jun 3, 2020

ghost closed this as completed Jun 17, 2020

This issue was closed.

Unable to connect with TLS encryption using STARTTLS 587 #17134

Unable to connect with TLS encryption using STARTTLS 587 #17134

Comments

JLueke commented Sep 13, 2019

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration detail

Client configuration

Logs

solracsf commented Sep 13, 2019 • edited Loading

JLueke commented Sep 13, 2019

j-ed commented Sep 13, 2019

JLueke commented Sep 13, 2019

JLueke commented Sep 13, 2019 • edited Loading

Certificate chain 0 s:CN = *.mailprovider.de i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1 1 s:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1 i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2

No client certificate CA names sent

SSL handshake has read 3599 bytes and written 345 bytes Verification: OK

j-ed commented Sep 13, 2019

JLueke commented Sep 13, 2019

j-ed commented Sep 13, 2019

JLueke commented Sep 13, 2019 • edited Loading

JLueke commented Sep 30, 2019 • edited Loading

JLueke commented Sep 30, 2019

kesselb commented Sep 30, 2019

JLueke commented Sep 30, 2019 • edited Loading

JLueke commented Sep 30, 2019 • edited Loading

kesselb commented Sep 30, 2019

JLueke commented Sep 30, 2019

kesselb commented Sep 30, 2019

JLueke commented Sep 30, 2019

kesselb commented Sep 30, 2019

JLueke commented Sep 30, 2019

guillaumv commented Oct 5, 2019

JLueke commented Oct 7, 2019

kesselb commented Oct 8, 2019

dimm0 commented Oct 19, 2019

guillaumv commented Oct 20, 2019

p0lyg0ne commented Dec 16, 2019

dimm0 commented Dec 16, 2019

kojo1984 commented Feb 17, 2020

vindic commented Feb 28, 2020 • edited Loading

wagnerfl85 commented May 3, 2020

kesselb commented May 3, 2020

wagnerfl85 commented May 4, 2020

ghost commented Jun 3, 2020

pwepwe973 commented Dec 2, 2020 • edited Loading

inverse commented Oct 5, 2021

pwepwe973 commented Oct 7, 2021

hugalafutro commented Dec 27, 2021

solracsf commented Sep 13, 2019 •

edited

Loading

JLueke commented Sep 13, 2019 •

edited

Loading

Certificate chain
0 s:CN = *.mailprovider.de
i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1
1 s:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1
i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2

SSL handshake has read 3599 bytes and written 345 bytes
Verification: OK

JLueke commented Sep 13, 2019 •

edited

Loading

JLueke commented Sep 30, 2019 •

edited

Loading

JLueke commented Sep 30, 2019 •

edited

Loading

JLueke commented Sep 30, 2019 •

edited

Loading

vindic commented Feb 28, 2020 •

edited

Loading

pwepwe973 commented Dec 2, 2020 •

edited

Loading