SMTP authentication does not occur #11528

barf · 2018-10-02T00:13:52Z

Steps to reproduce

Install Nextcloud following documentation.
Configure SMTP with authentication in Settings -> Basic Settings
Click 'Send email' to test configuration

Please note this is very easy to reproduce and probably the reason so many SMTP issues are floating around.

Expected behaviour

SMTP authentication attempt occurs

Actual behaviour

No SMTP authentication attempt occurs

Server configuration

Operating system:
Ubuntu 18.04

Web server:
Apache

Database:
MariaDB

PHP version:
7.2

Nextcloud version:
nextcloud-14.0.1

Postfix Log excerpt:
Oct 2 00:10:35 mail postfix/smtpd[14736]: NOQUEUE: reject: RCPT from nextcloud.example.com[10.146.234.197]: 554 5.7.1 foo.barf@example.com: Recipient address rejected: SMTP AUTH is required for users under this sender domain; from=nextcloud-outgoing@example.com to=foo.bar@example.com proto=ESMTP helo=<nextcloud.example.com>

nextcloud-bot · 2018-10-02T00:25:52Z

GitMate.io thinks possibly related issues are #11219 (SMTP ), #5240 (“Authentication Required” prompt after upgrade to 12.0.0 does not work), #3279 (user_external should log error message if authentication over imap does not work), #11085 (Relation does not exist), and #3158 (authentication prompt seems not to work).

rullzer · 2018-10-02T08:46:35Z

Please note this is very easy to reproduce and probably the reason so many SMTP issues are floating around.

Not so easy as it works for me @MorrisJobke just fine...

Could you try with another e-mail host?

barf · 2018-10-02T15:48:54Z

Same result on two different Postfix systems.

SMTP authentication is not attempted. I used debug_peer in Postfix to review the SMTP sessions to confirm that no auth is being attempted. No 'AUTH PLAIN' command is sent by the client. The chatter is as follows:

250-mail.example.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
MAIL FROM:<nextcloud-outgoing@example.com>

Per authenticated SMTP, the MAIL FROM: command should not occur at this time -- the AUTH PLAIN command should be sent, followed by the MAIL and RCPT commands.

Can you give me more information about your environment @rullzer ? Do you need any more information from me?

kesselb · 2018-10-02T16:17:23Z

Could you share php occ config:list?

server/lib/private/Mail/Mailer.php

Lines 253 to 257 in c2ef47e

    
           if ($this->config->getSystemValue('mail_smtpauth', false)) { 
        
           	$transport->setUsername($this->config->getSystemValue('mail_smtpname', '')); 
        
           	$transport->setPassword($this->config->getSystemValue('mail_smtppassword', '')); 
        
           	$transport->setAuthMode($this->config->getSystemValue('mail_smtpauthtype', 'LOGIN')); 
        
           }

Credentials for smtp are not passed when mail_smtpauth is not true. This would be my starting point for further investigation.

barf · 2018-10-02T16:47:43Z

Thanks @danielkesselberg -- I checked config.php to assert the SMTP credentials were correct in there.

FWIW unauthenticated SMTP works OK but authenticated is necessary in my environment. Also FYI the LDAP credentials were not redacted in this config dump, so I did that.

{
    "system": {
        "debug": true,
        "mail_smtpdebug": true,
        "app.mail.smtplog.enabled": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "nextcloud.example.com"
        ],  
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "14.0.1.1",
        "overwrite.cli.url": "https:\/\/nextcloud.example.com\/nextcloud",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "htaccess.RewriteBase": "\/nextcloud",
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "mail_smtpmode": "smtp",   
        "mail_smtpauthtype": "PLAIN",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "mail_from_address": "***REMOVED SENSITIVE VALUE***"
    },
    "apps": {
        "accessibility": {
            "enabled": "yes",
            "installed_version": "1.0.1",
            "types": ""
        },
        "activity": {
            "enabled": "yes",
            "installed_version": "2.7.0",
            "types": "filesystem"
        },
        "backgroundjob": {
            "lastjob": "303"
        },
        "cloud_federation_api": {
            "enabled": "yes",
            "installed_version": "0.0.1",
            "types": "filesystem"
        },
        "comments": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "logging"
        },
        "core": {
            "backgroundjobs_mode": "cron",
            "installedat": "1538162135.7035",
            "lastcron": "1538497813",
            "lastupdateResult": "[]",
            "lastupdatedat": "1538424426",
            "oc.integritycheck.checker": "[]",
            "public_files": "files_sharing\/public.php",
            "public_webdav": "dav\/appinfo\/v1\/publicwebdav.php",
            "scss.variables": "17469a460e0ff7dc6fbbf67b5806a9c4",
            "vendor": "nextcloud"
        },
        "dav": {
            "enabled": "yes",
            "installed_version": "1.6.0",
            "types": "filesystem"
        },
        "federatedfilesharing": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": ""
        },
        "federation": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "authentication"
        },
        "files": {
            "cronjob_scan_files": "500",
            "enabled": "yes",
            "installed_version": "1.9.0",
            "types": "filesystem"
        },
        "files_pdfviewer": {
            "enabled": "yes",
            "installed_version": "1.3.2",
            "types": ""
        },
        "files_sharing": {
            "enabled": "yes",
            "installed_version": "1.6.2",
            "types": "filesystem"
        },
        "files_texteditor": {
            "enabled": "yes",
            "installed_version": "2.6.0",
            "types": ""
        },
        "files_trashbin": {
            "enabled": "yes",
            "installed_version": "1.4.1",
            "types": "filesystem,dav"
        },
        "files_versions": {
            "enabled": "yes",
            "installed_version": "1.7.1",
            "types": "filesystem,dav"
        },
        "files_videoplayer": {
            "enabled": "yes",
            "installed_version": "1.3.0",
            "types": ""
        },
        "firstrunwizard": {
            "enabled": "yes",
            "installed_version": "2.3.0",
            "types": "logging"
        },
        "gallery": {
            "enabled": "yes",
            "installed_version": "18.1.0",
            "types": ""
        },
        "logreader": {
            "enabled": "yes",
            "installed_version": "2.0.0",
            "ocsid": "170871",
            "types": ""
        },
        "lookup_server_connector": {
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": "authentication"
        },
        "mail": {
            "enabled": "yes",
            "installed_version": "0.10.0",
            "types": ""
        },
        "nextcloud_announcements": {
            "enabled": "yes",
            "installed_version": "1.3.0",
            "pub_date": "Sat, 10 Dec 2016 00:00:00 +0100",
            "types": "logging"
        },
        "notifications": {
            "enabled": "yes",
            "installed_version": "2.2.1",
            "types": "logging"
        },
        "oauth2": {
            "enabled": "yes",
            "installed_version": "1.2.1",
            "types": "authentication"
        },
        "password_policy": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": ""
        },
        "provisioning_api": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "prevent_group_restriction"
        },
        "serverinfo": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": ""
        },
        "sharebymail": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "filesystem"
        },
        "support": {
            "enabled": "yes",
            "installed_version": "1.0.0",
            "types": ""
        },
        "survey_client": {
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": ""
        },
        "systemtags": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "logging"
        },
        "theming": {
            "enabled": "yes",
            "installed_version": "1.5.0",
            "types": "logging"
        },
        "twofactor_backupcodes": {
            "enabled": "yes",
            "installed_version": "1.3.1",
            "types": ""
        },
        "updatenotification": {
            "enabled": "yes",
            "installed_version": "1.4.1",
            "types": "",
            "update_check_errors": "0"
        },
        "user_ldap": {
            "background_sync_interval": "43200",
            "background_sync_offset": "0",
            "background_sync_prefix": "s01",
            "cleanUpJobOffset": "0",
            "enabled": "yes",
            "installed_version": "1.4.0",
            "s01_lastChange": "1538423752",
            "s01has_memberof_filter_support": "0",
            "s01home_folder_naming_rule": "",
            "s01last_jpegPhoto_lookup": "0",
            "s01ldap_agent_password": "xxx",
            "s01ldap_attributes_for_group_search": "",
            "s01ldap_attributes_for_user_search": "",
            "s01ldap_backup_host": "",
            "s01ldap_backup_port": "",
            "s01ldap_base": "domainName=example.com,o=domains,dc=example,dc=com",
            "s01ldap_base_groups": "ou=Users,domainName=example.com,o=domains,dc=example,dc=com",
            "s01ldap_base_users": "ou=Users,domainName=example.com,o=domains,dc=example,dc=com",
            "s01ldap_cache_ttl": "600",
            "s01ldap_configuration_active": "1",
            "s01ldap_default_ppolicy_dn": "",
            "s01ldap_display_name": "cn",
            "s01ldap_dn": "cn=nextcloud,dc=example,dc=com",
            "s01ldap_dynamic_group_member_url": "",
            "s01ldap_email_attr": "mail",
            "s01ldap_experienced_admin": "0",
            "s01ldap_expert_username_attr": "",
            "s01ldap_expert_uuid_group_attr": "",
            "s01ldap_expert_uuid_user_attr": "",
            "s01ldap_gid_number": "gidNumber",
            "s01ldap_group_display_name": "cn",
            "s01ldap_group_filter": "(&(|(objectclass=inetOrgPerson)))",
            "s01ldap_group_filter_mode": "0",
            "s01ldap_group_member_assoc_attribute": "uniqueMember",
            "s01ldap_groupfilter_groups": "",
            "s01ldap_groupfilter_objectclass": "inetOrgPerson",
            "s01ldap_host": "mail.example.com",
            "s01ldap_login_filter": "(&(|(objectclass=inetOrgPerson))(|(mailPrimaryAddress=%uid)(mail=%uid)))",
            "s01ldap_login_filter_mode": "0",
            "s01ldap_loginfilter_attributes": "",
            "s01ldap_loginfilter_email": "1",
            "s01ldap_loginfilter_username": "0",
            "s01ldap_nested_groups": "0",
            "s01ldap_override_main_server": "",
            "s01ldap_paging_size": "500",
            "s01ldap_port": "389",
            "s01ldap_quota_attr": "",
            "s01ldap_quota_def": "",
            "s01ldap_tls": "0",
            "s01ldap_turn_off_cert_check": "0",
            "s01ldap_turn_on_pwd_change": "0",
            "s01ldap_user_avatar_rule": "default",
            "s01ldap_user_display_name_2": "",
            "s01ldap_user_filter_mode": "0",
            "s01ldap_userfilter_groups": "",
            "s01ldap_userfilter_objectclass": "inetOrgPerson",
            "s01ldap_userlist_filter": "(|(objectclass=inetOrgPerson))",
            "s01use_memberof_to_detect_membership": "1",
            "types": "authentication"
        },
        "workflowengine": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "filesystem"
        }
    }
}

kesselb · 2018-10-02T17:20:58Z

When I understand http://www.huschi.net/archiv/esmtp-dialog-smtp-auth.html and https://www.fehcom.de/qmail/smtpauth.html##FRAMEWORK correctly your mailserver has to announce auth to client. Could you try with ssl or tls? It's common that mailserver don't allow unprotected auth on port 25.

barf · 2018-10-02T17:28:22Z

OK this might be the right direction, thanks!
SSL and STARTTLS are not working for me, the SMTP client rejects the certificate (LetsEncrypt).
I'll investigate that issue and close this if adding the LetsEncrypt CA to the certificate store resolves it.

kesselb · 2018-10-02T17:35:40Z

adding the LetsEncrypt CA to the certificate store resolves

I guess this could work 👍

barf · 2018-10-02T17:44:44Z

Can you please give me any hints about how to trust a CA?

I tried like this but it's not trusting LetsEncrypt yet:
# sudo -u www-data php /var/www/nextcloud/occ security:certificates:import /tmp/lets-encrypt-x3-cross-signed.pem

The SMTP client is not trusting the LetsEncrypt CA (from tshark):
TLSv1.2 73 Alert (Level: Fatal, Description: Unknown CA)

kesselb · 2018-10-02T17:51:20Z

I usually import these to the global certificate store. Then it should be available to all processes.

barf · 2018-10-02T17:58:17Z

Me too, I put the LetsEncrypt CA (PKCS#7) file in the /etc/ssl/certs directory (although it was already there), I confirmed the chain using 'openssl s_client' from the NextCloud LXD container too.

Does this SMTP client library have it's own trust store (like with Java and the JKS)?

barf · 2018-10-02T18:05:02Z

I got it! This was related to Ubuntu not trusting LetsEncrypt by default.

Thanks for your help @danielkesselberg

nextcloud-bot added the bug label Oct 2, 2018

barf closed this as completed Oct 2, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SMTP authentication does not occur #11528

SMTP authentication does not occur #11528

barf commented Oct 2, 2018

nextcloud-bot commented Oct 2, 2018

rullzer commented Oct 2, 2018

barf commented Oct 2, 2018

kesselb commented Oct 2, 2018 •

edited

Loading

barf commented Oct 2, 2018 •

edited

Loading

kesselb commented Oct 2, 2018 •

edited

Loading

barf commented Oct 2, 2018 •

edited

Loading

kesselb commented Oct 2, 2018

barf commented Oct 2, 2018

kesselb commented Oct 2, 2018

barf commented Oct 2, 2018

barf commented Oct 2, 2018

SMTP authentication does not occur #11528

SMTP authentication does not occur #11528

Comments

barf commented Oct 2, 2018

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

nextcloud-bot commented Oct 2, 2018

rullzer commented Oct 2, 2018

barf commented Oct 2, 2018

kesselb commented Oct 2, 2018 • edited Loading

barf commented Oct 2, 2018 • edited Loading

kesselb commented Oct 2, 2018 • edited Loading

barf commented Oct 2, 2018 • edited Loading

kesselb commented Oct 2, 2018

barf commented Oct 2, 2018

kesselb commented Oct 2, 2018

barf commented Oct 2, 2018

barf commented Oct 2, 2018

kesselb commented Oct 2, 2018 •

edited

Loading

barf commented Oct 2, 2018 •

edited

Loading

kesselb commented Oct 2, 2018 •

edited

Loading

barf commented Oct 2, 2018 •

edited

Loading