Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[9.5] New IMAP Library Does Not Handle No Reply-To in Email Headers #7643

Closed
b1scuit-thi3f opened this issue Jul 9, 2020 · 10 comments · Fixed by #7652
Closed

[9.5] New IMAP Library Does Not Handle No Reply-To in Email Headers #7643

b1scuit-thi3f opened this issue Jul 9, 2020 · 10 comments · Fixed by #7652
Assignees
Labels
Milestone

Comments

@b1scuit-thi3f
Copy link

b1scuit-thi3f commented Jul 9, 2020

Describe the bug

Since upgrading our system to 9.5.0, we have discovered that our email receiver does not work as expected. On further investigation, it appears that it now cannot handle emails in which we do not have a Reply-To header.

This is the error we get when trying to receive emails.

[2020-07-09 16:45:30] glpiphplog.CRITICAL:   *** Uncaught Exception Laminas\Mail\Storage\Exception\InvalidArgumentException: Header with Name reply-to or reply-to not found in /var/www/html/glpi/vendor/laminas/laminas-mail/src/Storage/Part.php at line 307
  Backtrace :
  inc/mailcollector.class.php:955                    Laminas\Mail\Storage\Part->getHeader()
  inc/mailcollector.class.php:711                    MailCollector->buildTicket()
  front/mailcollector.form.php:77                    MailCollector->collect()

It should be noted that we do have an In-Reply-To header. I would also like to note that it doesn't seem to handle emails where there is no To header (for example, all recipients are in the BCC header).

Page(s) URL

https://{GLPIHOST}/glpi/front/mailcollector.form.php

To reproduce

Set up a mail receiver.
Have an email with either no To header, or not Reply-To header in the inbox.
Attempt to retrieve the email.

Expected behavior

The email should be imported into GLPI.

Logs

[2020-07-09 16:45:30] glpiphplog.CRITICAL:   *** Uncaught Exception Laminas\Mail\Storage\Exception\InvalidArgumentException: Header with Name reply-to or reply-to not found in /var/www/html/glpi/vendor/laminas/laminas-mail/src/Storage/Part.php at line 307
  Backtrace :
  inc/mailcollector.class.php:955                    Laminas\Mail\Storage\Part->getHeader()
  inc/mailcollector.class.php:711                    MailCollector->buildTicket()
  front/mailcollector.form.php:77                    MailCollector->collect()  

Your GLPI setup (you can find it in Setup > General menu, System tab)

**General**
Operating system: Linux itsm01 18.04.1-Ubuntu SMP Tue Jun 30 23:04:16 UTC 2020 x86_64
PHP 7.2.24-0ubuntu apache2handler (Core, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, apache2handler, apc,
	apcu, calendar, ctype, curl, date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, imap, intl, json, ldap, libxml,
	mbstring, mysqli, mysqlnd, openssl, pcre, pdo_mysql, posix, readline, session, shmop, sockets, sodium, standard, sysvmsg,
	sysvsem, sysvshm, tokenizer, wddx, xml, xmlreader, xmlrpc, xmlwriter, xsl, zlib)
Setup: max_execution_time="600" memory_limit="256M" post_max_size="8M" safe_mode="" session.save_handler="files"
	upload_max_filesize="50M" 
Software: Apache/2.4.29 (Ubuntu) (Apache/2.4.29 (Ubuntu) Server at REDACTED Port 443)
	Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36
Server Software: (Ubuntu)
	Server Version: ubuntu
	Server SQL Mode: 
	Parameters: glpiuser@localhost/glpi
	Host info: Localhost via UNIX socket
	
PHP version is at least 7.2.0 - Perfect!PHP version is at least 7.2.0 - Perfect!
Sessions support is available - Perfect!Sessions support is available - Perfect!
Allocated memory > 64 Mio - Perfect!Allocated memory > 64 Mio - Perfect!
mysqli extension is installedmysqli extension is installed
ctype extension is installedctype extension is installed
fileinfo extension is installedfileinfo extension is installed
json extension is installedjson extension is installed
mbstring extension is installedmbstring extension is installed
iconv extension is installediconv extension is installed
zlib extension is installedzlib extension is installed
curl extension is installedcurl extension is installed
gd extension is installedgd extension is installed
simplexml extension is installedsimplexml extension is installed
intl extension is installedintl extension is installed
ldap extension is installedldap extension is installed
apcu extension is installedapcu extension is installed
Zend OPcache extension is installedZend OPcache extension is installed
xmlrpc extension is installedxmlrpc extension is installed
CAS extension is installedCAS extension is installed
exif extension is installedexif extension is installed
zip extension is not presentzip extension is not present
bz2 extension is not presentbz2 extension is not present
sodium extension is installedsodium extension is installed
Database version seems correct (5.7.30) - Perfect!Database version seems correct (5.7.30) - Perfect!
Timezones seems loaded in databaseTimezones seems loaded in database
The log file has been created successfully.The log file has been created successfully.
Write access to /etc/glpi/ has been validated.Write access to /etc/glpi/ has been validated.
Write access to /var/lib/glpi has been validated.Write access to /var/lib/glpi has been validated.
Write access to /var/lib/glpi/_dumps has been validated.Write access to /var/lib/glpi/_dumps has been validated.
Write access to /var/lib/glpi/_sessions has been validated.Write access to /var/lib/glpi/_sessions has been validated.
Write access to /var/lib/glpi/_cron has been validated.Write access to /var/lib/glpi/_cron has been validated.
Write access to /var/lib/glpi/_graphs has been validated.Write access to /var/lib/glpi/_graphs has been validated.
Write access to /var/lib/glpi/_lock has been validated.Write access to /var/lib/glpi/_lock has been validated.
Write access to /var/lib/glpi/_plugins has been validated.Write access to /var/lib/glpi/_plugins has been validated.
Write access to /var/lib/glpi/_tmp has been validated.Write access to /var/lib/glpi/_tmp has been validated.
Write access to /var/lib/glpi/_cache has been validated.Write access to /var/lib/glpi/_cache has been validated.
Write access to /var/lib/glpi/_rss has been validated.Write access to /var/lib/glpi/_rss has been validated.
Write access to /var/lib/glpi/_uploads has been validated.Write access to /var/lib/glpi/_uploads has been validated.
Write access to /var/lib/glpi/_pictures has been validated.Write access to /var/lib/glpi/_pictures has been validated.
Write access to /var/www/html/glpi/marketplace has been validated.Write access to /var/www/html/glpi/marketplace has been validated.

**Libraries**
tmlawed/htmlawed version 1.2.5 in (/var/www/html/glpi/vendor/htmlawed/htmlawed)
phpmailer/phpmailer version 6.1.6 in (/var/www/html/glpi/vendor/phpmailer/phpmailer/src)
simplepie/simplepie version 1.5.5 in (/var/www/html/glpi/vendor/simplepie/simplepie/library)
tecnickcom/tcpdf version 6.3.5 in (/var/www/html/glpi/vendor/tecnickcom/tcpdf)
michelf/php-markdown in (/var/www/html/glpi/vendor/michelf/php-markdown/Michelf)
true/punycode in (/var/www/html/glpi/vendor/true/punycode/src)
iamcal/lib_autolink in (/var/www/html/glpi/vendor/iamcal/lib_autolink)
sabre/dav in (/var/www/html/glpi/vendor/sabre/dav/lib/DAV)
sabre/http in (/var/www/html/glpi/vendor/sabre/http/lib)
sabre/uri in (/var/www/html/glpi/vendor/sabre/uri/lib)
sabre/vobject in (/var/www/html/glpi/vendor/sabre/vobject/lib)
laminas/laminas-cache in (/var/www/html/glpi/vendor/laminas/laminas-cache/src)
laminas/laminas-i18n in (/var/www/html/glpi/vendor/laminas/laminas-i18n/src)
laminas/laminas-serializer in (/var/www/html/glpi/vendor/laminas/laminas-serializer/src)
monolog/monolog in (/var/www/html/glpi/vendor/monolog/monolog/src/Monolog)
sebastian/diff in (/var/www/html/glpi/vendor/sebastian/diff/src)
elvanto/litemoji in (/var/www/html/glpi/vendor/elvanto/litemoji/src)
symfony/console in (/var/www/html/glpi/vendor/symfony/console)
scssphp/scssphp in (/var/www/html/glpi/vendor/scssphp/scssphp/src)
laminas/laminas-mail in (/var/www/html/glpi/vendor/laminas/laminas-mail/src/Protocol)
laminas/laminas-mime in (/var/www/html/glpi/vendor/laminas/laminas-mime/src)
rlanvin/php-rrule in (/var/www/html/glpi/vendor/rlanvin/php-rrule/src)
blueimp/jquery-file-upload in (/var/www/html/glpi/vendor/blueimp/jquery-file-upload/server/php)
ramsey/uuid in (/var/www/html/glpi/vendor/ramsey/uuid/src)
psr/log in (/var/www/html/glpi/vendor/psr/log/Psr/Log)
psr/simple-cache in (/var/www/html/glpi/vendor/psr/simple-cache/src)
mexitek/phpcolors in (/var/www/html/glpi/vendor/mexitek/phpcolors/src/Mexitek/PHPColors)
guzzlehttp/guzzle in (/var/www/html/glpi/vendor/guzzlehttp/guzzle/src)
wapmorgan/unified-archive in (/var/www/html/glpi/vendor/wapmorgan/unified-archive/src)
paragonie/sodium_compat in (/var/www/html/glpi/vendor/paragonie/sodium_compat/src)
phpCas version 1.3.3 in (/usr/share/php)

**Plugins**
actualtime           Name: ActualTime                     Version: 1.2.0      State: Installed / not activated
	fields               Name: Additionnal fields             Version: 1.11.0     State: Enabled
	behaviors            Name: Behaviours                     Version: 2.2.2      State: Enabled
	dashboard            Name: Dashboard                      Version: 0.9.6      State: Enabled
	escalade             Name: Escalation                     Version: 2.4.4      State: Enabled
	datainjection        Name: File injection                 Version: 2.7.0      State: Installed / not activated
	formcreator          Name: Form Creator                   Version: 2.10.0     State: Enabled
	itilcategorygroups   Name: ItilCategory Groups            Version: 2.4.0      State: Enabled
	mailanalyzer         Name: Mail Analyzer                  Version: 1.6.1      State: Enabled
	mreporting           Name: More Reporting                 Version: 1.7.0      State: Enabled
	genericobject        Name: Objects management             Version: 2.9.0      State: Enabled
	tag                  Name: Tag Management                 Version: 2.7.0      State: Enabled
	tasklists            Name: Tasks list                     Version: 1.5.1      State: Enabled
	timezones            Name: Timezones                      Version: 2.5.2      State: Installed / not activated
@raregtp
Copy link

raregtp commented Jul 10, 2020

Seeing this as well...identical in fact. Have tried both IMAP and POP....this is on a new install trying to retrieve email from an Office 365 mailbox.

@raregtp
Copy link

raregtp commented Jul 10, 2020

Ok....further digging....

Exchange environments do not set the Reply-To header. This goes for on-prem as well as Exchange Online. If when sending an email through an Outlook client you client the Options menu, and then click the 'Direct Replies To' button, under Delivery options there will be a checkbox called 'Have replies sent to', and it should have your email name/account pre-populated. Simply click close and send your email....it will now contain a Reply-To header. Upon doing this, the IMAP receiver picks up the mail just fine. Not sure how other mail server environments work, but at least for Exchange, there is no way to set a Reply-To header at the org level, even using transport rules.

That's enough about Exchange....just wanted to illustrate what I did to test this......with no changes to my IMAP receiver the above will allow the email to be picked up. If I don't perform the above, I'll receive the same error outlined by the op.

After some searching through the php files, what I've done to temporarily fix this, (and only because I'm not in prod with this new instance), I've commented out some lines in glpi/inc/mailcollector.class.php. Specifically, I've commented out lines 1965 and 1966. Without this case in the switch statement, the default option is fired when returns the From header as the requester field. With this change I'm pulling all emails that hit the inbox that my IMAP receiver is polling.

Again....I'm not in prod yet and would anticipate a permanent fix to this before I go to prod (if I don't dig further for the root cause myself....), but I also don't feel this is super-risky because I'm just not allowing the system to look for a reply-to header.

Just wanted to offer up more of what I've discovered this evening. Thanks all!

@b1scuit-thi3f
Copy link
Author

@raregtp Thanks for the insight! I had played around with the PHP myself, but not having used it ages and being very rusty, was making a lot of shots in the dark. I will look at disabling those lines too, as this may solve our issue temporarily. However, like you, I would hope for a permanent fix soon...

@cedric-anne
Copy link
Member

Hi @raregtp ,

Can you attach to this issue a sample email (in .eml format) that cause the problem ? So we will be able to use it in unit tests.
You can change address/names in email headers if you do not want them to be exposed in our repository.

Regards

@b1scuit-thi3f
Copy link
Author

b1scuit-thi3f commented Jul 10, 2020

Hi @cedric-anne

I've attached a sample email (redacted) that I believe causes the issue (as you will notice, it does not have a Reply-To header).
Please let me know if this helps.

SampleEmail.zip

@cedric-anne
Copy link
Member

Thanks for this sample email.

I successfully reproduce this problem on our test suite (see https://github.com/glpi-project/glpi/pull/7652/checks?check_run_id=857667428), so now we have to find a fix for it.

@b1scuit-thi3f
Copy link
Author

Hi,

Thanks for the update. Let me know if you need any more details in the meantime.

cedric-anne added a commit to cedric-anne/glpi that referenced this issue Jul 13, 2020
- Prevent exception on adresses parsing; fixes glpi-project#7643
- Correctly fetch additionnal headers that may be used in rules
- Fix exclusion of mailer-daemon/postmaster
cedric-anne added a commit to cedric-anne/glpi that referenced this issue Jul 13, 2020
- Prevent exception on adresses parsing; fixes glpi-project#7643
- Correctly fetch additionnal headers that may be used in rules
- Fix exclusion of mailer-daemon/postmaster
cedric-anne added a commit to cedric-anne/glpi that referenced this issue Jul 13, 2020
- Prevent exception on adresses parsing; fixes glpi-project#7643
- Correctly fetch additionnal headers that may be used in rules
- Fix exclusion of mailer-daemon/postmaster
cedric-anne added a commit to cedric-anne/glpi that referenced this issue Jul 13, 2020
- Prevent exception on adresses parsing; fixes glpi-project#7643
- Correctly fetch additionnal headers that may be used in rules
- Fix exclusion of mailer-daemon/postmaster
cedric-anne added a commit to cedric-anne/glpi that referenced this issue Jul 13, 2020
- Prevent exception on adresses parsing; fixes glpi-project#7643
- Correctly fetch additionnal headers that may be used in rules
- Fix exclusion of mailer-daemon/postmaster
cedric-anne added a commit to cedric-anne/glpi that referenced this issue Jul 13, 2020
- Prevent exception on adresses parsing; fixes glpi-project#7643
- Correctly fetch additionnal headers that may be used in rules
- Fix exclusion of mailer-daemon/postmaster
cedric-anne added a commit to cedric-anne/glpi that referenced this issue Jul 13, 2020
- Prevent exception on adresses parsing; fixes glpi-project#7643
- Correctly fetch additionnal headers that may be used in rules
- Fix exclusion of mailer-daemon/postmaster
- Prevent exception when "Content-Transfer-Encoding" is not set; fixes glpi-project#7649
cedric-anne added a commit to cedric-anne/glpi that referenced this issue Jul 13, 2020
- Prevent exception on addresses parsing; fixes glpi-project#7643
- Correctly fetch additionnal headers that may be used in rules
- Fix exclusion of mailer-daemon/postmaster
- Prevent exception when "Content-Transfer-Encoding" is not set; fixes glpi-project#7649
cedric-anne added a commit to cedric-anne/glpi that referenced this issue Jul 13, 2020
- Prevent exception on addresses parsing; fixes glpi-project#7643
- Correctly fetch additionnal headers that may be used in rules
- Fix exclusion of mailer-daemon/postmaster
- Prevent exception when "Content-Transfer-Encoding" is not set; fixes glpi-project#7649
@cedric-anne cedric-anne self-assigned this Jul 13, 2020
@cedric-anne cedric-anne added this to the 9.5.1 milestone Jul 13, 2020
cedric-anne added a commit that referenced this issue Jul 13, 2020
- Prevent exception on addresses parsing; fixes #7643
- Correctly fetch additionnal headers that may be used in rules
- Fix exclusion of mailer-daemon/postmaster
- Prevent exception when "Content-Transfer-Encoding" is not set; fixes #7649
@raregtp
Copy link

raregtp commented Jul 13, 2020

#7652 does appear to have resolved this issue. Thanks!

@b1scuit-thi3f
Copy link
Author

Yeah, this has fixed it for us too!
Unfortunately there are a few emails that have failed but it appears they don't have a content-type header. Not quite sure if GLPI should just handle these and mark them as failed imports, because at the moment it does appear to break the mailgate reciever.

@cedric-anne
Copy link
Member

Yeah, this has fixed it for us too!
Unfortunately there are a few emails that have failed but it appears they don't have a content-type header. Not quite sure if GLPI should just handle these and mark them as failed imports, because at the moment it does appear to break the mailgate reciever.

Hi,

Can you create another issue with the error trace and attach an example of email causing this issue ?
It will be easier for us to start a new thread for this new issue.

Regards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants