Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hebrew Characters turn to Gibberish when ticket is received via 0Auth with Office365 #17396

Open
2 tasks done
NegroniBlast opened this issue Jul 1, 2024 · 8 comments
Open
2 tasks done
Labels

Comments

@NegroniBlast
Copy link

NegroniBlast commented Jul 1, 2024

Code of Conduct

  • I agree to follow this project's Code of Conduct

Is there an existing issue for this?

  • I have searched the existing issues

Version

10.0.12

Bug description

I installed GLPI 10.0.12 and after connecting our ticketing office365 tenant mailbox via 0auth, I tried opening tickets via email with Hebrew characters and I get gibberish instead in the subject and the body.

When I open it manually on the platform there's no problem and Hebrew is shown fine.

Relevant log output

No response

Page URL

No response

Steps To reproduce

  1. Connect receiver office365 mailbox with 0Auth to GLPI
  2. Setup automatic mailgate to retrieve the new emails as tickets
  3. Send email with Hebrew characters to this mailbox
  4. The new ticket is created, but with Hebrew content in Gibberish

Your GLPI setup information

Information about system installation & configuration
GLPI 10.0.12 ( => /var/www/glpi)
Installation mode: TARBALL
Current language:en_US

Server
 
Operating system: Linux REDACTED WEBSITE ADDRESS 5.15.0-113-generic #​123-Ubuntu SMP Mon Jun 10 08:16:17 UTC 2024 x86_64
PHP 8.1.2-1ubuntu2.18 fpm-fcgi (Core, FFI, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, apcu, bz2, calendar, cgi-fcgi,
ctype, curl, date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, imagick, imap, intl, json, ldap, libxml,
mbstring, memcache, mysqli, mysqlnd, openssl, pcre, pdo_mysql, posix, pspell, readline, session, shmop, sockets, sodium,
standard, sysvmsg, sysvsem, sysvshm, tidy, tokenizer, xml, xmlreader, xmlrpc, xmlwriter, xsl, zip, zlib)
Setup: max_execution_time="30" memory_limit="128M" post_max_size="8M" safe_mode="" session.save_handler="files"
upload_max_filesize="2M" disable_functions=""
Software: nginx/1.18.0
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
Server Software: Ubuntu 22.04
Server Version: 10.6.18-MariaDB-0ubuntu0.22.04.1
Server SQL Mode: STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
Parameters: glpi_admin@localhost/glpi_db
Host info: Localhost via UNIX socket

PHP version (8.1.2-1ubuntu2.18) is supported.
Sessions configuration is OK.
Allocated memory is sufficient.
mysqli extension is installed.
Following extensions are installed: dom, fileinfo, filter, libxml, json, simplexml, xmlreader, xmlwriter.
curl extension is installed.
gd extension is installed.
intl extension is installed.
zlib extension is installed.
The constant SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES is present.
Database engine version (10.6.18) is supported.
No files from previous GLPI version detected.
The log file has been created successfully.
Write access to /var/www/glpi/files/_cache has been validated.
Write access to /var/www/glpi/files/_cron has been validated.
Write access to /var/www/glpi/files has been validated.
Write access to /var/www/glpi/files/_dumps has been validated.
Write access to /var/www/glpi/files/_graphs has been validated.
Write access to /var/www/glpi/files/_lock has been validated.
Write access to /var/www/glpi/files/_pictures has been validated.
Write access to /var/www/glpi/files/_plugins has been validated.
Write access to /var/www/glpi/files/_rss has been validated.
Write access to /var/www/glpi/files/_sessions has been validated.
Write access to /var/www/glpi/files/_tmp has been validated.
Write access to /var/www/glpi/files/_uploads has been validated.

Web server root directory configuration seems safe.
Sessions configuration is secured.
OS and PHP are relying on 64 bits integers.
exif extension is installed.
ldap extension is installed.
openssl extension is installed.
Following extensions are installed: bz2, Phar, zip.
Zend OPcache extension is installed.
Following extensions are installed: ctype, iconv, mbstring, sodium.
Write access to /var/www/glpi/marketplace has been validated.
Timezones seems loaded in database.

GLPI constants
 
GLPI_ROOT: "/var/www/glpi"
GLPI_CONFIG_DIR: "/var/www/glpi/config"
GLPI_VAR_DIR: "/var/www/glpi/files"
GLPI_MARKETPLACE_DIR: "/var/www/glpi/marketplace"
GLPI_USE_CSRF_CHECK: "1"
GLPI_CSRF_EXPIRES: "7200"
GLPI_CSRF_MAX_TOKENS: "100"
GLPI_USE_IDOR_CHECK: "1"
GLPI_IDOR_EXPIRES: "7200"
GLPI_ALLOW_IFRAME_IN_RICH_TEXT: false
GLPI_SERVERSIDE_URL_ALLOWLIST: ["/^(https?|feed):\/\/[^@:]+(\/.*)?$/"]
GLPI_TELEMETRY_URI: "https://telemetry.glpi-project.org"
GLPI_INSTALL_MODE: "TARBALL"
GLPI_NETWORK_MAIL: "glpi@teclib.com"
GLPI_NETWORK_SERVICES: "https://services.glpi-network.com"
GLPI_MARKETPLACE_ALLOW_OVERRIDE: true
GLPI_MARKETPLACE_MANUAL_DOWNLOADS: true
GLPI_USER_AGENT_EXTRA_COMMENTS: ""
GLPI_DISABLE_ONLY_FULL_GROUP_BY_SQL_MODE: "1"
GLPI_AJAX_DASHBOARD: "1"
GLPI_CALDAV_IMPORT_STATE: 0
GLPI_DEMO_MODE: "0"
GLPI_CENTRAL_WARNINGS: "1"
GLPI_TEXT_MAXSIZE: "4000"
GLPI_DOC_DIR: "/var/www/glpi/files"
GLPI_CACHE_DIR: "/var/www/glpi/files/_cache"
GLPI_CRON_DIR: "/var/www/glpi/files/_cron"
GLPI_DUMP_DIR: "/var/www/glpi/files/_dumps"
GLPI_GRAPH_DIR: "/var/www/glpi/files/_graphs"
GLPI_LOCAL_I18N_DIR: "/var/www/glpi/files/_locales"
GLPI_LOCK_DIR: "/var/www/glpi/files/_lock"
GLPI_LOG_DIR: "/var/www/glpi/files/_log"
GLPI_PICTURE_DIR: "/var/www/glpi/files/_pictures"
GLPI_PLUGIN_DOC_DIR: "/var/www/glpi/files/_plugins"
GLPI_RSS_DIR: "/var/www/glpi/files/_rss"
GLPI_SESSION_DIR: "/var/www/glpi/files/_sessions"
GLPI_TMP_DIR: "/var/www/glpi/files/_tmp"
GLPI_UPLOAD_DIR: "/var/www/glpi/files/_uploads"
GLPI_INVENTORY_DIR: "/var/www/glpi/files/_inventories"
GLPI_NETWORK_REGISTRATION_API_URL: "https://services.glpi-network.com/api/registration/"
GLPI_MARKETPLACE_PLUGINS_API_URI: "https://services.glpi-network.com/api/marketplace/"
GLPI_I18N_DIR: "/var/www/glpi/locales"
GLPI_VERSION: "10.0.12"
GLPI_SCHEMA_VERSION: "10.0.12@b902ef5f5da967c1a9d0826ee8a7dce07d57cf3d"
GLPI_MARKETPLACE_PRERELEASES: false
GLPI_MIN_PHP: "7.4.0"
GLPI_MAX_PHP: "8.4.0"
GLPI_YEAR: "2024"

Libraries
 
htmlawed/htmlawed version 1.2.14 in (/var/www/glpi/vendor/htmlawed/htmlawed)
phpmailer/phpmailer version 6.8.0 in (/var/www/glpi/vendor/phpmailer/phpmailer/src)
simplepie/simplepie version 1.5.8 in (/var/www/glpi/vendor/simplepie/simplepie/library)
tecnickcom/tcpdf version 6.6.2 in (/var/www/glpi/vendor/tecnickcom/tcpdf)
michelf/php-markdown in (/var/www/glpi/vendor/michelf/php-markdown/Michelf)
true/punycode in (/var/www/glpi/vendor/true/punycode/src)
iamcal/lib_autolink in (/var/www/glpi/vendor/iamcal/lib_autolink)
sabre/dav in (/var/www/glpi/vendor/sabre/dav/lib/DAV)
sabre/http in (/var/www/glpi/vendor/sabre/http/lib)
sabre/uri in (/var/www/glpi/vendor/sabre/uri/lib)
sabre/vobject in (/var/www/glpi/vendor/sabre/vobject/lib)
laminas/laminas-i18n in (/var/www/glpi/vendor/laminas/laminas-i18n/src)
laminas/laminas-servicemanager in (/var/www/glpi/vendor/laminas/laminas-servicemanager/src)
monolog/monolog in (/var/www/glpi/vendor/monolog/monolog/src/Monolog)
sebastian/diff in (/var/www/glpi/vendor/sebastian/diff/src)
donatj/phpuseragentparser in (/var/www/glpi/vendor/donatj/phpuseragentparser/src/UserAgent)
elvanto/litemoji in (/var/www/glpi/vendor/elvanto/litemoji/src)
symfony/console in (/var/www/glpi/vendor/symfony/console)
scssphp/scssphp in (/var/www/glpi/vendor/scssphp/scssphp/src)
laminas/laminas-mail in (/var/www/glpi/vendor/laminas/laminas-mail/src/Protocol)
laminas/laminas-mime in (/var/www/glpi/vendor/laminas/laminas-mime/src)
rlanvin/php-rrule in (/var/www/glpi/vendor/rlanvin/php-rrule/src)
ramsey/uuid in (/var/www/glpi/vendor/ramsey/uuid/src)
psr/log in (/var/www/glpi/vendor/psr/log/Psr/Log)
psr/simple-cache in (/var/www/glpi/vendor/psr/simple-cache/src)
psr/cache in (/var/www/glpi/vendor/psr/cache/src)
league/csv in (/var/www/glpi/vendor/league/csv/src)
mexitek/phpcolors in (/var/www/glpi/vendor/mexitek/phpcolors/src/Mexitek/PHPColors)
guzzlehttp/guzzle in (/var/www/glpi/vendor/guzzlehttp/guzzle/src)
guzzlehttp/psr7 in (/var/www/glpi/vendor/guzzlehttp/psr7/src)
glpi-project/inventory_format in (/var/www/glpi/vendor/glpi-project/inventory_format/lib/php)
wapmorgan/unified-archive in (/var/www/glpi/vendor/wapmorgan/unified-archive/src)
paragonie/sodium_compat in (/var/www/glpi/vendor/paragonie/sodium_compat/src)
symfony/cache in (/var/www/glpi/vendor/symfony/cache)
html2text/html2text in (/var/www/glpi/vendor/html2text/html2text/src)
symfony/css-selector in (/var/www/glpi/vendor/symfony/css-selector)
symfony/dom-crawler in (/var/www/glpi/vendor/symfony/dom-crawler)
twig/twig in (/var/www/glpi/vendor/twig/twig/src)
twig/string-extra in (/var/www/glpi/vendor/twig/string-extra)
symfony/polyfill-ctype not found
symfony/polyfill-iconv not found
symfony/polyfill-mbstring not found
symfony/polyfill-php80 not found
symfony/polyfill-php81 not found
symfony/polyfill-php82 in (/var/www/glpi/vendor/symfony/polyfill-php82)
league/oauth2-client in (/var/www/glpi/vendor/league/oauth2-client/src/Provider)
league/oauth2-google in (/var/www/glpi/vendor/league/oauth2-google/src/Provider)
thenetworg/oauth2-azure in (/var/www/glpi/vendor/thenetworg/oauth2-azure/src/Provider)
phpCas version 1.3.8 in (/usr/share/php/CAS/source)

SQL replicas
 
Not active

Notifications
 
Way of sending emails: SMTP+TLS (REDACTED@octopuscs.com@smtp.office365.com)

Plugins list
 
news Name: Alerts Version: 1.12.2 State: Installed / not activated
Install Method: Marketplace
behaviors Name: Behaviours Version: 2.7.3 State: Installed / not activated
Install Method: Marketplace
datainjection Name: Data injection Version: 2.13.5 State: Installed / not activated
Install Method: Marketplace
archimap Name: Diagrams Version: 3.3.4 State: Installed / not activated
Install Method: Marketplace
financialreports Name: Financial reports Version: 3.0.0 State: Installed / not activated
Install Method: Marketplace
addressing Name: IP Adressing Version: 3.0.2 State: Installed / not activated
Install Method: Marketplace
mfa Name: MFA Version: 1.0.2 State: Installed / not activated
Install Method: Marketplace
mreporting Name: More Reporting Version: 1.8.6 State: Installed / not activated
Install Method: Marketplace
oauthimap Name: Oauth IMAP Version: 1.4.3 State: Enabled
Install Method: Marketplace
onetimesecret Name: OneTimeSecret Version: 2.1.3 State: Installed / not activated
Install Method: Marketplace
pdf Name: Print to pdf Version: 3.0.0 State: Installed / not activated
Install Method: Marketplace
reports Name: Reports Version: 1.16.0 State: Installed / not activated
Install Method: Marketplace
screenshot Name: Screenshot Version: 2.0.3 State: Installed / not activated
Install Method: Marketplace
ticketfilter Name: Ticketfilter Version: 1.3.2 State: Installed / not activated
Install Method: Marketplace
timelineticket Name: Timeline of tickets Version: 10.0+1.2 State: Installed / not activated
Install Method: Marketplace
vip Name: VIP Version: 1.8.2 State: Installed / not activated
Install Method: Marketplace

Anything else?

Screenshot 2024-07-01 at 14 07 51

@NegroniBlast
Copy link
Author

these might be relevant but i dont know if these fixes were implemented into 10.0.12 so i decided to open a new issue here
#7748
#4844

@NegroniBlast
Copy link
Author

this is the closest issue I found on the subject #7709

@cedric-anne
Copy link
Member

Hi,

Please check if the problem persists in GLPI 10.0.15. If problem persists, please attache the message into .eml format so we could add it to our test suite and see the results.

@NegroniBlast
Copy link
Author

Hi Cedric thanks for the quick reply. we upgrade to 10.0.15 and now the gibberish is different haha, still an encoding issue.
Screenshot 2024-07-01 at 16 23 32
any recommendations?

@NegroniBlast
Copy link
Author

Testing after 10.0.15 בדיקה נוספת אחרי מעבר לגרסה חדשה.eml.zip

it was blocking the .eml file so it's attached inside this compressed zip file.

@cedric-anne
Copy link
Member

I validate the issue. It sees that the problem is related to usage of iso-8859-8-i encoding.

*** PHP Warning (2): iconv(): Wrong encoding, conversion from "iso-8859-8-i" to "UTF-8//IGNORE" is not allowed in /var/www/glpi/src/MailCollector.php at line 2470
  Backtrace :
  src/MailCollector.php:2470                         iconv()
  src/MailCollector.php:1808                         MailCollector->getDecodedContent()
  src/MailCollector.php:1150                         MailCollector->getBody()
  src/MailCollector.php:811                          MailCollector->buildTicket()
  front/mailcollector.form.php:106                   MailCollector->collect()
  public/index.php:82                                require()

@NegroniBlast
Copy link
Author

NegroniBlast commented Aug 1, 2024

Issue solved.

By editing Mailcollector.php I was able to edit the functions decodeMimeHeader and getDecodedContent to be able to handle Hebrew characters. the decoding function that works for incoming emails from M365 is mb_convert_encoding.

Attaching my code snippets of Mailcollector.php for your consideration. I will test some more next week but you can close this thread if you want.

  1. Subject wasn't using the subject decoding function - it was going to 'cleanSubject' instead. I replace cleanSubject with my decodeMimeHeader function.
  2. getDecodedContent had too many conditions at first, then i replaced it with a leaner version which is suitable for our needs.
    GLPI Mailcollector.php updated functions and code for Hebrew Encoding email messages.zip

@trasher
Copy link
Contributor

trasher commented Aug 2, 2024

Could you please open a pull request with your changes? Test suite will run with your changes; and it's easiest for us to review changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants