Provider is marking my emails as spam - seeing base64 encoding for subject

[Probedude]

Hello.
My question is - how do I turn off Base64 sending of To:, From:, and Subject: ?
I am using ESP-Mail-Client version 3.4.21

Background:
I have an ESP8266 set up on my mailbox to text me when the door is opened and closed. It does this by sending an email to my cellnumber@vtext.com. This has been working great for almost a year but now my email provider thinks my ESP8266 is sending spam and blocking the email. They're saying their spam filters are getting stuck on the base64 encoding in the subject line.

My code is a modification of the example "This example shows how to send text message without callback and debug."

Here's a snippet of the bounced email message when I click 'show source' (Note that I x'd out my cell #)

X-Priority: 5
X-MSMail-Priority: Low
Importance: Low
From: "=?utf-8?B?TWFpbGJveA==?=" housestatus@dschmidt.com
To: "=?utf-8?B?TWFpbGJveA==?=" xxxxxxxx@vtext.com
Subject: =?utf-8?B?TWFpbGJveA==?=
Date: Tue, 15 Oct 2024 11:43:58 -0759
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii";
Content-transfer-encoding: 7bit

Mailbox

What's weird is that the base64 in the To: decodes to 'Mailbox', same as the 'From:', yet the text after it reads different.

The problem is no matter what I set as the subject, I only ever see the base64 encoding, and not plain text.
Here's the relevant block of code

void processEmail(const char *destAddress, const char *textInfo)
{ Session_Config config;

config.server.host_name = SMTP_HOST;
config.server.port = SMTP_PORT;
config.login.email = AUTHOR_EMAIL;
config.login.password = AUTHOR_PASSWORD;

config.login.user_domain = F("127.0.0.1");

config.time.ntp_server = F("pool.ntp.org,time.nist.gov");
config.time.gmt_offset = -7;
config.time.day_light_offset = 1;

SMTP_Message message;

message.sender.name = F("Mailbox");
message.sender.email = AUTHOR_EMAIL;
message.subject = F("Mailbox");
message.addRecipient(F("Mailbox"), destAddress);

String textMsg = textInfo;
message.text.content = textMsg;
message.text.charSet = F("us-ascii");
message.text.transfer_encoding = Content_Transfer_Encoding::enc_7bit;
message.priority = esp_mail_smtp_priority::esp_mail_smtp_priority_low;

[Probedude]

Tried some stuff. Still no joy. Even though I have content transfer encoding set to 7bit, and character set to 'us-ascii', as you can see from the result message pasted above, and the most recent try below, the subject line is still coming up as base64 encoded.

Content-type: message/rfc822

Return-path: housestatus@dschmidt.com
Received: from mobile-166-170-44-139.mycingular.net ([166.170.44.139]:37521 helo=127.0.0.1)
by fawn.tchmachines.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
(Exim 4.97.1)
(envelope-from housestatus@dschmidt.com)
id 1t0qKX-0000000H1HQ-3EUd
for xxxxxxxxxx@vtext.com;
Tue, 15 Oct 2024 18:47:53 -0400
X-Priority: 5
X-MSMail-Priority: Low
Importance: Low
From: "=?utf-8?B?TWFpbGJveA==?=" housestatus@dschmidt.com
To: "=?utf-8?B?TWFpbGJveA==?=" xxxxxxxxxx@vtext.com
Subject: =?utf-8?B?TWFpbGJveCBEaXN0dXJiZWQ=?=
Date: Tue, 15 Oct 2024 15:48:53 -0659
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii";
Content-transfer-encoding: 7bit

Mailbox

--1729032475-eximdsn-552383372--

[mobizt]

In the name or text field e.g. message.sender.name, message.subject, message.author.name, before sending, that text should be encode in the correct form of RFC2047's encoded-words, e.g. ?US-ASCII?Q?Keith_Moore?=.

Library will find the string in that text field that begins with =? and ends with ?=". If both of them were not found, that text field will be encoded to the default encoding (base64) and charset (utf-8) string as you have seen.

[mobizt]

The content transfer encoding is for content (body) transfer encoding not a header text encoding.

For header charset and encoding, you have to encode it manually or leave the library for encoding.

For us-ascii character set and quoted printable for text encoding, your header fields can be the following.
message.sender.name = "?US-ASCII?Q?Mailbox?=";

This library does not provide any other encodings and charset as it stated in the library documentation.

And in the examples, you will see the following comment about charset and encoding for text fields.

ESP-Mail-Client/examples/SMTP/Send_Text/Send_Text.ino

Lines 232 to 235 in 566db6d

	// In case of sending non-ASCII characters in message envelope,
	// that non-ASCII words should be encoded with proper charsets and encodings
	// in form of `encoded-words` per RFC2047
	// https://datatracker.ietf.org/doc/html/rfc2047

[mobizt]

For spamming, you should read your mail server policy and comply with it.

[Probedude]

Made these changes to my code

message.sender.name = F("?US-ASCII?Q?Mailbox?=");
message.sender.email = AUTHOR_EMAIL;
message.subject = F("?US-ASCII?Q?Mailbox Disturbed?=");
message.addRecipient(F("?US-ASCII?Q?Mailbox?="), destAddress);

And still flagged as spam and not sent. Here is the bounceback. Note that now the message shows the ascii encoding, but also no text for subject after the =

--1729044849-eximdsn-475996278
Content-type: message/rfc822

Return-path: housestatus@dschmidt.com
Received: from syn-104-032-000-132.res.spectrum.com ([104.32.0.132]:62415 helo=127.0.0.1)
by fawn.tchmachines.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
(Exim 4.97.1)
(envelope-from housestatus@dschmidt.com)
id 1t0tY8-00000000EQi-0OMW
for xxxxxxxxxx@vtext.com;
Tue, 15 Oct 2024 22:14:07 -0400
X-Priority: 5
X-MSMail-Priority: Low
Importance: Low
From: "?US-ASCII?Q?Mailbox?=" housestatus@dschmidt.com
To: "?US-ASCII?Q?Mailbox?=" xxxxxxxxxx@vtext.com
Subject: ?US-ASCII?Q?Mailbox Disturbed?=
Date: Tue, 15 Oct 2024 18:14:07 -0800
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii";
Content-transfer-encoding: 7bit

Mailbox

--1729044849-eximdsn-475996278--

From this email acct, I manually sent an email to xxxxxxxxxx@vtext.com and it was sent okay. Looking in my sent folder at the message source, I see this:

MIME-Version: 1.0
Date: Tue, 15 Oct 2024 22:16:29 -0400
From: housestatus@dschmidt.com
To: xxxxxxxxxx@vtext.com
Subject: Mailbox
User-Agent: Roundcube Webmail/1.6.8
Message-ID: 1bece298d10b067c6f5a59022a8afe8b@dschmidt.com
X-Sender: housestatus@dschmidt.com
Content-Type: text/plain; charset=US-ASCII;
format=flowed
Content-Transfer-Encoding: 7bit

Mailbox disturbed

I don't see any special encoding in the to: from: subject: when manually sending an email.

[mobizt]

The charset and encoding are not related to the spam as your mail server should comply with RFC2047 encoded words otherwise do not use that server.

[Probedude]

I will write to tech support again then. This isn't a 2FA problem, and this is the company that hosts my domain so I won't be going elsewhere. This code has been working for almost a year and only stopped when they changed their spam filtering.

[mobizt]

The reply from mail server support does not make any sense.

[mobizt]

If they decode this =?utf-8?B?VGVzdCBzZW5kaW5nIEVtYWls?= to see real word is spamming or not. It is so easy.

[mobizt]

No MTAs (SMTP mail servers) have this problem.

[Probedude]

Here's the exact bounce back. Notice that the encoding I put it is showing up in the Subject, From and To. It's being passed in without being changed.

[mobizt]

If they don't get it send this to them.
https://datatracker.ietf.org/doc/html/rfc2047#section-8

[Probedude]

I'll see what they say. Email is working for all other purposes for 20 years now, only my ESP8266 devices are no longer working as of 2 weeks ago. (this mailbox notification and a power failure notification). I was using IFTTT until they did away with the free webhooks.

[mobizt]

The mail server may disable some capability that affected with the RFC2047 extension.

[mobizt]

The capability UTF8SMTP or SMTPUTF8 may be missing from mail server capability or they do not enable it.

Can you check the library debug message for server capabilities?

[Probedude]

I had a typo. I pasted in your reply verbatim and did not include the = in front of the first ?

After fixing that, I still get a spam return but now it looks like this:

So still triggering spam filters, but not showing the encoding as plain text.
I still don't get why I'm not seeing plain text of the subject text when I view the message source. Latest message source pasted below.

Return-Path: <>
Delivered-To: housestatus@dschmidt.com
Received: from fawn.tchmachines.com
by fawn.tchmachines.com with LMTP
id UDIJBvwrD2cUmQEAXKD5YQ
(envelope-from <>)
for housestatus@dschmidt.com; Tue, 15 Oct 2024 22:59:08 -0400
Return-path: <>
Envelope-to: housestatus@dschmidt.com
Delivery-date: Tue, 15 Oct 2024 22:59:08 -0400
Received: from mailnull by fawn.tchmachines.com with local (Exim 4.97.1)
id 1t0uFg-00000000TmH-0CPj
for housestatus@dschmidt.com;
Tue, 15 Oct 2024 22:59:08 -0400
X-Failed-Recipients: xxxxxxxxxx@vtext.com
Auto-Submitted: auto-replied
From: Mail Delivery System Mailer-Daemon@fawn.tchmachines.com
To: housestatus@dschmidt.com
Content-Type: multipart/report; report-type=delivery-status; boundary=1729047548-eximdsn-1919776723
MIME-Version: 1.0
Subject: Mail delivery failed: returning message to sender
Message-Id: E1t0uFg-00000000TmH-0CPj@fawn.tchmachines.com
Date: Tue, 15 Oct 2024 22:59:08 -0400

--1729047548-eximdsn-1919776723
Content-type: text/plain; charset=us-ascii

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

xxxxxxxxxx@vtext.com
host smtp.mailchannels.net [xx.xxx.xx.xxx]
SMTP error from remote mail server after end of data:
550 5.7.1 [CS] Message blocked. If this is a false positive, please report this to your hosting service provider. See https://console.mailchannels.net/insights/bounce?auid=totalchoicehosting&sender=xxxxxxxxxxxxxxxxxxxxxxxxx

--1729047548-eximdsn-1919776723
Content-type: message/delivery-status

Reporting-MTA: dns; fawn.tchmachines.com

Action: failed
Final-Recipient: rfc822;xxxxxxxxxx@vtext.com
Status: 5.0.0
Remote-MTA: dns; smtp.mailchannels.net
Diagnostic-Code: smtp; 550 5.7.1 [CS] Message blocked. If this is a false positive, please report this to your hosting service provider. See https://console.mailchannels.net/insights/bounce?auid=totalchoicehosting&sender=xxxxxxxxxxxxxxxxxxxxxx

--1729047548-eximdsn-1919776723
Content-type: message/rfc822

Return-path: housestatus@dschmidt.com
Received: from syn-xxx-xxx-xxx-xxx.res.spectrum.com ([xxx.x.x.xxx]:56200 helo=127.0.0.1)
by fawn.tchmachines.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
(Exim 4.97.1)
(envelope-from housestatus@dschmidt.com)
id 1t0uFe-00000000TlW-47Iz
for xxxxxxxxxx@vtext.com;
Tue, 15 Oct 2024 22:59:06 -0400
X-Priority: 5
X-MSMail-Priority: Low
Importance: Low
From: "=?US-ASCII?Q?Mailbox?=" housestatus@dschmidt.com
To: "=?US-ASCII?Q?Mailbox?=" xxxxxxxxx@vtext.com
Subject: =?US-ASCII?Q?Mailbox Disturbed?=
Date: Tue, 15 Oct 2024 18:59:06 -0800
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii";
Content-transfer-encoding: 7bit

Mailbox

--1729047548-eximdsn-1919776723--

[Probedude]

Still working on this. Latest reply from hosting services pasted below. Note that I intentionally removed 'message id' as they told me to the first time. I will re-add it back in and try and figure out the 'microsoft' issue they mention.

Hi,

This is the response we received when we analyzed the last header you provided.

From the details that you have provided, I could see that the email from your account " housestatus@dschmidt.com " is getting blocked with the error code"550 5.7.1 [CS] Message blocked" which triggers when the mail content contains any spam patterns.

While checking further, I could see that the emails getting blocked have a missing message-id issue. Please make sure that your mail client is adding a proper Message-Id header while sending emails. Please note that Message-IDs, are generated by the sending mail system. The Message ID is an essential part of the email header and is used by email servers to track the delivery status of an email. When an email is sent, the Message ID is included in the header, which is then verified by the recipient’s server. The Message ID helps the email provider to track the source of the email. Hence, please make sure that your mail client adds proper message IDs to the mail sent out from your account.

Also, In this case the message is pretending to be generated by a Microsoft email program which uses the extension header X-MSMail-Priority, but is missing the extension header X-MimeOLE which is characteristic of Microsoft email. That means the sender is using badly-written mailout software, rather than a genuine Microsoft email program.

Thank you,

Technical Support

[mobizt]

This is the smtp mail server (dschmidt.com) problem.

The message ID should be auto generated by smtp mail server and adding to the message envelope before sending out.

The other special headers may be required and it should done by smtp mail server.

You should send Microsoft response to your smtp mail server support.

[Probedude]

I know someone that runs his own mail server and he set up an acct for me. I had no problem sending my message through his server. Here is his reply when he inspected the information sent through his server.

My server hit on the date. Your message id might have also looked fishy. Both of those should be handled by the library, but I can see where they might be left to the user to do. I’ve seen libraries both ways.

The date should be specified as local time and include the offset. It is a specific format. Spam checkers frequently focus on the date (See RFC 2822 sec 3.4 to 3.4.) and sometimes on the message ID. Many spammers mess these up.

MessagIDs should be unique and not reused. If you are reusing one they may be caching them and hitting on them. Lots of spammers get caught this way because they hard code the ID. One easy way is to use the number of minutes since 1980 (or any arbitrary starting point or generated random number) on the left side, then a @ plus domain name.

I don’t save copies in eml format, and don’t save copies of outgoing mail at all for non-imap accounts. This is basically a relay.

I then sent my regular provider the information from above to them, and they took a look at a most recent bounce back before I switched servers. Here was their reply.

Hi,

The latest email bounce is a valid Subject but you need to check few more things on your mailing software, please read the update from Mailchannel support.

Hi,

The mails from your account " housestatus@dschmidt.com " is getting blocked with the error "CS". When the mail content shows any spam patterns, It will get blocked by our content filter with CS error.

Here, we can see that the emails are generated by older versions of Microsoft Outlook or certain email clients but do not include the X-Mailer or User-Agent header.

Hence kindly ensure that the email client or the sending system includes proper headers like X-Mailer or User-Agent to resolve this issue.

Thank You.
Regards,

For now I am just going to go through the mail server from my acquaintance, and give up going through my normal web hosting and email provider.