Handling signed messages (smime.p7m attachments)

[gasparez15]

Hi Benny,
I wanted to ask you if this library manages the signed Outlook message files.
Using the msgparser library I see that this message has an IPM.Note.SMIME class and that there is only one attachment called smime.p7m
This message has no body, but only an attachment
Is this library able to interpret the body and the attachments contained in the attachment smime?
Outlook Signed Message.zip
Thanks 4 Y time
Alex

[bbottema]

No, I'm not familiar with this and the underlying library used doesn't seem to do anything with it. I'll see what I can find out about it.

[bbottema]

I've had success in reading this smime attachment (using java-utils-mail-smime) and indeed found the content which is just an .eml, but I'm unsure how to handle this scenario properly.

If I open your supplied .eml in Thunderbird and that .msg in Outlook, both clients treat the attachment as the actual message. Is this behavior specified somewhere?

Currently I would be able to return the empty message with an .eml file attachment. This EML can easily be converted then to an Email object from Simple Java Mail, which provides a clean API for further content extraction.

OutlookFileAttachment eml = msg.fetchTrueAttachments().get(0);
MimeMessage mimeMessage = new MimeMessage(null, new ByteArrayInputStream(eml.getData()));

// Simple Java Mail Magic:
Email email = EmailConverter.mimeMessageToEmail(mimeMessage);

If this behavior would meet general expectation, I will go ahead with this solution.

[bbottema]

In fact, since this is not actually Outlook specific (signed attachments), I'm going to move S/MIME support over to Simple Java Mail. That's because to support your particular case (signed MimeMessage), outlook-message-parser would have to add javax.mail and bouncy castle as new dependencies, which are rather heavy weight.

Simple Java Mail already has javax.mail and works with optional dependencies, of which bouncy castle can be one. More to the point, I would like Simple Java Mail to allow new mails to be signed as well.

[gasparez15]

Ok Benny,
in fact I agree with your reasoning, since the SMIME attachment is actually the real message (only encrypted) it probably makes more sense to read this message in your main library.
I hope that inside that you will provide a convenient method to decipher the message and extract the attachments
Thanks for your time
Alex

[bbottema]

@gasparez15, can you tell me how you obtained the .eml file you posted in that zip? It contains more data than I get from the .msg parser (specifically ID, replyTo and bounceTo addresses as well as a bunch of Outlook original headers):

The most important one probably being the FROM name. Ignoring the minor fields, the following test shows the biggest problem:

Email emailParsedFromMsg = EmailConverter.outlookMsgToEmail("SMIME (signed and clear text).msg");
Email emailExpectedFromEml = EmailConverter.emlToEmailBuilder("SMIME (signed and clear text).eml")
		.clearId()
		.clearHeaders()
		.clearReplyTo()
		.clearBounceTo()
		.buildEmail();

EmailAssert.assertThat(emailParsedFromMsg).isEqualTo(emailExpectedFromEml);

[gasparez15]

Hi Benny, that message was sent by Outlook (MS Office 2016) and received with Thunderbird.
I saved it as an .eml file from Thunderbird

[gasparez15]

This applies to the .eml file.
About the .msg that has been saved by an application (outlook plugin) just before to be sent.
To tell the truth, analyzing the situation now, I realized that maybe the .msg file is not fully complete, in fact opening it with Outlook, I didn't see the signature icon.
Now I have downloaded it again from Outlook (sent messages) and the other copy of the file has the signature icon instead.
I send you the two copies
2copies-of-sent-message-outlook.zip

[bbottema]

Hi Benny, that message was sent by Outlook (MS Office 2016) and received with Thunderbird.
I saved it as an .eml file from Thunderbird

Aahhh, now I understand. Ok great, you helped me a lot with all this!

[bbottema]

But I still don't understand how this works.

First of all, you signed and encrypted a message, but not with a public key I gave you. So how is it that I can decrypt this message without any key? It's all supposed to work with keys and keystores and providers, but I haven't added any of that yet.

Second thing is, The Outlook message is parsed as a plain content message with subject, FROM and TO details and an S/MIME signed attachment. Is the resulting email supposed to be a combination/merge of the plain data and the decrypted message?

So:

• .msg
  • FROM name / address
  • TO names / addresses
  • SUBJECT
  • S/MIME signed attachment

Should result in the following?

• .msg
  • FROM name / address
  • TO names / addresses
  • SUBJECT
  • BODY taken from the decrypted attachment
  • attachments taken from the decrypted attachment

It's still confusing me a lot.

[gasparez15]

Hi Benny, the message is signed but not encrypted

[bbottema]

Hmm, the library I used reports it as encrypted, weird.

/edit: It's a bug in two libraries:

• This library, for not parsing the smime header properly (losing the smime-type parameters of the smime mime header
• java-utils-mail-smime, for not looking for the smime-type parameter (which according to S/MIME 3.2 should be used).

I fixed the bug in this library and raised a bug in the other.

[bbottema]

Hi @gasparez15, I finished implementation for reading signed .msg / .eml files in Simple Java Mail. Will be in the 6.0.0 release!

Note this doesn't support encrypted emails yet. Still working on that.

[bbottema]

I'm working on adding decryption now as well (only supports signed content currently). Are you able to provide me with additional examples?

I will need the following test set:

1. signed (already have that one)
2. encrypted
3. signed, then encrypted
4. encrypted, then signed
5. maybe: some combination of multiple encryption/sign passes (should be rare in real world usage, but in theory this is possible and should be handled correctly as well)

For testing decryption, you would need to a generate private/public key pair, encrypt with the public key key and I would then need the private key to decrypt it.

[bbottema]

I wasted many hours fooling around with self signed pkcs12 certificates in Outlook and Thunderbird, but it seems I have no clue what I'm doing :/

Seems like I was going about it the wrong way. When I get some more time, I'll try the following guide: https://www.dalesandro.net/create-self-signed-smime-certificates/

Alternatively, I could try the shorter https://gist.github.com/richieforeman/3166387

/edit nope, didn't work in Outlook / Thunderbird...

[bbottema]

Got everything figured in the meantime (test data here), it was quite complicated due to the way the S/MIME spec tries to maintain backwards compatibility with older emails and Markenwerk's smime library having bugs...

But, I've finished implemented both encrypted and / or signed messages. Implemented in Simple Java Mail, will be in the 6.0.0 release.

[bbottema]

And now added signing / encrypting as well! Last step is to make sure it works from the command line as well and final documentation tweaks.