SSLError(58, '[ASN1] nested asn1 error (_ssl.c:4174)') and SSLError(0, 'not enough data: cadata does not contain a certificate (_ssl.c:4159)')

[levicki]

If a CA encodes certificate serial number integer value 102 (0x66) as 02 04 00 00 00 66 instead of 02 01 66, which violates the DER standard the following error shows up when trying to load such a certificate:

SSLError(58, '[ASN1] nested asn1 error (_ssl.c:4174)') and SSLError(0, 'not enough data: cadata does not contain a certificate (_ssl.c:4159)')

There are some root certificates in circulation with such non-compliant DER encoding which apparently can't be replaced, and they are causing a host of issues in various Python based applications since Python uses OpenSSL.

I am reporting this as a bug even though I don't consider strict adherence to DER standard to be a bug, but something should be done to allow use of such certificates or there should be an option to relax validation so that this results in a warning instead of error.

Other implementations (Firefox, Chromium) accept those certificates.

Relevant Python issue can be found here.

Current workaround is to:

• Process certificates from Windows certificate store one by one and
• Skip certificates which raise an exception related to this error

@mattcaswell @t8m Please advise.

[mattcaswell]

Do we have some examples of such non-compliant CA certs?

[levicki]

@mattcaswell Original DER file can be found here:

http://crl.mup.gov.rs/MUPCARoot.crt

I converted it to .pem and attached as .txt here because OpenSSL refuses to convert it with the following error:

openssl x509 -inform DER -in MUPCARoot.cer -out MUPCARoot.pem
Could not find certificate from MUPCARoot.cer
10050000:error:1608010C:STORE routines:ossl_store_handle_load_result:unsupported:..\OpenSSL-source\crypto\store\store_result.c:151:

When we are at it, that's a rather cryptic and unhelpful error message.

Here's the problematic part of the DER encoding:

So apparently this issue affects command line tool as well.

Version used:

OpenSSL 3.3.0 9 Apr 2024 (Library: OpenSSL 3.3.0 9 Apr 2024)

Let me know if there's anything more I can help with.

mupcaroot.txt

[mattcaswell]

I converted it to .pem and attached as .txt here

Thanks. That was helpful.

I took a quick look to understand what would need to change in order to be able to parse these certificates. Here's a quick-and-dirty hack I did to make the certificate readable:

diff --git a/crypto/asn1/a_int.c b/crypto/asn1/a_int.c
index dc962290dd..0779fe8d9f 100644
--- a/crypto/asn1/a_int.c
+++ b/crypto/asn1/a_int.c
@@ -150,7 +150,7 @@ static size_t i2c_ibuf(const unsigned char *b, size_t blen, int neg,
  * of buffer or 0 on error: for malformed INTEGER. If output buffer is
  * NULL just return length.
  */
-
+#define HACK_ACCEPT_ILLEGAL_PADDING
 static size_t c2i_ibuf(unsigned char *b, int *pneg,
                        const unsigned char *p, size_t plen)
 {
@@ -176,7 +176,13 @@ static size_t c2i_ibuf(unsigned char *b, int *pneg,
 
     pad = 0;
     if (p[0] == 0) {
+#ifdef HACK_ACCEPT_ILLEGAL_PADDING
+        for (pad = 1; (size_t)pad < plen && p[pad] == 0; pad++);
+        if ((size_t)pad == plen)
+            pad--;
+#else
         pad = 1;
+#endif
     } else if (p[0] == 0xFF) {
         size_t i;
 
@@ -188,11 +194,13 @@ static size_t c2i_ibuf(unsigned char *b, int *pneg,
             pad |= p[i];
         pad = pad != 0 ? 1 : 0;
     }
+#ifndef HACK_ACCEPT_ILLEGAL_PADDING
     /* reject illegal padding: first two octets MSB can't match */
     if (pad && (neg == (p[1] & 0x80))) {
         ERR_raise(ERR_LIB_ASN1, ASN1_R_ILLEGAL_PADDING);
         return 0;
     }
+#endif
 
     /* skip over pad */
     p += pad;

Note that adding this hack causes various tests related to asn1 integer encoding/decoding to start failing (which is perhaps not surprising).

I agree, that I don't think we can characterise this as a bug. The code is operating as designed and in accordance with the standards. It might be reasonable to have some "relaxed" mode for DER parsing. OTOH this isn't without security consequences. Part of the point of DER is that there is only one possible encoding of the same thing. By being relaxed about such things we are allowing multiple encodings of the same certificate - although its difficult to see how this could be exploited in any meaningful way in this case.

This change is deep in the ASN1 parsing code. To make this kind of change acceptable I think we would need some mechanism for turning this on or off (default should be off, as now). That then throws up various questions as to how that would be achieved. It could be a compile time option - but then that means it would always be on if compiled that way. Which seems unwise. If we had some runtime option how would that work? Would it be global (probably the easiest to do - but again is it wise), or somehow local to a particular parsing attempt. If local how would you pass such an option down through the call stack without significant impact on the API?

This seems a somewhat niche feature so I think its unlikely that the dev team will prioritise this anytime soon. If someone from the community wanted to take this on then I think we would consider it - but I think getting the design right would be critical. Probably some outline design of how to turn it on/off would be required first.

Adding the "feature" and "help wanted" labels.

[levicki]

@mattcaswell I agree with the most of your assessment of the issue. Detailed reply below:

OTOH this isn't without security consequences.

If the user already has such a certificate in their trust store (which implies trust), then allowing them to override parsing strictness is a reasonable request even if it might not be wise thing to do.

I think we would need some mechanism for turning this on or off (default should be off, as now).

I agree it should be off by default, but I also think it should stay off by default forever.

My thoughts on the available options:

On one hand, this is something that a user having such a problem with their certificates will want to turn on for all instances of OpenSSL on their system, and on the other hand developers writing code in Python who are facing this issue could as well bundle custom OpenSSL build with compile-time option turned on. Finally, there's also an option of OpenSSL considering whether this DER enforcement in this particular case (serial number) makes sense, and either changing the default code to ignore the error like browsers and Microsoft certificate tools do, or adding a new API to load such certs with relaxed checking.

I am far from the security expert to tell which one of these 3 options is lower risk.

I'd prefer if @t8m also chimed in with his thoughts on the matter.

[mattcaswell]

@vdukhovni may also have a view.

[mattcaswell]

Another point to note about my quick-and-dirty hack, is that it modifies the way DER integer parsing is processed for all DER integers regardless of the context. To fix this problem we only really need to be relaxed for this one serial number integer. Perhaps we could declare that one integer as some special type of "relaxed" integer instead of a normal one which would reduce the scope of any change we are making here and perhaps make it much easier to justify having it on.