Login on windows is not working: non-zero custom status code considered fatal

[dani-weiss]

What happened?

I'm developing a flutter app that uses realm with device sync and atlas user management using anonymous login or custom function credentials.
It is working perfectly on iOS, Android, MacOS (with exactly the same code) but it is failing for Windows. This is the error I get upon calling the login function (with either credentials):

AppException (AppException: non-zero custom status code considered fatal, link to server logs: null)

There is nothing in the logs of the Atlas app. Could you help me narrow down what the issue might be? Since I don't know what the underlying error is, it is almost impossible for me to start debugging.

Repro steps

Call App.logIn with anonymous credentials or custom function credentials on Windows 10.

Version

1.3.0

What Atlas Services are you using?

Both Atlas Device Sync and Atlas App Services

What type of application is this?

Flutter Application

Client OS and version

Windows 10

Code snippets

Either:

final app = App(AppConfiguration("<app>"));
final credentials = Credentials.anonymous();
final user = await app.logIn(credentials);

or:

final app = App(AppConfiguration("<app>"));
final credentials = Credentials.function(jsonEncode({
  "email": email,
  "password": password
}));
final user = await app.logIn(credentials);

Stacktrace of the exception/crash you're getting

Empty stack trace: _StringStackTrace

Relevant log output

flutter: 2023-08-05T17:55:13.721611: [ERROR] Realm: App: log_in_with_credentials failed: 0 message: non-zero custom status code considered fatal


AppException (AppException: non-zero custom status code considered fatal, link to server logs: null) with status code 999

[nielsenko]

Can I ask you to put a breakpoint on

        responseRef.custom_status_code = _CustomErrorCode.unknown.code;

in realm_core.dart (If you are using v1.3.0 it is line 1730).and report back what the exact exception it is that causes the unknown error code to be set.

[dani-weiss]

Thank you for your suggestion. I did what you said and this is the underlying error:

HandshakeException (HandshakeException: Handshake error in client (OS Error: 
CERTIFICATE_VERIFY_FAILED: unable to get local issuer certificate(../../third_party/boringssl/src/ssl/[handshake.cc:393](http://handshake.cc:393/))))

This seems odd to me as it actually works on all other platforms. I even just set-up a new Windows 11 system on another laptop and it yields the same error. Would you have a suggestion what I should try next?

Edit:
Maybe this is related?
flutter/flutter#50699

[dani-weiss]

Update:
I added a global interceptor for bad certificates (badCertificateCallback) with the flutter http client using HttpOverrides. The failing host is:

eu-central-1.aws.realm.mongodb.com

I then went on to verify the ssl certificate used by the host with this result (using https://www.geocerts.com/ssl-checker):

There seems to be a configuration error concerning the certificate chain. See also this thread talking about the same problem but not related to flutter:
https://community.letsencrypt.org/t/handshake-verification-error/61614

[nielsenko]

On macos 13.5 it resolves fine:

❯ gnutls-cli eu-central-1.aws.realm.mongodb.com
Processed 138 CA certificate(s).
Resolving 'eu-central-1.aws.realm.mongodb.com:443'...
Connecting to '3.124.220.115:443'...
- Certificate type: X.509
- Got a certificate list of 3 certificates.
- Certificate[0] info:
 - subject `CN=*.aws.realm.mongodb.com', issuer `CN=R3,O=Let's Encrypt,C=US', serial 0x0415a544e6686b98a3a4f1068226dbb907fe, RSA key 2048 bits, signed using RSA-SHA256, activated `2023-07-26 11:31:23 UTC', expires `2023-10-24 11:31:22 UTC', pin-sha256="CVaXq2/zpuX8BaHTwVmlMV728SayahkassVuF1dxIYQ="
        Public Key ID:
                sha1:16a8f104ea29796f4eda59186070c3f0c0cfc254
                sha256:095697ab6ff3a6e5fc05a1d3c159a5315ef6f126b26a191ab2c56e1757712184
        Public Key PIN:
                pin-sha256:CVaXq2/zpuX8BaHTwVmlMV728SayahkassVuF1dxIYQ=

- Certificate[1] info:
 - subject `CN=R3,O=Let's Encrypt,C=US', issuer `CN=ISRG Root X1,O=Internet Security Research Group,C=US', serial 0x00912b084acf0c18a753f6d62e25a75f5a, RSA key 2048 bits, signed using RSA-SHA256, activated `2020-09-04 00:00:00 UTC', expires `2025-09-15 16:00:00 UTC', pin-sha256="jQJTbIh0grw0/1TkHSumWb+Fs0Ggogr621gT3PvPKG0="
- Certificate[2] info:
 - subject `CN=ISRG Root X1,O=Internet Security Research Group,C=US', issuer `CN=DST Root CA X3,O=Digital Signature Trust Co.', serial 0x4001772137d4e942b8ee76aa3c640ab7, RSA key 4096 bits, signed using RSA-SHA256, activated `2021-01-20 19:14:03 UTC', expires `2024-09-30 18:14:03 UTC', pin-sha256="C5+lpZ7tcVwmwQIMcRtPbsQtWLABXhQzejna0wHFr8M="
- Status: The certificate is trusted.
- Description: (TLS1.3-X.509)-(ECDHE-X25519)-(RSA-PSS-RSAE-SHA256)-(AES-256-GCM)
- Session ID: FC:98:D3:39:7D:81:9A:E9:C8:78:FB:87:21:92:39:C5:5B:0F:B1:35:30:4A:96:2B:91:67:98:56:6B:1E:70:9E
- Options:
- Handshake was completed

Will investigate

[nielsenko]

Could you try loading https://letsencrypt.org/ in a browser on the windows box in question? It is signed with the same root certificate ISRG Root X1.

Windows does some shenanigans downloading and installing certificates lazily, but I speculate that boringssl does not.

[nielsenko]

#1087 relates to a somewhat similar issue with Android 7 and earlier, and how you can work around it.

The PR #1088 never landed, but it illustrates a work-around.

[dani-weiss]

It's working now, thanks.

I opened https://letsencrypt.org in the browser on the windows machine as you suggested.

Then, I tried this code on the windows machine:

try {
  await http.get(Uri.https("letsencrypt.org"));
  await http.get(Uri.https("eu-central-1.aws.realm.mongodb.com")
}
catch(e) {
...
}

They both worked without any error. Then I tried the realm logIn again and it worked, too.

However, I tried the same on another freshly setup windows 10 machine (this is a vm, though) and it didn't work. I'm still getting the handshake error.

I then added the X1 and X2 root certificates of letsencrypt to a httpClient which I passed into the realm app configuration. It then worked like a charm. This is not an ideal solution as those certificates will expire (X1 in 2035) but it's the best solution I could find for the time being. Do you have a better idea?

The ISRG X1 is on the windows 11 machine now:

but it's not on the windows 10 machine (I did the exact same steps above):

So, to sum up, I guess it still won't work on some windows environments. A workaround is to include the certificate but it's not an ideal solution imho.

[nielsenko]

I think the most failsafe approach is to install the ISRG Root X1 root certificate on the SecurityContext of the HttpClient and try not to forget to update it before 2035. A lot can happen with the certificate chains before that anyway.

Other than that I guess you need to look into why and when Windows will automatically download a missing root certificate.

[nielsenko]

We will likely add it automatically on all platforms in the next version. Stay tuned.

[dani-weiss]

I think the most failsafe approach is to install the ISRG Root X1 root certificate on the SecurityContext of the HttpClient and try not to forget to update it before 2035. A lot can happen with the certificate chains before that anyway.

This is exactly what I did now. Yes, I set a reminder to repeatedly check the situation about the certificates.
Thanks for your help.

[dotjon0]

Just checking to see how #1378 is progressing?

[nirinchev]

@fealebenpae that's more in your alley - can you take a look and see if the approach in #1378 is what we want to do?

[milindgoel15]

I encountered this error after i did a password reset using bcrypt coded in nextjs and tried logging on windows on flutter side. But logging with plain passwords work.