Crash in encrypted realm

[allanweir]

How frequently does the bug occur?

All the time

Description

Hi, after upgrading Realm I noticed a crash happening once the app had started up which then seems to corrupt the realm object. I've narrowed it down to upgrading from realm swift 10.21.0 to 10.21.1.

The app will launch OK and seem to run until some kind of interaction happens with the realm. When the app is restarted the realm can no longer be opened and needs to be recreated. I've checked the password remains consistent and none of the code to do the setup has changed for a long time.

Stacktrace & log output

0   App                           0x000000010acc22dc _ZN5realm4utilL18terminate_internalERNSt3__118basic_stringstreamIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE + 28
1   App                           0x000000010acc259f _ZN5realm4util9terminateEPKcS2_lOSt16initializer_listINS0_9PrintableEE + 287
2   App                           0x000000010ac374ab _ZN5realm4util20EncryptedFileMapping13validate_pageEm + 1275
3   App                           0x000000010ac37524 _ZN5realm4util20EncryptedFileMapping8validateEv + 68
4   App                           0x000000010ac36043 _ZN5realm4util20EncryptedFileMapping5flushEv + 307
5   App                           0x000000010ac68826 _ZN5realm4util4File7MapBase5flushEv + 118
6   App                           0x000000010a46ba45 _ZN5realm4util4File3MapIcE5flushEv + 21
7   App                           0x000000010a46ba25 _ZN5realm11GroupWriter9MapWindow5flushEv + 21
8   App                           0x000000010a46c907 _ZN5realm11GroupWriter18flush_all_mappingsEv + 135
9   App                           0x000000010a470f4e _ZN5realm11GroupWriter6commitEm + 382
10  App                           0x000000010a42e2af _ZZN5realm11Transaction9async_endENS_4util14UniqueFunctionIFvvEEEENK3$_3clEv + 159
11  App                           0x000000010a42e205 _ZN5realm4util14UniqueFunctionIFvvEE17call_regular_voidIZNS_11Transaction9async_endES3_E3$_3EEvNSt3__117integral_constantIbLb1EEERT_ + 21
12  App                           0x000000010a42e169 _ZZN5realm4util14UniqueFunctionIFvvEE9make_implIZNS_11Transaction9async_endES3_E3$_3EEDaOT_EN12SpecificImpl4callEv + 25
13  App                           0x000000010a40db0a _ZNK5realm4util14UniqueFunctionIFvvEEclEv + 106
14  App                           0x000000010a40d6ed _ZN5realm2DB17AsyncCommitHelper4mainEv + 205
15  App                           0x000000010a41c6e8 _ZZN5realm2DB17AsyncCommitHelper12start_threadEvENKUlvE_clEv + 24
16  App                           0x000000010a41c68d _ZNSt3__1L8__invokeIZN5realm2DB17AsyncCommitHelper12start_threadEvEUlvE_JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS5_DpOS6_ + 29
17  App                           0x000000010a41c625 _ZNSt3__1L16__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEZN5realm2DB17AsyncCommitHelper12start_threadEvEUlvE_JEJEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE + 37
18  App                           0x000000010a41bea5 _ZNSt3__1L14__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEZN5realm2DB17AsyncCommitHelper12start_threadEvEUlvE_EEEEEPvSC_ + 101
19  libsystem_pthread.dylib             0x00007fff6da2b8fc _pthread_start + 224
20  libsystem_pthread.dylib             0x00007fff6da27443 thread_start + 15!!! IMPORTANT: Please report this at https://github.com/realm/realm-core/issues/new/choose

Can you reproduce the bug?

Yes, always

Reproduction Steps

The encryption of the Realm is handled as described here https://docs.mongodb.com/realm/sdk/swift/advanced-guides/encrypt-a-realm/

The configuration is

Realm.Configuration(
fileURL: Utils.documentsURLFor(fileName: "cache.realm"),
encryptionKey: {KEY},
schemaVersion: 113,
migrationBlock: {....}
)

And when accessing it the Realm initially with try Realm() there is no error thrown. The database has been integrated for a few years now and this basic flow hasn't changed in that time

Version

10.21.1

What SDK flavour are you using?

Local Database only

Are you using encryption?

Yes, using encryption

Platform OS and version(s)

iOS 15.2

Build environment

Xcode version: 13.2 & 13.2.1
Dependency manager and version: Swift Package Manager

[c128128]

same

[kordem]

same issue

[leemaguire]

Hi @allanweir

Sorry to hear that you are encountering this. To help narrow this down further could you tell us if you using Realm Sync? And does your model use EmbeddedObject's?

[bmt2018]

Same problem for me, as well. We are not using Realm Sync and our models do not use EmbeddedObject(s).

[Sethmr]

Same problem for me.

One other issue we are having is a similar and happening even on 10.21.0 where TouchID devices crash on launch.

[matthewjordanRiM]

This is happening to me as well.

1. I am using Real Encrypted, only local realm, not Realm Sync
2. I am using EmbeddedObjects(s).

Project Setup:
iOS Native Project
Realm and Realm Swift installed through a Pod v10.22.0

[matthewjordanRiM]

About the above, this is the error and the stack trace from firebase.

The issue is rising up quickly on devices

[matthewjordanRiM]

I was wondering if there is any follow-up on this one. This issue has been increasing in the last 8 days and I am afraid we cannot just use Realm Decrypted.

[bmt2018]

We're also still suffering crashes with encrypted Realms. We upgraded Realm to 10.21.1 and started experiencing these issues. Is there anyone from the Realm team actively investigating this issue?

[dianaafanador3]

Hi @bmt2018 we do have some changes in core in our next release which may solve this issue. We'll let you know any updates.

[bmt2018]

@dianaafanador3 Thanks, Diana. That's great to hear and I look forward to testing the next release!

[dianaafanador3]

@bmt2018 I'll try to investigate a little bit further into this next week when I get to do support, as it seems this affecting a lot of people.

[bmt2018]

@dianaafanador3 Thank you! That would be much appreciated! :-)

[cupcake-shukla]

Hi @dianaafanador3 is there any update on this issue?

[bmt2018]

I saw that they released 10.24.2 over the weekend with the following in the release notes: "Other issues indicating file corruption may also be fixed by this". Not sure if this was fix that @dianaafanador3 was referring to last week or not, though.

[dianaafanador3]

Hi @cupcake-shukla yep this fix will solve some issues that lead to a file corruption which may be related to this or not. I'm trying to get a reproduction for this issue, so we can be sure that this fix the problem.

[cupcake-shukla]

Hi, @dianaafanador3 any workaround for this issue? let's say I downgrade the realm SDK version and release a new version for my application, and if someone still faces this issue I can ask them to reinstall my application will this help?

My month crashlytics looked like 200-300 crashes after this update it has been 18k for this month only

[allanweir]

I have just tested upgrading from 10.21.0 to 10.25.0 and everything seems to be working as expected, no errors on the first or subsequent runs of the build.

Thanks for looking into this and getting it sorted!

I'll leave closing the issue for when you're happy it's resolved everyone.

EDIT: Actually not resolved.. more info in next comment

[allanweir]

Actually there is still an issue:

On one device, about 7 builds/runs in I got an error similar to my original post. There were 2 errors in sequence it looks like:
/Users/user/Library/Developer/Xcode/DerivedData/MyApp-gbughxsxqnhbrdfiuvzigofzgcyo/SourcePackages/checkouts/realm-core/src/realm/util/encrypted_file_mapping.cpp:622: [realm-core-11.13.0] 0 MyApp 0x000000010439b94c _ZN5realm4utilL18terminate_internalERNSt3__118basic_stringstreamIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE + 28 1 MyApp 0x000000010439bb9c _ZN5realm4util9terminateEPKcS2_lOSt16initializer_listINS0_9PrintableEE + 228 2 MyApp 0x0000000104318cd0 _ZN5realm4util20EncryptedFileMapping13validate_pageEm + 1048 3 MyApp 0x0000000104318d30 _ZN5realm4util20EncryptedFileMapping8validateEv + 76 4 MyApp 0x00000001043179a8 _ZN5realm4util20EncryptedFileMapping5flushEv + 300 5 MyApp 0x00000001043439e8 _ZN5realm4util4File7MapBase5flushEv + 108 6 MyApp 0x0000000103c78eec _ZN5realm4util4File3MapIcE5flushEv + 24 7 MyApp 0x0000000103c78ec8 _ZN5realm11GroupWriter9MapWindow5flushEv + 24 8 MyApp 0x0000000103c79be0 _ZN5realm11GroupWriter18flush_all_mappingsEv + 128 9 MyApp 0x0000000103c7d970 _ZN5realm11GroupWriter6commitEm + 404 10 MyApp 0x0000000103c13be0 _ZN5realm2DB16low_level_commitEyRNS_11TransactionEb + 716 11 MyApp 0x0000000103c13890 _ZN5realm2DB9do_commitERNS_11TransactionEb + 176 12 MyApp 0x0000000103c13f10 _ZN5realm11Transaction27commit_and_continue_as_readEb + 260 13 MyApp 0x0000000103daed64 _ZN5realm5_impl16RealmCoordinator12commit_writeERNS_5RealmEb + 216 14 MyApp 0x0000000103eefe40 _ZN5realm5Realm18commit_transactionEv + 276 15 MyApp 0x0000000103adbe8c -[RLMRealm commitWriteTransactionWithoutNotifying:error:] + 552 16 MyApp 0x00000001044eef60 $s10RealmSwift0A0V11commitWrite16withoutNotifyingySaySo20RLMNotificationTokenCG_tKF + 148 17 MyApp 0x00000001044eeab4 $s10RealmSwift0A0V5write16withoutNotifying_xSaySo20RLMNotificationTokenCG_xyKXEtKlF + 420 18 MyApp 0x00000001037b2b68 $s7AppCore12RealmManagerC5writeyyy0B5Swift0B0VXEF + 376 19 MyApp 0x00000001036a28e0 $s7AppCore14ContentDataSourceC7cleanupyyF + 88 20 MyApp 0x00000001037b39c0 $s7AppCore12RealmManagerC7cleanup10completionyyAC16StepResponseTypeOc_tFyycfU_ + 256 21 MyApp 0x000000010348d918 $sIeg_IeyB_TR + 52 22 libdispatch.dylib 0x000000010aa406d4 _dispatch_call_block_and_release + 32 23 libdispatch.dylib 0x000000010aa423b4 _dispatch_client_callout + 20 24 libdispatch.dylib 0x000000010aa44f2c _dispatch_queue_override_invoke + 1052 25 libdispatch.dylib 0x000000010aa56500 _dispatch_root_queue_drain + 408 26 libdispatch.dylib 0x000000010aa56f0c _dispatch_worker_thread2 + 196 27 libsystem_pthread.dylib 0x00000001f1bbf0b8 _pthread_wqthread + 228 28 libsystem_pthread.dylib 0x00000001f1bbee94 start_wqthread + 8!!! IMPORTANT: Please report this at https://github.com/realm/realm-core/issues/new/choose2022-04-04 11:42:29.603246+0100 MyApp[6637:2564589] /Users/user/Library/Developer/Xcode/DerivedData/MyApp-gbughxsxqnhbrdfiuvzigofzgcyo/SourcePackages/checkouts/realm-core/src/realm/util/encrypted_file_mapping.cpp:622: [realm-core-11.13.0]
and
libc++abi: terminating with uncaught exception of type realm::util::DecryptionFailed: Decryption failed Exception backtrace: 0 MyApp 0x0000000103b602c0 _ZN5realm4util6detail26ExceptionWithBacktraceBaseC2Ev + 48 1 MyApp 0x0000000103b6d428 _ZN5realm4util22ExceptionWithBacktraceISt13runtime_errorEC2IJRKNSt3__112basic_stringIcNS5_11char_traitsIcEENS5_9allocatorIcEEEEEEEDpOT_ + 88 2 MyApp 0x0000000103b6d208 _ZN5realm4util4File11AccessErrorC2ERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEESB_ + 68 3 MyApp 0x0000000104319ed4 _ZN5realm4util16DecryptionFailedC2Ev + 100 4 MyApp 0x0000000104316a30 _ZN5realm4util16DecryptionFailedC1Ev + 28 5 MyApp 0x0000000104316814 _ZN5realm4util10AESCryptor4readEixPcm + 504 6 MyApp 0x0000000104318460 _ZN5realm4util20EncryptedFileMapping12refresh_pageEm + 228 7 MyApp 0x0000000104319970 _ZN5realm4util20EncryptedFileMapping12read_barrierEPKvmPFmPKcE + 156 8 MyApp 0x0000000103b606dc _ZN5realm4util26do_encryption_read_barrierEPKvmPFmPKcEPNS0_20EncryptedFileMappingE + 68 9 MyApp 0x0000000103b5f9bc _ZN5realm4util23encryption_read_barrierEPKvmPNS0_20EncryptedFileMappingEPFmPKcE + 60 10 MyApp 0x0000000103b6cba8 _ZNK5realm9Allocator18translate_criticalEPNS0_14RefTranslationEm + 220 11 MyApp 0x0000000103b63014 _ZNK5realm9Allocator9translateEm + 72 12 MyApp 0x0000000103d2f200 _ZN5realm13ArrayBigBlobs3getEPKcmRNS_9AllocatorE + 80 13 MyApp 0x0000000103d2f0e8 _ZN5realm13ArrayBigBlobs10get_stringEPKcmRNS_9AllocatorEb + 48 14 MyApp 0x0000000103d221e0 _ZN5realm11ArrayString3getEPKcmRNS_9AllocatorE + 164 15 MyApp 0x0000000103d1ed58 _ZNK5realm3Obj4_getINS_10StringDataEEET_NS_6ColKey3IdxE + 384 16 MyApp 0x0000000103d1ebbc _ZNK5realm3Obj3getINS_10StringDataEEET_NS_6ColKeyE + 172 17 MyApp 0x00000001038962c4 _ZN12_GLOBAL__N_18getBoxedIN5realm10StringDataEEEP11objc_objectP13RLMObjectBasem + 124 18 MyApp 0x000000010389623c RLMGetSwiftPropertyString + 36 19 MyApp 0x0000000104471fc8 $sSS10RealmSwiftE23_rlmGetPropertyOptionalySSSgSo13RLMObjectBaseC_s6UInt16VtFZ + 40 20 MyApp 0x00000001044720e8 $sSS10RealmSwift26_PersistableInsideOptionalA2aBP015_rlmGetPropertyE0yxSgSo13RLMObjectBaseC_s6UInt16VtFZTW + 20 21 MyApp 0x0000000104477d98 $sSq10RealmSwiftAA26_PersistableInsideOptionalRzlE15_rlmGetPropertyyxSgSo13RLMObjectBaseC_s6UInt16VtFZ + 56 22 MyApp 0x0000000104478078 $sxSg10RealmSwift12_PersistableA2B01_C14InsideOptionalRzlAbCP15_rlmGetPropertyyxSo13RLMObjectBaseC_s6UInt16VtFZTW + 20 23 MyApp 0x00000001044d2158 $s10RealmSwift9PersistedV3getyxSo13RLMObjectBaseCF + 3436 24 MyApp 0x00000001044d13b0 $s10RealmSwift9PersistedV18_enclosingInstance7wrapped7storagexqd___s24ReferenceWritableKeyPathCyqd__xGAHyqd__ACyxGGtcSo13RLMObjectBaseCRbd__luigZ + 204 25 MyApp 0x000000010375a1ac $s7AppCore7ItemStepC14imageURLStringSSSgvg + 100 26 MyApp 0x00000001037b5f58 $s7AppCore15UserFileManagerC11fileCleanupyyF + 1288 27 MyApp 0x00000001037b3bc0 $s7AppCore12RealmManagerC7cleanup10completionyyAC16StepResponseTypeOc_tFyycfU_ + 768 28 MyApp 0x000000010348d918 $sIeg_IeyB_TR + 52 29 libdispatch.dylib 0x000000010aa406d4 _dispatch_call_block_and_release + 32 30 libdispatch.dylib 0x000000010aa423b4 _dispatch_client_callout + 20 31 libdispatch.dylib 0x000000010aa44f2c _dispatch_queue_override_invoke + 1052 32 libdispatch.dylib 0x000000010aa56500 _dispatch_root_queue_drain + 408 33 libdispatch.dylib 0x000000010aa56f0c _dispatch_worker_thread2 + 196 34 libsystem_pthread.dylib 0x00000001f1bbf0b8 _pthread_wqthread + 228 35 libsystem_pthread.dylib 0x00000001f1bbee94 start_wqthread + 8

When running another device which is used more frequently, so will have more data and more of a history I get:

libc++abi: terminating with uncaught exception of type realm::util::DecryptionFailed: Decryption failed Exception backtrace: 0 MyApp 0x0000000101edc280 _ZN5realm4util6detail26ExceptionWithBacktraceBaseC2Ev + 48 1 MyApp 0x0000000101ee93e8 _ZN5realm4util22ExceptionWithBacktraceISt13runtime_errorEC2IJRKNSt3__112basic_stringIcNS5_11char_traitsIcEENS5_9allocatorIcEEEEEEEDpOT_ + 88 2 MyApp 0x0000000101ee91c8 _ZN5realm4util4File11AccessErrorC2ERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEESB_ + 68 3 MyApp 0x0000000102695e94 _ZN5realm4util16DecryptionFailedC2Ev + 100 4 MyApp 0x00000001026929f0 _ZN5realm4util16DecryptionFailedC1Ev + 28 5 MyApp 0x00000001026927d4 _ZN5realm4util10AESCryptor4readEixPcm + 504 6 MyApp 0x0000000102694984 _ZN5realm4util20EncryptedFileMapping13validate_pageEm + 268 7 MyApp 0x0000000102694cf0 _ZN5realm4util20EncryptedFileMapping8validateEv + 76 8 MyApp 0x0000000102693968 _ZN5realm4util20EncryptedFileMapping5flushEv + 300 9 MyApp 0x00000001026bf9a8 _ZN5realm4util4File7MapBase5flushEv + 108 10 MyApp 0x0000000101ff4eac _ZN5realm4util4File3MapIcE5flushEv + 24 11 MyApp 0x0000000101ff4e88 _ZN5realm11GroupWriter9MapWindow5flushEv + 24 12 MyApp 0x0000000101ff5ba0 _ZN5realm11GroupWriter18flush_all_mappingsEv + 128 13 MyApp 0x0000000101ff9930 _ZN5realm11GroupWriter6commitEm + 404 14 MyApp 0x0000000101f8fba0 _ZN5realm2DB16low_level_commitEyRNS_11TransactionEb + 716 15 MyApp 0x0000000101f8f850 _ZN5realm2DB9do_commitERNS_11TransactionEb + 176 16 MyApp 0x0000000101f8fed0 _ZN5realm11Transaction27commit_and_continue_as_readEb + 260 17 MyApp 0x000000010212ad24 _ZN5realm5_impl16RealmCoordinator12commit_writeERNS_5RealmEb + 216 18 MyApp 0x000000010226be00 _ZN5realm5Realm18commit_transactionEv + 276 19 MyApp 0x0000000101e57e4c -[RLMRealm commitWriteTransactionWithoutNotifying:error:] + 552 20 MyApp 0x000000010286af20 $s10RealmSwift0A0V11commitWrite16withoutNotifyingySaySo20RLMNotificationTokenCG_tKF + 148 21 MyApp 0x000000010286aa74 $s10RealmSwift0A0V5write16withoutNotifying_xSaySo20RLMNotificationTokenCG_xyKXEtKlF + 420 22 MyApp 0x0000000101b2eb28 $s7AppCore12RealmManagerC5writeyyy0B5Swift0B0VXEF + 376 23 MyApp 0x0000000101a1e8a0 $s7AppCore14ContentDataSourceC7cleanupyyF + 88 24 MyApp 0x0000000101b2f980 $s7AppCore12RealmManagerC7cleanup10completionyyAC16StepResponseTypeOc_tFyycfU_ + 256 25 MyApp 0x00000001018098d8 $sIeg_IeyB_TR + 52 26 libdispatch.dylib 0x0000000108f086d4 _dispatch_call_block_and_release + 32 27 libdispatch.dylib 0x0000000108f0a3b4 _dispatch_client_callout + 20 28 libdispatch.dylib 0x0000000108f0cf2c _dispatch_queue_override_invoke + 1052 29 libdispatch.dylib 0x0000000108f1e500 _dispatch_root_queue_drain + 408 30 libdispatch.dylib 0x0000000108f1ef0c _dispatch_worker_thread2 + 196 31 libsystem_pthread.dylib 0x00000001f1bbf0b8 _pthread_wqthread + 228 32 libsystem_pthread.dylib 0x00000001f1bbee94 start_wqthread + 8

The sequence of events on my app starting up are:

• Launch
• Wait for Realm to be setup (i.e. let realm = try Realm())
• When the Realm object is available run a cleanup on old content straight away, before the rest of the app can do anything
• During cleanup items older than 30 days may be removed, or marked for deletion on an object property

The first crash sequence happened during this cleanup. All the cleanup is ran on the main thread in a sequence rather than concurrently. In that sequence the first error happens opening a write block (realm.write { .. } ). The second comes accessing an object property (.imageURLString)

[pavel-ship-it]

@allanweir, @cupcake-shukla, @bmt2018, @matthewjordanRiM, @c128128, @kordem, @Sethmr
This issue is fixed in Realm v10.25.1
Please let us know if you still see the problem.

[allanweir]

Hi Pavel, looks to be working well! Thanks for the update and getting the fix out

[bmt2018]

Updated this morning and so far, so good. Thank you for the update on this!