Assertion failed: has_refs()

[xiles]

SDK and version

SDK : Swift
Version:
Realm 10.30.0
RealmDatabase 12.7.0
Xcode 14.0.1

Observations

• Not very often but it happens
• Happened in production

Crash log / stacktrace

Thread 1 Queue : com.apple.main-thread (serial)
#0 0x00000001ae50fe60 in __pthread_kill ()
#1 0x00000001ae5623c0 in pthread_kill ()
#2 0x0000000114258e38 in abort ()
#3 0x0000000102dbd924 in ::please_report_this_issue_in_github_realm_realm_core_v_12_7_0() at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/util/terminate.cpp:65
#4 0x0000000102dbdca8 in realm::util::terminate_internal(std::__1::basic_stringstream<char, std::__1::char_traits, std::__1::allocator >&) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/util/terminate.cpp:142
#5 0x0000000102dbda08 in realm::util::terminate(char const*, char const*, long, std::initializer_listrealm::util::Printable&&) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/util/terminate.cpp:152
#6 0x0000000102578124 in realm::Array::get_as_ref_or_tagged(unsigned long) const at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/array.hpp:754
#7 0x00000001025e34d0 in realm::Group::set_size() const at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/group.cpp:219
#8 0x00000001025e2584 in realm::Group::attach(unsigned long, bool, bool) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/group.cpp:552
#9 0x00000001025e5858 in realm::Group::attach_shared(unsigned long, unsigned long, bool) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/group.cpp:618
#10 0x0000000102d1a478 in realm::Transaction::Transaction(std::__1::shared_ptrrealm::DB, realm::SlabAlloc*, realm::DB::ReadLockInfo&, realm::DB::TransactStage) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/transaction.cpp:114
#11 0x0000000102d1a6b0 in realm::Transaction::Transaction(std::__1::shared_ptrrealm::DB, realm::SlabAlloc*, realm::DB::ReadLockInfo&, realm::DB::TransactStage) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/transaction.cpp:108
#12 0x00000001025b1254 in std::__1::shared_ptrrealm::Transaction (anonymous namespace)::make_transaction_ref<std::__1::shared_ptrrealm::DB, realm::SlabAlloc*, realm::DB::ReadLockInfo&, realm::DB::TransactStage>(std::__1::shared_ptrrealm::DB&&, realm::SlabAlloc*&&, realm::DB::ReadLockInfo&, realm::DB::TransactStage&&) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/db.cpp:406
#13 0x00000001025ab228 in realm::DB::start_read(realm::VersionID) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/db.cpp:2390
#14 0x00000001025a9d80 in realm::DB::open(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, bool, realm::DBOptions) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/db.cpp:1180
#15 0x00000001025abca0 in realm::DB::open(realm::Replication&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, realm::DBOptions) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/db.cpp:1215
#16 0x00000001025b20b8 in realm::DB::create(std::__1::unique_ptr<realm::Replication, std::__1::default_deleterealm::Replication >, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, realm::DBOptions) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/db.cpp:2529
#17 0x0000000102782ed4 in realm::_impl::RealmCoordinator::open_db() at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/object-store/impl/realm_coordinator.cpp:459
#18 0x0000000102784498 in realm::_impl::RealmCoordinator::do_get_realm(realm::RealmConfig, std::__1::shared_ptrrealm::Realm&, std::__1::optionalrealm::VersionID, realm::util::CheckedUniqueLock&) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/object-store/impl/realm_coordinator.cpp:280
#19 0x00000001027842ac in realm::_impl::RealmCoordinator::get_realm(realm::RealmConfig, std::__1::optionalrealm::VersionID) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/object-store/impl/realm_coordinator.cpp:252
#20 0x00000001028d3fbc in realm::Realm::get_shared_realm(realm::RealmConfig) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-core/src/realm/object-store/shared_realm.cpp:157
#21 0x00000001023141ac in +[RLMRealm realmWithConfiguration:queue:error:] at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-cocoa/Realm/RLMRealm.mm:532
#22 0x00000001023ccb88 in @nonobjc RLMRealm.__allocating_init(configuration:queue:) ()
#23 0x000000010248621c in Realm.init(queue:) at /Users/xiles/Library/Developer/Xcode/DerivedData/HashPhotos-emjpzodbkpdtdgcztdgogetflflc/SourcePackages/checkouts/realm-cocoa/RealmSwift/Realm.swift:79

Steps & Code to Reproduce

I got this broken database file from a user of my app.
I don't exactly know how come the database file broken.

Is there any way to fix the broken database file?
Users lost all of their data when this happens.

[nicola-cab]

Hello, thanks for reporting this and sharing the realm file with us. To answer your question, unfortunately there is no way to restore a broken realm file.
We are aware of this uncommon file corruptions, and we are working in order to fix these issues.

Most probably the corruption has happened before, looking at your stack trace it seems like we were opening the file, thus either starting the application or re-initializing it. At that stage we realized that the file was not right and terminated.

Knowing what the application was doing when it wrote down something to the file, could help us. In the meantime, I'll keep digging and keep you posted in case we find out a possible solution to this problem.

If you are able to reproduce the crash, please add some code snipped to this issue. It will help us a lot.

[xiles]

Not exact but a database file is broken after updating an app from App Store.

• Added some realm objects.
• An app goes to background.
• An app is updated from App Store.
• Open an app => crash (database file is broken)

I've been using realm for a long time. It's very fast and easy to use.. but broken database file is too critical... for database. It keeps happening. And once it is broken, there is no way to restore data at all. not a part of them.
In last 1~ 2 months, about 4 ~5 users reported a crash with a broken database file. Maybe there are more but they doesn't report and just reinstalls an app.

[jedelbo]

@xiles Unfortunately we are seeing an uptake in the number of issues like this reported by users of the Swift SDK. It is of course high on our priority list to find the root cause of these issues. Thank you for sharing the file - it might bring us new insights into what could be wrong.

[BlueCobold]

The file still contains a regular header structure and the top-refs do not look that bad either. The main table-information block can also be found directly after the header. Manual reconstruction could still be tricky, and probably not all the data can be restored. I have modified versions of the dump- and browser-tools and in case that file is super important to restore, I could try to take a closer look. Please note, however, that I am not part of the realm team with dedicated time for that. I'm just a realm-using dev, having faced and recovered corrupted realm files before.

[xiles]

@BlueCobold Could you help to restore the broken file? All user's data is super important. A user have built those data for a long period and with many effort. Thanks in advance.

[BlueCobold]

@xiles I can't make any promises, but I can take a look to check what might be salvageable.

[xiles]

@BlueCobold Of course, I totally understand.

[BlueCobold]

@xiles How many tables should the file have? Because it's trying to load 10, where I can only see 7 (+1 internal). Do you by chance have an empty example file to compare the broken file to?

[xiles]

@BlueCobold Here is the normal database file.
The broken file is have been migrated from old versions and there could be tables not using anymore.
(HPPrivateAssetCollection, HPPrivateTag, HPTag)

Here is the realm configuration using in a current version.

Realm.Configuration(fileURL: dataFileURL, inMemoryIdentifier: nil, encryptionKey: nil, readOnly: false, schemaVersion: HashPhotos.RealmSchemaVersion, migrationBlock: migrationBlock, shouldCompactOnLaunch: shouldCompactBlock, objectTypes: [HLAssetVault.self, HLAssetCollection.self, HLSmartAssetCollection.self, HLAsset.self, HLTag.self, HLEvent.self,  PHAssetMetadata.self])

default.realm.zip

[BlueCobold]

@xiles Can you check if this file makes any sense? It might be an old or outdated version of the file's content, because I just simply used one of the old "commits", but I have no idea if this makes any sense. There are various versions of the data in the file, does your application make heavy use of parallel transactions?
I could possibly export all working old "commit" versions of the file for you or your customer to check which one makes the most sense, but it might be quite a few.
database_backup-1.realm.zip

@jedelbo I simply used another top-ref which made more sense. The two ones in the corrupted file point to invalid table_key arrays, but to the correct table_name array. From my first glance, I could not find any overwritten chunks, but they could still be there.

[BlueCobold]

@xiles: Since the given file is still kind of "broken", I don't recommend using that file for further productive use. So in case you want to give it to your customer, I would recommend reading all entries from this file and write them to a fresh new realm, so that no bad references remain which could damage the file again straight away.

[xiles]

@BlueCobold I'm not sure there are data losses or not but at least the file is opened and tables look ok.
I'll recreate the data file and check with a user.
An app doesn't perform parallel transactions.. but there are codes using try! realm.write { .. } in a loop.
Does this count as parallel transactions?

I really really appreciate you doing this for me and my user.

By any chance, could you teach(?) me how you restore the broken file if you do not mind and it's possible?
There were more users experienced this broken database crash.. and I guess that there will be more until something is corrected in my app.

[BlueCobold]

@xiles That's not parallel, that's sequential access. Parallel would be if you used multiple async blocks to read/write your realm. If that's true, it would mean the issue is not related to multi-threading, which I hoped to solve by making all my transactions sequential in a single thread - and so far it seemed to be mostly solving my corruption issues.

About what I did to the file, it is basically a heavily nested list of trees and dictionaries of references. Too complex and too sparsely documented even for me to fully comprehend. However, at the start of the file there are 2 major references. One pointing to an old version, one pointing to a new version. Based on a flag in the header (4th byte after 'T-DB') defines which one is the current top_ref. It points to the table-definition array which points to the tables_names array and table_keys array respectively. Realm writes new versions of these during transactions and then updates internal top-/references. Sometimes this gets corrupted for some unknown reason, but you can try to find one of these old top_refs due to their distinct "signature" (41414141 4600000B 18000000 D85E0C00 - 41414141 being the head-marker of the array, 4600000B is the length of the array and type of the entries, 18000000 is the table_name ref. D85E0C00 is the table_key ref, which is different for most top_refs and thus may contain different stuff). I tried a few of them to be used in the header as top_ref and this one allowed reading the file. I'm not sure it it allows reading all entries correctly, though, because that top_ref is probably an outdated left-over reference from previous transactions. What I do to recover files, is first to find a valid top_ref to point to table_names and table_keys and then basically recursing the same procedure, because all columns are again arrays of references, lists and maps. It is... complex and I am debugging through the exec-tools a lot, adding additional parameters to change parsing or adding or removing certain validity-checks to bypass potential crashes and read the remaining data or to skip broken entries.
I could possibly check which readable top_ref version of the file points to the most entries in the tables.
PS: "refs" are basically just "references", meaning offsets in the file. 18000000 means the array starts at offset 0x00000018 and D85E0C00 means the table_key array starts at offset 0x000C5ED8.

[BlueCobold]

@xiles If more users are affected, I'm afraid the corruptions might be very different for each one and may require different approaches. Sometimes the references are invalid, sometimes memory areas are overwritten and unrecoverable. Depending on what happened, other references can be used or entries can only partly be read with specifically changed tools/code to export the columns that still exist and skip others or setting them to null. If you have another file, I could potentially have a look at it as well. Or you start debugging into the various exec tools (realm2json, realmTrawler, realmBrowser) in the core repository to see how deep you can go into the file and/or to identify corrupted entries/columns/tables.

[xiles]

@BlueCobold You are super!!! Thanks for your explanation in detail. I'll try what I can do.

PS:
Do you use iPhone or iPad?
I know that it's not enough for your help but I would like to give you a redeem code of my app as a token of appreciation.
let me know your email If you want to try. It's a photo manager app for iOS.

[BlueCobold]

@xiles I just develop for iOS, I am mainly using Android. But thanks for the offer ;)

[jedelbo]

Closed by #5993