App crashes with FORMAT_UPGRADE_REQUIRED while using Realm 7.0.0

[73sravanthi]

We have been using Realm from the past year, We recently upgraded the application from V6.1.0 to V7.0.0, post this upgrade we are witnessing the crash mentioned, we are using Kotlin for application development with Realm java and lastly, we have enabled DB encryption

Process: com.com.myapp.Application.qa, PID: 31558
java.lang.RuntimeException: Unable to create application com.myapp.Application: io.realm.exceptions.RealmFileException: Opening Realm files of format version 0 is not supported by this version of Realm (Database has an unsupported version and cannot be upgraded
Exception backtrace:
(/data/data/com.myapp.Application:/files/myapp.realm) in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 107 Kind: FORMAT_UPGRADE_REQUIRED.
at io.realm.internal.OsSharedRealm.nativeGetSharedRealm(Native Method)
at io.realm.internal.OsSharedRealm.(OsSharedRealm.java:175)
at io.realm.internal.OsSharedRealm.getInstance(OsSharedRealm.java:251)
at io.realm.BaseRealm.(BaseRealm.java:137)
at io.realm.BaseRealm.(BaseRealm.java:104)
at io.realm.Realm.(Realm.java:163)
at io.realm.Realm.createInstance(Realm.java:499)
at io.realm.RealmCache.createInstance(RealmCache.java:507)
at io.realm.RealmCache.doCreateRealmOrGetFromCache(RealmCache.java:473)
at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:414)
at io.realm.Realm.getDefaultInstance(Realm.java:407)

[cmelchior]

How are you opening the Realm?

[73sravanthi]

DB configuration :
RealmLivedataWrapper.init(context)
val config = RealmConfiguration.Builder()
.name(BuildConfig.REALM_FILE_NAME)
.modules(Realm.getDefaultModule(), DataRealmModules())
.encryptionKey(StorageKeyUtils.getKey(sharedPreferences, context))
.schemaVersion(BuildConfig.REALM_SCHEMA_VERSION)
.migration(DbMigration())
.build()
Realm.setDefaultConfiguration(config)

realmLivedataWrapper = RealmLivedataWrapper.Builder()
  .setRealmConfiguration(config)
  .build()

While navigating between the modules its crashing at
val realm: Realm = Realm.getDefaultInstance()

[cmelchior]

Weird. The error message seems to indicate that you are opening an older already existing file. Is that the case?

[73sravanthi]

It's crashing while navigating from one module to another after some modifications. We have used Realm v6.1.0. This issue is observed when upgraded to v7.0.0 .

[73sravanthi]

The same scenario is working fine with Realm v6.1.0

[73sravanthi]

Will there be any issue if we use following method in 7.0.0

val realm: Realm = Realm.getDefaultInstance()
realm.sharedRealm.refresh()
try {
realm.executeTransaction { realmInstance ->
realmInstance.copyToRealmOrUpdate(dataObjects)
}
} catch (ex: Exception) {
Log.d("DOWNLOAD_ERROR", "storeToLocal DB store EXCEPTION")
Log.d(this::class.java.simpleName, ex.message)
} finally {
realm.refreshAndClose()
}

[73sravanthi]

Hi.Any update on the above issue? We are facing the above crash with above error in multiple places .

[edgar-zigis]

I have noticed this error too. We have only upgraded realm version and did not touch schema itself.
Mostly it occurs on Android 10 devices. Most popular devices which failed to upgrade:

Samsung Galaxy A70
Xiaomi Mi9 SE
Nokia 5

Other devices seemed to work fine, but this is quite critical.

[ganfra]

Exactly the same issue on our side, reproduced on my Pixel 3 on Android 9.

[cmelchior]

@ganfra Can you share how to reproduce? If you want to keep it private you can email me at christian.melchior@mongodb.com

[ganfra]

If you are brave enough you can test it on our open-source project: https://github.com/vector-im/riotX-android
Updating realm to 7.0.0, then after some initial launches it will crash with this issue.
(Also, not related at all, but we are having massive performance issues when db is getting big)

[ganfra]

So I created a test account where you will be able to reproduce the error:
Login: realm7
Password: realm7.0help

You should use the develop branch.

So here is how to reproduce

1. compile the app with realm 6.1.0 (in the matrix-sdk-android gradle file).
2. launch and log in with given credentials.
3. Compile the app with realm 7.0.0
4. Relaunch the app, use the app a bit.
5. kill the app and relaunch => it should crash. If not, kill and relaunch until it crashes

Tell me if you need more information :)

[ganfra]

@cmelchior : any chance you could look at it?

[cmelchior]

@ganfra Thank you for the project. I am able to reproduce the problem. We are looking into it.

A few things ruled out:

• It isn't a wrong encryption key causing this issue or lack of key completely.

[ganfra]

Did you have any lead on what could be wrong?

[patelnishantk]

@cmelchior Would appreciate any update on this. Unfortunately, we too face this issue after upgrading from v6.1.0 to v7.0.0. It has resulted into hundreds of crashes reported in Firebase in barely few hours.

[cmelchior]

@patelnishantk Unfortunately we still don't have a root cause for this.

[ppamorim]

We are experiencing the same issue with a single client.

[ppamorim]

@ganfra Be aware that already corrupted databases won't be restored (in my case). We had to create a special app to export critical data from our apps. Far from ideal and a bit humiliating. :( I would like to see the Realm team speaking more about it. @jedelbo Could you please refer to the PR? Was it in the core?

[jedelbo]

Be aware that the file is not corrupted.We just fail to read the file correctly. The issue was fixed in realm/realm-core#3910.

[ppamorim]

@jedelbo I did export 3 databases + encryption keys and I could not open any of them on Realm Studio. All of them where around 700mb for an unknown reason. That's why I suspect that the file was corrupt.

[ganfra]

Same issue of 700mb size, but I did manage to see this issue before releasing the app, so no problem for me...

[jedelbo]

@ganfra so you see an unexpected increase in size? How big a difference is there compared to earlier?

[ganfra]

Generally the file is 150mb in this usecase. Multiple time after this crash it grows to 700mb as I said.

[ppamorim]

@jedelbo Could you please confirm if the latest version is capable of opening these large databases and if it's able to reduce the file size?

I saw that you recently did a change in the core but this version on this library is not out, is there any way to release this version anytime soon?

[jedelbo]

The Java team is working out getting the release out later today.

We are able to handle realms of any practical size. You can use CompactOnLaunch to reduce the size of a realm. However we are of course interested in keeping the size of the realm down, so we would be interested in investigating why you experience this growth in size. So would you be willing and able to share a 'before' and 'after' realm with us? You can share the files privately by mail (jorgen.edelbo@mongodb.com).

[ppamorim]

I think @ganfra will be able to provide a database, I could not replicate the issue on my side and the data that I have cannot be shared.

[ppamorim]

@jedelbo Any update?

[ppamorim]

@cmelchior The version 7.0.2 didn't sort the issue. We are going to update it to version 7.0.5. The situation is bad.

[cmelchior]

We will be making a 7.0.6 release in the next few hours that should have a fix for this.

[jedelbo]

You should wait for v7.0.6. It should be just around the corner.

[prettyClouds]

Is there any kind of public pipeline we can monitor to get an idea of when things will release?

[cmelchior]

You can watch releases on Bintray here: https://bintray.com/realm/maven/realm-android-library

[cmelchior]

7.0.6 was just released which should fix this. Please reopen the issue if you still see it after upgrading.

[ppamorim]

@cmelchior The version 7.0.6 is still not available, any timeframe or workaround for the moment?

[cmelchior]

It should be available now.

[ganfra]

Ok so now I can't reproduce, looking good! Thanks for this long-awaited fix!

[ppamorim]

@ganfra Hi, were you able to open that 700mb realm file too?

[ganfra]

Didn't try, but I got a SEGFAULT 11 and a corrupted file just after, so shout victory too quickly...

[ppamorim]

@ganfra Very nice, imagine living in a world where the database of your application gets corrupted due to a bug.

[ppamorim]

@ganfra Some users were able to restore the app. I got one case where the app wasn't able to load the database.

[ganfra]

@ppamorim thanks for letting me know, but I just keep getting native crashes on some other parts

[sampathkumar1988]

Hi Team, We are getting this crash in our production. Not able to reproduce though. Any solutions ?

CRASH TYPE SUMMARY
LOCATION
SourceFile line -2 in io.realm.internal.OsSharedRealm.nativeGetSharedRealm()
EXCEPTION
io.realm.exceptions.RealmFileException
MESSAGE
Opening Realm files of format version 22 is not supported by this version of Realm (Database has an unsupported version and cannot be upgraded Exception backtrace: ) (/data/data/com.company.au.gma/files/company-sdk-account.storage) in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 107

Note: We are using the 7.0.8 realm version

Thanks

[jedelbo]

@sampathkumar1988 Seems like you are opening a file produced by a newer version of realm with a rather old version.