Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to open a realm at path. ACCESS_ERROR: Invalid mnemonic #6257

Closed
TTransmit opened this issue Oct 29, 2018 · 16 comments
Closed

Unable to open a realm at path. ACCESS_ERROR: Invalid mnemonic #6257

TTransmit opened this issue Oct 29, 2018 · 16 comments

Comments

@TTransmit
Copy link

TTransmit commented Oct 29, 2018

Goal

We have been receiving reports in Crashlytics of crashes due to issues with Realm. These seem to have gotten more frequent in switching from Realm 5.5.0 to 5.7.0. I was able to find the crash myself while looking into a different issue. I have a copy of the corrupted Realm database, which will not open in Realm browser - it says it needs a password, which is not needed for uncorrupted versions of the database. I can send the corrupted database via email. I'm told by another developer that we only access Realm from a single thread.

Actual Results

10-29 15:29:02.574 31851-31851/com.mycompany.dev E/AndroidRuntime: FATAL EXCEPTION: main Process: com.mycompany.dev, PID: 31851 java.lang.RuntimeException: Unable to start activity ComponentInfo{com.mycompany.dev/com.mycompany.myfeature.myfeatureActivity}: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.mycompany.dev/files/myfeaturedb.realm': Invalid mnemonic. top_ref[0]: 109600030316, top_ref[1]: F3267A703049192, mnemonic: 1F EE EE 35, fmt[0]: 146, fmt[1]: 94, flags: 11. (Invalid mnemonic. top_ref[0]: 109600030316, top_ref[1]: F3267A703049192, mnemonic: 1F EE EE 35, fmt[0]: 146, fmt[1]: 94, flags: 11) (/data/data/com.mycompany.dev/files/myfeaturedb.realm) in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 101 Kind: ACCESS_ERROR.

Steps & Code to Reproduce

The crash occurred first when trying to replicate another issue. The process was starting the app and rapidly backgrounding it many times in a row. The device was connected to Charles Proxy at the time and Charles seemed to be playing up: it looked like it was returning 200 but with no body or headers. I'm not sure if I'm interpreting what Charles was returning correctly.

Version of Realm and tooling

Realm version(s): 5.7.0

Realm Sync feature enabled: No

Android Studio version: 3.2.1

Android Build Tools version: 3.2.1 ? target/compileSdk 28

Gradle version: 4.10.2

Which Android version and device(s): 5.0, Asus Z00AD (Though other Android versions and devices in Fabric Crashlytics)

@bmunkholm
Copy link
Contributor

Thanks @TTransmit! If you can send that file to help@realm.io and mention this github issue we will be grateful.

@cmelchior
Copy link
Contributor

cmelchior commented Oct 29, 2018

Also, If you can post as many of stack traces as possible it would really help a lot.
We are especially interested in this error message of this kind:
'/data/data/com.mycompany.dev/files/myfeaturedb.realm': Invalid mnemonic. top_ref[0]: 109600030316, top_ref[1]: F3267A703049192, mnemonic: 1F EE EE 35, fmt[0]: 146, fmt[1]: 94, flags: 11. (Invalid mnemonic. top_ref[0]: 109600030316, top_ref[1]: F3267A703049192, mnemonic: 1F EE EE 35, fmt[0]: 146, fmt[1]: 94, flags: 11)

The different values here can help us narrow down where the problem might be

@cmelchior cmelchior changed the title Unable to open a realm at path ...realm Invalid mnemonic Unable to open a realm at path. ACCESS_ERROR: Invalid mnemonic Oct 29, 2018
@cmelchior
Copy link
Contributor

cmelchior commented Oct 30, 2018

This bug is now being tracked in Core here realm/realm-core#3134

Some notes about this error:

  • The mnemonic is a special sentinel value at the beginning of a Realm file. Its purpose is to mark the file as a Realm file. For a correct Realm file, this value is T-DB or 54 2D 44 42 in Hex.

image

If this value isn't present we will throw the error seen in this issue.

The most likely cause of this value being wrong is that something else than Realm manipulated the file in a wrong way, examples:

  • Copying the file.
  • Deleting the file.

Note that Realm API's exists that that does both of these things but to our knowledge, these should all be safe to use. This includes RealmConfiguration.assetFile() which will copy the Realm file from the APK asset folder and Realm.deleteRealm(config) which will delete the Realm file on disk.

Android provides automatic API's that can backup and restore files: https://developer.android.com/guide/topics/data/autobackup but according to its documentation it should only run when the app is killed preventing race conditions here as well.

@cmelchior
Copy link
Contributor

@TTransmit Any information you have regarding how the Realms are being opened/manipulated would be very valuable. Especially if you copying Realms from assets, deleting or copying the files. If you can share your RealmConfiguration that would also help.

From the stack traces you sent, it seems you have different errors happening to librarydb.realm and userdb.realm which indicates that they are somehow being handled differently?

@cmelchior
Copy link
Contributor

cmelchior commented Nov 6, 2018

Info from #6284

OS Version: 7.0
Device: Redmi Note 4
RAM Free: 45.9%
Disk Free: 67.3%

Fatal Exception: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.virinchi.mychat/files/docquity.realm': Invalid mnemonic. top_ref[0]: 0, top_ref[1]: B1490, mnemonic: 0 0 0 0, fmt[0]: 0, fmt[1]: 9, flags: 1. (Invalid mnemonic. top_ref[0]: 0, top_ref[1]: B1490, mnemonic: 0 0 0 0, fmt[0]: 0, fmt[1]: 9, flags: 1) (/data/data/com.virinchi.mychat/files/docquity.realm) in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 101

@imvikasrai The mnemonic is all zeros, which indicate a very corrupt file. But without access to the file it is hard to say exactly how much.

A few extra questions:

  • Are you deleting the Realm file in any way in your app?
  • Are you copying a Realm file around in any way? This includes using RealmConfiguration.assetFile()?

@imvikasrai
Copy link

imvikasrai commented Nov 6, 2018

I m only deleting all the tables only like this

realm.beginTransaction();
realm.delete(DocquityLog.class);
realm.delete(ChatDialogDb.class);
realm.delete(ChatQB.class);
realm.delete(EventLogDb.class);
realm.delete(SaveDb.class);
realm.delete(UserDb.class);
realm.delete(ChannelDb.class);
realm.delete(CMEDb.class);
realm.delete(MediaDb.class);
realm.delete(OptionDb.class);
realm.delete(PointsDb.class);
realm.delete(QuestionDb.class);
realm.delete(ShareContentDb.class);
realm.delete(SpecialityDb.class);
realm.commitTransaction();
realm.close();

I m not deleting the database

but realm tables is only deleted when user logged out and this is very rare condition I don't think so user got logged out

and I. m not using RealmConfiguration.assetFile() anywhere

@imvikasrai
Copy link

@cmelchior, How can I reproduce this issue?.I m not able to reproduce this issue. At least I want to handle this exception.Will the exception will be thrown when accessing realm??

@guilhermekrz
Copy link

guilhermekrz commented Nov 19, 2018

I'm also experiencing this issue with 50 users:

  • 94% running Android 6.0.1
  • 2% running Android 4.4.4
  • 1% running Android 5.1.1
  • 1% running Android 6.0.0
  • 1% running Android 7.0.0
  • 1% running Android 7.1.1

And devices:

  • 95% Samsung SM-G532MT or SM-J5500M
  • 2% Motorola
  • 1% others

None of them are rooted.

Some examples (actually, now I see that Crashlytics grouped two type of issues: invalid mnemonic and top ref outside file, are they related?):
Invalid mnemonic. top_ref[0]: 104600030316, top_ref[1]: 70AD0, mnemonic: 6B 2B 79 31, fmt[0]: 19, fmt[1]: 9, flags: F5. (Invalid mnemonic. top_ref[0]: 104600030316, top_ref[1]: 70AD0, mnemonic: 6B 2B 79 31, fmt[0]: 19, fmt[1]: 9, flags: F5)
Invalid mnemonic. top_ref[0]: 104600030316, top_ref[1]: 70AD0, mnemonic: 6B 2B 79 31, fmt[0]: 19, fmt[1]: 9, flags: F5. (Invalid mnemonic. top_ref[0]: 104600030316, top_ref[1]: 70AD0, mnemonic: 6B 2B 79 31, fmt[0]: 19, fmt[1]: 9, flags: F5)
Invalid mnemonic. top_ref[0]: 104600030316, top_ref[1]: 70AD0, mnemonic: 6B 2B 79 31, fmt[0]: 19, fmt[1]: 9, flags: F5. (Invalid mnemonic. top_ref[0]: 104600030316, top_ref[1]: 70AD0, mnemonic: 6B 2B 79 31, fmt[0]: 19, fmt[1]: 9, flags: F5)

Invalid mnemonic. top_ref[0]: 104600030316, top_ref[1]: F8118460FA044142, mnemonic: 4A 64 84 D0, fmt[0]: 243, fmt[1]: 192, flags: C8. (Invalid mnemonic. top_ref[0]: 104600030316, top_ref[1]: F8118460FA044142, mnemonic: 4A 64 84 D0, fmt[0]: 243, fmt[1]: 192, flags: C8)

Invalid mnemonic. top_ref[0]: 104600030316, top_ref[1]: C1A215DD31044142, mnemonic: 8A CE 26 8A, fmt[0]: 246, fmt[1]: 131, flags: 8. (Invalid mnemonic. top_ref[0]: 104600030316, top_ref[1]: C1A215DD31044142, mnemonic: 8A CE 26 8A, fmt[0]: 246, fmt[1]: 131, flags: 8)

Top ref outside file (size = 1179648). top_ref[0]: 3031500000840A8, top_ref[1]: 46020200, mnemonic: 54 2D 44 42, fmt[0]: 9, fmt[1]: 9, flags: 0. (Top ref outside file (size = 1179648). top_ref[0]: 3031500000840A8, top_ref[1]: 46020200, mnemonic: 54 2D 44 42, fmt[0]: 9, fmt[1]: 9, flags: 0)
Top ref outside file (size = 1310720). top_ref[0]: 3031500001274D0, top_ref[1]: 46020200, mnemonic: 54 2D 44 42, fmt[0]: 9, fmt[1]: 9, flags: 1. (Top ref outside file (size = 1310720). top_ref[0]: 3031500001274D0, top_ref[1]: 46020200, mnemonic: 54 2D 44 42, fmt[0]: 9, fmt[1]: 9, flags: 1)
Top ref outside file (size = 1179648). top_ref[0]: 303150000058DA8, top_ref[1]: 46020200, mnemonic: 54 2D 44 42, fmt[0]: 9, fmt[1]: 9, flags: 0. (Top ref outside file (size = 1179648). top_ref[0]: 303150000058DA8, top_ref[1]: 46020200, mnemonic: 54 2D 44 42, fmt[0]: 9, fmt[1]: 9, flags: 0) (/data/data/com.brokoli.campeonatobrasileiro/files/db.realm)

It seems this issue started when I upgraded from classpath "io.realm:realm-gradle-plugin:5.1.0" to classpath "io.realm:realm-gradle-plugin:5.7.0", persisting when I upgraded to classpath "io.realm:realm-gradle-plugin:5.8.0".

Are you deleting the Realm file in any way in your app? No.
Are you copying a Realm file around in any way? This includes using RealmConfiguration.assetFile()? No.

Let me know if I can help with additional information. For now, is it safe for me downgrading to 5.1.0 to see if this issue disappear, or are there any breaking changes?

@guilhermekrz
Copy link

guilhermekrz commented Nov 19, 2018

It also started at the same time that I did a Realm migration, deleting an entire table, like this: schema.remove("Abc");

This is the way I am initializing Realm:

Realm.init(
RealmConfiguration config = new RealmConfiguration.Builder()
       .schemaVersion(9)
       .migration(new MyRealmMigration())
       .name("db.realm")
       .build();
Realm.setDefaultConfiguration(config);

@imvikasrai
Copy link

imvikasrai commented Nov 20, 2018

Now I m this handling this exception by accessing realm in onActivityResumed of ActivityLifecycleCallbacks and deleting all files in the internal storage.like this

`try{
SingleInstace.getInstace().getRealm();
Log.e(TAG, "isRealmReadyToUse: success" );
}catch (UnsatisfiedLinkError unsatisfiedLinkError){
deleterealmFiles();
Log.e(TAG, "isRealmReadyToUse: ex"+unsatisfiedLinkError.getMessage() );
}catch (RealmFileException ex){
deleterealmFiles();
Log.e(TAG, "isRealmReadyToUse: ex2 "+ex.getMessage() );
``
}catch (RealmError error){
deleterealmFiles();
Log.e(TAG, "isRealmReadyToUse: ex2 "+error.getMessage() );

    }`

This issue can be reproduced by corrupting your realm file on any online file corrupt platform and replacing that file with your current file.

@guilhermekrz
Copy link

guilhermekrz commented Nov 20, 2018

After downgrading to 5.1.0 I am still seeing these crashes, but I think they are from the same 50 users from before, which already have the files corrupted.

I think I will have to do the same as @imvikasrai to solve for these users, but ideally, I would like to remove this code in the near future.

@aritrobanerjee93
Copy link

Hi,
I faced a similar issue and documented the issue here

https://stackoverflow.com/questions/53498692/io-realm-exceptions-realmfileexception-invalid-mnemonic-error?noredirect=1#comment93878320_53498692

I added the stack trace and some steps I took to reproduce the error.
Please let me know if it helps.
I have been searching for a solution for this for a while now.

@anggrayudi
Copy link

anggrayudi commented Mar 27, 2019

Also affects these devices (I will update it if I found new affected devices):

  • SPA Condor Electronics (model PGN611). Android 7 Nougat.

      RealmFileException: Unable to open a realm at path '/data/data/com.anggrayudi.watercat/files/default.realm': Invalid mnemonic. top_ref[0]: 8AF9A5D0000000A, top_ref[1]: 79196033FCD5D094, mnemonic: E0 F0 68 F2, fmt[0]: 189, fmt[1]: 170, flags: C3 Path:. (Invalid mnemonic. top_ref[0]: 8AF9A5D0000000A, top_ref[1]: 79196033FCD5D094, mnemonic: E0 F0 68 F2, fmt[0]: 189, fmt[1]: 170, flags: C3 Path: /data/data/com.anggrayudi.watercat/files/default.realm) (/data/data/com.anggrayudi.watercat/files/default.realm) in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 101
    
  • Samsung Galaxy S5. Android 6.0.1

      RealmFileException: Unable to open a realm at path '/data/data/com.anggrayudi.watercat/files/default.realm': Invalid mnemonic. top_ref[0]: 169DD5B100000092, top_ref[1]: 476335EFE44869F1, mnemonic: B9 16 44 A9, fmt[0]: 128, fmt[1]: 50, flags: 2B Path:. (Invalid mnemonic. top_ref[0]: 169DD5B100000092, top_ref[1]: 476335EFE44869F1, mnemonic: B9 16 44 A9, fmt[0]: 128, fmt[1]: 50, flags: 2B Path: /data/data/com.anggrayudi.watercat/files/default.realm) (/data/data/com.anggrayudi.watercat/files/default.realm) in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 101
    

@cmelchior
Copy link
Contributor

Should be fixed in 6.0.1 (not released yet), but 6.0.1-SNAPSHOT should be available. See https://github.com/realm/realm-java/blob/master/README.md#using-snapshots

@OlegNovosad
Copy link

Hello there. We are still observing the same issue with Realm 6.0.2:

java.lang.RuntimeException: Unable to create application <app_id>.MyApplication: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/<app_id>/files/encrypted.realm': Invalid mnemonic. top_ref[0]: D6C5AF7400000011, top_ref[1]: 88591B63C8C6044D, mnemonic: C7 35 2 54, fmt[0]: 132, fmt[1]: 174, flags: BC Path:Exception backtrace:
    <backtrace not supported on this platform>. (Invalid mnemonic. top_ref[0]: D6C5AF7400000011, top_ref[1]: 88591B63C8C6044D, mnemonic: C7 35 2 54, fmt[0]: 132, fmt[1]: 174, flags: BC Path: /data/data/<app_id>/files/encrypted.realm
    Exception backtrace:
    <backtrace not supported on this platform>) (/data/data/<app_id>/files/encrypted.realm) in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 101 Kind: ACCESS_ERROR.
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5876)
        at android.app.ActivityThread.access$1100(ActivityThread.java:199)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1650)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:193)
        at android.app.ActivityThread.main(ActivityThread.java:6669)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
     Caused by: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/<app_id>/files/encrypted.realm': Invalid mnemonic. top_ref[0]: D6C5AF7400000011, top_ref[1]: 88591B63C8C6044D, mnemonic: C7 35 2 54, fmt[0]: 132, fmt[1]: 174, flags: BC Path:Exception backtrace:
    <backtrace not supported on this platform>. (Invalid mnemonic. top_ref[0]: D6C5AF7400000011, top_ref[1]: 88591B63C8C6044D, mnemonic: C7 35 2 54, fmt[0]: 132, fmt[1]: 174, flags: BC Path: /data/data/<app_id>/files/encrypted.realm
    Exception backtrace:
    <backtrace not supported on this platform>) (/data/data/<app_id>/files/encrypted.realm) in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 101 Kind: ACCESS_ERROR.
        at io.realm.internal.OsSharedRealm.nativeGetSharedRealm(Native Method)
        at io.realm.internal.OsSharedRealm.<init>(OsSharedRealm.java:171)
        at io.realm.internal.OsSharedRealm.getInstance(OsSharedRealm.java:241)
        at io.realm.internal.OsSharedRealm.getInstance(OsSharedRealm.java:231)
        at io.realm.RealmCache.doCreateRealmOrGetFromCache(RealmCache.java:337)
        at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:290)
        at io.realm.Realm.getInstance(Realm.java:428)

Unfortunately I cannot provide more logs due to NDA. Basically we are injecting Realm Configuration to data layers which have control over Realm.getInstance(configuration) (injection performed with Dagger 2).

@kibotu
Copy link

kibotu commented Jan 17, 2020

having same issue on 6.0.2; is it related to encryption?

Caused by io.realm.exceptions.RealmFileException
Unable to open a realm at path '/data/data/***/files/***.realm': Top ref outside file (size = 20971520). top_ref[0]: 303150000204CB8, top_ref[1]: 46020200, mnemonic: 54 2D 44 42, fmt[0]: 9, fmt[1]: 9, flags: 1 Path:Exception backtrace: <backtrace not supported on this platform>. (Top ref outside file (size = 20971520). top_ref[0]: 303150000204CB8, top_ref[1]: 46020200, mnemonic: 54 2D 44 42, fmt[0]: 9, fmt[1]: 9, flags: 1 Path: /data/data/***/files/***.realm Exception backtrace: <backtrace not supported on this platform>) (/data/data/***/files/***.realm) in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 101
io.realm.internal.OsSharedRealm.nativeGetSharedRealm (OsSharedRealm.java)
io.realm.internal.OsSharedRealm.<init> (OsSharedRealm.java:171)
io.realm.internal.OsSharedRealm.getInstance (OsSharedRealm.java:241)
io.realm.internal.OsSharedRealm.getInstance (OsSharedRealm.java:231)
io.realm.RealmCache.doCreateRealmOrGetFromCache (RealmCache.java:332)
io.realm.RealmCache.createRealmOrGetFromCache (RealmCache.java:285)
io.realm.Realm.getInstance (Realm.java:428)
****.data.database.RealmDatabase$realm$2.invoke (RealmDatabase.kt:42)
****.data.database.RealmDatabase$realm$2.invoke (RealmDatabase.kt:14)
kotlin.SynchronizedLazyImpl.getValue (LazyJVM.kt:74)
****.data.database.RealmDatabase.getRealm (Unknown Source:7)
****.data.database.Database$seasonDao$2.invoke (Database.kt:78)
****.data.database.Database$seasonDao$2.invoke (Database.kt:11)
kotlin.SynchronizedLazyImpl.getValue (LazyJVM.kt:74)
****.data.database.Database.getSeasonDao (Unknown Source:8)
****.data.repositories.SeasonRepository.getDb (SeasonRepository.kt:22)
****.services.repositories.RepositoryViewModel.getSeasons (RepositoryViewModel.kt:355)
****.services.repositories.RepositoryViewModel.<init> (RepositoryViewModel.kt:102)
java.lang.Class.newInstance (Class.java)
androidx.lifecycle.ViewModelProvider.get (ViewModelProvider.java:145)
****.MainActivity$$special$$inlined$viewModel$1.invoke (ViewModel+Extensions.kt:10)
****.MainActivity$$special$$inlined$viewModel$1.invoke
kotlin.SynchronizedLazyImpl.getValue (LazyJVM.kt:74)
com.****.MainActivity.getRepositoryViewModel (Unknown Source:7)
com.****.MainActivity.onCreate (MainActivity.kt:145)
android.app.Activity.performCreate (Activity.java:7183)
com.android.internal.os.ZygoteInit.main (ZygoteInit.java:1374)

initializing realm like

protected open val config: RealmConfiguration by lazy {

       val key = ByteArray(64)
       SecureRandom().nextBytes(key)
       secret = key

       val name = "***.realm"
       with(RealmConfiguration.Builder()) {
           schemaVersion(schemaVersion)
           deleteRealmIfMigrationNeeded()
           name(name)
           modules(module)
           build()
       }
   }

crashlytics meta:

76% samsung
10% HUAWEI
7% Google
2% BullittGroupLimited
5% Other (2)

43% Android 8
41% Android 9
12% Android 10
2% Android 6
2% Other (1)

14% In background
0% Rooted
100% Proximity on
Free space 21.36 GB
Free RAM 1.46 GB

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

10 participants