Illegal Argument Exception encountered when compressing a Realm file created by version 0.83.0 #1581

creativepsyco · 2015-10-12T09:14:55Z

Hi peeps,

I am currently running into an IllegalArgumentException indicating that the format of the Realm file is invalid when I am trying to call Realm.compactRealm(realmConfiguration...) before I use the realm file.

I have a sample project to demonstrate the crash here: https://github.com/creativepsyco/sample-realm-crash/

Essentially I am trying to compact the realm file before it is ever used in the app.

Steps to reproduce:

Clone the project from https://github.com/creativepsyco/sample-realm-crash/
Compile and run. It will not crash because realm version is 0.82.2
Now in build.gradle edit the version to 0.83.0
Compile and run the app.
It crashes with the stack trace:

 FATAL EXCEPTION: main
 Process: mohitkanwal.com.examplerealm, PID: 28687
 java.lang.RuntimeException: Unable to start activity
ComponentInfo{mohitkanwal.com.examplerealm/mohitkanwal.com.examplerealm.MainActivity}:\
java.lang.IllegalArgumentException: Illegal Argument: Invalid format of Realm file.
     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3119)
     at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3218)
     at android.app.ActivityThread.access$1000(ActivityThread.java:198)
     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1676)
     at android.os.Handler.dispatchMessage(Handler.java:102)
     at android.os.Looper.loop(Looper.java:145)
     at android.app.ActivityThread.main(ActivityThread.java:6837)
     at java.lang.reflect.Method.invoke(Native Method)
     at java.lang.reflect.Method.invoke(Method.java:372)
     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1404)
     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1199)
  Caused by: java.lang.IllegalArgumentException: Illegal Argument: Invalid format of Realm file.
     at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(Native Method)
     at io.realm.internal.SharedGroup.<init>(SharedGroup.java:68)
     at io.realm.internal.SharedGroupManager.<init>(SharedGroupManager.java:49)
     at io.realm.BaseRealm.<init>(BaseRealm.java:87)
     at io.realm.Realm.<init>(Realm.java:151)
     at io.realm.Realm.createAndValidate(Realm.java:279)
     at io.realm.Realm.create(Realm.java:247)
     at io.realm.Realm.getInstance(Realm.java:219)
     at mohitkanwal.com.examplerealm.ContactHelper.getMeARealmFile(ContactHelper.java:53)
     at mohitkanwal.com.examplerealm.ContactHelper.getAllContacts(ContactHelper.java:64)
     at mohitkanwal.com.examplerealm.MainActivity.onCreate(MainActivity.java:25)
     at android.app.Activity.performCreate(Activity.java:6500)
     at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1120)
     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3072)
     at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3218)

Is the SharedGroupManager.compact() implementation compatible with the new format of the Realm File? Right now it seems to be unusable with any Realm created by v0.83 or v0.82.2

Regards
Mohit

The text was updated successfully, but these errors were encountered:

dalinaum · 2015-10-13T02:37:25Z

Hello @creativepsyco I can reproduce this probem too. We are investigating this.

creativepsyco · 2015-10-13T02:57:20Z

👍 Sounds good.

kneth · 2015-10-13T05:45:50Z

@creativepsyco When you don't call Realm.compactRealm() everything is fine, right? We are currently investigating the underlying method in the storage engine. My recommendation is to disable compaction. I hope we will be able to release a patch to fix it.

creativepsyco · 2015-10-13T07:43:10Z

Yes everything is fine if I don't call Realm.compactRealm(). I suppose some format related issues with the core storage engine. Thanks for the prompt reply 👍

cmelchior · 2015-10-15T18:55:10Z

0.83.1 was released today with a fix for this.

creativepsyco · 2015-10-16T03:27:39Z

👍 Verified and it works.

erichkleung · 2015-12-16T19:28:18Z

Hi, I'm running into this problem with 0.86.0. Old Realm was dropped and new one was created on this version.


Fatal Exception: java.lang.RuntimeException: Unable to create application com.company.myapp.app: java.lang.IllegalArgumentException: Illegal Argument: Invalid format of Realm file.
       at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5116)
       at android.app.ActivityThread.access$1600(ActivityThread.java:177)
       at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1509)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:145)
       at android.app.ActivityThread.main(ActivityThread.java:5942)
       at java.lang.reflect.Method.invoke(Method.java)
       at java.lang.reflect.Method.invoke(Method.java:372)
       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1399)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1194)
Caused by java.lang.IllegalArgumentException: Illegal Argument: Invalid format of Realm file.
       at io.realm.internal.SharedGroup.nativeCreate(SharedGroup.java)
       at io.realm.internal.SharedGroup.<init>(SharedGroup.java:71)
       at io.realm.internal.SharedGroupManager.compact(SharedGroupManager.java:174)
       at io.realm.BaseRealm.compactRealm(BaseRealm.java:547)
       at io.realm.Realm.compactRealm(Realm.java:1095)
       at com.company.myapp.app.checkForMigration(app.java:122)
       at com.company.myapp.app.onCreate(app.java:91)
       at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1020)
       at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5113)
       at android.app.ActivityThread.access$1600(ActivityThread.java:177)
       at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1509)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:145)
       at android.app.ActivityThread.main(ActivityThread.java:5942)
       at java.lang.reflect.Method.invoke(Method.java)
       at java.lang.reflect.Method.invoke(Method.java:372)
       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1399)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1194)

kneth · 2015-12-17T08:57:47Z

The Invalid format of Realm file. message indicate that the Realm file is damaged in some way. Can you elaborate a bit on your setup?

erichkleung · 2015-12-17T09:01:58Z

We had a previous version of Realm installed, and decided it would be easier to drop the old Realm and create a new one.

This is what we do immediately prior to attempting to compact Realm.

        RealmConfiguration oldRealm = new RealmConfiguration.Builder(this)
                .name("default.realm")
                .build();

        boolean needToRebuildRealm = false;
        File f = new File(getFilesDir().getAbsolutePath(), "default.realm");
        if (f.exists()) {
            FSLog.log("old realm exists, will be rebuilding");
            needToRebuildRealm = true;
        }

        Realm.deleteRealm(oldRealm);

        RealmConfiguration.Builder builder = new RealmConfiguration.Builder(this)
                .name("cache")
                .migration(new MigrationToLatest())
                .schemaVersion(MigrationToLatest.LATEST_VERSION);

        RealmConfiguration realmConfiguration = builder.build();
        Realm.setDefaultConfiguration(realmConfiguration);

I'm not really sure how I can help elaborate on the setup, what exactly are you looking for?

kneth · 2015-12-17T09:08:58Z

The error happens when you call compactRealm(). How do you call that method?

erichkleung · 2015-12-17T09:10:45Z

Oh, immediately below the block above with this:

Realm.compactRealm(realmConfiguration);

kneth · 2015-12-17T12:16:36Z

@erichkleung I have tried to reproduce your case without luck. Please take a look at #1958 and tell me if my test differs from your code.

LiuDeng · 2016-02-11T02:02:59Z

@kneth
i still get this problem in io.realm:realm-android:0.87.4 if i do not use compactRealm the realm file increase fast
when will you fix this！

cmelchior · 2016-02-11T06:21:17Z

Hi @LiuDeng. This issue is unrelated to any file size increase. Please create a new issue with all relevant information instead.

LiuDeng · 2016-02-11T06:29:06Z

@cmelchior i mean if i do not use compactRealm the realm size increase fast

dalinaum · 2016-02-12T10:51:51Z

Hello, @LiuDeng. Could you create another issue for your problem? It would be helpful to handle your problem.

puneetagarwal · 2016-04-11T15:14:59Z

Hello,
I am not using this method Realm.compactRealm() but still I am facing the same crash here.

Caused by: java.lang.IllegalArgumentException: Illegal Argument: Invalid format of Realm file.
at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(Native Method)
at io.realm.internal.SharedGroup.(Unknown Source)
at io.realm.internal.SharedGroupManager.(Unknown Source)
at io.realm.BaseRealm.(Unknown Source)
at io.realm.Realm.(Unknown Source)
at io.realm.Realm.createAndValidate(Unknown Source)
at io.realm.Realm.create(Unknown Source)
at io.realm.Realm.getDefaultInstance(Unknown Source)

I am using Realm 0.85.

Can you please help me with this. Thanks.

kneth · 2016-04-12T07:20:31Z

@puneetagarwal Many things can cause this exception. For example, if you have an encrypted Realm file, and open it with the wrong key. Can you share your configuration?

puneetagarwal · 2016-04-12T09:34:31Z

Hi Kneth,
Here is the code for my realm configuration:-

`private void configureRealm() {
byte[] key = null;

    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.JELLY_BEAN_MR2) {
        SecurePreferences securePreferences = new SecurePreferences(this);

        if (!securePreferences.contains("DBAlias")) {
            // generate a key
            key = new byte[64];
            new SecureRandom().nextBytes(key);

            // store the key in the encrypted shared prefs
            securePreferences.edit().putString("DBAlias", Base64.encodeToString(key,  Base64.DEFAULT)).apply();
        } else {
            String keyString = securePreferences.getString("DBAlias", null);
            key = Base64.decode(keyString, Base64.DEFAULT);
        }
    } else {
        try {
            KeyStore keyStore = KeyStore.getInstance("AndroidKeyStore");
            keyStore.load(null);

            List<String> keyAliases = new ArrayList<>();
            Enumeration<String> aliases = keyStore.aliases();
            while (aliases.hasMoreElements()) {
                keyAliases.add(aliases.nextElement());
            }

            SharedPreferences sharedPreferences = getSharedPreferences("DBPrefs", Context.MODE_PRIVATE);

            if (!keyStore.containsAlias("DBAlias")) {
                // create a keystore alias
                Calendar start = Calendar.getInstance();
                Calendar end = Calendar.getInstance();
                end.add(Calendar.YEAR, 1);
                KeyPairGeneratorSpec spec = new KeyPairGeneratorSpec.Builder(this)
                        .setAlias("DBAlias")
                        .setSubject(new X500Principal("CN=Sample Name, O=Android Authority"))
                        .setSerialNumber(BigInteger.ONE)
                        .setStartDate(start.getTime())
                        .setEndDate(end.getTime())
                        .build();
                KeyPairGenerator generator = KeyPairGenerator.getInstance("RSA", "AndroidKeyStore");
                generator.initialize(spec);

                KeyPair keyPair = generator.generateKeyPair();
                RSAPublicKey publicKey = (RSAPublicKey) keyPair.getPublic();

                // generate a key
                key = new byte[64];
                new SecureRandom().nextBytes(key);

                // encrypt the new key
                Cipher input = Cipher.getInstance("RSA/ECB/PKCS1Padding", "AndroidOpenSSL");
                input.init(Cipher.ENCRYPT_MODE, publicKey);

                ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
                CipherOutputStream cipherOutputStream = new CipherOutputStream(outputStream, input);
                cipherOutputStream.write(key);
                cipherOutputStream.close();

                // store the encrypted key in the shared prefs
                sharedPreferences.edit().putString("DBAlias", Base64.encodeToString(outputStream.toByteArray(), Base64.DEFAULT)).apply();
            } else {
                String encryptedString = sharedPreferences.getString("DBAlias", null);

                if (encryptedString != null) {
                    KeyStore.PrivateKeyEntry privateKeyEntry = (KeyStore.PrivateKeyEntry) keyStore.getEntry("DBAlias", null);

                    Cipher output = Cipher.getInstance("RSA/ECB/PKCS1Padding");
                    output.init(Cipher.DECRYPT_MODE, privateKeyEntry.getPrivateKey());

                    CipherInputStream cipherInputStream = new CipherInputStream(new ByteArrayInputStream(Base64.decode(encryptedString, Base64.DEFAULT)), output);
                    List<Byte> values = new ArrayList<>();
                    int nextByte;
                    while ((nextByte = cipherInputStream.read()) != -1) {
                        values.add((byte) nextByte);
                    }

                    key = new byte[values.size()];
                    for (int i = 0; i < key.length; i++) {
                        key[i] = values.get(i);
                    }
                }
            }

        } catch (NoSuchAlgorithmException |
                KeyStoreException |
                CertificateException |
                IOException |
                NoSuchProviderException |
                InvalidAlgorithmParameterException |
                InvalidKeyException |
                UnrecoverableEntryException |
                NoSuchPaddingException e) {
            e.printStackTrace();
        }
    }

    RealmConfiguration config = null;
    if (key != null) {
        config = new RealmConfiguration.Builder(getApplicationContext())
                .encryptionKey(key)
                .schemaVersion(AppRealmMigration.REALM_VERSION) // Must be bumped when the schema changes
                .deleteRealmIfMigrationNeeded()
                .migration(new AppRealmMigration()) // Migration to run
                .build();
    } else {
        config = new RealmConfiguration.Builder(getApplicationContext())
                .schemaVersion(AppRealmMigration.REALM_VERSION) // Must be bumped when the schema changes
                .deleteRealmIfMigrationNeeded()
                .migration(new AppRealmMigration()) // Migration to run
                .build();
    }

    Realm.setDefaultConfiguration(config);
}`

I just added deleteRealmIfMigrationNeeded () method to resolve this issue in case if there is any issue with the DB migration. Does it helped you to understand if I am doing something wrong.
Thanks.

kneth · 2016-04-12T12:49:42Z

deleteRealmIfMigrationIsNeeded will delete your Realm file, and all (Realm) data will be wiped.

If you can reproduce it on your local devices, it might be useful to log the key to see if you get the exact key every time.

puneetagarwal · 2016-04-13T11:26:39Z

I tried to reproduce this issue at our end and it's not crashing but it crashes in the play store version only on some devices.

Is this possible that if I use deleteRealmIfMigrationIsNeeded() and migration() methods at the same time while creating Realm configuration, it throws this exception "IllegalArgumentException: Illegal Argument: Invalid format of Realm file."

Also, if I add name() method to create a realm file, will it create a new Realm file with that name or it just renames the previous one. As I am not using name() method right now and Realm creates a file with a default name.

Hope to hear from you soon. Thanks.

dalinaum added T-Help T-Bug and removed T-Help labels Oct 12, 2015

cmelchior added the P1 label Oct 13, 2015

kneth mentioned this issue Oct 13, 2015

Compaction results in corrupt file realm/realm-core#1216

Closed

cmelchior closed this as completed Oct 15, 2015

cmelchior removed the P1 label Oct 15, 2015

zaki50 mentioned this issue Oct 18, 2015

Add test to open Realm database after compactRealm #1613

Merged

kneth added a commit that referenced this issue Dec 17, 2015

Unit test to match #1581

0587c95

github-actions bot locked as resolved and limited conversation to collaborators Mar 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Illegal Argument Exception encountered when compressing a Realm file created by version 0.83.0 #1581

Illegal Argument Exception encountered when compressing a Realm file created by version 0.83.0 #1581

creativepsyco commented Oct 12, 2015

dalinaum commented Oct 13, 2015

creativepsyco commented Oct 13, 2015

kneth commented Oct 13, 2015

creativepsyco commented Oct 13, 2015

cmelchior commented Oct 15, 2015

creativepsyco commented Oct 16, 2015

erichkleung commented Dec 16, 2015

kneth commented Dec 17, 2015

erichkleung commented Dec 17, 2015

kneth commented Dec 17, 2015

erichkleung commented Dec 17, 2015

kneth commented Dec 17, 2015

LiuDeng commented Feb 11, 2016

cmelchior commented Feb 11, 2016

LiuDeng commented Feb 11, 2016

dalinaum commented Feb 12, 2016

puneetagarwal commented Apr 11, 2016

kneth commented Apr 12, 2016

puneetagarwal commented Apr 12, 2016

kneth commented Apr 12, 2016

puneetagarwal commented Apr 13, 2016

Illegal Argument Exception encountered when compressing a Realm file created by version 0.83.0 #1581

Illegal Argument Exception encountered when compressing a Realm file created by version 0.83.0 #1581

Comments

creativepsyco commented Oct 12, 2015

dalinaum commented Oct 13, 2015

creativepsyco commented Oct 13, 2015

kneth commented Oct 13, 2015

creativepsyco commented Oct 13, 2015

cmelchior commented Oct 15, 2015

creativepsyco commented Oct 16, 2015

erichkleung commented Dec 16, 2015

kneth commented Dec 17, 2015

erichkleung commented Dec 17, 2015

kneth commented Dec 17, 2015

erichkleung commented Dec 17, 2015

kneth commented Dec 17, 2015

LiuDeng commented Feb 11, 2016

cmelchior commented Feb 11, 2016

LiuDeng commented Feb 11, 2016

dalinaum commented Feb 12, 2016

puneetagarwal commented Apr 11, 2016

kneth commented Apr 12, 2016

puneetagarwal commented Apr 12, 2016

kneth commented Apr 12, 2016

puneetagarwal commented Apr 13, 2016