ANR and Segfault during large transaction #1080

dshen6 · 2015-05-01T16:59:26Z

Hey, keep it up Realm team! Awesome work.

I've got 8k update operations running in the same transaction in a background service, which leads to ANRs and a Segfault. I broke up it up into 200 operation chunks, which doesn't ANR/crash, but this seems like a bandaid fix. Any advice for how to handle this?

emanuelez · 2015-05-01T17:10:16Z

Hello,

the funny thing about Services is that most times they don't run on a separate thread, and that's most likely why you see the ANRs. I would suggest using a worker thread as you can see in the examples in our repo.

dshen6 · 2015-05-01T23:00:11Z

Thanks for the help! I've tried it as an Async task as well, and even an entirely separate process. To be fair, this is on a very memory-limited device, which garbage collects extremely often. Trying another process / Async task doesn't stop the garbage collection from ANR'ing the app.

emanuelez · 2015-05-04T08:10:45Z

I see. So it's a matter of the GC stopping the world in order to be able to keep up.
Can you show what kind of operations your transactions contain?

One known problem is that when iterating through RealmResults we are creating objects so fast that the GC is not able to keep up. We saw this kind of issue only internally so far, and we already have a fix for it, but it comes at a performance cost, which is the reason it has not been merged yet.

Seeing the code of your transaction can help us evaluate it.

dshen6 · 2015-05-04T19:55:07Z

Sure!

So the user data comes in as a 4.5 MB String. I use JacksonParser to handle the converting of this huge String into JSONNodes, instead of the built-in JSONObject, because I'll frequently run into OutOfMemory Exceptions if I go the JSONObject route. In a for loop, I run this createOrUpdate function 8k times.

public static User createOrUpdate(Realm realm, JsonNode data) {
    UserDB userDB = realm.where(UserDB.class).equalTo("_id", data.get("_id").getTextValue()).findFirst();
    if (userDB == null) {
        userDB = realm.createObject(UserDB.class);
    }
    update(userDB, data);
    return new User(realm, userDB);
}

private static void update(UserDB userDB, JsonNode data) {
    if (data == null) {
        return;
    }
    JsonNode test = data.get("about");
    if (test != null) {
        userDB.setAbout(test.getTextValue());
    }
    test = data.get("avatar");
    if (test != null) {
        userDB.setAvatar(test.getTextValue());
    }
    test = data.get("coverPhoto");
    if (test != null) {
        userDB.setCoverPhoto(test.getTextValue());
    }
    test = data.get("email");
    if (test != null) {
        userDB.setEmail(test.getTextValue());
    }
    test = data.get("firstname");
    if (test != null) {
        userDB.setFirstname(test.getTextValue());
    }
    test = data.get("lastname");
    if (test != null) {
        userDB.setLastname(test.getTextValue());
    }
    test = data.get("lastActivity");
    if (test != null) {
        userDB.setLastActivity(test.getLongValue());
    }
    test = data.get("goal");
    if (test != null) {
        userDB.setGoal(test.getTextValue());
    }
    test = data.get("position");
    if (test != null) {
        userDB.setPosition(test.getTextValue());
    }
    test = data.get("status");
    if (test != null) {
        userDB.setStatus(test.getTextValue());
    }
    test = data.get("mood_sticker");
    if (test != null) {
        userDB.setMood_sticker(test.getTextValue());
    }
    test = data.get("isNetworkAdmin");
    if (test != null) {
        userDB.setNetworkAdmin(test.getBooleanValue());
    }
    test = data.get("isVerified");
    if (test != null) {
        userDB.setVerified(test.getBooleanValue());
    }
    test = data.get("level");
    if (test != null) {
        userDB.setLastActivity(test.getIntValue());
    }
}

dshen6 · 2015-05-04T21:43:52Z

Also, here's the stacktrace from ANR.

05-05 04:40:07.771 26684-26684/com.ekocustom.true A/libc﹕ Fatal signal 6 (SIGABRT) at 0x000001d4 (code=0), thread 26684 (.ekocustom.true)
05-05 04:40:07.771 468-540/? E/ActivityManager﹕ ANR in com.ekocustom.true (com.ekocustom.true/com.ekoapp.Thread.ThreadActivity)
Reason: keyDispatchingTimedOut
Load: 15.87 / 21.3 / 21.43
CPU usage from 5342ms to 132ms ago:
17% 26684/com.ekocustom.true: 16% user + 0.9% kernel / faults: 1196 minor 4 major
0.9% 107/mmcqd/0: 0% user + 0.9% kernel
0.9% 223/adbd: 0% user + 0.9% kernel / faults: 252 minor
0.9% 468/system_server: 0.7% user + 0.1% kernel / faults: 78 minor
0.5% 1073/mpdecision: 0% user + 0.5% kernel
0.3% 25999/kworker/0:1: 0% user + 0.3% kernel
0% 2/kthreadd: 0% user + 0% kernel
0.1% 3/ksoftirqd/0: 0% user + 0.1% kernel
0.1% 23/irq/53-msmdatam: 0% user + 0.1% kernel
0.1% 116/jbd2/mmcblk0p21: 0% user + 0.1% kernel
0.1% 151/surfaceflinger: 0.1% user + 0% kernel
0.1% 154/mediaserver: 0% user + 0.1% kernel
0% 173/alljoyn-daemon: 0% user + 0% kernel
0% 630/com.android.systemui: 0% user + 0% kernel
0.1% 2734/ksdioirqd/mmc2: 0% user + 0.1% kernel
0.1% 3255/kworker/u:5: 0% user + 0.1% kernel
0% 10476/jp.naver.line.android: 0% user + 0% kernel / faults: 9 minor
0% 19515/kworker/u:2: 0% user + 0% kernel
0.1% 21063/kworker/0:4: 0% user + 0.1% kernel
0.1% 26275/logcat: 0.1% user + 0% kernel
0.1% 27436/com.google.android.gms: 0.1% user + 0% kernel
32% TOTAL: 5.3% user + 1.9% kernel + 24% iowait
CPU usage from 952ms to 1480ms later:
73% 26684/com.ekocustom.true: 73% user + 0% kernel / faults: 235 minor 2 major
52% 26718/AsyncTask #2: 52% user + 0% kernel
20% 26684/.ekocustom.true: 20% user + 0% kernel
7.6% 468/system_server: 1.9% user + 5.7% kernel / faults: 11 minor
3.8% 540/InputDispatcher: 0% user + 3.8% kernel
3.7% 107/mmcqd/0: 0% user + 3.7% kernel
0.9% 3/ksoftirqd/0: 0% user + 0.9% kernel
1% 151/surfaceflinger: 1% user + 0% kernel
1% 439/SurfaceFlinger: 1% user + 0% kernel
1% 449/hwcVsyncThread: 0% user + 1% kernel
1.8% 223/adbd: 0% user + 1.8% kernel
1.3% 25999/kworker/0:1: 0% user + 1.3% kernel
40% TOTAL: 20% user + 2.9% kernel + 16% iowait
05-05 04:40:07.841 26684-26718/com.ekocustom.true D/PEOPLE_VIEW_CONTROLLER﹕ commit transaction at i 5799
05-05 04:40:07.841 26684-26718/com.ekocustom.true D/PEOPLE_VIEW_CONTROLLER﹕ begin transaction at i 5800
05-05 04:40:07.871 149-149/? I/DEBUG﹕ *** *** *** *** *** *** *** *** *** *** *** *** *** *** * ***
05-05 04:40:07.871 149-149/? I/DEBUG﹕ Build fingerprint: 'qcom/msm8625/msm8625:4.1.2/JZO54K/TBW593332_8771_V006115:user/test-keys'
05-05 04:40:07.871 149-149/? I/DEBUG﹕ pid: 26684, tid: 26684, name: .ekocustom.true >>> com.ekocustom.true <<<
05-05 04:40:07.871 149-149/? I/DEBUG﹕ signal 6 (SIGABRT), code -6 (?), fault addr --------
05-05 04:40:07.981 149-149/? I/DEBUG﹕ r0 fffffffc r1 bead9558 r2 00000010 r3 0000089d
05-05 04:40:07.981 149-149/? I/DEBUG﹕ r4 51aebe00 r5 0000089d r6 00000000 r7 000000fc
05-05 04:40:07.981 149-149/? I/DEBUG﹕ r8 00000000 r9 00000014 sl 51aebe14 fp bead96cc
05-05 04:40:07.981 149-149/? I/DEBUG﹕ ip 40379ff0 sp bead9510 lr 403755b5 pc 40188ab0 cpsr 20000010
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d0 430b3d7142d06e1e d1 0000000042d06e1e
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d2 0000000800000317 d3 531790c000000053
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d4 0000000000000000 d5 0000000000000000
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d6 4387000043870000 d7 43f00000c3f00000
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d8 3f8000003f800000 d9 43ff000041700000
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d10 4026000041880000 d11 0000000000000000
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d12 0000000000000000 d13 0000000000000000
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d14 0000000000000000 d15 0000000000000000
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d16 7fffffffffffffff d17 7fffffffffffffff
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d18 0000000000000000 d19 0701070100700798
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d20 0000000000000c07 d21 0000043f00890000
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d22 0000000000080008 d23 0000000000000008
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d24 3ff0000000000000 d25 0000000000000000
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d26 402d969680000000 d27 0000000000000000
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d28 001e001d001c001b d29 0020001f001e001c
05-05 04:40:07.981 149-149/? I/DEBUG﹕ d30 002a002a002a002a d31 0000000000000000
05-05 04:40:07.981 149-149/? I/DEBUG﹕ scr 88000017
05-05 04:40:07.991 149-149/? I/DEBUG﹕ backtrace:
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #00 pc 0000dab0 /system/lib/libc.so (epoll_wait+12)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #1 pc 000145b1 /system/lib/libutils.so (android::Looper::pollInner(int)+96)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #2 pc 00014819 /system/lib/libutils.so (android::Looper::pollOnce(int, int_, int_, void)+104)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #3 pc 0005f157 /system/lib/libandroid_runtime.so (android::NativeMessageQueue::pollOnce(JNIEnv, int)+22)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #4 pc 0001f330 /system/lib/libdvm.so (dvmPlatformInvoke+112)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #5 pc 0004e079 /system/lib/libdvm.so (dvmCallJNIMethod(unsigned int const, JValue_, Method const_, Thread_)+360)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #6 pc 000287e0 /system/lib/libdvm.so
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #7 pc 0002cfa8 /system/lib/libdvm.so (dvmInterpret(Thread_, Method const_, JValue_)+180)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #8 pc 0005f93f /system/lib/libdvm.so (dvmInvokeMethod(Object_, Method const_, ArrayObject_, ArrayObject_, ClassObject_, bool)+374)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #9 pc 000668a9 /system/lib/libdvm.so
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #10 pc 000287e0 /system/lib/libdvm.so
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #11 pc 0002cfa8 /system/lib/libdvm.so (dvmInterpret(Thread_, Method const_, JValue_)+180)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #12 pc 0005f695 /system/lib/libdvm.so (dvmCallMethodV(Thread_, Method const_, Object_, bool, JValue_, std::va_list)+272)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #13 pc 0004a6a7 /system/lib/libdvm.so
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #14 pc 00048bd9 /system/lib/libandroid_runtime.so
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #15 pc 000495f5 /system/lib/libandroid_runtime.so (android::AndroidRuntime::start(char const, char const)+368)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #16 pc 00000dcf /system/bin/app_process
05-05 04:40:07.991 149-149/? I/DEBUG﹕ stack:
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94d0 00000000
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94d4 00000000
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94d8 00000000
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94dc 512a1bf0
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94e0 00000000
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94e4 40192005 /system/lib/libc.so (free+12)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94e8 00000001
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94ec 4036fff5 /system/lib/libutils.so (android::SharedBuffer::dealloc(android::SharedBuffer const_)+10)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94f0 51aebe48
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94f4 40372e21 /system/lib/libutils.so (android::VectorImpl::_shrink(unsigned int, unsigned int)+138)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94f8 00000000
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead94fc 51aebe00
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9500 0000089d
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9504 00000000
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9508 df0027ad
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead950c 00000000
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #00 bead9510 51aebe00
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9514 00000000
05-05 04:40:07.991 149-149/? I/DEBUG﹕ #1 bead9518 00000000
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead951c 4f2d12e4 /system/lib/egl/libEGL_adreno200.so (eglTimestampWait)
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9520 4f2d9020 /system/lib/egl/libEGL_adreno200.so
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9524 00000002
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9528 00000000
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead952c 51aebe48
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9530 000025c8
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9534 000025f5
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9538 51886478
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead953c 4eeb7bf0
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9540 00000002
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9544 4f012a51 /system/lib/egl/eglsubAndroid.so
05-05 04:40:07.991 149-149/? I/DEBUG﹕ bead9548 4f2d9020 /system/lib/egl/libEGL_adreno200.so
05-05 04:40:08.001 149-149/? I/DEBUG﹕ bead954c 00000002
05-05 04:40:08.001 149-149/? I/DEBUG﹕ bead9550 001d4d3a
05-05 04:40:08.001 149-149/? I/DEBUG﹕ bead9554 00000000
05-05 04:40:08.001 149-149/? I/DEBUG﹕ ........ ........
05-05 04:40:08.001 149-149/? I/DEBUG﹕ #2 bead9680 00000000
05-05 04:40:08.001 149-149/? I/DEBUG﹕ bead9684 51ac6f08
05-05 04:40:08.001 149-149/? I/DEBUG﹕ bead9688 00000000
05-05 04:40:08.001 149-149/? I/DEBUG﹕ bead968c 41217c38 [heap]
05-05 04:40:08.001 149-149/? I/DEBUG﹕ bead9690 40339e24
05-05 04:40:08.001 149-149/? I/DEBUG﹕ bead9694 bead96b8 [stack]
05-05 04:40:08.001 149-149/? I/DEBUG﹕ bead9698 40339e1c
05-05 04:40:08.001 149-149/? I/DEBUG﹕ bead969c 405af15b /system/lib/libandroid_runtime.so (android::NativeMessageQueue::pollOnce(_JNIEnv, int)+26)

kneth · 2015-05-12T13:03:09Z

Parsing a 4.5 MB JSON file might be too large for most Android devices: http://stackoverflow.com/questions/9205821/android-parsing-5mb-json-file-causing-out-of-memory-exception

cmelchior · 2015-06-02T07:51:45Z

I assume your question was answered. Please reopen if that was not the case.

A86S · 2016-07-25T10:20:49Z

I am having the same issue while i am trying to modify DB with 13K records. It blocks the UI.
Here is my scenario :

*Using JobManager for Background operation
*Retrofit for parse JSON to realm Model
*copyToRealmOrUpdate(list_of_objects) for insert and update
*Realm Version - 1.1.0

Zhuinden · 2016-07-26T11:58:51Z

@A86S do you insert the elements within the list in one transaction?

Can you replace copyToRealmOrUpdate() with insertOrUpdate()?

A86S · 2016-07-26T12:19:36Z

@Zhuinden yes i am trying insert/update the list in one transaction. i have changed the code with insertOrUpdate() but still freeze UI with 7k records.

Zhuinden · 2016-07-26T13:06:58Z

....are you sure you're executing this on a background thread?

A86S · 2016-07-27T05:18:25Z

@Zhuinden Yes i am sure i have tested it with both AsyncTask and JobManager.

Zhuinden · 2016-07-27T06:46:00Z

You should probably consider streaming the JSON and using only 1 element at a time instead of obtaining every element in a list

It's a bit of a mess but it can reduce your 7000 objects to 1

emanuelez added the T-Help label May 1, 2015

emanuelez added the Pending label May 4, 2015

cmelchior closed this as completed Jun 2, 2015

cmelchior removed the Pending label Jun 2, 2015

A86S mentioned this issue Jul 26, 2016

findAll() and findAllAsync() get different result for same query. #3214

Closed

github-actions bot locked as resolved and limited conversation to collaborators Mar 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ANR and Segfault during large transaction #1080

ANR and Segfault during large transaction #1080

dshen6 commented May 1, 2015

emanuelez commented May 1, 2015

dshen6 commented May 1, 2015

emanuelez commented May 4, 2015

dshen6 commented May 4, 2015

dshen6 commented May 4, 2015

kneth commented May 12, 2015

cmelchior commented Jun 2, 2015

A86S commented Jul 25, 2016

Zhuinden commented Jul 26, 2016

A86S commented Jul 26, 2016

Zhuinden commented Jul 26, 2016

A86S commented Jul 27, 2016

Zhuinden commented Jul 27, 2016

ANR and Segfault during large transaction #1080

ANR and Segfault during large transaction #1080

Comments

dshen6 commented May 1, 2015

emanuelez commented May 1, 2015

dshen6 commented May 1, 2015

emanuelez commented May 4, 2015

dshen6 commented May 4, 2015

dshen6 commented May 4, 2015

kneth commented May 12, 2015

cmelchior commented Jun 2, 2015

A86S commented Jul 25, 2016

Zhuinden commented Jul 26, 2016

A86S commented Jul 26, 2016

Zhuinden commented Jul 26, 2016

A86S commented Jul 27, 2016

Zhuinden commented Jul 27, 2016