"mmap() failed: Cannot allocate memory size: 61865984 offset: 0"

[petehoch]

!!! MANDATORY TO FILL OUT !!!

Goals

I want to stop having all of these version of the db in the realm file. We just turned off encryption thinking that would help, but it did not.

I have been reading everything I can find, but I find contradictory messages. For example, we have wrapped every use of Realm in an autoreleasepool {} but then I saw a comment (on one issue, sorry lost the ref) that said NOT to use autoreleasepools on background threads with Realm.

Expected Results

I expect Realm to not crash after 12 hours of continuous use.

Actual Results

#0 0x00000001b0e28194 in swift_runtime_on_report ()
#1 0x00000001b0e7e738 in swift_stdlib_reportFatalErrorInFile ()
#2 0x00000001b0bc7d20 in specialized String.withUnsafeBufferPointerToUTF8(:) ()
#3 0x00000001b0da3b98 in specialized assertionFailure(::file:line🎏) ()
#4 0x00000001b0c07588 in swift_unexpectedError ()
#5 0x00000001005eddd0 in closure #2 in closure #4 in InteractionBranch.recordWifiInteraction(ssid:bssid:interval:) at /Users/hoch/Dev/GitHub/covid19-ios/NoCov/Persistence/Realm/Branches/InteractionBranch.swift:248
#6 0x000000010037f5c4 in thunk for @callee_guaranteed () -> (@error @owned Error) ()
#7 0x00000001005f0a6c in thunk for @callee_guaranteed () -> (@error @owned Error)partial apply ()
#8 0x00000001b111bcb0 in autoreleasepool(invoking:) ()
#9 0x00000001005ed6e8 in closure #4 in InteractionBranch.recordWifiInteraction(ssid:bssid:interval:) at /Users/hoch/Dev/GitHub/covid19-ios/NoCov/Persistence/Realm/Branches/InteractionBranch.swift:246
#10 0x00000001004a415c in closure #1 in API.postWifi(:callback:) at /Users/hoch/Dev/GitHub/covid19-ios/NoCov/Persistence/API/API.swift:970
#11 0x0000000100487280 in thunk for @escaping @callee_guaranteed (@guaranteed Data?, @guaranteed NSURLResponse?, @guaranteed Error?) -> () ()
#12 0x0000000183a00688 in **75-[**NSURLSessionLocal taskForClass:request:uploadFile:bodyData:completion:]_block_invoke ()
#13 0x0000000183a14220 in **49-[**NSCFLocalSessionTask _task_onqueue_didFinish]_block_invoke ()
#14 0x0000000183e90cf4 in NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK ()
#15 0x0000000183d9cd90 in -[NSBlockOperation main] ()
#16 0x0000000183d9c278 in -[__NSOperationInternal _start:] ()
#17 0x0000000183e92a78 in __NSOQSchedule_f ()
#18 0x000000010461f6f4 in _dispatch_call_block_and_release ()
#19 0x0000000104620c78 in _dispatch_client_callout ()
#20 0x0000000104623ffc in _dispatch_continuation_pop ()
#21 0x0000000104623458 in _dispatch_async_redirect_invoke ()
#22 0x0000000104631dc8 in _dispatch_root_queue_drain ()
#23 0x00000001046327ac in _dispatch_worker_thread2 ()
#24 0x000000018304a1b4 in _pthread_wqthread ()
#25 0x000000018304ccd4 in start_wqthread ()
Enqueued from com.apple.NSURLSession-work (Thread 3578) Queue : com.apple.NSURLSession-work (serial)
#0 0x0000000104625a1c in dispatch_async ()
#1 0x0000000183e90434 in __NSOQSchedule ()
#2 0x0000000183e91860 in __addOperations ()
#3 0x00000001839d78c4 in -[__NSCFLocalSessionTask _task_onqueue_didFinish] ()
#4 0x00000001839d7530 in -[__NSCFLocalSessionTask connection:didFinishLoadingWithError:] ()
#5 0x0000000183a8d9c8 in **51-[**NSCFURLLocalSessionConnection _task_sendFinish]_block_invoke ()
#6 0x00000001839d6a1c in -[__NSCFURLLocalSessionConnection _task_sendFinish] ()
#7 0x000000010462ee90 in _dispatch_block_async_invoke2 ()
#8 0x0000000104620c78 in _dispatch_client_callout ()
#9 0x0000000104628bf4 in _dispatch_lane_serial_drain ()
#10 0x00000001046298b4 in _dispatch_lane_invoke ()
#11 0x000000010463377c in _dispatch_workloop_worker_thread ()
#12 0x000000018304a114 in _pthread_wqthread ()
#13 0x000000018304ccd4 in start_wqthread ()
Enqueued from com.apple.NSURLSession-work (Thread 3578) Queue : com.apple.NSURLSession-work (serial)
#0 0x0000000104625a1c in dispatch_async ()
#1 0x00000001839d2d7c in -[__NSCFURLSessionConnection withWorkQueueAsync:] ()
#2 0x00000001839d3750 in -[__NSCFURLLocalSessionConnection afterDelegateWithTick:] ()
#3 0x0000000183a8db08 in **50-[**NSCFURLLocalSessionConnection _tick_finishing]_block_invoke_2 ()
#4 0x00000001839e266c in -[__NSCFLocalSessionTask connection:willCacheResponse:responseCallback:] ()
#5 0x0000000183a8da9c in **50-[**NSCFURLLocalSessionConnection _tick_finishing]_block_invoke ()
#6 0x00000001839d8b54 in -[__NSCFURLLocalSessionConnection _tick_finishing] ()
#7 0x000000010462ee90 in _dispatch_block_async_invoke2 ()
#8 0x0000000104620c78 in _dispatch_client_callout ()
#9 0x0000000104628bf4 in _dispatch_lane_serial_drain ()
#10 0x00000001046298b4 in _dispatch_lane_invoke ()
#11 0x000000010463377c in _dispatch_workloop_worker_thread ()
#12 0x000000018304a114 in _pthread_wqthread ()
#13 0x000000018304ccd4 in start_wqthread ()
Enqueued from com.apple.NSURLSession-work (Thread 3578) Queue : com.apple.NSURLSession-work (serial)
#0 0x0000000104625a1c in dispatch_async ()
#1 0x00000001839d2d7c in -[__NSCFURLSessionConnection withWorkQueueAsync:] ()
#2 0x00000001839d3750 in -[__NSCFURLLocalSessionConnection afterDelegateWithTick:] ()
#3 0x0000000183a8d8f8 in **48-[**NSCFURLLocalSessionConnection _tick_running]_block_invoke_2 ()
#4 0x00000001839d26e8 in -[__NSCFLocalSessionTask connection:didReceiveData:completion:] ()
#5 0x0000000183a8d844 in **48-[**NSCFURLLocalSessionConnection _tick_running]_block_invoke ()
#6 0x00000001839d2470 in -[__NSCFURLLocalSessionConnection _tick_running] ()
#7 0x000000010462ee90 in _dispatch_block_async_invoke2 ()
#8 0x0000000104620c78 in _dispatch_client_callout ()
#9 0x0000000104628bf4 in _dispatch_lane_serial_drain ()
#10 0x00000001046298b4 in _dispatch_lane_invoke ()
#11 0x000000010463377c in _dispatch_workloop_worker_thread ()
#12 0x000000018304a114 in _pthread_wqthread ()
#13 0x000000018304ccd4 in start_wqthread ()
Enqueued from com.apple.NSURLSession-work (Thread 3578) Queue : com.apple.NSURLSession-work (serial)
#0 0x0000000104625a1c in dispatch_async ()
#1 0x00000001839d2d7c in -[__NSCFURLSessionConnection withWorkQueueAsync:] ()
#2 0x00000001839d3750 in -[__NSCFURLLocalSessionConnection afterDelegateWithTick:] ()
#3 0x0000000183a8dccc in **61-[**NSCFURLLocalSessionConnection _didReceiveResponse:sniff:]_block_invoke_2 ()
#4 0x0000000183a15004 in **67-[**NSCFLocalSessionTask connection:didReceiveResponse:completion:]_block_invoke_2 ()
#5 0x00000001839d683c in -[__NSCFLocalDataTask _onqueue_didReceiveResponse:completion:] ()
#6 0x0000000183a14fac in *67-[NSCFLocalSessionTask connection:didReceiveResponse:completion:]_block_invoke ()
#7 0x00000001839d63f4 in -[__NSCFLocalSessionTask connection:didReceiveResponse:completion:] ()
#8 0x0000000183a8dc1c in 61-[NSCFURLLocalSessionConnection _didReceiveResponse:sniff:]_block_invoke ()
#9 0x00000001839d5ae4 in -[__NSCFURLLocalSessionConnection _didReceiveResponse:sniff:] ()
#10 0x0000000183b588bc in ___ZN19URLConnectionLoader26protocolDidReceiveResponseEP14_CFURLResponse_block_invoke ()
#11 0x0000000183a8fed0 in ___ZN25SessionConnectionLoadable21withLoaderClientAsyncEU13block_pointerFvP21LoaderClientInterfaceE_block_invoke ()
#12 0x000000010462ee90 in _dispatch_block_async_invoke2 ()
#13 0x0000000104620c78 in _dispatch_client_callout ()
#14 0x0000000104628bf4 in _dispatch_lane_serial_drain ()
#15 0x00000001046298b4 in _dispatch_lane_invoke ()
#16 0x000000010463377c in _dispatch_workloop_worker_thread ()
#17 0x000000018304a114 in _pthread_wqthread ()
#18 0x000000018304ccd4 in start_wqthread ()
Enqueued from com.apple.NSURLSession-work (Thread 3578) Queue : com.apple.NSURLSession-work (serial)
#0 0x0000000104625a1c in dispatch_async ()
#1 0x00000001839d2d7c in -[__NSCFURLSessionConnection withWorkQueueAsync:] ()
#2 0x0000000183a8f5b8 in SessionConnectionLoadable::withLoaderClientAsync(void (LoaderClientInterface) block_pointer) ()
#3 0x0000000183b586e4 in URLConnectionLoader::protocolDidReceiveResponse(_CFURLResponse) ()
#4 0x0000000183b5dc28 in ___ZN27URLConnectionLoader_Classic26protocolDidReceiveResponseEP14_CFURLResponse_block_invoke ()
#5 0x0000000183b4cc08 in ___ZNK25URLConnectionInstanceData18withWorkQueueAsyncEU13block_pointerFvvE_block_invoke ()
#6 0x000000010461f6f4 in _dispatch_call_block_and_release ()
#7 0x0000000104620c78 in _dispatch_client_callout ()
#8 0x0000000104628bf4 in _dispatch_lane_serial_drain ()
#9 0x00000001046298b4 in _dispatch_lane_invoke ()
#10 0x000000010463377c in _dispatch_workloop_worker_thread ()
#11 0x000000018304a114 in _pthread_wqthread ()
#12 0x000000018304ccd4 in start_wqthread ()
Enqueued from com.apple.CFNetwork.LoaderQ (Thread 3578) Queue : com.apple.CFNetwork.LoaderQ (serial)
#0 0x0000000104625a1c in dispatch_async ()
#1 0x0000000183b4cbd8 in URLConnectionInstanceData::withWorkQueueAsync(void () block_pointer) const ()
#2 0x0000000183b5dbfc in URLConnectionLoader_Classic::protocolDidReceiveResponse(_CFURLResponse) ()
#3 0x0000000183b05794 in HTTPProtocol::checkAndSendDidReceiveResponse() ()
#4 0x0000000183b0b258 in HTTPProtocol::performHeaderReadPostProcessing(__CFHTTPMessage, unsigned char) ()
#5 0x0000000183b06894 in HTTPProtocol::performHeaderRead(__CFHTTPMessage) ()
#6 0x0000000183b06274 in HTTPProtocol::handleStreamEvent(CFHTTPMessage, dispatch_data_s, CFStreamError const) ()
#7 0x0000000183b33084 in HTTP2Stream::_onschedulingset_notifyDataAvailable(dispatch_data_s) ()
#8 0x0000000183b33018 in ___ZN11HTTP2Stream28_onqueue_notifyDataAvailableEv_block_invoke ()
#9 0x0000000183c25e0c in ___ZNK18QCoreSchedulingSet12performAsyncEU13block_pointerFvvE_block_invoke ()
#10 0x000000010461f6f4 in _dispatch_call_block_and_release ()
#11 0x0000000104620c78 in _dispatch_client_callout ()
#12 0x0000000104628bf4 in _dispatch_lane_serial_drain ()
#13 0x00000001046298ec in _dispatch_lane_invoke ()
#14 0x000000010462af68 in _dispatch_workloop_invoke ()
#15 0x000000010463377c in _dispatch_workloop_worker_thread ()
#16 0x000000018304a114 in _pthread_wqthread ()
#17 0x000000018304ccd4 in start_wqthread ()

Code at that point:

        autoreleasepool {
            let realm = try! Realm(configuration: self.config)
            try! realm.write {
                realm.add(interaction, update: .modified)
            }
        }

Steps for others to Reproduce

Code Sample

        autoreleasepool {
            let realm = try! Realm(configuration: self.config)
            try! realm.write {
                realm.add(interaction, update: .modified)
            }
        }

Version of Realm and Tooling

Realm framework version: pod 'RealmSwift'
Realm Object Server version: whatever the pod gives us

Xcode version: Version 11.6 (11E708

iOS/OSX version: Happening on iOS 12.x & 13.x

Dependency manager + version: No idea what this is.

Look, I realize that this crash is the symptom and not the cause, but we have made sure to wrap every use of Realm in

autorelease {
}

We also have set up shouldCompactOnLaunch, and it gets called multiple times, for a while, but then it stops and so we get pinned versions in the file and what should be a file of 0.5 MB tops grows to 65 GB and then we crash. I have a bad DB file and the lock file, but I couldn't find anything to tell me how to open and read the lock file.

Can we get better tools to help users figure out where the real bug is?

[lasseporsch]

We see the same behavior in our App. Just like @petehoch we wrapped every API call to Realm from background threads in autoreleasepools. However, it is just a matter of time when Realm crashes with "mmap() failed: Cannot allocate memory size: 61865984 offset: 0". We experience this behavior especially after a fresh install of our app when a lot of data is fetched from the cloud and stored in Realm (we do not use RealmCloud or Object Server, we just manually download and write to Realm).

Things we tried but did not solve the problem were:

• Called Realm.refresh() before, after or before AND after background writes
• Reducing the number of writes by writing data in bulk. This just drags out the inevitable, but doesn't prevent the crash.
• Serialize background writes via a dedicated DispatchQueue

We have been seeing this behavior in practically all 5.* versions, up to and including 5.3.3.

[ZComwiz]

@lasseporsch Any chance we can roll back to the good old days of 4.x? What are you doing to solve it in the field/production?

[lasseporsch]

@lasseporsch Any chance we can roll back to the good old days of 4.x? What are you doing to solve it in the field/production?

@ZComwiz In our case we decided to 'downgrade' to 4.4.1. This however works for us only because our currently released app version is on an even older Realm version.

[johyunmin88]

@realm-support , @RealmBot Is there no other way but to downgrade the version? Aren't you going to designate someone in charge? Please tell me how to solve it.

[jsflax]

We have been talking internally about a tool for reverse migrations, but nothing is set in stone yet. We'll keep the community informed if there is any progress there.

In the meantime, could you try upgrading to 5.4.4?

[johyunmin88]

@jsflax Okay, Thank you. I hope this is resolved soon.

[finnschiermer]

If you can distill this into a repro-case based on the latest release, we'd be very happy to investigate in detail.

[johyunmin88]

@finnschiermer Occurs when writing to the DB continuously. We keep saving data in DB as an application that takes notes. In the process, the DB expands and dies.
It continues to occur even after upgrading to Realm 5.4.4. Users complain of inconvenience. How can I submit a reproducible case?

[leemaguire]

@johyunmin88 you can send an app which reproduces your issue to lee.maguire@mongodb.com Thanks

[johyunmin88]

@leemaguire I can't give you an example of our company's project. Our error is only possible if we have a Neo Smart Pen. Do I just need to give you an error log?

[leemaguire]

@johyunmin88 If you could give us a stack trace & some context of how you are using Realm (your setup, how you're writing to it, etc.)

[johyunmin88]

@leemaguire I left a question by email. Please check. Thank you.

[leemaguire]

Closing due to inactivity. If you have any other questions please open a new issue. Thanks.

[iwwwh]

@johyunmin88 @lasseporsch

Hello guys,

Please tell me if you could find any solution? Faced the same problem.

Thanks!

[FitzAfful]

Did you guys manage to fix it? How did you do it? Im currently facing the same problems with version 10.15.1

[bmunkholm]

@igorvoytovich @FitzAfful Please post new issues with all your details, and feel free to reference this one. "mmap() failed" can happen for multiple reasons - one being you are actually just out of memory... So it's important to look at the details of your particular issue.

[Hartistic]

We're having the same issue too

[telipskiy]

the same issue too

[olegrussu1]

having the same issue. All creations of Realm in non-main threads are enclosured in autoreleasepool { }
But in my case offset is not 0:

Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=io.realm Code=9 "mmap() failed: Cannot allocate memory size: 29081600 offset: 67108864" UserInfo={NSLocalizedDescription=mmap() failed: Cannot allocate memory size: 29081600 offset: 67108864, Error Code=9}

[theoks]

We are having the same issue with version 10.26.0

A sample error is something like this:

Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=io.realm Code=9 "mmap() failed: Cannot allocate memory size: 55672832 offset: 335544320" UserInfo={NSLocalizedDescription=mmap() failed: Cannot allocate memory size: 55672832 offset: 335544320, Error Code=9}

Any ideas why this is happening?

[mr-grey]

the same issue realm swift 10.28.0

realm::AddressSpaceExhausted: mmap() failed: Cannot allocate memory size: 64520192 offset: 201326592

[saad-arrivy]

Experiencing the same issue.
Tried all workarounds including the :

• refreshing realm after backgroundwrites
• updating realm version upto 5.3.3

Couldn't reproduce it. Got such error from production logs..

Please provide a rectification for the issue.

[bmunkholm]

@saad-arrivy @mr-grey @theoks @olegrussu1 @telipskiy @emtjoshhart Please see #6690 (comment) above.
I'll lock this issue and kindly request new observations to be created in separate issues. Thanks!