[Question] iOS 12

[joediv]

👋 Hey Matt, anything else we should be cautious of when updating to Swift 4.2? Saw this thread #20 and was successfully able to migrate to Swift 4.2.

iOS 12
Unfortunately with our existing build on the latest released version of LocoKit, we're experiencing weird/inaccurate location data on iOS 12. I saw this comment, looks like you were/are experiencing similar issues #29 (comment)

I'm seeing much higher rates of background fails in iOS 12 (in Arc's analytics, and on my test devices). And much higher rates of nonsense location data, with some devices "sleepwalking" on almost a daily basis, several kilometres away from where they really are.

Are you still seeing issues with LocoKit on iOS 12? Did Apple end up fixing some of their issues with the patch release? Anything I can potentially help with? (don't know how much time I have)

For what it's worth, I'm also experiencing quite a few crashes in the background on TimelineStore.save(). The app crashes after some time in the background. Anything potentially wrong with my setup or are you noticing the same thing? Any help is much appreciated, thanks.

0  libswiftCore.dylib             0x10143e480 _stdlib_destroyTLS + 207848
1  libswiftCore.dylib             0x101493f2c swift_unexpectedError + 280
2  LocoKit                        0x101064b10 TimelineStore.save() (TimelineStore.swift:325)
3  LocoKit                        0x101066858 closure #1 in TimelineStore.process(changes:) (TimelineStore.swift:348)
4  LocoKit                        0x100ff2a68 thunk for @escaping @callee_guaranteed () -> () (<compiler-generated>)
5  libdispatch.dylib              0x1db5c36c8 _dispatch_call_block_and_release + 24
6  libdispatch.dylib              0x1db5c4484 _dispatch_client_callout + 16
7  libdispatch.dylib              0x1db56bc18 _dispatch_lane_serial_drain$VARIANT$mp + 592
8  libdispatch.dylib              0x1db56c760 _dispatch_lane_invoke$VARIANT$mp + 432
9  libdispatch.dylib              0x1db574f00 _dispatch_workloop_worker_thread + 600
10 libsystem_pthread.dylib        0x1db7a60f0 _pthread_wqthread + 312
11 libsystem_pthread.dylib        0x1db7a8d00 start_wqthread + 4

[sobri909]

Unfortunately with our existing build on the latest released version of LocoKit, we're experiencing weird/inaccurate location data on iOS 12.

Hmm. Can you confirm whether it was the LocoKit version change, or the iOS version change that caused the differences in location data accuracy?

Basically what I've been seeing is some devices are affected by a weird "sleepwalking" effect on iOS 12, where the provided locations have horizontalAccuracy values that are orders of magnitude apart from their real accuracy levels. For example locations with horizontalAccuracy of 30 metres, but with a coordinate that's 300 metres away from the real location (and often even a kilometre or more away from the real location).

So for those affected devices, the phone effectively sleepwalks, being convinced it's going to a leisurely stroll around the neighbourhood, with high accuracy.

I've only seen it on two of my test devices so far. My main phone is the most affected, and sleepwalks almost every night. Another test device exhibits the problem about once a week. And the other test devices are unaffected. (All test devices currently are on either iOS 12.0, 12.0.1, or 12.1). The problem started with the early iOS 12 betas, and appears to remain in the iOS 12.1 betas (although I haven't done extensive testing on iOS 12.1 yet).

For what it's worth, I'm also experiencing quite a few crashes in the background on TimelineStore.save(). The app crashes after some time in the background. Anything potentially wrong with my setup or are you noticing the same thing?

[sobri909]

Ugh. I tapped the post button by accident. Let me continue that reply...

For what it's worth, I'm also experiencing quite a few crashes in the background on TimelineStore.save(). The app crashes after some time in the background. Anything potentially wrong with my setup or are you noticing the same thing?

I haven't seen that in the LocoKit Demo App at all. However I have seen it in Arc App. Arc App does a lot of fiddling with the database outside of LocoKit's managed save methods, so it risks upsetting the foreign keys in some cases. But that shouldn't be possible in plain LocoKit, as long as the timeline items aren't being manipulated and saved outside of the processing queue.

To deal with the crashes in Arc App I've been using a bit of a hack, in my TimelineStore subclass.

    override func save() {
        var savingItems: Set<TimelineItem> = []
        var savingSamples: Set<PersistentSample> = []

        mutex.sync {
            savingItems = itemsToSave.filter { $0.needsSave }
            itemsToSave.removeAll(keepingCapacity: true)

            savingSamples = samplesToSave.filter { $0.needsSave }
            samplesToSave.removeAll(keepingCapacity: true)
        }

        if !savingItems.isEmpty {
            do {
                try pool.write { db in
                    let now = Date()
                    for case let item as TimelineObject in savingItems {
                        item.transactionDate = now
                        do { try item.save(in: db) }
                        catch PersistenceError.recordNotFound { os_log("PersistenceError.recordNotFound", type: .error) }
                        catch let error as DatabaseError where error.resultCode == .SQLITE_CONSTRAINT {
                            logToFile("ERROR: \(error)")

                            Crashlytics.sharedInstance().recordError(error, withAdditionalUserInfo:
                                ["extendedResultCode": String(describing: error.extendedResultCode)])

                            // break the edges and put it back in the queue
                            logToFile("BREAKING ITEM EDGES")
                            (item as? ArcItem)?.previousItemId = nil
                            (item as? ArcItem)?.nextItemId = nil
                            save(item, immediate: false)
                            
                        } catch {
                            logToFile("ERROR: \(error)")
                            Crashlytics.sharedInstance().recordError(error)
                            save(item, immediate: false)
                        }
                    }
                    db.afterNextTransactionCommit { db in
                        for case let item as TimelineObject in savingItems where !item.hasChanges {
                            item.lastSaved = item.transactionDate
                        }
                    }
                }

            } catch {
                logToFile("ERROR: \(error)")
                Crashlytics.sharedInstance().recordError(error)
            }
        }

        if !savingSamples.isEmpty {
            do {
                try pool.write { db in
                    let now = Date()
                    for case let sample as TimelineObject in savingSamples {
                        sample.transactionDate = now
                        do { try sample.save(in: db) }
                        catch PersistenceError.recordNotFound { os_log("PersistenceError.recordNotFound", type: .error) }
                        catch {
                            logToFile("ERROR: \(error)")
                            Crashlytics.sharedInstance().recordError(error)
                            save(sample, immediate: false)
                        }
                    }
                    db.afterNextTransactionCommit { db in
                        for case let sample as TimelineObject in savingSamples where !sample.hasChanges {
                            sample.lastSaved = sample.transactionDate
                        }
                    }
                }

            } catch {
                logToFile("ERROR: \(error)")
                Crashlytics.sharedInstance().recordError(error)
            }
        }
    }

That shouldn't be necessary in plain LocoKit, but it sounds like you're perhaps running into a similar situation as Arc's, so it's worth giving that a try.

[sobri909]

I'll do some more testing with the LocoKit Demo App, to see if I can reproduce the crashes in that. And will either look into moving some portion of my hack upstream into LocoKit, or refactoring the transactions to be more foreign key safe.

It'll definitely be worth having a look at the specific SQLite error codes though, to see whether it's really a constraint failure error, or something else.

[sobri909]

Oh, for the iOS 12 sleepwalking location data, that's definitely not Arc / LocoKit specific, on my test devices. Once the phone is sleepwalking, the same drifting data is visible in Apple Maps, Google Maps, etc. You can see the location drift across the neighbourhood, with a small accuracy radius, showing that the phone is convinced it's going for a lovely stroll when really it's sitting on a table 😏

[joediv]

Well, the issue was on my end 🤦‍♂️ . I enabled data protection and the device would crash when attempting to write to SQLite in the background. Missed that the first time around, my bad. Still going to add that bit of code from Arc to avoid crashes.

Looks like we're getting sleepwalking location data on one device at least. 🤞Apple figures something out soon. Thanks for all the help

[sobri909]

Aah, yeah, data protection would do that 😂 I got excited about turning that on in Arc, until I realised it was completely impossible to use.

Looks like we're getting sleepwalking location data on one device at least. 🤞Apple figures something out soon.

Yeah, fingers crossed. Still no sign of improvement in iOS 12.0.1 or 12.1 though, which is concerning. It's probably worth filing a Radar for it, although I only ever do that in the most extreme cases, because it's such a disheartening experience.

If Apple don't solve it, I'm planning an ML based approach of auto detecting the bogus data and automatically flagging it. But it will require user input to train the ML model. So in Arc's case, it will train the model each time the user marks a segment with the "This location data is bogus" feature.

But I hope it doesn't come to needing something like that.