networkEnabled = YES has no effect when using authenticateWithAppID:token:

[martinoneutrino]

If I set networkEnabled = NO before I call authenticateWithAppID:token:, I cannot get the network to restart (and syncing to occur), by a subsequent call to networkEnabled= YES.

It appears that the code uses different paths when using tokens vs user IDs. FYI, my app is using tokens, and not prompting for usernames/passwords, and I am using the 0.6.9 release.

(The reason I am disabling the network is because I'm trying to find the cause of a separate bug where not all existing entities are being uploaded.)

Martin

[jleandroperez]

@martinoneutrino hello there,

The authentication method authenticateWithAppID:token: wasn't designed to store, in any point, the token used to initialize the library. Which means that, if, for whatever reason it doesn't succeed, the ssimperium:didFailWithError: method will get executed, and retries will not be attempted by the library itself.

May i ask you how many entities are you currently handling?. All of your legacy entities should get caught by this routine, which is in charge of detecting entities without simperiumKey or ghostData. That would be a great start to begin debugging!.

Thank you!

[martinoneutrino]

I haven't had any problems using authenticateWithAppID:token:. The authentication works and entities are synced if I haven't first called networkEnabled=NO. I will make sure I set the delegate before I authenticate, in case there is something else I'm missing.
Simperium correctly detects 11 legacy entties, each of which have somewhere between 1 and 300 instances (which the managing X count gets correct). None have simperiumKey or ghostData. (I start with a fresh database and user each time.) It detects the correct number of entities, but doesn't finish uploading a few of them, based on SPBucket's numChangesPending. It is possible that I'm just not waiting long enough, or that value isn't being updated accurately...

[jleandroperez]

@martinoneutrino please, do not use SPBucket's numChangesPending property to verify sync'ing status. That property was meant for internal usage, and may have a negative impact on performance.

In order to avoid any confusions, this property will be moved over to the SPBucket+Internals category in the next release (sorry about that!).

Regarding the sync'ing, are you observing any errors in the console?. Please, try enabling verbose logging first.

Thanks!

[martinoneutrino]

What is the best method of getting sync status and/or completion events?
Simperium has nothing.
SimperiumDelegate has only methods relating to failure and authentication.
SPBucketDelegate can tell me lots about individual objects, but can't tell me when it's done syncing.
SPBucket provides access to all its objects, but only one method relating to pending changes. (Make that zero in the next release.)

If I am performing a lengthy operation on a full dataset, I need to be able to provide feedback to the user. Even a fast sync still only does 20-30 objects/second.

Here is the only part that indicates any kind of error (I did have verbose logging enabled the first time, but it was just too much information for just trying to get an idea of the flow of updates. I'll try it again with that enabled to see if anything interesting shows up):

Simperium successfully reloaded local entity (Settings): f149a8e9d865406287e96180d14ce592.1
Simperium rebasing local changes for object (Settings): f149a8e9d865406287e96180d14ce592.1
Simperium error: transform diff for a ghost member (ghost <SPGhost: 0x1da7b950>, memberData (null)) that doesn't exist (lastObjectURL): {
o = "+";
v = "x-coredata://E1E8ECEF-941E-4838-AB81-282A15581A9C/StorageType/p24";
}
Simperium error: transform diff for a ghost member (ghost <SPGhost: 0x1da7b950>, memberData (null)) that doesn't exist (appVersion): {
o = "+";
v = "4.67";
}

[martinoneutrino]

OK, it turns out that I was not waiting long enough. However, it took a 5 minute timeout before it resumed (and finished) the sync.

2014-09-19 18:03:30.163 Simperium performing change operation: M
2014-09-19 18:03:30.167 Simperium received version = (null), previous version = (null)
2014-09-19 18:03:30.170 Simperium MODIFIED ghost version 1 (EntityName-EntityName)
2014-09-19 18:08:31.689 Simperium connection closed (will retry): 1001, Stream end encountered
2014-09-19 18:08:33.698 Pinned Certificate is Valid: Proceeding with extra checks!
2014-09-19 18:08:33.701 Simperium opening WebSocket connection...
2014-09-19 18:08:34.043 Simperium initializing websocket channel 0
...

[jleandroperez]

We have an issue open to implement sync'ing status.

One of the main caveats to this is: Simperium is a streaming protocol, which allows multiple devices to update any of the objects, at any time. The real sync'ing status would be given by... [Local Changes] + [Remote Changes].

Trick to this is: local changes (might) be rejected by the backend / rebased, and as for Remote Changes, we might have devices that actually modified the dataset, but are offline. Real sync'ing status would be though to implement.

We could, perhaps, expose the Local Pending Changes (which would be roughly the same property you've been using, numChangesPending, but with a couple updates so it's a non expensive operation (the private method can be quite expensive in terms of time!).

Regarding the error you've posted few days ago:

Simperium error: transform diff for a ghost member (ghost , memberData (null)) that doesn't exist (appVersion):

That would (possibly) be caused by an entity that got inserted in the backend dataset, but the iOS client didn't receive the acknowledge.

Would you please confirm if this 5 minute timespan goes by, without anything in the console ? (verbose=on), meaning... "isn't anything happening" ?

If possible, it'd be awesome if you could send us some console logs. Would love to take a look:

jorge.perez [at] automattic (dot com)

Thanks in advance!

[martinoneutrino]

I don't think it is necessary for the purposes of the sync status to be aware of unknown (offline remote) changes.
It is important know if there are local changes still pending. It is also important to know how many changes the server knows are still pending to send. I haven't worked at that aspect, so I can't say if the current system is sufficient.
I can confirm that there was nothing else in the logs (it was verbose); no lines are missing as posted.
I will send the logs via email.

[jleandroperez]

@martinoneutrino You can learn about the Simperium Streaming Protocol Here. The backend + clients simply receive the new changes via websocket, yet, the server is never aware of How many changes are pending to be sent. That's just a client implementation detail.

I've just moved issue #9 to release 0.7.0, and added a note to implement a non-expensive Local Pending Changes getter.

Regarding the logs, we'd need to check if sync'ing is getting stalled for some reason (nothing is being printed out for a long period of time), or it actually takes up to 5 minutes with un-interruped messages going through.

Thank you again!

[martinoneutrino]

I look forward to getting an inexpensive local pending count.

My original issue still stands (re-enabling the network using networkEnabled=YES after authenicateWithAppID:token:), however I will not be doing that in my app.

[jleandroperez]

@martinoneutrino Would you please verify if you're still having this issue with the latest develop code?.

networkEnabled property has been updated, this should no longer be an issue.

Thank you!

[martinoneutrino]

The issue no longer occurs when using the develop branch.

[jleandroperez]

@martinoneutrino thanks for verifying that!. Regarding the 5 minutes upload, please, set verboseLogging = true;, and check if the lib gets stalled at any point. If so, it'd be awesome if we could take a look at the logs.

Thanks!

[jleandroperez]

Let's move the Sync'ing glitch hunt over to #356

Thanks for your feedback @martinoneutrino !