ReadKit crashes when used with coldsweat

[bl1nk]

This seems to be the only app for OS X supporting Fever at the moment. It crashes after you add coldsweat as a Fever source, though.
I also tested my coldsweat installation with Reeder on iOS. It works.

I know that this is most likely related to ReadKit, but I think that the devs might not support third party Fever API implementations. I'll also submit a bug report to them anyway.

[passiomatic]

Damn, ReadKit looks sleek. I didn't know about it. Too bad it requires Lion - I'm still on Snow Leopard here - or I'll download it immediately.

Regarding the crash, it would be interesting to have a log of what ReadKit sends to Coldsweat. To get more log details: edit etc/config file and set level to DEBUG instead of INFO (the default).

[bl1nk]

Alright. Got the debug log.

localhost - - [10/Sep/2013:00:34:32 +0000] 31802 DEBUG client from 130.180.125.129 requested: NestedMultiDict([(u'api', u''), ('api_key', u'c760a32e9837014c534cca249896c5e6')])
localhost - - [10/Sep/2013:00:34:33 +0000] 31802 DEBUG client from 130.180.125.129 requested: NestedMultiDict([(u'api', u''), (u'groups', u''), ('api_key', u'c760a32e9837014c534cca249896c5e6')])

That's all.

[passiomatic]

Thank you. I've found a bug in the Fever implementation. Could you please update Coldsweat to latest commit and try again? It may be that your issue is entirely Coldsweat fault.

[bl1nk]

Still crashing.

[passiomatic]

Do you see the same log lines, that is auth and then groups?

[bl1nk]

Yes.

[passiomatic]

I have no clues on what is causing the crash. "Groups" command seems to be working as expected now since after the fix Reeder for iPhone shows me groups for my subscriptions. It would be great if you can file a bug to ReadKits devs. BTW this is a sample JSON response sent by Coldsweat for the "groups" command:

Underworld:coldsweat$ python -m coldsweat.tests.fever http://coldsweat.lan/index.cgi/fever -s groups

{
    "groups": [
        {
            "id": 1, 
            "title": "All entries"
        }, 
        {
            "id": 2, 
            "title": "New group"
        }
    ], 
    "feeds_groups": [
        {
            "group_id": 1, 
            "feed_ids": "1,3,4,7,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77"
        }, 
        {
            "group_id": 2, 
            "feed_ids": "2,5,6,8"
        }
    ], 
    "api_version": 2, 
    "auth": 1, 
    "last_refreshed_on_time": 1378711436
}

[bl1nk]

I already submitted a bug report to them and I also linked to this bug report. We'll see what happens.

[weichsel]

It seems that ReadKit stumbles over missing saved/unread item keys in the ?api&saved_item_ids/?api&unread_item_ids responses.
With the small changes in the above pull request, I am able to sync ReadKit (OS X) with Reeder (iOS) via coldsweat.

[passiomatic]

This is great, thank you! I've noticed that Reeder is quite forgiving when it comes to parse Fever's API JSON output, thus having different clients is helping to make the Fever implementation more robust.

@bl1nk could you please test you ReadKit too using the latest commits? If it runs fine I guess I'll make another release this weekend.

[weichsel]

Looking forward to the next release - Thanks for your work on coldsweat!

[bl1nk]

@passiomatic I tried, still crashes. Also nothing new in the debug log.

[passiomatic]

Thank you, must be some corner case with your data then. Are you using sqlite? Could you please send me your database file (data/coldsweat.db) - andrea [at] passiomatic [dot] com Or alternatively, it would be useful to have some sqlite database filled with data that can crash ReadKit.

[bl1nk]

I am using sqlite. I'll send you my database in a sec.

[passiomatic]

Thanks. I will look at it over the weekend.

[weichsel]

Just to make sure that my fix isn't snake oil, I tested it against the sample subscriptions provided in tests/subscriptions.xml.
It does indeed fix a reproducible bug related to save/read-state setting that would otherwise crash ReadKit :)

@bl1nk Maybe you can find out the stage at which ReadKit is crashing by monitoring the requests dispatched to coldsweat (e.g. with Charles Proxy, http://www.charlesproxy.com)

[bl1nk]

@weichsel I did try to debug it with charlesproxy, it shows the same request I see in the coldsweat debug log, nothing more.

[weichsel]

So it seems ReadKit crashes right after receiving the groups response.
My coldsweat.db is really simple (I just have 1 group that contains all feeds) so the groups JSON response is straightforward:

{
    "groups": [{
        "id": 1,
        "title": "All entries"
    }],
    "feeds_groups": [{
        "group_id": 1,
        "feed_ids": "1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90"
    }],
    "api_version": 2,
    "auth": 1,
    "last_refreshed_on_time": 1380128579
}

Do you have a group that doesn't contain any feeds?
ReadKit obviously has problems when coldsweat omits keys &values for empty lists (like for the read/saved items list) so this could be the culprit here as well.

[bl1nk]

My response looks like this:

{
    "groups": [{
        "id": 1,
        "title": "All entries"
    }],
    "feeds_groups": [{
        "group": "1",
        "feed_ids": "1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34"
    }],
    "api_version": 2,
    "auth": 1,
    "last_refreshed_on_time": 1380183308
}

So it doesn't seem like I have an empty group.

For comparison, the response for my Fever installation looks like this:

{
    "api_version": 3,
    "auth": 1,
    "last_refreshed_on_time": "1380183317",
    "groups": [{
        "id": 1,
        "title": "Menschen"
    }, {
        "id": 3,
        "title": "Customize"
    }, {
        "id": 4,
        "title": "Apple"
    }, {
        "id": 5,
        "title": "Blogs"
    }, {
        "id": 6,
        "title": "Web und Apps"
    }, {
        "id": 7,
        "title": "Twitter"
    }, {
        "id": 8,
        "title": "Linux"
    }, {
        "id": 9,
        "title": "Fotografie"
    }, {
        "id": 10,
        "title": "IRC"
    }, {
        "id": 11,
        "title": "Gaming"
    }],
    "feeds_groups": [{
        "group_id": 1,
        "feed_ids": "2,4,5,6,7,36"
    }, {
        "group_id": 3,
        "feed_ids": "10,11,12,13"
    }, {
        "group_id": 4,
        "feed_ids": "14"
    }, {
        "group_id": 5,
        "feed_ids": "15,35"
    }, {
        "group_id": 6,
        "feed_ids": "16,17,18,19"
    }, {
        "group_id": 7,
        "feed_ids": "20,21,22,23,24,25,29,31,37,38"
    }, {
        "group_id": 8,
        "feed_ids": "26,30,34,39"
    }, {
        "group_id": 9,
        "feed_ids": "27"
    }, {
        "group_id": 10,
        "feed_ids": "28,33"
    }, {
        "group_id": 11,
        "feed_ids": "32"
    }]
}

[passiomatic]

   "feeds_groups": [{
        "group": "1",
        "feed_ids": "1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34"
    }],

This looks wrong. That "group" should be "group_id" with an integer value. The bug has been fixed in this commit: 165930c - Are you sure you are running the most recent code?

PS: So there's a version 3 of the API. Published docs do not mention that at http://www.feedafever.com/api

[bl1nk]

@passiomatic I thought I was running the most recent code because I always pulled all changes. I didn't re-setup the database which might have been my fault?

At least I can say that it works now.

[passiomatic]

I didn't re-setup the database which might have been my fault?

Nope, but if are running Coldsweat in FastCGI or Passenger environments you have to restart the server/process or older code gets executed.