offline database

[magsout]

This is not a request but just a question. Is it possible to have a database offline? I work a lot in train without 3G

[miketaylr]

http://pouchdb.com/ might be a cool way to do this. It would require us to use couchdb on the server, however. But, we do want to have our own store of the GitHub data anyways.

There are probably other options--and we should explore them--but I think this would also benefit people who have slow connections to GitHub (cc @karlcow).

[miketaylr]

(this might make a cool hack/work/week project).

[magsout]

Oh yeaaaah coool

[miketaylr]

See also http://mozilla.github.io/localForage/.

[miketaylr]

Thinking more about this, either using something like localForage or just indexeddb directly, storing API data offline/locally would be relatively easy. The tricky part comes with allowing an offline workflow and syncing with the db.

We could probably split this into two tasks: 1) offline 2) sync--if we think the 2nd is the worth the effort.

@magsout Can you explain a bit more what would be useful to have offline? The stuff in /issues, and the data for individual issues? If you had offline access to that that was read-only, would that be helpful or just annoying?

[magsout]

@miketaylr

The stuff in /issues, and the data for individual issues?

Exactly,

If you had offline access to that that was read-only, would that be helpful or just annoying?

Not a problem. Just need to see list of issue when I code

[miketaylr]

Cool, let's start with that. :)

[magsout]

<3

[miketaylr]

This relates to #222, which I sort of punted on. I'm leaning towards using @mozilla's localForage and the backbone adapter: https://github.com/mozilla/localForage-backbone

[karlcow]

@magsout because I like the idea and I'm trying also to see what would be beneficial.

Could you explain in terms of a user scenario what would you do offline with webcompat.com data?

[magsout]

Best scenario (no cache, no database, no localStorage, profile clean) :

1. go to webcompat.com , loading data (list of issue)
2. no connexion, no need to reload list of issue which are already in cache (localForage, localStorage, wahetever)
3. connexion avalaibale, send a request if, no-modified no need to reload list of issue, if changed, reload list of issue

[karlcow]

I guess it relates to #165 with regards to what @magsout says.
Some questions

1. Do you expect to have the full list of all issues?
2. Do you expect to have only the list of issues accessed when browsing the home page?
3. Something else?
4. What is the minimum information do you require for each issue on that list?

back of the napkin evaluation:

→ curl 'https://webcompat.com/api/issues/897' -H 'Host: webcompat.com' -H 'User-Agent: Mozilla/5.0 (Android; Mobile; rv:38.0) Gecko/38.0 Firefox/38.0' -H 'Accept: application/json' --compressed -H 'X-Requested-With: XMLHttpRequest' -H 'Referer: https://webcompat.com/issues/897' -H 'Cookie: _ga=GA1.2.543739634.1428993312; _gat=1' -H 'Connection: keep-alive' -H 'If-None-Match: W/"3a211cb9e9d68b3eeb6202603dcd785a"' -H 'Cache-Control: max-age=0' > 52.json

→ wc -c 52.json 
    2256 52.json

900 (issues) x 2256 = 2030400, around 2 Mo (storage)

→ gzip 52.json
→ wc -c 52.json.gz 
     861 52.json.gz

which is around 0.74 Mo (transfert)

(This without the comments obviously)

[miketaylr]

The "simple" way I'm looking at starting to implement this is to cache things as they're requested. So that might only be the first 50 issues on /issues, or more depending on what you clicked on. We could get more advanced and do crazier things, of course.

[magsout]

Your simple way is perfect

[miketaylr]

According to https://blog.wanderview.com/blog/2015/03/24/service-workers-in-firefox-nightly/, service workers will be in Dev Edition at some point soon (though not progressing to beta/stable yet). Might be fun to (re)write this functionality using them.

[karlcow]

Might be fun to (re)write this functionality using them.

Maybe fun to write, but not necessary for users. ;) http://caniuse.com/#feat=serviceworkers
Yeah I know ;) 💩

[miketaylr]

:p Given the niche use-case, I think it's probably OK. But we're neither here nor there right now anyways.

[magsout]

any update about this issue ? ☺️

[miketaylr]

Update: I'd still love to do this but haven't done much work on it. Something to discuss in Paris. ^_^

[magsout]

ok ❤️

[magsout]

I'm playing with LocalForage. Very good lib. Good solution for this issue.

[miketaylr]

Saw this today: https://www.talater.com/upup/. Also looks pretty cool.

[karlcow]

@miketaylr said we need to break this up in manageable tasks

[zoepage]

This is a tool a friend of mine build. Maybe we can adapt this and use it for webcompat.com
https://github.com/schultyy/offline-issues

[miketaylr]

This is not a request but just a question. Is it possible to have a database offline? I work a lot in train without 3G

@magsout is this still relevant, and what kind of task are you trying to do on the train? Work on webcompat.com dev bugs? Triage issues (which gets hard w/o internet lol)?

[magsout]

@miketaylr old issue ;)

we can close for now.

[magsout]

I wanted to have access to the test issues because the internet connection is bad. But it's ok. I sleep on the train ;)

[zoepage]

@magsout So, to web-bugs? :)

[magsout]

@zoepage yes, exactly

[zoepage]

@magsout We talked about that use case yesterday. Would you be able to test web-bugs without an internet connection, because you wouldn't be able to check the websites? Or am I missing something?

Otherwise you could use https://schultyy.github.io/offline-issues/ ?