Realtime API

[ishitatsuyuki]

There are some known services (mainly docs.rs) that continuously watch updates on crates.io to trigger jobs (doc build, dependency update PR, etc).

This will probably need a way for baseline fetch as well, which could be achieved with a database dump (#630).

The requirements for the APIs are:

• Streams (probably WebSocket) of changes, including relations (document-based NoSQL excels at this)
• A window of backlog of changes (to allow consistency when a short disconnect happens)

Some ways to implement reactiveness are:

• Use a database that is designed for direct sync (bypasses backend) (CouchDB)
• Use a database that is capable of realtime watches (MongoDB, RethinkDB)
• Roll our own implementation based on some PubSub and filtering based on timestamp datas

[samueltardieu]

There is also the possibility of using a RSS/Atom feed + WebSub. Work would be minimal on crates.io (generate XML feed with the 100 latest applications + ping the hub and let it do its work).

[sgrif]

This seems like it would be much better handled by watching the index, rather than the registry. Either way, I certainly don't see this being prioritized any time in the near future.