chore(archive): allow request concurrency towards PostgreSQL #1604
Labels
E:PostgreSQL
See https://github.com/waku-org/pm/issues/84 for details
good first issue
Good for newcomers
In order to allow for a scalable and fault-resistant Waku Archive technology as set out in waku-org/pm#12, we have decided to implement a PostgreSQL backend.
Work on adding basic support for PostgreSQL has started: #1590
A related work item, required for scalability, is to allow processing store requests concurrently. The standard library integration will, similar to the previous SQLite-based implementation, block on processing a store query, which is not scalable at high store query rates.
A simple solution may be to adapt https://github.com/treeform/pg to work with
nim-chronos
, i.e. an async wrapper functioning as an extension of the std library API.cc @cammellos @LNSD
Tasks
[ ] Keep the sql queries in external .sql plain text files ( comment ) . Also, important to pay attention to the @LNSD comment: Add postgres basic support #1590 (comment)-> created a separate issue[ ] Optimize the database. We need a rapid response when duplicate msgs happens.-> created a separate issue[ ] Apply @Menduist 's enhancement suggestions for a more appropriate asynchronous handling: feat(common): added postgress async pool wrapper #1631 (comment)-> created a separate issue[ ] Protect the nwaku compilation process so that the user has the-> created a separate issuelibpq
library properly installed.[ ] Create a docker file to bring nwaku + PostgreSQL simultaneously. Ping @LordGhostX when completed.-> created a separate issueThe text was updated successfully, but these errors were encountered: