-
Notifications
You must be signed in to change notification settings - Fork 43
More powerful caching mechanics #157
Comments
Probably not worth trying to setup for profit stats. I'm not sure I see the benefit to using a separate DB as a cache, and I'd rather not add dependencies unnecessarily. The only real annoyance I can recall us having with data that falls into 1 is the requirement to cleaning it up later on (like with block credits), which is pretty minor. |
The benefit of a separate datastore to hold type #1 is that it could get quite large, but not be fetched that often, therefore Redis is probably not the best choice since the dataset is held in memory. For now it would be trivial to use the redis cache and switch it later if it becomes an issue at all. The main reason I'm discussing this is that you mentioned that generating the data took significant amount of CPU, and it if has expiry set (required by Flask-Cache) then we will be recomputing it regularly. |
I guess I'm looking for an example of where something like SSDB might be better than just using Postgres. Our use cases for redis/flask-cache seems pretty straightforward imo. In the branch I made profit data is cached in redis keys, and its a pretty small number of keys so no worries there. When profit is computed it loops through ALL the credits of the last X hours, which is expensive. This is probably OK in the short term, if we recompute it every couple hours or w/e. But I'm not sure the best way to handle this moving forward - as its clearly sub-optimal. One easy/straightforward way to minimize the performance hit is caching some of the data needed from the This optimization would probably be total trash if we want to move towards a more robust profit tracking system. I can't find the ticket for it, but at some point you suggested a table that functions similarly to the share_slice table, except for profits. |
It's been apparent to me for a while that we need more refined caching mechanisms. I believe we can boil down our caching needs into three main categories:
So really we're lacking a good solution for 1. Worth implementing this for profitability?
The text was updated successfully, but these errors were encountered: