You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Due to the importance of having these indices up-to-date from a performance perspective (API providers), we should make sure that we always have these databases fully backfilled.
Currently we don't automatically backfill the msgindex/events/txhas sqlite databases but instead rely on the user knowing to backfill manually. Having these databases fully backfilled is important to the performance of several RPC methods which otherwise have a huge latency (over 10+ minutes on archival nodes).
Instead, on Lotus startup, we should just always check if we have missing data (handling also cases where config was changed from disabled to enabled for an index) and then handle the backfilling in the background automatically. Only in some rare edge cases should user need to manually run backfill via lotus-shed, for example if they are migrating dbs and/or editing the raw sqlite data.
The text was updated successfully, but these errors were encountered:
This probably needs something like MaxBackfillLookback for the case of an archival node turning this on late in the game; maybe you don't want to go back to FEVM and reprocess all tipsets to get events that you don't have.
Due to the importance of having these indices up-to-date from a performance perspective (API providers), we should make sure that we always have these databases fully backfilled.
Currently we don't automatically backfill the msgindex/events/txhas sqlite databases but instead rely on the user knowing to backfill manually. Having these databases fully backfilled is important to the performance of several RPC methods which otherwise have a huge latency (over 10+ minutes on archival nodes).
Instead, on Lotus startup, we should just always check if we have missing data (handling also cases where config was changed from disabled to enabled for an index) and then handle the backfilling in the background automatically. Only in some rare edge cases should user need to manually run backfill via lotus-shed, for example if they are migrating dbs and/or editing the raw sqlite data.
The text was updated successfully, but these errors were encountered: