Design and list out possible approaches for Data Archiving

[shashwat-egov]

DIGIT Core modules like billing service generate recurring data(Bills are generated against an entity in each billing period). This data is stored in Postgres. Over time this data accumulates and the data size increases. This causes a slowdown in the query operations which impacts performance of demand and billing APIs. Also as data size increases more resources are needed by the database which increases the operating cost. For such use cases we need to devise a data archiving strategy. The old data of the system should be moved from the main RDBMS storage to long-term storage solutions like Amazon S3 Glacier. A data retrieval mechanism should be provided to retrieve data from this archive as per need.