Proposal: FIP on Indexer

[honghaoq]

We recently shipped Indexer to Production and released with Lotus 1.15. We'd like to propose a FIP to standardize index providing: for content to be considered available for online retrieval, it needs to be advertised through the indexer advertisement protocol.

Context
Indexers store mappings of content multi-hashes to provider data records. A client that wants to know where a piece of information is stored can query the indexer, using the CID or multi-hash of the content, and receive a provider record that tells where the client can retrieve the content and via which data transfer protocols.
Content can be queried at cid.contact.

To provide index to indexer node, there is an Indexer Provider service that runs alongside with Storage Provider, which tells the Indexer what contents this storage provider has. Index Provider sends updates to the Indexer via a series of [Advertisement messages. Each message references a previous advertisement so that as a whole it forms an advertisement chain. The state of the indexer is a function of consuming this chain from the initial Advertisement to the latest one, and delta of indexer content is updated periodically.

Indexer enables content discovery within Filecoin network today, it is also a fundamental component toward the end goal of universal retrieval across IPFS/Filecoin, i.e. to be able to efficiently retrieve Filecoin content from IPFS. Combining Indexer work for content discovery with IPFS/Filecoin interoperability work for content transfer, we aim to reach the end state of efficient and universal retrieval across IPFS and Filecoin network.

Proposal
Our draft FIP can be found in our public notion here. We will propose it formally in FIP repo after this discussion.