Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Peers should cache provider information to spread it around the network. #14

Open
Stebalien opened this issue Jul 30, 2019 · 2 comments

Comments

@Stebalien
Copy link
Member

Stebalien commented Jul 30, 2019

After finding new providers for a piece of content, peers should store those providers locally. This:

  1. Speeds up future (initial) find provider queries for the same thing.
  2. Distributes frequently requested provider records around the network.

However, I'm a bit concerned about provider record expiration given a FindProviders call won't actually return when the provider record was initially submitted. This means that popular provider records would live forever.

One solution is to do this (record that some peer provides some content/service) inside the end service itself. For example, when we get a block from a peer, we could locally record a provider record for the peer from which we got the block. This would, incidentally, help us resume downloading a file if we restart.

@Stebalien Stebalien changed the title Peers should _cache_ provider information to spread it around the network. Peers should cache provider information to spread it around the network. Jul 30, 2019
@aarshkshah1992
Copy link

@Stebalien Why do we want to do this just for the provider information ? Why not for the values passed around as a part of PutValue as well ?

@Stebalien Stebalien transferred this issue from libp2p/go-libp2p-kad-dht Aug 13, 2019
@yiannisbot
Copy link
Contributor

This is a very good idea, which will improve performance and resolution time and which we have discussed briefly with @raulk. In order for this to be efficient, there should be a relation between the time when a block was initially requested and cached and the "expiration time" from the cache. In other words, there should be a link between the position of the provider record in the peer cache (let's call it "provider record cache") and the actual block/data in the peer cache where the provider record is pointing to.

This can only be done probabilistically and should probably also take into account topological issues (e.g., how many nodes can point to an end-peer), but we have shown in [1] that it can be very efficient. (In [1], replace router with peer cache and "EFIB" (ephemeral FIB) with "provider record cache".)

Could the gateway help keep those provider record caches?

[1] A Native Content Discovery Mechanism for Information-Centric Networks, ACM ICN 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants