Describe how IPFS bookmarks will be handled in browsers

[flyingzumwalt]

Question from @jefft0:

Is there an issue for how IPFS bookmarks would be handled in the browser?

[jefft0]

A goal of IPFS is to solve the problem of link rot. IPFS solves this problem when I bookmark a URL like fs:/ipfs/QmAAA/content.html (or send the link to a friend). A year later the bookmark will work as someone else has a copy. Hooray!

But my concern is that many web sites will be dynamic and use IPNS. A bookmark to an IPNS link like fs:/ipns/QmBBB/content.html will be broken a year later if there is no longer a content.html at QmBBB or of the publisher at QmBBB is offline. (The DHT will not remember a published IPNS value forever.) Is that right? I have some ideas to solve this problem, but I wanted to confirm that it is a problem.

[Kubuxu]

@jefft0 yes.

I had some ideas how to do it here: ipfs/notes#109 (comment)

In included the IPNS/IPRS having two cache timeouts: timeout after which the record is not fresh anymore (query for it will be run but the cached version would be served right away) and second timeout after which record is removed from caches. The second timeout could be infinity.

Other solution (but they are not exclusive) would be storing old IPFS link along the IPNS link and fallback to the old IPFS link when IPNS link is not longer accessible.

[jefft0]

storing old IPFS link along the IPNS link and fallback to the old IPFS link when IPNS link is no longer accessible.

Yes. The IPNS link that I bookmark (or send to a friend or put in a blog post or cite in a paper) should be an IPFS link to an IPLD object which contains both the IPFS multihash of exactly what I'm looking at, as well as the IPNS link for getting the current version (if possible). Is there already a proposal for the format of the object which has both the IPFS snapshot plus the IPNS link for the mutable current version?

[Kubuxu]

Not that I know of.

[jefft0]

An object which has the IPNS peer ID well as the current value is (almost) the IpnsEntry value . So, we could extend this to define an object:

message IpnsEntryWithPeerId {
  required IpnsEntry entry = 1; // Contains the current value.
  required bytes peerId = 2;
}

A proposal: When the publisher does ipfs name publish, in addition to publishing an IpnsEntry for the peer ID, IPFS also makes an IpnsEntryWithPeerId object and adds it to the publisher's IPFS content at /ipfs/<hash of IpnsEntryWithPeerId> . (This step is actually optional, see below.)

Here's the magic. When I look at a page in the browser, I see the URI /ipns/<peer ID>. The browser has already done the IPNS lookup and internally stored the IpnsEntry. So the browser can construct the IpnsEntryWithPeerId object and get its hash. This should be the same /ipfs/<hash of IpnsEntryWithPeerId> which the publisher (optionally) made. The browser also adds this object to my local IPFS content along with the cache of the web page content. So, when I bookmark the page (or send the link to a friend) the actual link copied is /ipfs/<hash of IpnsEntryWithPeerId>, basically a permanent snapshot .

When this link is clicked later, the browser fetches the IpnsEntryWithPeerId, from the original publisher or anyone else who cached the content. If the IpnsEntry is still current, the browser can immediately display the value without needing to access IPNS. If the IpnsEntry is out of date, the browser can use the peer ID to fetch and display the latest version from IPNS. But, most importantly, if there is no longer an IPNS entry or the user is offline, then the browser can display using the permanent IPFS value in the IpnsEntry. Humanity's history no longer deleted due to IPNS link rot!

[jefft0]

What is the current thinking to solve the IPNS link rot problem? One solution discussed above is to change how browsers save an IPNS link, by also saving the IPFS link that it resolves to. Should a solution be part of the Q1 objectives? Maybe Firefox/Chrome doesn't have a bookmarks API to solve the IPNS link rot problem, but maybe it can be done in Brave?

[lidel]

Well, if there is an API for additional metadata (and for now, Brave and Chrome have none, Firefox has.. plans), we could save both IPNS and current IPFS (+automatically pin latest version at a local node).
Having that, we could open an intermediary page that checks if IPNS is alive and if not, falls back to last-working-IPFS. If it sounds like a kludge, it's because it is one.

However, a better approach is in the works.
As @Stebalien hinted in discuss.ipfs.io/t/can-ipns-addresses-be-pinned:

[..] we're working on a system that will let you subscribe to IPNS updates (via a system called pubsub) so this should get better in the future.

At some point in the future it will be enough to bookmark+pin IPNS path and your node will take care of following updates.

The bookmark thing won't happen in Q1 but feel free to subscribe to:

• Normalize URL of newly created bookmark ipfs-companion#254 to track IPFS/IPNS bookmarking status in browser extension.
• IPNS pinning: ipfs name follow kubo#4435 to track dynamic IPNS pins

(+referencing a bunch of related issues to improve discoverability)

• Advanced Pinning notes#49: Advanced Pinning
• idea: pin through ipns names notes#97: idea: pin through ipns names
• Ability to pin ipns names kubo#1467: Ability to pin ipns names

[jefft0]

Thanks for the info on pinning IPNS. Here is the use case: A scientist makes a web page with research data and puts an IPNS link in a paper. Someone else ("Alice") pins the IPNS link. A year later, the scientist is offline and no longer refreshing the IPNS record. I view the paper in a browser and click the IPNS link. Will my browser resolve the IPNS link from Alice's pinned cache and fetch the web page?

[ghost]

I suggest using ipfs name follow which is being worked on at least for go-ipfs: ipfs/kubo#4435

[jefft0]

So, some other user ("Alice") may track IPNS changes for herself. The question is, a year later when I click the IPNS link for the first time, will my browser find Alice and resolve the IPNS name using her pinned data? If not, then for me the IPNS link has rotted.

[victorb]

Subscribing to the IPNS record does not solve the problem. Once I pinned something from IPNS, we should keep the value of what it resolved to, so I don't have to ask the network for the IPFS hash which it resolved to. Guess the question is if it's up to the user application (browsers bookmark functionality in this case) or in the core of IPFS.

Well, if there is an API for additional metadata (and for now, Brave and Chrome have none, Firefox has.. plans), we could save both IPNS and current IPFS (+automatically pin latest version at a local node).
Having that, we could open an intermediary page that checks if IPNS is alive and if not, falls back to last-working-IPFS. If it sounds like a kludge, it's because it is one.

Yeah, something like that. But I don't think we need a intermediary page, if I bookmark a IPNS link, I would like it to always show the IPFS link, since that's what I saw at the moment. Could be just my perspective though, as I use bookmarks for "archiving" pages, not for having a shortcut to something I visit often and want the latest update of.

[jefft0]

So, it sounds like the answer "no", none of the proposed solutions (including IPNS pinning) will help me when I click the IPNS link for the first time a year after the author stops publishing updates. Is there a proposed solution to address this use case?

[ghost]

none of the proposed solutions (including IPNS pinning) will help me when I click the IPNS link for the first time a year after the author stops publishing updates

ipfs name follow as described in ipfs/kubo#4435 should help with that.

[jefft0]

How does ipfs name follow help? The original publisher has stopped publishing IPNS records into the DHT, so when I try to resolve the IPNS name for the first time I get nothing from the DHT. Am I able to resolve the IPNS name using the records that you have pinned on your computer with ipfs name follow?

[Stebalien]

@jefft0 the trick is the --allow-expired flag. That will allow your node to use the last seen IPNS record.

[jefft0]

@Stebalien, the use case is were I download a research paper with an IPNS link and click it for the first time. I have never pinned this IPNS name before on my computer. The original publisher is not publishing IPNS records any more. Now if I do ipfs name follow --allow-expired, what happens? Nothing, right?

[jefft0]

Case in point: The IPFS Weekly for January 22 has an entry for PirlTube. It is an IPNS link:
https://ipfs.io/ipns/QmPq5wsJx28EJaGHHxkBzbRgufVK3scJey2oM886iUQ175/#/videos

I didn't click this link when it was good. But surely lots of other people clicked it and cached the data. But it just doesn't matter. An ipns link is born rotting. If I had an ipfs link then it would hit their cache, but not an ipns link, if the original publisher is not refreshing the ipns entry.

I've mentioned a way to solve this a few times, but I still don't think that the IPFS community sees IPNS immediate link rot as a problem. Admittedly, I'm not really in the developer loop. Maybe people assume something like the Ethereum Name Service will take over for ipns links?