Ability for any website to opt-out from IPFS integrations

[lidel]

Problem

Some websites may provide degraded UX when loaded from local gateway,
and may prefer to load from canonical Origin by default.

Example:

• https://app.fleek.co/ does not work correctly because https://stripe.com (payment method they use) does not allow non-HTTPS Origins.

Looking for a solution

Opt-out per website

We already inspect HTTP headers while looking for X-IPFS-Path header, so my initial idea is that we could support opt-out based on HTTP header without adding new overhead. Thanks to this header, any IPFS-powered website could turn the implicit behavior (when the user did not make any manual choice yet) into explicit one.

Unfortunately:

• HTTP-header approach means we need to delay DNSLink lookup to occur after headers are read.
• This sends potentially undesired HTTP requests in situation when user prefers to not send HTTP traffic unless necessary
• Perhaps instead of HTTP header we could leverage DNSLink (extend spec with optional hints)

Opt-in per website / globally

There is an argument that installing IPFS Companion should not degrade browsing performance, but augument it, and user should make the decision when local node is used for DNSLink website (opt-in and not opt-out).

This got extracted to #1072

Additional notes

• this needs feat: remember manual opt-ins and opt-outs per site #929 to land first
• my main worry: we may need to refactor a bunch of things because so far we've been doing a redirects in onBeforeRequest, before we send anything. Response headers are available in onHeadersReceived hook.
  • If this proves to be time-consuming, it may be a better idea to punt this until we rewrite everything to conform with Manifest V3 (What to do about Manifest V3 #666)

[Gozala]

At the high level it appears to me that problem we have is as follows:

• Site can have DNS record (dnslink) identifying that same content can be loaded from IPFS.
• When companion discovering dnslinks proactively redirects to local gateway.
• Some sites may not be fully functional when loaded from different origin (or is it just limited to localhost ?)

I think fundamental problem here is that:

1. Content of site been replicated on IPFS does not necessarily mean that loading it from other host will make it fully functional.
2. There is no way for sites to say that you can load content from IPFS, but some things won't work.

Above makes me wonder if:

1. Sites that can not be fully functional should even have a dnslink setup ?
   • Maybe they should map specific subpaths or something ?
2. Should there be an alternative (or additional) record to dnslink that can tell (companion or other client) what the consequences of using other origin would be ?

I think instead of opt-out mechanism it would make more sense to amend dnslink in some ways so that client could choose better default (other than redirect or not). E.g. companion instead of redirecting could:

1. Present a UI telling the user that there is on option to load it from local gateway.
2. Persist user choice for the next load, but still have an option to change the choice.
3. Ideally have some means for the site to communicate consequence of using local / other gateway.
   • Maybe .well-knows/dnslink-problems could hold such message ?

[lidel]

Thanks, this is useful! Fleshing out current thoughts on this:

• Long term
  • I suspect Google's push towards Manifest V3 (What to do about Manifest V3 #666) will force us to reimplement the way we handle DNSLink in web browsers, and we may end up with UI close to what you suggested 👍
• Short term
  • (A) Checking for presence of .well-known/dnslink/keep-origin, when DNSLink is detected,
    sounds like a better alternative to HTTP header
    • it still requires execution of one HTTP request, but does not pollute regular HTTP responses
    • main selling point: "configuration lives with data":
      • site owner does not need to configure HTTP header (eg. Fleek users won't have that power)
      • configuration is gateway-agnostic (I can move from Fleek to Cloudflare without migrating HTTP headers)
  • (B) extending DNSLink spec is another way to do it, and feels more future-proof
    • main selling point here is that in theory, no HTTP request has to be sent if DNSLink and origin preservation policy are both stored in DNS TXT record
      • if user has DNS over HTTPS/TLS then its better for privacy than leaking own IP to a third party HTTP server (I'm thinking about the future where anonymizing libp2p transports exist)
    • I have mixed feelings about introducing web-specific hint to something that is web-agnostic (dnslink only deals with DNS→content path mapping)
      • perhaps the hint could be as simple as marking DNSLink "test"?
        • this would be useful for big websites that want to experiment with DNSLink but don't want to disruption unless they are sure everything works 100% when loaded from IPFS
        • user agents such as IPFS Companion would then interpret "test" flag and modify their behavior. For example, in case of IPFS Companion it would not redirect by default if DNSLink is marked as "test", or when asking where to load it from, change which option is the default.

I like (B) bit more, because it removes the need for sending a HTTP request to every DNSLink-enabled website we discover, and works even when original server is down, enabling us to build better recovery UIs.

[olizilla]

protoschool could do something fancy like have a subdomain that doesn't have a dnslink and use and iframe and postMessage to store scores against an origin that is stable regardless of how the user fetches it.

More simply, if the DNSLink redirection was opt-in, where companion offered you the "Hey, this site is available via IPFS" then the site would also get an opportunity to tell you the trade offs. It would be most helpful to have a way for a site to detect if companion is enabled, so it could post a banner like "get this on IPFS, learn more..." and explain the trade-off (super awesome co-hosting, but you gotta pick a side, or else your scores will get messed up)

Either way, i'm mildly against both A and B, when making DNSLink redirection opt-in could solve it without adding more nuance to either the dnslink spec or companion.

[Gozala]

Main intent behind my proposition was to:

1. Empower user with a choice (as opposed to empowering a host)
2. Empower user to make to make an educated decision by
   • Enabling a host to communicate tradeoffs at play.
   • Potentially client (companion / brave etc..) in future could provide even more information in the future.
3. Create a room for a reasoned user choice to exist
   • Currently host is put in position of binary choice (provide DNSLink or not). This reduces user choice to to only hosts that opt-in.
   • By allowing hosts to communicate constraints of using DNSLink, they are no longer forced to make a choice but pass that responsibility onto a client. This creates a room for companion, brave etc... to help user make an informed choice.

I do however do not want to empower sites / hosts with more tools to do tracking, that is to say I disagree that they should recognize if you have companion and do something specific then. In fact I think it is important that they don not know what choice user has made despite given constraints.

I'm also less interested in implementation details of all this. I have a mild preference towards complementary solutions, which is how I viewed .well-known/dnslink/keep-origin. Client (companion, next get browser) can communicate tradeoffs to a user without necessarily fetching the host provided reasoning, and / or it could fetch it on demand e.g. on user action etc...

Another thought I had coming back to this is we could have another TXT record (can we query both in one request?) pointing to CID a with tradeoffs. I imagine eventually those will boil down to handful of CIDs.

[lidel]

We will try to tackle aspects this as part of ManifestV3 rewrite #666 work, most likely switching the default behavior to prioritize user choice where use of local gateway being opt-in rather than opt-out, similar to Brave:

This issue was about opt-out from the server side, so closing it in favour of client-side opt-in tracked in #1072