Epic: service-worker-gateway inbrowser.link MVP

[lidel]

For general use

Minimal feature set

• Missing support for index.html in HAMT-subdirectory #19
• Enumerating HAMT directory retrieves too many blocks? #18
• Support the same DNSLink TLDs as Kubo and Rainbow #23
• @helia/verified-fetch: HTTP Range request support helia-verified-fetch#9
• Trailing / for UnixFS directory normalization is missing  #62 (broken relative links on index.html pages)
  • Support http(s):// gateway URLs helia-verified-fetch#11
• fix: handle helia-sw query from _redirects #67
• Bug: percent-encoding-like filenames fail to load #83
• Update mechanisms #26

Optimizations

• Cache final Responses #73
• Subdomain loads require 30MiB download every time #93
• CI should not [fail when no need to publish to ipfs]
• reduce code transferring from subdomain gateway urls and other http/https to ipfs:// and ipns:// urls #110
• Slow iframe requests to root domain (unintended cache bypass) #88
• Support direct HTTP retrieval from /https providers #72 (pick up after others are tackled first)
• Send Ipfs-Path-Affinity (IPIP-462) with requests to trustless HTTP gateways helia#678
• https://github.com/ipshipyard/waterworks-infra/pull/82

Security and Compliance

• Origin isolation warnings, tests and enforcement #30
• flesh out how we use gateway-conformance for testing subdomain and path modes #181
• Remove hacks like changeCssFontPath #33
• Original URL is passed to @helia/verified-fetch as-is (minimized delta for conformance tests)
• Bug: DoH client is missing in-memory cache #59
• Missing LICENSE #116

UX / Configuration

• Return meaningful errors #133
• Improve UX of the root domain landing page #58
• Rename and move to ipfs/ org #80
• feat: Service worker configuration #22
• Updating global config on root domain doesn't affect subdomain urls once SW is installed  #91
• User-friendly error on redundant service worker registration #32
• Documentation for how to self-host
  • Nginx, DNS and TLS setup (expand gist from Russell, replace Fleek with local build + updating DNSLink. via Cloudflare API + token limited to single domain as example)

For inbrowser.link

• https://github.com/ipshipyard/waterworks-infra/issues/54
• https://github.com/ipshipyard/waterworks-infra/issues/55
• https://github.com/ipshipyard/waterworks-infra/pull/44
• https://github.com/ipshipyard/waterworks-infra/pull/42
• fix: dynamic subdomain gateway detection #53 (remove need for passing BASE_URL during build)
• Subdomain isolation enforced on inbrowser.link (subset of Origin isolation warnings, tests and enforcement #30)
  • Partially broken, needs fix: handle helia-sw query from _redirects #67
• inbrowser.tld being blocked #78
• Works on localhost gateway and when loaded via ipns:// in Brave
  • Add _redirects file for SPA behavior #28)
• CI: set up auto-deployment of helia-service-worker gateway with nginx config #20
  • feat(ci): build car, pin to cluster, update dnslink #69 (remove dependency on fleek)
  • pin in more places than just our collab cluster
    • chore(ci): additional pin to web3.storage #105
    • chore(ci): additional pin to scaleway #107
  • Deploy releases to inbrowser.link #79
  • Migrate Fleek from Ignite team to ipfs.tech infra team account. (low priority, we will use fleek oly for PR reviews)

---

Until we get automated testing in place, here's a list of CIDs of websites for testing the Service Worker Gateway

Site	CID	IPNS subdomain
Wikipedia on IPFS	bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze	n/a
Uniswap Interface	bafybeifaiclxh6pc3bdtrrkpbvvqqxq6hz5r6htdzxaga4fikfpu2u56qi	n/a
CID tool	bafybeiae366charqmeewxags5b2jxtkhfmqyyagvqhrr5l7l7xfpp5ikpa	cid-ipfs-tech.ipns
IPFS Docs	bafybeiedlhslivmuj2iinnpd24ulx3fyd7cjenddbkeoxbf3snjiz3npda	docs-ipfs-tech.ipns

[2color]

I think we should consider how we name this and call it in the docs and materials we will create before we make a splash. I propose we call it "Service Worker IPFS Gateway" or just "Service Worker Gateway" since the fact that it uses Helia is an implementation detail for most users. Moreover, it should be pretty clear that it's powered by Helia since it's the most actively maintained JS IPFS implementation.

I figured it's a good time be bring it up since we're going to be using the inbrowser.link domain for it (which already doesn't have Helia in it).

[lidel]

@2color agree, we can mention Helia on the landing page (#58) but this project should be branded as "Service Worker [IPFS] Gateway" or ipfs-swgw.

We already have to move to ipfs/ org, renaming repo could happen at the same time.
Let's discuss details in #80

[2color]

I added a list of CID of sites for manual testing to the issue body. Feel free to add any additional ones.

[SgtPooki]

NOTE: subdomains don't automatically refresh the service worker.. so we need some clever logic for checking for service worker assets, and then redirecting to /#/ipfs-sw-config again.

see #96 for some changes that make this more likely to happen

[SgtPooki]

Question: should we default to an ipfs favicon when favicon.ico doesn't exist...?

[lidel]

We should not modify immutable data. If the DAG that user tries to load does not have favicon, we should not add it.

[2color]

Add the following examples to the list and make sure are working well enough before sharing:

• Video
• ENS link
• IPNS name

[SgtPooki]

@lidel Some additional items we may want to think about adding here:

• Use recursive gateways (HTTPGatewayRouter) as a fallback when other routers fails helia#665
• Video streaming performance helia-verified-fetch#52
• feat: add server timings to responses helia-verified-fetch#160
• feat: perf measuring via onProgress events #503
• test: add test for comparing rendered content vs public gateway #502

[lidel]

I think this is greatly outdated, most of things got resolved, and remaining ones changed priority. We did not maintain list in this issue since March.

Let's close this to reduce noise – if we need new milestone/epic we will fill new issue

[lidel]

@SgtPooki thanks for flagging, for issues that exist in other repos:

• Use recursive gateways (HTTPGatewayRouter) as a fallback when other routers fails helia#665
• Video streaming performance helia-verified-fetch#52
• feat: add server timings to responses helia-verified-fetch#160

we should have dedicated tickets here that can be closed once a dependency update with fix is applied. Mind creating them? (using same title + link to upstream bug is enough).

[SgtPooki]

@lidel done:

• feat: deprioritize recursive gateways #512
• feat: dynamic dag traversal #513
• feat: responses contain server timing info #514