Expose trustless gateway and delegated routing under new domains with certain policies

[BigLep]

Background

As part of ipfs/helia#255, there is the need for:

1. (trustless gateway) an endpoint that only exposes the trustless gateway functionality
2. (delegate routing) an endpoint that only exposes /routing/v1

Both of these enable reliable retrieval from the browser in different ways:

1. (trustless gateway) Fallback to a trustless gateway that isn't (and can't be) soiled like ipfs.io is (thus avoiding browser red screen)
2. (delegate routing) Empowering a browser node to do p2p retrieval

Kubo 0.23+ supports all of this functionality, which means the existing "ipfs.io" fleet can be used.

These efforts have similar work and so are being grouped together for efficiency:

1. Secure a domain
2. Add TLS cert to nginx LB
3. Setup proper nginx config concerning paths/headers/caching

Trustless Gateway requirements/suggestions:

nginx config that only allows responses with one of these content types:

1. https://www.iana.org/assignments/media-types/application/vnd.ipld.raw
2. https://www.iana.org/assignments/media-types/application/vnd.ipld.car
3. https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record

Caching: ???

Delegate routing needs/suggestions

Some more context is in https://github.com/protocol/bifrost-infra/issues/2142

Specific suggestions are in https://github.com/protocol/bifrost-infra/issues/2758#issuecomment-1716761794

Tasks

• Deploy Kubo 0.23 to "ipfs.io" fleet
• Determine domains for "trustless gateway" and "delegate routing"
• Register/configure "trustless gateway" and "delegate routing" domains
• "trustless gateway" nginx config deployed
• Enable /routing/v1 on the "preload" boxes (https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewayexposeroutingapi)
• "delegated routing" nginx config deployed (hitting the "preload" boxes)
• redirect ipfs.io/routing/v1 to delegated-ipfs.dev/routing/v1
• fix CORS missing CORS header on trustless-gateway.link #4
• https://github.com/ipshipyard/waterworks-infra/issues/9
• (@ns4plabs) Deploy someguy to previous/existing preload-staging nodes
• (@ns4plabs) "delegated routing" nginx config points to these nodes that have someguy deployed
• (@2color) ipfs/ipfs-docs about these new community good endpoints
  • delegated routing
    • Add a section above https://docs.ipfs.tech/how-to/peering-with-content-providers/#content-provider-list for "Delegated routing endpoints": Document delegated routing concept ipfs/ipfs-docs#1752
    • Added a redirect from https://delegated-ipfs.dev to https://docs.ipfs.tech/concepts/public-utilities/
  • Trustless gateway
    • Update https://docs.ipfs.tech/reference/http/gateway/#trusted-vs-trustless to use new domain instead of ipfs.io
    • Add entry to https://github.com/ipfs/public-gateway-checker
    • Added a redirect from the root of https://trustless-gateway.link and https://dweb.link to https://docs.ipfs.tech/concepts/public-utilities/

[BigLep]

Concerning domain names, we had said trustless-gateway.link

That said, I'm wondering if we instead want an overarching domain for IPFS-related network utilities that we can subdomain underneath. The parent domain can function as a landing page (or redirect to the right docs).

Ideas:

• ipfspublicutilities.org (verbose and descriptive)
• ipfspud.org (for public utility district)
• ipfswaterworks.org (maybe more brand potential but maybe requires more attention as to why we call the suite of network services "water works")

(I went with .org but good to go with another tld)

@lidel: do you have feedback/suggestions here?

[lidel]

For trustless gateway, the same caching config as we use for existing ipfs.io gateway will suffice.
(Responses already have correct Cache-Control HTTP headers, nginx cache can leverage them)

For delegated routing at /routing/v1 we should set up nginx cache with different expiration for hits and misses, but not sure what are the ideal values yet. We can start with values from https://github.com/protocol/bifrost-infra/issues/2758#issuecomment-1716761794 and see how it goes.

IPFS-related network utilities that we can subdomain underneath

Putting all eggs in one DNS basket runs into similar risk ipfs.io was under – ISPs or "antivirus" blocking entire domain kills unrelated services that are on subdomains. There is also a general risk of human error shutting down entire infrastructure due to DNS mishap, like we've seen in Saturn.

The main risk is around hosting third-party data. It is way way lower with limiting things to trustless responses, but given enough time, is non-zero.

👉 @BigLep Due to this we should keep separate trustless-gateway.link and just to be safe, also separate delegated-ipfs.dev for routing when we hard-code it as implicit default in Helia and other places (requests for delegated-ipfs.dev/routing/v1 in dev-tools will be self-explanatory, we have "delegated" "ipfs" and "routing" in the URL, which is good UX)

ps. I really like "IPFS Waterworks" as potential nucleation name for dev/infra team (ipfs-waterworks.dev?) :)
but it feels separate from trustless gateway and delegated routing needs we discuss in this issue

[BigLep]

2023-10-26 maintainer conversation:
We need to give CURL or CLI commands so that @ns4plabs can verify
This conversation will happen in #bifrost-community

[BigLep]

2023-10-31 maintainer discussion about docs

(@BigLep will flush this out further)

• delegated routing
  • https://docs.ipfs.tech/how-to/peering-with-content-providers/#content-provider-list
  • New section up top
• trustless gateway
  • Update https://docs.ipfs.tech/reference/http/gateway/#trusted-vs-trustless to use new domain instead of ipfs.io
  • Add it to the public gateway checker list

General delegated routing

• Also should fill in https://docs.ipfs.tech/reference/js/api/
• Delegated routing glossary item

[BigLep]

I cleaned up the minimum done criteria around docs.

A better treatment of delegated content routing docs is started in ipfs/ipfs-docs#1752

[cewood]

Per the planning sync in 2024-01-04-IPFS-Shipyard-Kubo-0-26-planning we decided to repurpose the preload staging nodes to run someguy, and point the delegated routing domains to these nodes. If appropriate, we can split that into a separate ticket, or just handle it here.

[2color]

Update from @ns4plabs:

The following branch of Someguy is deployed to a new host (not replacing any of the preload nodes) and is available on https://delegated-ipfs.dev/routing/v1

[2color]

Main challenge right now is that the endpoint is slow, e.g. 8 seconds for the following request: https://delegated-ipfs.dev/routing/v1/providers/bafkreia2xtwwdys4dxonlzjod5yxdz7tkiut5l2sgrdrh4d52d3qpstrpy and there's no caching in place.

[2color]

@ns4plabs added the cache-control header with the max-age=60 to the reverse proxy for the hosted version of this. I think this is fine for now, since this will also apply to empty responses, which we'd want cached for only 15 seconds.

[2color]

For us to intelligently set the cache-control (depending on wether the response is empty or not), we'd need to either:

• Return a 204 with no body for empty responses, instead of an empty array and HTTP 200
• Implement the cache-control header in someguy.

More info here ipfs/someguy#26

[2color]

trustless-gateway.link still doesn't work when passing the format GET parameter. It only works when passing the Accept header.

This has two implications:

• It isn't fully implementing the trustless gateway spec
• It can't be tested in browsers as there's no way to set the Accept header

Example:

curl -i "https://trustless-gateway.link/ipfs/bafybeicklkqcnlvtiscr2hzkubjwnwjinvskffn4xorqeduft3wq7vm5u4?format=raw"
HTTP/2 400
server: openresty
date: Wed, 31 Jan 2024 14:44:49 GMT
content-type: text/html
content-length: 154
x-ipfs-datasize: 154
x-ipfs-lb-pop: gateway-bank2-fr2
x-bfid: 117b28ca94345718171f0e92c856146d
strict-transport-security: max-age=31536000; includeSubDomains; preload

<html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<hr><center>openresty</center>
</body>
</html>

I think it'd be quite important to add support passing the format GET parameter

[lidel]

We should move filtering from Nginx to rainbow.
Proposed solution in ipfs/rainbow#71 and https://github.com/ipshipyard/waterworks-infra/issues/9