Configuration Flag to run Gateway in Trustless-Only Mode

[lidel]

tldr

Add a configuration option that limits Gateway to trustless Block and CAR response types.

More details in the specification at /ipfs/specs/http-gateways/TRUSTLESS_GATEWAY.md

Details

Context

Kubo 0.13 shipped support for Block and CAR response types (#8758).

These "trustless" response types disable IPLD deserialization and website hosting, and enable clients to verify the data is matching the requested CID, removing the need to trust that the gateway sent the correct bytes.

Why

Right now, Kubo supports both trusted and trustless response types, and there is no way to disable trusted ones: text/html and other deserialized responses have to be blocked on reverse proxy running in front of Kubo.

I believe we should have a clear config Flag that allows people to limit their gateway to Block and CAR responses for direct CID at /ipfs/{cid} (and nothing more).

Value added:

• allows more people to run public gateway without worrying about their DNS name being used for hosting questionable content such as text/html used in phishing campaigns
• encourages ecosystem to shift towards fetching verifiable HTTP responses (light clients, service workers)

How

We already have Gateway.NoDNSLink (bool) which can be used globally or be limited to specific hostname defined in Gateway.PublicGateways (see docs/condif.md)

The idea here is to add Gateway.NoTrust (name TBD, ideas welcome), working in analogous fashion.

Some pointer for implementer:

• detecting request type: https://github.com/ipfs/kubo/blob/v0.14.0/core/corehttp/gateway_handler.go#L397-L404
  • check if in "trustless" mode, and error if request was for anything other than things described in /ipfs/specs/http-gateways/TRUSTLESS_GATEWAY.md
• routing to specific handler: https://github.com/ipfs/kubo/blob/v0.14.0/core/corehttp/gateway_handler.go#L430-L444

[Jorropo]

I'm surprised people would care about that, it seems like it's a client issue if they aren't using trustless but if someone needs it then fair we can have it.

[lidel]

People running gateways start to care, especially when their domain name gets blocked by Google. This is not limited to IPFS ecosystem.

This flag enables a safer deployment option. A Trustless-Only Gateway provides content-addressed data to HTTP contexts (CLI, light clients, JS, Service Workers), but can't be used for website hosting, so won't be marked as the Origin of deserialized, third-party data.

[eleijonmarck]

@Jorropo @lidel do you agree that this option would be valuable to have? If so; I would love to take a stab at this.

Also curious to hear your thoughts on making kudo more secure by default by instead making this an opt-out feature?

[lidel]

This was implemented in #9789 as opt-out Gateway.DeserializedResponses config flag.
It can be tested in Kubo v0.21.0-rc1.
See release notes here.

The upstream boxo/gateway library has this flag disabled by default: ipfs/boxo#252