feat: *.localhost subdomain gateway support with http proxy #853
Conversation
lidel
force-pushed
the
feat/subdomain-proxy-support
branch
3 times, most recently
from
43e6494
to
1d922de
Compare
lidel
force-pushed
the
feat/subdomain-proxy-support
branch
2 times, most recently
from
e3ffed5
to
f948443
Compare
|
Surfacing some notes on additional fixes that had to be applies to keep subresource redirects working on HTTPS websites:
In other news:
|
lidel
force-pushed
the
feat/subdomain-proxy-support
branch
from
a4c9dce
to
bfa59c3
Compare
| "option_useSubdomains_title": { | ||
| "message": "Use Subdomains", | ||
| "description": "An option title on the Preferences screen (option_useSubdomains_title)" | ||
| }, | ||
| "option_useSubdomains_description": { | ||
| "message": "Isolate content roots from each other by loading them from subdomains at *.localhost and creating a unique Origin for each CID, IPNS or DNSLink record. Requires a local go-ipfs 0.5.0 or later.", | ||
| "description": "An option description on the Preferences screen (option_useSubdomains_description)" | ||
| }, |
There was a problem hiding this comment.
On Preferences screen the above labels look like this:
@jessicaschilling @autonome I suspect this could be phrased in a more human way, but this is the best I could do for next Beta release. Suggestions welcome before we ship to Stable.
This is a description of a toggle that lets people to switch to old path-based gateway on 127.0.0.1 if they choose to do so for some reason. (disabling it stops HTTP proxy and swaps Custom Gateway URL to use raw IP)
|
Shipped this in a new Beta: v2.10.0.902. Will play with it during the weekend. I do not expect big changes apart from tweaking labels mentioned in previous comment. |
This adds support for subdomain gateways introduced in go-ipfs v0.5.0, specifically, one running at *.localhost subdomains They key challenge was to ensure *.localhost DNS names resolve to 127.0.0.1 on all platforms. We do that by setting up HTTP Gateway port of local go-ipfs to act as HTTP Proxy. This removes DNS lookup step from the browser, and go-ipfs ships with implicit support for subdomain gateway when request comes with "Host: <cid>.ipfs.localhost:8080" or similar. We register HTTP proxy using Firefox and Chromium-specific APIs, but the end result is the same. When enableid, default gateway uses 'localhost' hostname (subdomain gateway) instead of '127.0.0.1' (path gateway) and every path-pased request gets redirected to subdomain by go-ipfs itself, which decreases complexity on browser extension side. By default, extension will now redirect from public subdomain gateways such as dweb.link: https://bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq.ipfs.dweb.link/wiki/ to the local one: http://bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq.ipfs.localhost:8080/wiki/ Redirect to gateway logic was path based, this is wip surgical refactor to unify subdomain and path handling, hopefully decreasing maintenance buden going forward This is work in progress. Fixes will be applied in separate commits.
This avoids ibreaking IPFS content paths by converting them to URI params
It should not block the init()
Firefox 74 does not mark *.localhost subdomains as Secure Context yet (https://bugzilla.mozilla.org/show_bug.cgi?id=1220810#c23) so we can't redirect there when we have IPFS resource embedded on HTTPS page (eg. image loaded from a public gateway) because that would cause mixed-content warning and subresource would fail to load. Given the fact that localhost/ipfs/* provided by go-ipfs 0.5+ returns a redirect to *.ipfs.localhost subdomain we need to check requests for subresources, and manually replace 'localhost' hostname with '127.0.0.1' (IP is hardcoded as Secure Context in Firefox). The need for this workaround can be revisited when Firefox closes mentioned bug. Chromium 80 seems to force HTTPS in the final URL (after all redirects) so https://*.localhost fails. This needs additional research (could be a bug in Chromium). For now we reuse the same workaround as Firefox. To unify use of 127.0.0.1 and localhost in address bar (eg. when user opens an image in a new tab etc) when Subdomain Proxy is enabled we normalize address bar requests made to the local gateway and replace raw IP with 'localhost' hostname to take advantage of subdomain redirect provided by go-ipfs >= 0.5
Chromium 80 already hardcodes localhost to loopback IPs, and we don't want to add unnecessary permission as it may cause us fail review at Chrome Web Store. I also renamed the setting as its about overall Subdomain support on localhost, even when HTTP proxy is not used.
lidel
force-pushed
the
feat/subdomain-proxy-support
branch
from
bfa59c3
to
eb66dfa
Compare
| it('should keep the "Origin: null" header ', async function () { | ||
| // Presence of Origin header is important as it protects API from XSS via sandboxed iframe | ||
| // NOTE: Chromium <72 was setting this header in requests sent by browser extension, | ||
| // but they fixed it since then. |
There was a problem hiding this comment.
Removed unused code related to loading webui from gateway port without setting CORS and added test to ensure Companion does not remove Origin: null
lidel
force-pushed
the
feat/subdomain-proxy-support
branch
2 times, most recently
from
a94e46b
to
c3a5d93
Compare
This restores the proper way of opening webui in version provided by IPFS node. Context: Before subdomain gateway support was added, we loaded webui from gateway port. Why? API port has a hardcoded list of whitelisted webui versions and it is not possible to load non-whitelisted CID when new webui was released. To enable API access from webui loaded via Gateway port, the Companion extension removed Origin header for requests coming from its background page. This let us avoid the need for manual CORS setup, but was seen in the diff, was pretty complex process. Webui is stable now, so to decrease maintenance burden we move away from that complexity and just load version whitelisted on API port. What if someone wants to run newest webui? They can now load it from webui.ipfs.io.ipns.localhost:8080 (whitelist API access from that specific Origin by appending it to API.HTTPHeaders.Access-Control-Allow-Origin in go-ipfs config) Closes #736
lidel
force-pushed
the
feat/subdomain-proxy-support
branch
from
c3a5d93
to
4a81bf1
Compare
|
After feedback from two beta releases I feel this is stable enough to merge to master and ship beta+stable. We need this not only for subdomains, but also to ensure POST-only API (ipfs/kubo#7097) scheduled for go-ipfs 0.5 does break DNSLink lookup and ipfs-webui. |
lidel
force-pushed
the
feat/subdomain-proxy-support
branch
from
e174d74
to
5bce2a2
Compare
|
Released to Beta: v2.11.0.904 Subdomains are supported out of the box. If you have go-ipfs 0.5.0-dev you should see interesting change in the address bar: Backward-compatibleVarious edge cases are handled properly now in a way that is backward-compatible. Users of local gateways without subdomains won't see any regressions, but will get a nicer address bar with Subdomain preferenceJust in case, it is possible to restore old school path-based gateway via Preferences screen: It will switch Custom Gateway URL from ExamplesThere will be a longer write up, for now enjoy some test links:
Testing with DockerDocker makes it easy to test with go-ipfs v0.5.0-dev without changing your local setup. $ docker pull ipfs/go-ipfs:master-latest
$ docker run --rm -it --net=host ipfs/go-ipfs:master-latest |
This PR adds support for subdomain gateways introduced in go-ipfs v0.5.0-dev (ipfs/kubo#6096), specifically, one running at
*.localhostsubdomains.Um.. what?
TL;DR we will change the URL present in the address bar when user loads content from the local node (ipfs-desktop or standalone go-ipfs):
to:
Why?
How?
They key challenge was to ensure *.localhost DNS names resolve to 127.0.0.1 on all platforms, specifically Linux distribution which ship with very basic DNS resolver that returns "not found" when trying to resolve localhost subdomains.
Browsers are moving in the direction of hardcoding
localhostnames to loopback interface, but we are not there yet. We need to solve this problem with tools at hand.We do that by setting up HTTP Gateway port of local go-ipfs to act as HTTP Proxy. This removes DNS lookup step from the browser, and go-ipfs ships with implicit support for subdomain gateway when request comes with "Host: .ipfs.localhost:8080" or
similar.
We register HTTP proxy using Firefox and Chromium-specific APIs, but the end result is the same. When enables, default gateway uses 'localhost' hostname (subdomain gateway) instead of '127.0.0.1' (path gateway) and every path-pased request gets redirected to subdomain by go-ipfs itself, which decreases complexity on browser extension side.
TODO
This is work in progress:
localhostthe default127.0.0.1tolocalhostis-ipfswithisIPFS.dnslinkSubdomain(url): feat: support subdomains in isIPFS.url(url) ipfs-shipyard/is-ipfs#32127.0.0.1tolocalhostwhen proxy is enabled (unify user experience)*.dweb.link) to*.locahost<fqdn>.ipns.localhost<fqdn>.ipns.localhostlocalhostports, only the one of current gatewaycaveats
localhostname without port. This is by design, match patterns do not seem to support ports and are executed on all of them. We need to check port inside of handler, and returndirectproxy route if it does not matchlocalhostand*.localhostto loopback IPsRelated
http://<cidv1b32>.ipfs.localhost: Act as HTTP PROXY for http://<cidv1b32>.ipfs.localhost kubo#5982localhostGateway #328