Forwarding requested dat archives to shortname subdomain

[rjcorwin]

This PR was inspired by a discussion with @RangerMauve who is also working exciting additions to dat-gateway to allow DatArchive support through a websockets connection to dat-gateway bunsenbrowser/bunsen#40

This refactors how dat archives are accessed by placing them on subdomains in order to ensure that browser memory is sandboxed between dat archives. This works by forwarding a request to http://dat.<top level domain name>/<dat archive uuid> to http://<first 6 characters of dat archive uuid>.<top level domain name>/.

I wish it could be as simple as http://<dat archive uuid>.<top level domain name>/, but alas, as @creationix pointed out to me, DNS doesn't support segments longer than 63 characters..

@pfrazee I'm open to ways that this can be done better! The exciting part is that this satisfies the security needs that we need for Bunsen Browser.

[rjcorwin]

[pfrazee]

Why are you using a gateway at all? Are you not able to run Dat and provide a custom dat:// protocol inside bunsen?

[rjcorwin]

Why are you using a gateway at all? Are you not able to run Dat and provide a custom dat:// protocol inside bunsen?

Secrets out ;). We've been running a custom node based dat gateway running in the APK this whole time. Moving to dat-gateway would be nice for collaborating, especially since it's the go to for dat-fox and we're working on the same problem of how to implement DatArchive.

[rjcorwin]

A bit more info, the Beaker UI is an app that interacts with the gateway. When you request a dat archive address in the UI, it transforms it to what's needed for the gateway and then opens an iframe pointing at the gateway.

[rjcorwin]

@RangerMauve No luck with playing with Base32 stuff, but there is a magic library for compressing strings.

The nice part about this is that it would do away with the need for maintaining that map of shortnames to dat archive addresses while staying under the 63 character limit required by DNS.

[RangerMauve]

Are you sure the compressed string will work for URLs? AFAIK URLs only support ASCII and other utf8 characters get converted to punycode which is actually a lot longer.

[rjcorwin]

Oooooh right. Damn.

[RangerMauve]

https://stackoverflow.com/questions/33844325/javascript-hex-to-base32-encoding?rq=1 Maybe use this?

[pfrazee]

Secrets out ;). We've been running a custom node based dat gateway running in the APK this whole time. Moving to dat-gateway would be nice for collaborating, especially since it's the go to for dat-fox and we're working on the same problem of how to implement DatArchive.

@rjsteinert I'm a little concerned about that. It's not really using the p2p network, is it? It's a gateway dressed up as a browser

[RangerMauve]

It's a gateway dressed up as a browser

From what I understand the gateway is running locally within the app. The gateway itself is interacting with the p2p network.

[chrisekelley]

@pfrazee Hey Paul, we are using the p2p network, specifically, dat-node to download the archive to ram and then pass it on to mirror-folder to persist to the file system. (Hmmm, couldn't I just go straight to fs??? May be time to refactor...) Anyway I see that we're doing dat.joinNetwork() and have some stubs setup to monitor network connections:

dat.network.on('connection', function () {
      console.log('I connected to someone for ' + datId)
      console.log('connected to', network.connections.length, 'peers')
    })

Here's the code:
https://github.com/bunsenbrowser/bunsen/blob/master/www/nodejs-project/index.js

Anyway, Bunsen currently can open a dat:// scheme link from sites like hashbase.io (via Android intents), save the dat archive to the Android fs, make it available on the network, and display them in an iframe as RJ mentioned. RJ identified a security issue with that approach (bunsenbrowser/bunsen#30) so he's been looking at ways to ameliorate this situation. Plus, it sure would be nice to host more than one dat in Bunsen! So dat-gateway may be our gateway to sorting out these issues.

[RangerMauve]

Working on the hex-to-32 library now.

[rjcorwin]

Thanks @RangerMauve - that would be awesome.

[pfrazee]

@chrisekelley Oh I see, the lvh.me domain resolves to localhost. Hm, ok, I get that

[pfrazee]

Thanks for explaining it to me!

[rjcorwin]

lvh.me domain resolves to localhost

I didn't know about that either until @fuzzyTew pointed to it in that security issue. Such a great suggestion! But @fuzzyTew does have concerns about lvh.me being hijacked but I honestly don't quite understand. I've been trying to look up how to programmatically manipulate DNS in Android but my google fu has failed me thus far. Either way, this PR brings more security to dat-gateway on any domain by making sure that cookies and localStorage are going to be sandboxed as you move from Dat Archive to Dat Archive on the gateway.

[RangerMauve]

@rjsteinert https://github.com/RangerMauve/hex-to-32

Also, would you mind forking off of my websocket branch instead so that there's less to keep track of?