Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timthelion's opinion on the dweb:/ipfs vs /ipfs/ debate #153

Closed
wants to merge 1 commit into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
189 changes: 189 additions & 0 deletions addresses-scheme/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,192 @@ In [this cryptic comment](https://github.com/ipfs/in-web-browsers/issues/28#issu
In the end this hack would let you have addresses that look like `ipfs:/QmdyWzsrBvSkPYPU1ScBpwzfCcegzbc6c2hkEBLLJ6VcPV` while also allowing people to address that same content as `dweb:/ipfs/QmdyWzsrBvSkPYPU1ScBpwzfCcegzbc6c2hkEBLLJ6VcPV` or, in a unix/posic contenxt just `/ipfs/QmdyWzsrBvSkPYPU1ScBpwzfCcegzbc6c2hkEBLLJ6VcPV`.

to quote him from an offline conversation, @nicola poses this as the baseline -- we have to beat this in terms of simplicity of use. Calling it an ugly hack isn't good enough. You need to pose a better solution that creates **cleaner, more reliable, or more powerful addresses**.

### @timthelion's viewpoint (relates only to native POSIX, and not to HTML/the web)

@timthelion does not see the debate of `fs:/...` vs `dweb:/...` vs `ipfs:/...` to be worldshaking. All of these mechanisms would, according to him, work. The only real difference between them, that he can see, is asthetic. He is, however, strongly opposed to `/ipfs/...`. He sees it as mixing namespaces. @jbenet writes of `/ipfs/...` "addressing of data broke when URLs and URIs were defined as a space OUTSIDE unix/posix paths, instead of INSIDE unix/posix paths (unlike say plan9's 9p transparent addressing)". @timthelion, however, takes the opposite view. He does not see the fact of URLs and URIs having their own namespace as being problematic.

The reason why he sees this mixture of namespacing as being problematic is that current POSIX utilities, when asked to open a file, see a string which represents a path to a file. They then take this string, dumbly, without examining it, and pass that string to the system's `open` function. If you add a new layer to the POSIX `/` namespace, which cannot be understood by `open` then every program must decide whether to send that string onto `open` or to send it somewhere else. That decision should be easy, and in the case of URLs it is. Paths NEVER contain `:` unless they are a URL and therefore, it is trivial to figure out what can be sent to `open` and what must be sent elsewhere. Of course, timthelion understands that it is sad that the vast majority of POSIX utilities don't support opening URLs, but he sees this as a problem that can be solved, by adding a few lines of code to each utility, or, more universaly, by adding URL support to the OS's `open` function. He sees, adding a new layer to the `/` namespace as being a move that only complicates this transition and actually makes it harder, and not easier, to re-unite POSIX with the web.

Here is a peice of python code which is able to open both http urls and POSIX files.

````
if filename.startswith( "http://" ) or filename.startswith( "https://" ):
import urllib.request
try:
with urllib.request.urlopen(filename) as webgraph:
self.json = webgraph.read().decode("utf-8")
except urllib.error.URLError as e:
raise OSError(str(e))
else:
try:
with open(filename) as fd:
self.json = fd.read()
except FileNotFoundError:
pass
````

Modifying this code in order to support `/ipfs/` paths is not left as an excercise to the reader, read on:

@timthelion wrote a wrapper for the `diff` utility which supports ipfs so as to demonstrate his point. The diff util has two implemenations. The first interprets `/ipfs/..` style paths, and works in the naive case:

````
timothy@yoga ~/c/ipfs-multiaddr> ./multiaddr-diff ../subuser/COPYING.LGPL /ipfs/QmSRrBvLXvYQRdQ3kZtJ5oJicKMcNQzC3CwH6bJDbEKWYp
127a128,130
> f) Sacrifice your first born child to the GOD Laurath on the first
> full moon of the following even numbered year.
>
````

However, it contains a bug, if an `/ipfs/..` folder actually exists on the system. That folder is unnaccsessable to the modified diff util:

````
timothy@yoga ~/c/ipfs-multiaddr> su
Password:
root@yoga:/home/timothy/current/ipfs-multiaddr# mkdir /ipfs/
root@yoga:/home/timothy/current/ipfs-multiaddr# echo "foo">/ipfs/foo
root@yoga:/home/timothy/current/ipfs-multiaddr# exit
exit
timothy@yoga ~/c/ipfs-multiaddr> ./multiaddr-diff ../subuser/COPYING.LGPL /ipfs/foo
Error: selected encoding not supported
....
````

Here is the sorce code to the `/ipfs/..` version of the utility:

````
timothy@yoga ~/c/ipfs-multiaddr> cat multiaddr-diff
#!/bin/bash
get_multiaddr_or_normal_file_getter(){
if [[ $1 == /ipfs/* ]] ;
then
file_getter="ipfs cat $1"
else
file_getter="cat $1"
fi
echo $file_getter
}

file1=`get_multiaddr_or_normal_file_getter $1`
file2=`get_multiaddr_or_normal_file_getter $2`

diff <($file1) <($file2)
````

He also wrote a second utility which iterprets paths which start with `dweb:/ipfs/` and sends only those paths on to ipfs.

Like the first version, this version works in the naive case:

````
timothy@yoga ~/c/ipfs-multiaddr> ./url-syntax-diff ../subuser/COPYING.LGPL dweb:/ipfs/QmSRrBvLXvYQRdQ3kZtJ5oJicKMcNQzC3CwH6bJDbEKWYp
127a128,130
> f) Sacrifice your first born child to the GOD Laurath on the first
> full moon of the following even numbered year.
>
timothy@yoga ~/c/ipfs-multiaddr>
````

Unlike the first version, it is **also** capable of accessing a real existing `/ipfs/` directory:

````
timothy@yoga ~/c/ipfs-multiaddr> echo "bar" >bar
timothy@yoga ~/c/ipfs-multiaddr> ./url-syntax-diff bar /ipfs/foo
cat bar
cat /ipfs/foo
1c1
< bar
---
> foo
````

Here is the source code to the second version, so you can play with it.

````
timothy@yoga ~/c/ipfs-multiaddr> cat url-syntax-diff
#!/bin/bash
get_multiaddr_or_normal_file_getter(){
if [[ $1 == dweb:* ]] ;
then
prefix="dweb:"
internal_path=${1#$prefix}
file_getter="ipfs cat $internal_path"
else
file_getter="cat $1"
fi
echo $file_getter
}

file1=`get_multiaddr_or_normal_file_getter $1`
file2=`get_multiaddr_or_normal_file_getter $2`

diff <($file1) <($file2)
````

Another example that @timthelion came up with was the case of a hypothetical markdown to pdf utility which supports ipfs. Imagine we have a utilty named `md-to-pdf-with-ipfs-support` and we pass it a markdown file like so:

**atic.md**
````
Stuff I found in my attic
----------------------------------

![An old box of rocks](/ipfs/QmdyWzsrBvSkPYPU1ScBpwzfCcegzbc6c2hkEBLLJ6VcPV)

An old box of rocks.

![A can of oil for water-proofing leather](/ipfs/QmUPC5xbVtu6NxMwFBtmWVjrVM3XffuPtSMLpmDFGfTaKG)
````

````
$ md-to-pdf-with-ipfs-support attic.md > attic.pdf
````

How is this utility supposed to recognise which paths to resolve using `ipfs` and which paths to resolve normally? @flyingzumwalt suggests that prefixing `dweb:` for ipfs paths resolves this ambiguity and @timthelion concures.

````
Stuff I found in my attic
----------------------------------

![An old box of rocks](dweb:/ipfs/QmdyWzsrBvSkPYPU1ScBpwzfCcegzbc6c2hkEBLLJ6VcPV)

An old box of rocks.

![A can of oil for water-proofing leather](dweb:/ipfs/QmUPC5xbVtu6NxMwFBtmWVjrVM3XffuPtSMLpmDFGfTaKG)
````

@timthelion does not see resolving `/ipfs/` paths to be difficult merely for computers. As a human he is confused as well. When he sees `/ipfs` written on his local machine, he imagines that this means that the ipfs filesystem must be mounted in root for that path to be accessable. After reading the [multiaddr specs](https://github.com/multiformats/multiaddr) he had the feeling that this system would create unimaginable clutter on his system.

"""
Today, when I do `ls /` I get:

````
$ ls /
bin/ dev/ home/ lib/ media/ opt/ root/ sbin/ sys/ usr/ vmlinuz@
boot/ etc/ initrd.img@ lib64/ mnt/ proc/ run/ srv/ tmp/ var/
````

With the paths proposed by multiaddr I would instead see:

````
$ ls /
bin/ bitcoin/ boot/ dev/ dns/ dns4/ dns6/ etc/ home/ http/ https/ initrd.img@ ipfs/ lib/ lib64/
libp2p-circuit-relay/ libp2p-webrtc-direct/ libp2p-webrtc-star/ media/ mnt/ onion/ opt/ p2p/
proc/ root/ run/ sbin/ sctp/ srv/ sys/ tmp/ udt/ unix/ usr/ utp/ var/ vmlinuz@
````
[multiaddr-protocols](https://github.com/multiformats/multiaddr/blob/master/protocols.csv)
"""

While this may be a misunderstanding on his part, other users may be confused as well.

@timthelion suggests, that if the ipfs devs insist on mixing special paths into the POSIX `/` namespace, they at least use a subdirectory and not polute all of the filesystem root `/`.

### @timthelion's compromise: place all `multiaddr` filesystems under a special directory `/webfs/` and treat the prefix `/webfs/` as a magic path prefix
Copy link
Member

@lidel lidel Mar 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kinda like this aesthetic. No matter what prefix is picked, it looks better than anything with ://
/webfs/ipfs/QmT272yei1Zn1eAUq5P9nZyeaKP4oJmVv7CbYvUPyk3aLj/hobby.jpg
/dweb/ipfs/QmT272yei1Zn1eAUq5P9nZyeaKP4oJmVv7CbYvUPyk3aLj/hobby.jpg
/x/ipfs/QmT272yei1Zn1eAUq5P9nZyeaKP4oJmVv7CbYvUPyk3aLj/hobby.jpg


If all of the "magic" `multiaddr` filesystems were placed in a subdirectory of root named `/webfs/` then it would be possible for utilities to treat paths starting with `/webfs/` as being magical and non-standard. @timthelion doesn't see this as being ideal, but it is better then having N new magic subdirectories of root `/`. In @timthelion`s oppinion, this would be somewhat analogous to how we sometimes mount flash disks and CDRoms to `/mnt/` or `/media/` but we never mount them in the root directory.

In that case, `ls /` would look like:

````
$ ls /
bin/ dev/ home/ lib/ media/ opt/ root/ sbin/ sys/ usr/ vmlinuz@
boot/ etc/ initrd.img@ lib64/ mnt/ proc/ run/ srv/ tmp/ var/ webfs/
````