--allow-paths does not resolve relative paths correctly

[lorenzb]

I ran into some ergonomics issues related to --allow-paths and symlinks.

Problem description

Say I have two contracts Foo (in foo.sol) and Bar (in bar/bar.sol) and assume that foo.sol imports bar/bar.sol.

This obviously works:

$ tree -l
.
├── bar
│   └── bar.sol
└── foo.sol

$ solc foo.sol --metadata
[...]

If we move bar.sol behind a symlink, everything still works:

$ tree -l
.
├── bar -> bar2
│   └── bar.sol
├── bar2
│   └── bar.sol
└── foo.sol

$ solc foo.sol --metadata
[...]

... or not, depending on where the symlink points to:

$ tree -l
.
├── bar -> ../bar2/
│   └── bar.sol
└── foo.sol

$ solc foo.sol --metadata
foo.sol:1:1: Error: Source "bar/bar.sol" not found: File outside of allowed directories.

Weird, maybe I just need to add bar as an allowed directory?

$ solc foo.sol --metadata --allow-paths bar
foo.sol:1:1: Error: Source "bar/bar.sol" not found: File outside of allowed directories.

Nope, not working. Maybe I should try resolving the symlink?

$ solc foo.sol --metadata --allow-paths ../bar2
foo.sol:1:1: Error: Source "bar/bar.sol" not found: File outside of allowed directories.

Hmm, that doesn't work. (See also this issue.)
But this does:

$ solc foo.sol --metadata --allow-paths /absolute/path/to/bar2
[...]

Note that solc generates identical metadata in all three cases that compile:

======= bar/bar.sol:Bar =======
Metadata: 
{"compiler":{"version":"0.4.24+commit.e67f0147"},"language":"Solidity","output":{"abi":[],"devdoc":{"methods":{}},"userdoc":{"methods":{}}},"settings":{"compilationTarget":{"bar/bar.sol":"Bar"},"evmVersion":"byzantium","libraries":{},"optimizer":{"enabled":false,"runs":200},"remappings":[]},"sources":{"bar/bar.sol":{"keccak256":"0x126b05c69a314dd4dc83ce9e9ffcb9832a1cb228f8d594b35ee9845d6cfa9a44","urls":["bzzr://09984546db62920903b6d6f760674a79af34d16306b6a963e19ad4ae35151d75"]}},"version":1}

======= foo.sol:Foo =======
Metadata: 
{"compiler":{"version":"0.4.24+commit.e67f0147"},"language":"Solidity","output":{"abi":[],"devdoc":{"methods":{}},"userdoc":{"methods":{}}},"settings":{"compilationTarget":{"foo.sol":"Foo"},"evmVersion":"byzantium","libraries":{},"optimizer":{"enabled":false,"runs":200},"remappings":[]},"sources":{"bar/bar.sol":{"keccak256":"0x126b05c69a314dd4dc83ce9e9ffcb9832a1cb228f8d594b35ee9845d6cfa9a44","urls":["bzzr://09984546db62920903b6d6f760674a79af34d16306b6a963e19ad4ae35151d75"]},"foo.sol":{"keccak256":"0x32ef0f192269bf14bcb7466f6f069813a9dc371eb9dca5d87426e8554bed0433","urls":["bzzr://ae450631ec9fe187bdfcbf2f01266637fddbe2b2ade5060ea50810e47a7f6703"]}},"version":1}

Suggestions

Can we improve this situation to make it more developer friendly?

Here are a few suggestions for resolving this issue:

1. Follow symlinks (from what I gather, symlinks are currently not supported for security reasons. I must confess that I don't understand those reasons. Either you can trust your FS or you can't.)
2. Allow --allow-paths to:
   2.1 resolve symlinks
   2.2 resolve relative paths
3. If (1) and (2) are too controversial, at least:
   3.1 document the behavior
   3.2 detect that compilation failed due to a symlink/relative path, warn the user, and point them to the docs

[relyt29]

4. Add a "force-follow-symlinks-unsafe-who-needs-security-anyways" option

(Edit: note I don't think this option is the best idea, just wanted to make it known that it is also an approach)

[chriseth]

Note that the metadata is identical because the physical paths on the machine you compile from are irrelevant for the metadata, only the virtual directory structure (with foo.sol at the root and bar/bar.sol inside that) is visible.

As far as I understand the problem, this just looks like relative paths are not properly taken into account for --allowed-paths, so basically my proposal would be to fix it according to (2).

[cameel]

The reason symlinks are not followed is simply that it would make the whole feature pointless. You use it to whitelist specific directories and be sure that nothing outside them cannot be included in the contract. If the compiler followed symlinks, this could be trivially by passed by placing a symlink to any file on your filesystem inside one of the whitelisted directories.

[cameel]

Just to clarify this issue: it's not really a problem with symlinks, relative paths without symlinks do not work properly either:

$ tree -l
.
├── bar2
│   └── bar.sol
└── foo.sol

$ solc foo.sol --metadata --allow-paths ../bar2
[...]