Proposal: Extending the Filesystem of World

[ludns]

Introduction

The World has an implicit filesystem described with Routes. In this Filesystem, two types of "files" are stored: Tables and Systems.
Currently, the Filesystem is used to facilitate two features:

• Access control for Store: Systems in the same folder as a Table can write to the Table
• Access control for Systems: Systems in the same folder as a System can call the System

As planned in #393 (World framework), nested routes are used to proxy write access (ie: Systems having access to the parent route of a Table or System in a child route can call / write them). This is however not currently implemented.

I propose three modifications to the Filesystem:

1. Rename subRoute to File
2. Add file types to differentiate between directories (a File that contains other files), Systems, and Tables.
3. Add one additional file type

1) Rename subRoute to File and store all File related data into a FileTable

I have found subRoute to be a confusing name; and I propose renaming it to File to match expectations. This change -- combined with adding file types -- turns the current pseudo-Filesystem into something more akin to a traditional filesystem.
An initial FileTable schema could look like such:

• primary key: routeId (uint256)
• name: string
• parent: routeId (uint256)
• content: uint256 (stores an address for all file type but the TABLE one, where it instead stores the Schema)
• fileType: DELEGATECALL, CALL, TABLE, ASSET

2) Add file types to differentiate between directories (a File that contains other files), Systems, and Tables.

Currently, file types are implicit; and different types of resources can have the same full-name (eg: there can be a system at /mudswap/SwapSystem and a table at /mudswap/SwapSystem, which is IMO very confusing).
I propose creating a unified FileTable which enforces unique File names, even if the Filetype is different.
Enforcing unique names across different file types mean we can drop the file type when operating over files.
The initial set of proposed file types are found in the Appendix: Description of the file types

3) Add one additional file type

There is a case to be made for non-executable data stored in the filesystem. Additionally, we should pave the way for more file types than the ones we have right now.
I propose adding one additional file types over the three implicit ones existing (Folder, System, and Table)

• ASSET: a piece of data stored under an address with SSTORE2 that is not executable.
  This would allow each World to store static files in the style EthFS.
  Systems will be able to write to these files (in practice: deploy a new contract and change the content field of the file to the new address). This can be used by clients to load assets from the chain.

Appendix: Description of the file types

• FOLDER: a file that can be used as a baseRoute. created with createdFolder (current registerRoute)
• SYSTEM: the EVM byte code at the corresponding address will be CALLed called from the World (or DELEGATECALLed for root Systems); some context will be appended on the calldata (for now: the original msg.sender)
• TABLE: a Store table, where its Schema can be found in the content column. It doesn’t have an address because it lives on the OS
• ASSET: anything else stored via SSTORE2 on-chain.

[alvrs]

TL;DR: I agree paving the way for more file types is interesting and important to integrate while working on the foundations. Have left some thoughts on implementation details like the FileTable and access control below.

I have found subRoute to be a confusing name; and I propose renaming it to File to match expectations.

In the current approach the subRoute can be a nested route too. Access to /baseRoute also grants access to /baseRoute/nested/sub/route/file with an O(1) access control check (the World only needs to check access on /baseRoute, because the subRoute is guaranteed to be a sub route since it is concatenated with the base route in the World contract). If we change this to file, then the current access control check wouldn't work for nested routes anymore. We can explore alternative approaches for access control, but so far I haven't found any that are as gas efficient as the current approach (which only requires a single storage access and two keccak256 for any depth of baseRoute or subRoute).

An initial FileTable schema could look like such:

primary key: routeId (uint256)
name: string
parent: routeId (uint256)
content: uint256 (stores an address for all file type but the TABLE one, where it instead stores the Schema)
fileType: DELEGATECALL, CALL, TABLE, ASSET

I generally like the approach of making more file types possible. One thing on my mind is whether we would check the fileType in functions like call/setRecord. If we do, it would lead to an additional sload. If we don't it might lead to undefined behavior if someone tries to call a table or setRecord on a system (because the content is interpreted differently). The advantage of splitting up data relevant for certain file types into different tables is invalid actions on file types would fail explicitly, without the need for an additional sload to check the file type, and we would have the benefit of leveraging our typed libraries for working with the content data. (Eg for systems we probably want to store the openAccess flag in the content field to minimize sload - with its own table we can work with bool and address, with the unified table we'd have to manually encode/decode it from the raw content field). And with limiting content to 32 bytes we introduce a limitation for future file types (maybe we want to add one that requires more than 32 bytes of content).

Currently, file types are implicit; and different types of resources can have the same full-name (eg: there can be a system at /mudswap/SwapSystem and a table at /mudswap/SwapSystem, which is IMO very confusing).

I agree this is confusing, and it's unintended. It is possible because of registerSchema, which registers a table at an existing route to match the IStore interface. I agree we should prevent the registration of a table (or any other file type) at a route that already has a file. We can also achieve this without a unified FileTable (see thoughts on limitations of unifying everything in the paragraph above), eg by only storing route -> fileType in a unified table, and having separate content tables per file type.

There is a case to be made for non-executable data stored in the filesystem. Additionally, we should pave the way for more file types than the ones we have right now.
I propose adding one additional file types over the three implicit ones existing (Folder, System, and Table)

• ASSET: a piece of data stored under an address with SSTORE2 that is not executable.
  This would allow each World to store static files in the style EthFS.
  Systems will be able to write to these files (in practice: deploy a new contract and change the content field of the file to the new address). This can be used by clients to load assets from the chain.

Agree this is interesting, @holic probably has thoughts on how to best integrate something like EthFS / support static files.