sys/vfs: add file-system auto-mount

[benpicco]

Contribution description

This adds a way to let boards define automatic mount points for file systems, similar to what fstab provides.
This also bring the handling of the FAT VFS integration more in line with the other file systems.
The user no longer has to manually manage the fatfs_mtd_devs array when using the VFS layer.

Testing procedure

In e.g. examples/default add

USEMODULE += vfs
USEMODULE += vfs_auto_format
USEMODULE += littlefs2

This is now all that is needed to add file system support to an application. The vfs_auto_format module will attempt to format the file system if mounting fails (no file system present yet).

ls and the vfs commands should work on the shell.

You can add auto-mounts to an existing board by adding a single macro to the compilation unit in which the MTD is defined:

VFS_AUTO_MOUNT(littlefs2, &mtd0_dev.base, "/", 0);

Issues/PRs references

vfs_auto_format requires #14430 to work on fatfs

[fjmolinas]

A couple of nitpicks.

[benpicco]

I added a vfs_auto_mount default module that gets disabled by the example.

[benpicco]

I changed

VFS_AUTO_MOUNT(littlefs2, mtd0_dev, "/", 0);

to

VFS_AUTO_MOUNT(littlefs2, &mtd0_dev.base, "/", 0);

to reduce preprocessor usage

[chrysn]

OK, please squash while I'm doing some more testing. If you like, drop a3fa01b (the .dev to .base rename) while doing so, but the consistency doesn't do any harm.

[chrysn]

My tests passed, both on native and with a modified mtd_flash on the microbit-v2, both with littlefs2. One test I'd like to add but failed so far was with fatfs (at least on native; on emulated MTD there seems to be major SNAFU when the block device doesn't have 512 byte blocks/sectors/pages/whatever and fatfs just copies a page into its 512-sized buffer...); while on master I did succeed mounting a FS created using #14430's formatting, the same test failed on this branch (even only looking at the examples/filesystem example that shouldn't have changed much); can you confirm this works for you?

(Not relevant any more, but for the record: The switch over from using .base in VSF_AUTO_MOUNT toward passing the pointer to the actual device *mtd_dev_t was done on to avoid starting off a preprocessor-based duck-typing system; see some Matrix chat from 2022-02-08).

During my tests I've wound up in some loop like this once (from a board's mount line VFS_AUTO_MOUNT(fatfs, &some_mtd, "/internal", 0);, likely from the above-mentioned SNAFU):

vfs_mount: -> "/internal" (0xb155), 0x20000200
vfs_bind: bound 1
vfs_bind: 2, 1, 0x2000048c, 0x2
vfs_bind: bound 2
vfs0: mounting as '/internal'
vfs_mount: 0x2000050c

I wonder whether a warning is warranted that anything that sends a device into a reboot loop (while running with DEVELHELP=0) might cause quick flash wear with file systems that perform writes on mount. (I can't tell quickly whether ours really do; I know that there is some writing going on with Linux file systems, judging from warning notes about data forensics I've read some time ago).

Other than that, things are looking good; the new in-code commends are just that (I'd be OK going ahead with this even without them merged, but I'd like them to be considered).

[chrysn]

The automounter only depending on VFS and not on MTD (which, above, defines mtd0 in the first place) may look odd in later review, but is really OK: All mentions later to mtd0 are behind further ifdefs that (transitively) do depend on MTD because their backend is MTD.

If, at some time, someone were to write a native filesystem pass-through, its backend would not be an MTD device but a host POSIX path, thus its second argument to VFS_AUTO_MOUNT would become just that, and no dependency on the MTD module is there for that case.

[edit: highlighting that things are OK here, primarily for the benefit of people coming back to this issue, wondering]

[benpicco]

I also noticed that fatfs was broken on native but would work with tests/pkg_fatfs_vfs. This made me scratch my head, but your comment about the sector size made me look closer: And indeed the Makefile of tests/pkg_fatfs_vfs does

CFLAGS += -DMTD_PAGE_SIZE=$(MTD_PAGE_SIZE)
CFLAGS += -DMTD_SECTOR_SIZE=$(MTD_SECTOR_SIZE)

and even the Readme of examples/filesystem has a Section about this.

Now I also don't like reading Readmes, should we just

#ifndef MTD_PAGE_SIZE
#ifdef MODULE_FATFS_DISKIO_MTD
#define MTD_PAGE_SIZE           (512)
#else
#define MTD_PAGE_SIZE           (256)
#endif
#endif

#ifndef MTD_SECTOR_SIZE
#ifdef MODULE_FATFS_DISKIO_MTD
#define MTD_SECTOR_SIZE         (512)
#else
#define MTD_SECTOR_SIZE         (4096)
#endif
#endif

[chrysn]

Now I also don't like reading Readmes, should we just

... and I missed that too.

I have a soft preference for fatfs_diskio_mtd to produce an (unfortunately runtime) error if the MTD device's sizes don't agree with it. Tuning the defaults for native's MTD is certainly a good idea, the check could be done independently as well.

[chrysn]

Thank you, please squash.

(Reading that PR I also realized that fatfs can not be expected to work on many flash-backed MTDs given that thee have 4k erasure units but FAT overwrites in 512 byte blocks; well, probably no reason to use FAT there anyway, for its main selling point is "plug the thing into a PC and it'll work with it").

[benpicco]

All green here :)

[chrysn]

And green here. Thank you!

[benpicco]

Thank you for the review!