Implement object model for files

[devsaurus]

• This PR is for the dev branch rather than for master.
• This PR is compliant with the other contributing guidelines as well (if not, please describe why).
• I have thoroughly tested my contribution.
• The code changes are reflected in the documentation at docs/en/*.

Current file access functions operate on a default file and restrict the user to only one opened file at a time. This PR adds an object model to the file module which removes this constraint. The basic model is kept for compatibility.

Compared to the current implementation, this extension comes at costs regarding

• memory - opening a file now allocates a Lua userdata (containing a single int)
• run time - functions like file_read() etc. call lua_type() to determine the used model
• dev complexity - both models have to be maintained in parallel
• user complexity - users have to understand the differences to select and consistently use one of the models

This topic was briefly mentioned in the course of the FAT concept discussion, but I'd like to re-check whether there's a common agreement that file should evolve in this direction.

[marcelstoer]

It's probably no surprise that I like this model way better than the existing one. It always felt odd calling functions on "the module" rather than on the "the thing" (i.e. the file) affected by those function calls. Thus the file module was also not consistent with other modules like .e.g. MQTT.

dev complexity - both models have to be maintained in parallel

That should only be temporary. If you clearly mark the old style as being deprecated in the docs and attach a "will be removed in 6 months"-label to it we have a way forward. Which reminds me...we did the same for tmr but without giving a time frame, which was not a good idea.

P.S. "Only one file can be open at any given time." in the intro of the docs needs to be removed or rephrased.

[jmattsson]

Quick note: SPIFFs working buffer might need to grow, I cut those back to bare minimum (supporting 2 files in parallel) way back when. This comment might already be out of date, however.

[pjsg]

I thought that we allowed 64 open spiffs files. I don't think that we need to increase the buffers for multiple files, but I'm not sure.

[devsaurus]

Thanks for highlighting this. I'll set up some stress tests with many open files to probe for the upper limit.

[devsaurus]

I get SPIFFS_ERR_OUT_OF_FILE_DESCS error when opening the 4th file 😞 Wasn't aware that SPIFFS uses a static buffer for the descriptors, but it's pretty obvious from https://github.com/nodemcu/nodemcu-firmware/blob/dev/app/spiffs/spiffs_hydrogen.c#L100:

fs->fd_count = (fd_space_size/sizeof(spiffs_fd));

Our spiffs.c provides 32*4 bytes for descriptor storage which fits 3 instances of spiffs_fd (size 36). Should I turn this into 36 * SPIFFS_MAX_OPEN_FILES and place the define in user_config.h (set to 4)?

[pjsg]

I'd use sizeof(spiffs_fd) * SPIFFS_MAX_OPEN_FILES

[TerryE]

Thanks Arnim, this sounds like great improvement. I suspect that most Apps won't use it, but when they do it will prove extremely useful. 😄

[devsaurus]

Rebased onto current dev.

[danjohn]

Fine extension to file.* module, appreciate this!

Please add a function to get details out of a file object... maybe I'm wrong, but it seems to be very hidden by your approach, as there are no getters().
If e.g. there is to be making a backup copy of a file, tha filename of the encapsulated object has to be evalueable. Or think of a file object, thats passed between functions.. that like to know, if it's W/R.

I suppose, that can be found somewhere in _G, but a simple file:getStat() (or the like) would be a great help for the file module UI!

[devsaurus]

@danjohn see #1724, adds the file.stat() function to propagate file attributes to Lua level.