SPIFFS issues with 1.5.4 branch?

[marcelstoer]

I'm trying to finally put some old 512KB modules to use. I flashed them with a binary from the frozen 1.5.4 branch. Upon start the firmware fails to format the file system.

Output from esptool.py

./esptool.py --port /dev/cu.SLAB_USBtoUART write_flash -fm dio 0x00000 /Users/marcelstoer/Downloads/nodemcu-1.5.4.1-spi-i2c-bme280-dht-mqtt-http-float.bin
esptool.py v2.0-beta3
Connecting....
Detecting chip type... ESP8266
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 512KB
Flash params set to 0x0200
Compressed 483380 bytes to 315100...
Wrote 483380 bytes (315100 compressed) at 0x00000000 in 27.7 seconds (effective 139.5 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting...

Output from NodeMCU start
Note: debug is enabled as you'll see

Formatting file system. Please wait...
_flash_used_end:40278174
_flash_used_end:40278174
mount res: 0, 0
Formatting: size 0x1000, addr 0x7a000
_flash_used_end:40278174
...more of the same...
_flash_used_end:40278174

*** ERROR ***: unable to format. FS might be compromised.
It is advised to re-flash the NodeMCU image.
Task task_lua started.
nul mode, fpm auto sleep set:enalbe

NodeMCU custom build by frightanic.com
	branch: 1.5.4.1-final
	commit: 1885a30bd99aec338479aaed77c992dfd97fa8e2
	SSL: false
	modules: bme280,dht,file,gpio,http,i2c,mqtt,net,node,spi,tmr,uart,wifi
 build 	built on: 2017-06-04 22:22
 powered by Lua 5.1.4 on SDK 1.5.4.1(39cb9a32)
lua: cannot open init.lua

I tried this with two ESP-01 and a ESP-201, the behavior is the same for all. I also tried with esptool.py 1.3 (instead of 2.0 beta). It doesn't make any difference whether the flash mode is QIO or DIO. And I did try re-flashing and esptool.py erase_flash before flashing.
Apart from the SPIFFS issue the firmware seems to run fine (tried WiFi and I2C with BME280).

@pjsg any ideas? Since a significant number of people use binaries from the 1.5.4 branch I somehow doubt this is a firmware issue - we would have learned about it long ago. Yet, it irritates me that I'm seeing this behavior on three different devices.

[pjsg]

How much space is left for the filesystem? What does file.fscfg() return? I'm wondering if the min space needed grew.....

[marcelstoer]

Spot on, thanks! It never occurred to me that lack of space could be the reason. I would have expected other types of failures in such cases. I stripped the firmware of a few modules and now life is good (apart from the missing modules 🙁 ).

Results for: file.fscfg()

• good binary: 425984 73728
• bad binary: 491520 12288

Couldn't/shouldn't the firmware check for min size at https://github.com/nodemcu/nodemcu-firmware/blob/1.5.4.1-final/app/modules/file.c#L146 or before format is even invoked and report a constraint violation accordingly?

[pjsg]

I've been looking at the code, and (of course) it isn't that simple. My feeling is that fixing the message would help, and adding some output that reported exactly what was going on would be helpful. I'm thinking that just adding a print in the innards of the format code to say that there is insufficient space -- that this will be enough.

[marcelstoer]

I agree and I'd appreciate a fix for this. So, you'd add to app/platform/vfs.c vfs_format() or just tweak the message(s) at file.c#L152?

[marcelstoer]

Philip, since the 1.5.4 branch is labeled frozen do you think this should be fixed at all?

[pjsg]

No -- this is a curious error case. There are other fixes in the core spiffs codebase that we ought to pick up.....

[pjsg]

I propose that we fix this when we merge in the new version of spiffs.....

[marcelstoer]

I propose that we fix this when we merge in the new version of spiffs

How? Do you plan to back-port #2172 to the frozen 1.5.4 branch?

[pjsg]

I would only fix for the 2.1.0 branch going forwards.....