v2.0
What's new?
Big things
-
Metadata logging
Metadata pairs now act as small two block logs. This allows incremental updates without erasing metadata blocks, which is a significant performance improvement. Additionally, this enables dynamically sized entries, which is necessary for the two following features.
-
Custom attributes
It's now possible to get and set custom attributes on files, directories, and the even the filesystem itself. Custom attributes use a byte as an identifier and can be up to 1022 bytes large. This is useful for things such as timestamps, permissions, hashes, or other attributes.
-
Inline files
Files can now be inlined directly in the directory block. This allows multiple small files to be coalesced into a single block instead of wasting an entire block for each file. This opens up the possibility for littlefs to be used on NAND flash and MCU internal flash, both which often have block sizes >16KiB. Note that inline files are limited by the cache size.
-
Independent cache sizes
littlefs now has a separate cache_size config option. This config option determines the size of the read cache, prog cache, and file caches independently of the storage's physical read/prog size. This allows more granular reads and writes without artificially limiting the cache, improving performance and log utilization.
-
XOR global state, no more move problem
XOR global state is a mechanism that allows littlefs to store a small amount of global state distributed across the filesystem. This global state is now used internal to fix moves in O(1), a significant improvement over the previous O(n) search. Additionally XOR global state may be useful for future improvements.
-
True dynamic wear-leveling
littlefs now proactively levels wear over dynamic blocks instead of reacting only to bad blocks. This offers better protection against wear that causes stored data to deteriorate.
Dynamic wear-leveling is provided by evicting blocks after a bounded number of writes determined by the block_cycles config option. A combination of random offset and incremental allocation of blocks provide a uniform distribution of wear over the storage.
-
Expanding superblocks
The superblock no longer has a flat cost of 2 blocks. Instead, the superblock doubles as the root directory until block_cycles are passed. After every block_cycles number of erases, the superblock is expanded to include another metadata pair if space is available. This allows littlefs to use as few superblocks as is needed to avoid an early death.
At the smallest, littlefs can now fit in only 2 blocks.
Small things
-
Configurable name max, file max, and attr max
These limits can now be set during format and should be respected on other devices. This means that the RAM cost of stat can be reduced without losing portability.
-
Added lfs_fs_size
A function to get a count of the used blocks on the filesystem. Likely easier to use than lfs_fs_traverse.
-
Better update tracking
Metadata logging, with resizable entries, means that there's a lot of state changes flying around. This required a review and ultimately a rewrite of the way littlefs manages open files and dirs. This should be much more robust moving forward and also has a small performance improvement.
Important changes
- Must set cache_size. If read_buffer or prog_buffer are provided manually, these must be cache_size.
- Highly suggested to set block_cycles in the range of 100-1000. This enables wear leveling.
- Change lookahead -> lookahead_size. The lookahead buffer must now be aligned to a 64-bit boundary.
- Drop file_buffer in favor of per-file buffers.
- Renamed lfs_traverse -> lfs_fs_traverse. Note the addition of lfs_fs_size can replace the most common use of lfs_traverse.
- Change lfs_crc to not use a pointer if it was overridden.
Migration
Following the discussion on #127, releases now have three special branch/tags that are generated as a part of CI:
- A patch-specific tag (
v1.7.2), immutable - A major version branch (
v1), automatically updated every patch release - A major version prefix branch (
v1-prefix) aromatically generated every patch release. Note that this is generated as a merge commit, making it easy to update code that needs to avoid name conflicts between multiple versions of littlefs.
To help adoption with the breaking disk changes, this version includes a migration function designed to help migrate v1 filesystems to v2.
Note that this migration is best-effort, if it fails, an error is returned and the v1 filesystem is left unmodified.
Enabling the migration requires two steps:
- Defining LFS_MIGRATE
- Call lfs_migrate (only available with the above macro)
More info about this here.
Documentation
Documentation has been brought up to date with the changes. More info on how littlefs v2 works can be found in the updated DESIGN.md and SPEC.md.
Performance comparison of v1 and v2
Calculated by simulating v1 and v2 with various file counts and file sizes. Results are divided by the count*size of the files to get an easier-to-compare multiplicative cost. Note that these results are still on the smaller scale for storage. Smaller numbers are better.
As expected, the performance does not change much for reading (and even gets worse on NAND). However, the prog/erase performance is much better. This is desired as erasing has a much higher runtime penalty than reading,
There's also a moderate improvement to storage consumption thanks to inline files, though note that the storage consumption converges as the file size increases.
