Use @nodelib/fs.walk

[mrmlnc]

Just a topic to control the move to the @nodelib/fs.walk package.

[mrmlnc]

Just for history. Windows 10. NVMe. Node.js 11. 20k entries in the node_modules directory. Right now only synchronous version.

$'NODE'; node ./fs.walk.sync.js; ''; 'PYTHON'; python ./glob.py; ''; 'RUST'; ./rr/target/release/rr.exe

NODE
readdir-enhanced: 2580.425ms
readdir-enhanced: 2506.904ms
readdir-enhanced: 2439.665ms
readdir-enhanced: 2425.582ms
readdir-enhanced: 2427.237ms
readdir-enhanced: 2411.686ms
readdir-enhanced: 2414.529ms
readdir-enhanced: 2400.802ms
readdir-enhanced: 2396.740ms
readdir-enhanced: 2410.073ms
readdir-enhanced: 2448.938ms
'---'
rrdir: 354.530ms
rrdir: 329.836ms
rrdir: 315.849ms
rrdir: 319.082ms
rrdir: 330.218ms
rrdir: 320.658ms
rrdir: 319.012ms
rrdir: 317.120ms
rrdir: 326.768ms
rrdir: 340.706ms
rrdir: 329.340ms
'---'
fs.walk: 309.706ms
fs.walk: 308.196ms
fs.walk: 285.142ms
fs.walk: 286.360ms
fs.walk: 288.033ms
fs.walk: 284.356ms
fs.walk: 285.615ms
fs.walk: 282.716ms
fs.walk: 286.125ms
fs.walk: 284.812ms
fs.walk: 291.108ms

PYTHON
os.walk 567.9525ms
os.walk 568.9956ms
os.walk 568.0302ms
os.walk 583.9758ms
os.walk 570.9993ms
os.walk 609.0039ms
os.walk 570.0322ms
os.walk 559.0091ms
os.walk 554.9662ms
os.walk 566.0604ms

RUST
rust 284ms
rust 281ms
rust 288ms
rust 303ms
rust 281ms
rust 296ms
rust 302ms
rust 286ms
rust 280ms
rust 291ms

[mrmlnc]

fs.walk.async: 119.546ms
19675
fs.walk.stream: 124.888ms
19675
(node:16688) ExperimentalWarning: Readable[Symbol.asyncIterator] is an experimental feature. This feature could change at any time
fs.walk.stream.iterator: 148.545ms
19675

[mrmlnc]

[root@bench fast-glob]# BENCHMARK_MAX_STDEV=7 BENCHMARK_LAUNCHES=200 npm run bench

> fast-glob@2.2.6 bench /root/fast-glob
> npm run bench-async && npm run bench-sync


> fast-glob@2.2.6 bench-async /root/fast-glob
> npm run bench-async-flatten && npm run bench-async-deep


> fast-glob@2.2.6 bench-async-flatten /root/fast-glob
> node ./out/benchmark --type async --pattern "*"

===> Benchmark pattern (*) with 200 launches (async)

fast-glob-current.js
(TIME) 5.019ms ±0.079% | (MEMORY) 3.115MB ±0.004% | Entries: 6 | Errors: 0 | Retries: 1
fast-glob-previous.js
(TIME) 7.784ms ±0.235% | (MEMORY) 5.078MB ±0.019% | Entries: 6 | Errors: 0 | Retries: 1
node-glob.js
(TIME) 3.213ms ±0.071% | (MEMORY) 2.512MB ±0.046% | Entries: 6 | Errors: 0 | Retries: 1
tiny-glob.js
(TIME) 79.112ms ±4.143% | (MEMORY) 7.507MB ±0.087% | Entries: 14 | Errors: 0 | Retries: 1

> fast-glob@2.2.6 bench-async-deep /root/fast-glob
> node ./out/benchmark --type async --pattern "**"

===> Benchmark pattern (**) with 200 launches (async)

fast-glob-current.js
(TIME) 63.784ms ±3.199% | (MEMORY) 4.948MB ±0.122% | Entries: 4940 | Errors: 0 | Retries: 1
fast-glob-previous.js
(TIME) 99.950ms ±3.877% | (MEMORY) 7.108MB ±0.262% | Entries: 4940 | Errors: 0 | Retries: 1
node-glob.js
(TIME) 253.212ms ±6.499% | (MEMORY) 19.404MB ±2.885% | Entries: 4940 | Errors: 0 | Retries: 1
tiny-glob.js
(TIME) 91.043ms ±4.889% | (MEMORY) 9.730MB ±0.059% | Entries: 5736 | Errors: 0 | Retries: 1

> fast-glob@2.2.6 bench-sync /root/fast-glob
> npm run bench-sync-flatten && npm run bench-sync-deep


> fast-glob@2.2.6 bench-sync-flatten /root/fast-glob
> node ./out/benchmark --type sync --pattern "*"

===> Benchmark pattern (*) with 200 launches (sync)

fast-glob-current.js
(TIME) 2.663ms ±0.027% | (MEMORY) 3.027MB ±0.004% | Entries: 6 | Errors: 0 | Retries: 1
fast-glob-previous.js
(TIME) 7.010ms ±0.318% | (MEMORY) 5.054MB ±0.026% | Entries: 6 | Errors: 0 | Retries: 1
node-glob.js
(TIME) 2.257ms ±0.096% | (MEMORY) 2.484MB ±0.070% | Entries: 6 | Errors: 0 | Retries: 1

> fast-glob@2.2.6 bench-sync-deep /root/fast-glob
> node ./out/benchmark --type sync --pattern "**"

===> Benchmark pattern (**) with 200 launches (sync)

fast-glob-current.js
(TIME) 50.545ms ±2.279% | (MEMORY) 6.014MB ±0.060% | Entries: 4940 | Errors: 0 | Retries: 1
fast-glob-previous.js
(TIME) 78.331ms ±3.296% | (MEMORY) 11.575MB ±0.129% | Entries: 4940 | Errors: 0 | Retries: 1
node-glob.js
(TIME) 174.265ms ±3.838% | (MEMORY) 12.080MB ±3.233% | Entries: 4940 | Errors: 0 | Retries: 1