Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: use fs.walk instead of fs.walkStream for async provider #361

Merged
merged 1 commit into from
Jun 4, 2022

Conversation

mrmlnc
Copy link
Owner

@mrmlnc mrmlnc commented May 31, 2022

What is the purpose of this pull request?

Improving the performance of the fg('**') method.

Something like ~20% for medium directories and trees. Practically nothing for large directories and trees (1 million and more).

// *
===> Benchmark pattern "*" with 200 launches (regression, async)
===> Max stdev: 7 | Retries: 5 | Options: {}

Name                   Time, ms  Time stdev, %  Memory, MB  Memory stdev, %  Entries  Errors  Retries
---------------------  --------  -------------  ----------  ---------------  -------  ------  -------
fast-glob-current.js   6.970     0.402          4.388       0.014            5        0       1
fast-glob-previous.js  8.571     0.430          4.531       0.015            5        0       1

// **
===> Benchmark pattern "**" with 200 launches (regression, async)
===> Max stdev: 7 | Retries: 5 | Options: {}

Name                   Time, ms  Time stdev, %  Memory, MB  Memory stdev, %  Entries  Errors  Retries
---------------------  --------  -------------  ----------  ---------------  -------  ------  -------
fast-glob-current.js   89.665    3.884          9.649       0.380            11882    0       1
fast-glob-previous.js  112.966   5.370          7.760       0.112            11882    0       1

What changes did you make? (Give an overview)

Instead of using the ReaderStream reader, now we use the ReaderAsync reader. This avoids the overhead of stream processing.

But we still use ReaderStream for static method.

@mrmlnc mrmlnc force-pushed the use_async_method_instead_of_stream branch from a6e691c to b9f74dd Compare May 31, 2022 11:57
@mrmlnc mrmlnc marked this pull request as ready for review May 31, 2022 21:10
@mrmlnc mrmlnc changed the title pref: use fs.walk instead of fs.walkStream for async provider perf: use fs.walk instead of fs.walkStream for async provider May 31, 2022
@mrmlnc mrmlnc force-pushed the use_async_method_instead_of_stream branch from b9f74dd to 7268b50 Compare June 4, 2022 08:49
@mrmlnc mrmlnc merged commit 2ff5a2f into master Jun 4, 2022
@mrmlnc mrmlnc deleted the use_async_method_instead_of_stream branch June 4, 2022 09:36
@mrmlnc mrmlnc modified the milestones: 3.3.0, 3.2.12 Jun 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant