Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix slow qualified name perf in 0.4.2+ #698

Merged
merged 1 commit into from
Jun 16, 2022

Conversation

lpetre
Copy link
Contributor

@lpetre lpetre commented Jun 14, 2022

Summary

This is a performance fix for a change I made to correct compute qualified names in #682

Test Plan

I hacked three versions of the node_accesses set creation in Scope.get_qualified_names_for and ran a perf test on them:

(libcstvenv) lpetre@lpetre-mbp LibCST % cat /tmp/fastero.json 
{
    "setup": "from libcst import safe_parse_module, MetadataWrapper; from libcst.metadata import QualifiedNameProvider; import os; from wcmatch import wcmatch; env = os.environ; paths = list(wcmatch.WcMatch('/Users/lpetre/dev/src/github.com/python/cpython/Lib', '*.py', exclude_pattern='tests|test', flags=wcmatch.RECURSIVE).imatch()); paths = paths[:100]; mods = [MetadataWrapper(safe_parse_module(open(path).read())) for path in paths];",
    "results": [
        {
            "snippet_name": "0.4.2",
            "snippet_code": "env['FIX_QNAME'] = '0.4.2'; [mod.resolve(QualifiedNameProvider) for mod in mods]"
         },
         {
            "snippet_name": "0.4.4",
            "snippet_code": "env['FIX_QNAME'] = '0.4.4'; [mod.resolve(QualifiedNameProvider) for mod in mods]"
         },
         {
            "snippet_name": "0.4.5",
            "snippet_code": "env['FIX_QNAME'] = '0.4.5'; [mod.resolve(QualifiedNameProvider) for mod in mods]"
         }
    ]
 }

(libcstvenv) lpetre@lpetre-mbp LibCST % python -m fastero -f /tmp/fastero.json -m 5
╭───────────────────────────── Setup code ─────────────────────────────╮
│   1 from libcst import safe_parse_module, MetadataWrapper; from      │
│     libcst.metadata import QualifiedNameProvider; import os; from    │
│     wcmatch import wcmatch; env = os.environ; paths = list(wcmatch.W │
│     Match('/Users/lpetre/dev/src/github.com/python/cpython/Lib',     │
│     '*.py', exclude_pattern='tests|test',                            │
│     flags=wcmatch.RECURSIVE).imatch()); paths = paths[:100]; mods =  │
│     [MetadataWrapper(safe_parse_module(open(path).read())) for path  │
│     in paths];                                                       │
╰──────────────────────────────────────────────────────────────────────╯
────────────────────────────────────────────────────────────────────────────────────────────────────────────── Benchmark started… ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
0.4.2: env['FIX_QNAME'] = '0.4.2'; [mod.resolve(QualifiedNameProvider) for mod in mods]
  Time  (mean ± σ):       34.785 s ± 19.85 ms
  Range (min  … max):     34.753 s … 34.805 s    [runs: 5]
0.4.4: env['FIX_QNAME'] = '0.4.4'; [mod.resolve(QualifiedNameProvider) for mod in mods]
  Time  (mean ± σ):        38.538 s ± 628.35 ms
  Range (min  … max):      37.862 s …  39.197 s    [runs: 5]
0.4.5: env['FIX_QNAME'] = '0.4.5'; [mod.resolve(QualifiedNameProvider) for mod in mods]
  Time  (mean ± σ):       34.278 s ± 50.14 ms
  Range (min  … max):     34.224 s … 34.339 s    [runs: 5]

Summary:
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Bar Chart ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ 0.4.2 [34.753 s]: ▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆     ┃
┃ 0.4.4 [37.862 s]: ▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆ ┃
┃ 0.4.5 [34.224 s]: ▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆      ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━ (lower is better) ━━━━━━━━━━━━━━━━━━━━━━━━━━┛
  0.4.5 is the fastest.
    1.01 (1.02 … 1.01) times faster than 0.4.2
    1.12 (1.11 … 1.14) times faster than 0.4.4

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 14, 2022
@lpetre lpetre force-pushed the fix_qname_performance branch from b166e11 to 3b7da63 Compare June 14, 2022 12:58
@codecov-commenter
Copy link

codecov-commenter commented Jun 14, 2022

Codecov Report

Merging #698 (c9e57b3) into main (69a4f4e) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #698      +/-   ##
==========================================
- Coverage   94.82%   94.79%   -0.03%     
==========================================
  Files         246      246              
  Lines       25653    25692      +39     
==========================================
+ Hits        24325    24356      +31     
- Misses       1328     1336       +8     
Impacted Files Coverage Δ
libcst/metadata/scope_provider.py 95.52% <100.00%> (+0.02%) ⬆️
libcst/metadata/tests/test_scope_provider.py 99.88% <100.00%> (ø)
libcst/_nodes/tests/test_funcdef.py 86.66% <0.00%> (-6.89%) ⬇️
libcst/_nodes/expression.py 96.66% <0.00%> (-0.13%) ⬇️
libcst/_typed_visitor.py 97.00% <0.00%> (-0.04%) ⬇️
libcst/matchers/__init__.py 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 69a4f4e...c9e57b3. Read the comment docs.

@lpetre lpetre force-pushed the fix_qname_performance branch 2 times, most recently from c9e57b3 to ff3965f Compare June 14, 2022 14:54
@lpetre lpetre marked this pull request as ready for review June 14, 2022 22:23
Copy link
Member

@zsol zsol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for working on this

Comment on lines 410 to 411
__accesses_by_name: MutableMapping[str, Set[Access]]
__accesses_by_node: MutableMapping[cst.CSTNode, Set[Access]]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the double underscore?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because there were other examples (in Access and BaseAssignment) where we use double underscores. Fixed

@lpetre lpetre force-pushed the fix_qname_performance branch from ff3965f to f9c8ca8 Compare June 16, 2022 11:29
@lpetre lpetre merged commit 6f28c79 into Instagram:main Jun 16, 2022
@lpetre lpetre deleted the fix_qname_performance branch June 16, 2022 11:48
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Sep 16, 2022
0.4.7 - 2022-07-12

Fixed
* Fix get_qualified_names_for matching on prefixes of the given name by @lpetre in Instagram/LibCST#719

Added
* Implement lazy loading mechanism for expensive metadata providers by @Chenguang-Zhu in Instagram/LibCST#720


0.4.6 - 2022-07-04

New Contributors
- @superbobry made their first contribution in Instagram/LibCST#702

Fixed
- convert_type_comments now preserves comments following type comments by @superbobry in Instagram/LibCST#702
- QualifiedNameProvider optimizations
  - Cache the scope name prefix to prevent scope traversal in a tight loop by @lpetre in Instagram/LibCST#708
  - Faster qualified name formatting by @lpetre in Instagram/LibCST#710
  - Prevent unnecessary work in Scope.get_qualified_names_for_ by @lpetre in Instagram/LibCST#709
- Fix parsing of parenthesized empty tuples by @zsol in Instagram/LibCST#712
- Support whitespace after ParamSlash by @zsol in Instagram/LibCST#713
- [parser] bail on deeply nested expressions by @zsol in Instagram/LibCST#718


0.4.5 - 2022-06-17

New Contributors

-   @zzl0 made their first contribution in Instagram/LibCST#704

Fixed

-   Only skip supported escaped characters in f-strings by @zsol in Instagram/LibCST#700
-   Escaping quote characters in raw string literals causes a tokenizer error by @zsol in Instagram/LibCST#668
-   Corrected a code example in the documentation by @zzl0 in Instagram/LibCST#703
-   Handle multiline strings that start with quotes by @zzl0 in Instagram/LibCST#704
-   Fixed a performance regression in libcst.metadata.ScopeProvider by @lpetre in Instagram/LibCST#698


0.4.4 - 2022-06-13

New Contributors

-   @adamchainz made their first contribution in Instagram/LibCST#688

Added

-   Add package links to PyPI by @adamchainz in Instagram/LibCST#688
-   native: add overall benchmark by @zsol in Instagram/LibCST#692
-   Add support for PEP-646 by @zsol in Instagram/LibCST#696

Updated

-   parser: use references instead of smart pointers for Tokens by @zsol in Instagram/LibCST#691
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants