Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize Target and FieldSet operations (Cherry-pick of #18917) #18949

Merged
merged 1 commit into from
May 9, 2023

Conversation

stuhood
Copy link
Member

@stuhood stuhood commented May 9, 2023

Recent profiles (from #18911, and reproduced locally) highlighted:

  • FieldSet.create was repeatedly calling the unmemoized _get_field_set_fields, which used the surprisingly expensive from typing import get_type_hints.
    • This change moves type hint extraction into a memoized property.
  • Target.has_fields was doing an N^2 lookup of fields in a computed collection of registered fields.
    • Moved to using the field_values dict to represent the set of present fields, and removed the plugin_fields property (which is always incorporated into the field_values at construction time anyway).

Performance for ./pants --no-pantsd dependencies :: in pantsbuild/pants improves by 9% (or 15%, if the time for rule graph solving is ignored).

Recent profiles (from pantsbuild#18911, and reproduced locally) highlighted:
* `FieldSet.create` was repeatedly calling the unmemoized
`_get_field_set_fields`, which used the surprisingly expensive `from
typing import get_type_hints`.
    * This change moves type hint extraction into a memoized property.
* `Target.has_fields` was doing an N^2 lookup of fields in a computed
collection of registered fields.
* Moved to using the `field_values` `dict` to represent the set of
present fields, and removed the `plugin_fields` property (which is
always incorporated into the `field_values` at construction time
anyway).

----

Performance for `./pants --no-pantsd dependencies ::` in
`pantsbuild/pants` improves by 9% (or 15%, if the time for rule graph
solving is ignored).
@stuhood stuhood merged commit 56a60be into pantsbuild:2.15.x May 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants