Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: collisions don't contain reserved names by default #684

Merged
merged 3 commits into from
Oct 27, 2020

Conversation

software-dov
Copy link
Contributor

@software-dov software-dov commented Oct 23, 2020

The 'collisions' set does NOT contain RESERVED_NAMES; they are
combined at runtime when needed.

For a large real-world, this results in an order of magnitude
reduction in memory usage.
I'm not joking: Google Ads v5 uses 2.45 GB peak before this change
and 223 MB after.

Also contains changes to add __slots__ attributes to Metadata and Address. These are ancillary, optional, and open to negotiation.
In the above scenario, they reduce memory usage from 223 MB to 177 MB. If other people feel that the reduction in readability does not warrant the reduction in memory usage I am absolutely open to dropping that particular commit.

Includes other minor memory usage optimizations that collectively shave about 5 MB.

The 'collisions' set does NOT contain RESERVED_NAMES; they are
combined at runtime when needed.

For a large real-world, this results in an order of magnitude
reduction in memory usage.
I'm not joking: Google Ads v5 uses 2.45 GB before this change
and 223 MB after.
@software-dov software-dov requested a review from a team as a code owner October 23, 2020 21:49
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Oct 23, 2020
@codecov
Copy link

codecov bot commented Oct 23, 2020

Codecov Report

Merging #684 into master will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff            @@
##            master      #684   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           26        26           
  Lines         1561      1560    -1     
  Branches       315       316    +1     
=========================================
- Hits          1561      1560    -1     
Impacted Files Coverage Δ
gapic/schema/metadata.py 100.00% <100.00%> (ø)
gapic/schema/wrappers.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 18425e2...6821a97. Read the comment docs.

Copy link
Contributor

@busunkim96 busunkim96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also contains changes to add slots attributes to Metadata and Address. These are ancillary, optional, and open to negotiation.
In the above scenario, they reduce memory usage from 223 MB to 177 MB. If other people feel that the reduction in readability does not warrant the reduction in memory usage I am absolutely open to dropping that particular commit.

I would personally prefer preserving readability.

Comment on lines 151 to 152
# Don't keep RESERVED_NAMES in self.collisions,
# just combine the two when necessary.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps add a note that this is for memory usage?

@software-dov software-dov added the automerge Merge the pull request once unit tests and other checks pass. label Oct 27, 2020
@gcf-merge-on-green gcf-merge-on-green bot merged commit 2ec6ea6 into googleapis:master Oct 27, 2020
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Oct 27, 2020
gcf-merge-on-green bot pushed a commit that referenced this pull request Oct 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants