Speed up starting compression #9169
Conversation
Enumerating a enum and accessing all the .value is not performant. Switching to a dict is significantly faster
bdraco
added
backport-3.10
Codecov ReportAll modified and coverable lines are covered by tests ✅
✅ All tests successful. No failed tests found. Additional details and impacted files@@ Coverage Diff @@
## master #9169 +/- ##
==========================================
- Coverage 98.31% 98.24% -0.07%
==========================================
Files 107 107
Lines 34485 34486 +1
Branches 4095 4096 +1
==========================================
- Hits 33903 33882 -21
- Misses 411 429 +18
- Partials 171 175 +4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
psf-chronographer
bot
added
the
bot:chronographer:provided
| for coding in ContentCoding: | ||
| if coding.value in accept_encoding: | ||
| for value, coding in CONTENT_CODINGS.items(): | ||
| if value in accept_encoding: |
There was a problem hiding this comment.
I'm struggling with this one. It looks to me like all we've saved is one attribute access...
With the dict, I was expecting a reversal of logic or something here, like for enc in accept_encoding: CONTENT_CODINGS.get(enc).
There was a problem hiding this comment.
Iterating enums in python is surprisingly expensive and good chunk of its the __get__. _start_compression was the most expensive call in _prepare_headers before:
We save:
- creating the iterator for ContentCoding enum (although it might be an even exchange to iterate
.items(), but thats all going to be in native code for sure) - creating a enum singleton for each one in the loop
- accessing the value which does some magic under the hood (its not a simple attribute)
... maybe we can save more here but I was thinking the in 3x was cheaper than a dict lookup
Zoomed in on _prepare_headers before the change:
Zoomed in on _start_compression before the change:
There was a problem hiding this comment.
a dict get might make sense if we parsed the string but parsing is probably more expensive Accept-Encoding: deflate, gzip;q=1.0, *;q=0.5
What do these changes do?
Enumerating a enum and accessing all the .value is not performant. Switching to a pre-built dict is significantly faster
Are there changes in behavior for the user?
no
Is it a substantial burden for the maintainers to support this?
no
related issue #2779
before
after
