Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-117578: Introduce _PyType_GetModuleByDef2 private function #117661

Merged
merged 13 commits into from
Apr 25, 2024
Merged

gh-117578: Introduce _PyType_GetModuleByDef2 private function #117661

merged 13 commits into from
Apr 25, 2024

Conversation

neonene
Copy link
Contributor

@neonene neonene commented Apr 9, 2024

_PyType_GetModuleByDefInPairs(provisional name: outdated) is a private function that takes two types, runs PyType_GetModuleByDef() twice, and finally raises an error if any.

Objects/typeobject.c Outdated Show resolved Hide resolved
Objects/typeobject.c Outdated Show resolved Hide resolved
neonene and others added 3 commits April 9, 2024 19:17
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Objects/typeobject.c Outdated Show resolved Hide resolved
@neonene neonene changed the title gh-117578: Add PyType_GetModuleByDef() repeater for two types gh-117578: Introduce _PyType_GetModuleByDef2 private function Apr 9, 2024
@encukou
Copy link
Member

encukou commented Apr 23, 2024

Please don't change the documented behaviour of PyType_GetModuleByDef. It should return the first match.
That said, if you have several different modules with the same def in a MRO, things are probably not going the way you wanted. The distinction between the first/last matching entry would make more sense if you want the base type itself, not its ht_module.

@neonene
Copy link
Contributor Author

neonene commented Apr 23, 2024

I can also revert the starting value of the loop from 1 to 0, if you prefer. I've seen a slight performance change there, which may be negligible.

@rhettinger rhettinger removed their request for review April 24, 2024 03:41
Objects/typeobject.c Outdated Show resolved Hide resolved
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Copy link
Member

@encukou encukou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!
As for tests, this should be covered by the decimal test suite, but if you want to write a GetModuleByDef test please ping me for a review.

@encukou encukou merged commit 2c45148 into python:main Apr 25, 2024
36 checks passed
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL7 LTO + PGO 3.x has failed when building commit 2c45148.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/96/builds/7358) and take a look at the build logs.
  4. Check if the failure is related to this commit (2c45148) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/96/builds/7358

Failed tests:

  • test_capi

Summary of the results of the build (if available):

==

Click to see traceback logs
remote: Enumerating objects: 10, done.        
remote: Counting objects:  10% (1/10)        
remote: Counting objects:  20% (2/10)        
remote: Counting objects:  30% (3/10)        
remote: Counting objects:  40% (4/10)        
remote: Counting objects:  50% (5/10)        
remote: Counting objects:  60% (6/10)        
remote: Counting objects:  70% (7/10)        
remote: Counting objects:  80% (8/10)        
remote: Counting objects:  90% (9/10)        
remote: Counting objects: 100% (10/10)        
remote: Counting objects: 100% (10/10), done.        
remote: Compressing objects:  11% (1/9)        
remote: Compressing objects:  22% (2/9)        
remote: Compressing objects:  33% (3/9)        
remote: Compressing objects:  44% (4/9)        
remote: Compressing objects:  55% (5/9)        
remote: Compressing objects:  66% (6/9)        
remote: Compressing objects:  77% (7/9)        
remote: Compressing objects:  88% (8/9)        
remote: Compressing objects: 100% (9/9)        
remote: Compressing objects: 100% (9/9), done.        
remote: Total 10 (delta 1), reused 3 (delta 1), pack-reused 0        
From https://github.com/python/cpython
 * branch            main       -> FETCH_HEAD
Note: checking out '2c451489122d539080c8d674b391dedc1dedcb53'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 2c45148... gh-117578: Introduce _PyType_GetModuleByDef2 private function (GH-117661)
Switched to and reset branch 'main'

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make[2]: [Makefile:3101: clean-retain-profile] Error 1 (ignored)

make: *** [Makefile:2232: buildbottest] Error 2

@neonene neonene deleted the getmod branch April 25, 2024 12:50
@mdboom
Copy link
Contributor

mdboom commented May 29, 2024

@neonene: We should have measured the performance impact of this at the time, but it came up recently in conversation with @markshannon, so I decided to measure it.

While overall on the benchmarking suite it's kind of inconclusive (1% faster, which is below the usual noise threshold), for the one benchmark that uses _PyType_GetModuleByDef heavily (via that fraction extension), it saw a 5% improvement, as well as a 3% decrease in memory usage. Results here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants