Propagate qualname from the compiler unit to code objects for finer grained profiling data

[P403n1x87]

BPO	44530
Nosy	@markshannon, @zooba, @pablogsal, @P403n1x87
PRs	bpo-44530: Add co_qualname field to PyCodeObject #26941 bpo-44530: Document the new CodeObject.co_qualname attribute #27052 bpo-44530: Reverts a change to the 'code.__new__' audit event #29809
Files	py_main.png

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-12-03.19:50:26.033>
created_at = <Date 2021-06-28.22:35:33.377>
labels = ['expert-C-API', 'type-feature', '3.11']
title = 'Propagate qualname from the compiler unit to code objects for finer grained profiling data'
updated_at = <Date 2021-12-03.19:50:26.033>
user = 'https://github.com/P403n1x87'

bugs.python.org fields:

activity = <Date 2021-12-03.19:50:26.033>
actor = 'steve.dower'
assignee = 'none'
closed = True
closed_date = <Date 2021-12-03.19:50:26.033>
closer = 'steve.dower'
components = ['C API']
creation = <Date 2021-06-28.22:35:33.377>
creator = 'Gabriele Tornetta'
dependencies = []
files = ['50128']
hgrepos = []
issue_num = 44530
keywords = ['patch']
message_count = 15.0
messages = ['396667', '396706', '396730', '396754', '396927', '396930', '396932', '396933', '396940', '397071', '397072', '397079', '407105', '407106', '407120']
nosy_count = 4.0
nosy_names = ['Mark.Shannon', 'steve.dower', 'pablogsal', 'Gabriele Tornetta']
pr_nums = ['26941', '27052', '29809']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue44530'
versions = ['Python 3.11']

Linked PRs

• gh-88696: clean up dead argument to compiler_make_closure #100806

[P403n1x87]

When dumping profiling data out of code objects using out-of-process tools like Austin (https://github.com/p403n1x87/austin) one has access only to file name, function name, and line number. Consider the flame graph generated by running the following script, and aggregating on function names

----

class Foo:
    def on_cpu(self, n):
        a = []
        for i in range(n):
            a.append(i)


class Bar:
    def on_cpu(self, n):
        a = []
        for i in range(n):
            a.append(i)


if __name__ == "__main__":
    f = Foo()
    b = Bar()

    f.on_cpu(1_000_000)
    b.on_cpu(5_000_000)

---

Without the extra information coming from the actual Python source, one would not be able to tell, by looking at the flame graph alone, that on_cpu has contributions from two different methods. By propagating the qualname information from the compiler to code objects, such names would be disambiguated and the resulting flame graph would be clearer.

I would like to propose adding the co_qualname field to the PyCodeObject structure, which is to be set to NULL except for when the code object is created by the compiler in compile.c.

[markshannon]

I think this is a worthwhile improvement.
A few comments on this issue and your PR.

1. We already have qualified names on functions and generators and those can be modified by @decorators.
   In those cases, the code object and the function would disagree.
   Code objects are immutable, so any thoughts on the usability implications of qualname differing between code object and function?

2. If we are adding a new attribute, it should be at the Python level. The internal layout of the code object is likely to change. (I have no problem with it being visible to tools like austin, just it won't be part of the API.)

3. As it stands, this is beneficial to only a small set of tools, but has a negative impact on memory use.
   If we were to leverage the new field to improve the performance of function creation (important for closures) by omitting the qualname (and defaulting to the code object's) then it would benefit everyone.

[pablogsal]

This also has the side effect of making marshalled code objects bigger, and we keep adding to this in 3.11 with the new exception table and soon with PEP-657. Given that a lot of people are concerned about this and that this change only affects a very small set of tools, we need to analyze very carefully the balance.

[P403n1x87]

Thanks for the feedback. I certainly appreciate all the concerns around this proposed change.

@Mark.Shannon

1. This is the desired behaviour as profiling data should be attached to the actual code objects that correlate with the actual source content. Besides, in this respect, given the immutability of code objects, qualname behaves like name.

2. At the Python level it is already possible to get the __qualname__ of a function (but not of a code object). As this is all I needed, I kept my initial implementation to the bare minimum.

3. I agree with this analysis. Whilst this increases the memory footprint, it might have the benefit of making the derivation of __qualname__ faster as this would now be cached on code objects.

However, I also appreciate @pablogsal's point of view of evaluating the actual benefits. The extra point in favour of this change that I can make is that it would resolve name clashes in profiling data from tools that have minimal impact on the runtime. Whether this is enough to justify the extra memory cost is certainly up for debate. From my side, I don't have the numbers to make this case more concrete.

As a next step, I'll look into exposing the new field to Python to see what it actually ends up looking like.

[P403n1x87]

@pablogsal

Commit a0252ab exposes co_qualname to Python. I've added a test case to check that the result coincides with the current implementation of __qualname__. Is there a "standard" way for me to quantify the memory impact of these kinds of changes?

@Mark.Shannon

I'll look into making __qualname__ use __code__.co_qualname as a next step.

[pablogsal]

Is there a "standard" way for me to quantify the memory impact of these kinds of changes?

That is a pointer size per code object. The most standard way is to calculate the size of all pyc files in the stdlib after compiling them all with "-m compileall -r 1000 Lib".

[P403n1x87]

That is a pointer size per code object. The most standard way is to calculate the size of all pyc files in the stdlib after compiling them all with "-m compileall -r 1000 Lib".

This yields the following numbers between what was main when I branched off and the tip of my branch:

# main (7569c0f): 25_059_438 bytes
# bpo-445303-code-qualname (a0252ab): 25_511_492 bytes

So that seems to be about half a MB increase over 25 MB (about 2% relative increase).

This is potentially just an interim result, as at first sight it looks like MAKE_FUNCTION could require one less argument (the qualname), which can now be taken from the code object. So not quite the final picture yet.

[pablogsal]

So that seems to be about half a MB increase over 25 MB (about 2% relative increase).

I personally think that is acceptable, so I would be supportive of the patch but for context, many folks have indicated that they are worried about this size getting bigger during previous discussions, so we need to still think collectively :)

[P403n1x87]

With commit 7a12d31a8c22cae7a9c1a2239a1bb54adee33622 the new figures are

# main (7569c0f): 25_059_438 bytes
# bpo-445303-code-qualname (7a12d31a8c): 25_089_917 bytes

which is a 0.1% relative increase :). This is likely due to the fact that with the change MAKE_FUNCTION needs to pop one less value from the TOS, as the qualname now comes from the code object.

I think the PR is now good for a first review. I think I'd need some help with the documentation sources.