gh-117142: Port _ctypes to multi-phase init

[neonene]

Port _ctypes to multi-phase init with the module state enabled.

Module state access:

• Rely on metaclass's mro, which needs to be walked when the C-API metaclass is overridden:

class _swapped_struct_meta(_swapped_meta, type(Structure)):
    pass
class LittleEndianStructure(Structure, metaclass=_swapped_struct_meta):
    ...

# Normal type's mro is less reliable:
type(_SimpleCData)("", (list,), {'_type_': 'i'})

From	access
ctype	PyType_GetModuleByDef(Py_TYPE(type))
cdata	PyType_GetModuleByDef(Py_TYPE(Py_TYPE(obj)))
CFieldObject	_PyType_GetModuleState(Py_TYPE(obj))	Internal
CThunkObject	_PyType_GetModuleState(Py_TYPE(obj))	Internal

• Issue: Convert _ctypes extension module to multi-phase init #117142

[neonene]

Changing this to a draft. See a separated PR: #117189.

[erlend-aasland]

FWIW, I've used the practice of defining a "clinic state" macro for getting the module state in generated clinic code. It has proven useful if you want to tweak stuff afterwards (smaller diffs, less churn). You can git grep clinic_state **/*.c for inspiration.

[neonene]

IIUC, *_impl() should be passed a module state if the AC gets it in advance with some overhead. Also, _ctypes would need a module state getter to be specified in each function clinic input, making a class input have a type name without any operator, EDIT: or making the getter return st.

[neonene]

Ah, _PyType_GetModuleState(cls) or _PyType_GetModuleState(cls->tp_base) can be applied to each class. I'll try.

[neonene]

Is it acceptable to optimize PyType_GetModuleByDef (limited API) by checking the given type before the MRO walk? This PR passes C-API types to it in almost all cases. The tuning in _ctypes (outside PyType_GetModuleByDef) would be fine as well, which should be inlined. I would prefer the former, though.

[encukou]

Is it acceptable to optimize PyType_GetModuleByDef (limited API) by checking the given type before the MRO walk?

I seem to recall it had such an optimization at some point. I'm not sure why it's not there now; perhaps we simply went with simpler code in the first PR. If you can measure the speedup, definitely add it :)

[encukou]

This looks amazing, thank you!
I found one more snag, other than that it looks like we're almost home.

[encukou]

And back to optimizing PyType_GetModuleByDef,
a similar optimization was added in: #25505
and later removed with no reason in: #30795

So it seems the current code isn't quite carefully tuned, feel free to make it faster. (But note that getting a type's MRO and getting an item from a tuple should be very fast already.)

[neonene]

Regarding PyType_GetModuleByDef, I've applied optimization at #117661, which is not beneficial for the getset of ctypes. _decimal got around 7% faster in some cases with noises.

[encukou]

Thank you!

[neonene]

Thank you very much for the review.

[erlend-aasland]

Good job!