Define a public "maximize performance" API and separate it better from the regular "maximize compatibility" API

[vstinner]

For me, it became clear that there are two categories of C extensions users:

• Prioritize compatibility: single code base and single binary working on most Python versions, accept to pay some cost onf the performance
• Prioritize performance: use any available "hack" to get even minor performance win, ok to rebuild the C extension for each Python version.

Both usages are valid and we should design an API which fit them.

Currently, it's unclear which implementation you get. It can be optimized or not depending on the C compiler options, on the Python version, and other things. There is clear opt-in macro or other mechanism.

Before, when building Python, using PyThreadState_GET() was optimized or not depending if C file included some header files or not. I dislike that, it's easy to forget to include a specific include. I chose to use a different function name: the _PyThreadState_GET() function (static inline function in practice) with an additional underscore prefix is always "optimized" (inlined), and is only available if you opt-in for the internal C API.

Maybe we should clarify which implementation is provided by having a mechanism to more explicitly opt-in for maximum compatibility, or opt-in for maximum performance.

I think that the limited C API is a clear opt-in for "maximize compatibility", and using the internal C API is a clear opt-in for "maximize performance."

I would prefer the internal C API to not be used outside Python. I would prefer a public C API for performance.

---

Discussion splitted from the issue Avoid macros and static inline functions.

[encukou]

It seems that this is strictly a performance vs. compatibility trade-off. If that's the case, then users should only select the desired ABI stability, and they should get the fastest available implementation for that.

I agree that this shouldn't depend on whether a header is included or not.

[vstinner]

I read again my own issue and I don't see a clear action that can be triggered to solve it. So I prefer to close the issue. If I manage to design a more concrete action, I will open a new issue.