distutils access ['headers'] from sysconfig scheme but the key in sysconfig is ['include']

[asottile]

I suspect this is the cause of this trace, but I have yet to track down a reproduction: deadsnakes/issues#191 (comment)

here's where the key is accessed:

distutils/distutils/command/install.py

Lines 154 to 155 in 5ed4fe5

	f'install_{key}': scheme[key]
	for key in SCHEME_KEYS

and here's where that key list is:

distutils/distutils/command/install.py

Line 92 in 5ed4fe5

SCHEME_KEYS = ('purelib', 'platlib', 'headers', 'scripts', 'data')

here's an example sysconfig entry:
https://github.com/python/cpython/blob/31ff96712e8f89ac1056c2da880b44650002219f/Lib/sysconfig.py#L27-L38

here's what cpython does for this:
https://github.com/python/cpython/blob/31ff96712e8f89ac1056c2da880b44650002219f/Lib/distutils/command/install.py#L45-L51

[FFY00]

No, headers and include are two different keys with different values. I think the bug that lead to the issue you linked was introduced in #79, which I did not have a chance to review.

cc @jaraco

[jaraco]

Probably #79 was implicated but also #23, where we learned that sysconfig supplies include and distutils requires headers and they're different things. The solution was to provide compatibility by continuing to supply the legacy install schemes with headers definitions. This also means that if a custom scheme is provided, it also needs to provide headers or needs to opt out of this distutils with SETUPTOOLS_USE_DISTUTILS=stdlib (for now).

If someone has an idea how distutils could rely on include instead of headers, I'd be interested in a patch to support that transition.

[jaraco]

I can see where CPython's copy of distutils simply treats 'include' as 'headers' and then appends the dist name. Probably this version of distutils could do the same.

[FFY00]

That is not correct, I didn't notice it earlier, but it's a bug in the CPython implementation.

See https://bugs.python.org/issue44445 for some context.

[jaraco]

Thanks Filipe for pointing me to that bug (again). That's really helpful.

As I understand it, the long term fix (in that bug) is to provide another key such as site-include.

The medium-term fix is for platforms to include a headers key for any custom scheme. Perhaps CPython should include some guidance on the need for this key until site-include exists.

A short-term fix might be for distutils to use the headers key from the default scheme (ignoring preferred) whenever the preferred scheme does not define headers.

[jaraco]

Setuptools PR at pypa/setuptools#2960.