40.1.0 breaks version detection on some packages

[aduskett]

Hello;

I tried updating setuptools to 40.1.0 in the BuildRoot project, only to find a few packages such as
python-cryptography no longer seem to think setuptools is a version high enough to work.
Another package that has the same issue is python-funcsigs as well.

Downgrading to 40.0.0 fixes the issue. Any idea as to what could be going on?

Thanks!

[benoit-pierre]

You'll have to give more information than that: environment? Python version? What command are you using? What are the exact errors?

[pganssle]

@aduskett Is it possible that this is actually an issue with your local environment and that "downgrading" to version 40.0.0 actually upgraded some version on your python path from something < 18.5?

I can't reproduce this - I cloned cryptography and had no problem with either pip install -e . or python setup.py install with setuptools==40.1.0 on either master or the tag 2.3.1.

[aduskett]

Python 3.7.0 is the version:

Here is the command used by BuildRoot with python-cryptography-2.3.1 and setuptools 40.1.0:

(cd /home/vagrant/Downloads/br/output/build/python-cryptography-2.3.1//; _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata_m_linux_x86_64-linux-gnu" PATH="/home/vagrant/Downloads/br/output/host/bin:/home/vagrant/Downloads/br/output/host/sbin:/opt/rh/rh-git29/root/usr/bin:/opt/rh/devtoolset-7/root/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/vagrant/.local/bin:/home/vagrant/bin" PYTHONPATH="/home/vagrant/Downloads/br/output/target/usr/lib/python3.7/" PYTHONNOUSERSITE=1 _python_sysroot=/home/vagrant/Downloads/br/output/host/x86_64-buildroot-linux-gnu/sysroot _python_prefix=/usr _python_exec_prefix=/usr /home/vagrant/Downloads/br/output/host/bin/python setup.py build )

Here is the output:

Traceback (most recent call last): File "setup.py", line 28, in <module> "cryptography requires setuptools 18.5 or newer, please upgrade to a " RuntimeError: cryptography requires setuptools 18.5 or newer, please upgrade to a newer version of setuptools make[1]: *** [package/pkg-generic.mk:232: /home/vagrant/Downloads/br/output/build/python-cryptography-2.3.1/.stamp_built] Error 1 make: *** [Makefile:79: _all] Error 2

I am more than happy to make a github project with a defconfig so anybody who wishes to test the output could do so.

[benoit-pierre]

In [1]: import pkg_resources, setuptools
In [2]: pkg_resources.parse_version(setuptools.__version__)
Out[2]: <Version('40.0.0')>
In [3]: pkg_resources.parse_version(setuptools.__version__) < pkg_resources.parse_version("18.5")
Out[3]: False

It's unfortunate the error message does not show the version of setuptools available, but simplest explanation is that an older version get picked up. A buildroot (configuration?) bug? How did you update setuptools to 40.1.0?

[aduskett]

Setuptools is installed directly. It's never upgraded.
I modified python-cryptographies setup.py to print the found package version:

if (
pkg_resources.parse_version(setuptools.version) <
pkg_resources.parse_version("18.5")
):

raise RuntimeError(
"cryptography requires setuptools 18.5 or newer, please upgrade to a newer version of setuptools: " + str(pkg_resources.parse_version(setuptools.version))
)

RuntimeError: cryptography requires setuptools 18.5 or newer, please upgrade to a newer version of setuptools: 40.1.0.post20180820.post-20180820

[benoit-pierre]

I don't understand, I've not used buildroot in a while, and not with Python packages, but isn't setuptools itself supposed to be provided (built/installed) first by buildroot (maybe be with both a host and a build version if cross-compiling)?

As for the version of setuptools you're getting, I've to ask where you got it, because that's certainly not a PEP 440 compliant version number (which explains why the comparison is failing).

[aduskett]

Yes, setuptools is provided by buildroot itself.

Setup tools are grabbed from this repository in fact!

PYTHON_SETUPTOOLS_VERSION = v40.1.0
PYTHON_SETUPTOOLS_SITE = $(call github,pypa,setuptools,$(PYTHON_SETUPTOOLS_VERSION))

[benoit-pierre]

OK, so you did upgrade the setuptools version by patching the buildroot package for setuptools. What version of buildroot is that? I've not looked at the particular of the $(call github,...) macro, but if one of the zip/tar.gz of the Github release section is used, then I don't see how you could end up with that version number.

[aduskett]

Indeed, it's the latest next branch.

It ends up downloading from this url:
https://github.com/pypa/setuptools/archive/v40.1.0/python-setuptools-v40.1.0.tar.gz

[aduskett]

Interesting fact, If I downgrade to 40.0, this is the output for cryptography:
40.0.0.post20180820

[benoit-pierre]

I still don't see what's happening, can you give some verbose traces for setuptools' build? Maybe add an additional trace with the output of python setup.py --version?

[benoit-pierre]

So even with '40.0.0', the version is not right.

[benoit-pierre]

Do you have a minimal buildroot configuration to reproduce the problem?

[aduskett]

Yes I do:
Clone this: https://github.com/aduskett/buildroot

and run make pythontest_defconfig then run make python-cryptography

That should reproduce the issue.

[benoit-pierre]

OK, so I can't reproduce the issue because the build fails earlier; when building host-m4:

freadahead.c: In function 'freadahead':
freadahead.c:92:3: error: #error "Please port gnulib freadahead.c to your platform! Look at the definition of fflush, fread, ungetc on your system, then report this to bug-gnulib."                               
  #error "Please port gnulib freadahead.c to your platform! Look at the definition of fflush, fread, ungetc on your system, then report this to bug-gnulib."

Anyway, setuptools.egg-info is supposed to be generated by the call to bootstrap.py, so what's the output with:

 package/python-setuptools/python-setuptools.mk | 1 +
 1 file changed, 1 insertion(+)

diff --git i/package/python-setuptools/python-setuptools.mk w/package/python-setuptools/python-setuptools.mk
index 590e9829..e3980a40 100644
--- i/package/python-setuptools/python-setuptools.mk
+++ w/package/python-setuptools/python-setuptools.mk
@@ -14,6 +14,7 @@ PYTHON_SETUPTOOLS_SETUP_TYPE = setuptools
 # before the standard setup process.
 define PYTHON_SETUPTOOLS_RUN_BOOTSTRAP
 	cd  $(@D) && $(HOST_DIR)/bin/python ./bootstrap.py
+	cd  $(@D) && $(HOST_DIR)/bin/python ./setup.py --version
 endef
 
 PYTHON_SETUPTOOLS_PRE_CONFIGURE_HOOKS = PYTHON_SETUPTOOLS_RUN_BOOTSTRAP

?