Fails to build on Python 3.8

[cheeseandcereal]

Using Python 3.8, cytoolz cannot build/install.

It appears to fail while compiling the dicttoolz.c file.

See here for the full log:
https://pastebin.com/hrYA77um

If you have docker, you can quickly/easily replicate this with docker run python:3.8-rc python -m pip install cytoolz==0.10.0

Note that this appears to be related (and would be resolved) by fixing #134, as I can confirm that cloning the repository and using the 0.10.0 tag, I can successfully build and install the package with python setup.py build --cython after removing all of the generated .c files on python 3.8.

[eriknw]

Good to know, thanks for the report! I'll try to get this resolved soon.

[palkeo]

If it helps, I think it's the same problem as there:
cython/cython#2938 (comment)

[jonringer]

correct, they are related. The signature for a few functions changed:

cytoolz/dicttoolz.c: In function ‘__Pyx_CreateCodeObjectForTraceback’:
cytoolz/dicttoolz.c:11394:5: error: macro "__Pyx_PyCode_New" requires 16 arguments, but only 15 given
     );
     ^
cytoolz/dicttoolz.c:11378:15: error: ‘__Pyx_PyCode_New’ undeclared (first use in this function); did you mean ‘PyCode_New’?
     py_code = __Pyx_PyCode_New(
               ^~~~~~~~~~~~~~~~
               PyCode_New
error: command 'gcc' failed with exit status 1

[bashalex]

@eriknw any update on this?

[eriknw]

I'll give this attention today. Sorry for the delay.

[eriknw]

See #137

I can push a bugfix release later today. Thanks all for the attention and patience.

Sorry again for the delay! My future in-laws were in town (😁), and then my car decided to go bottoms up (I just sold it--phew!).

[mrocklin]

My future in-laws were in town

Whoa. You have news to share apparently :) Congrats!

…

On Sun, Nov 3, 2019 at 6:43 AM Erik Welch ***@***.***> wrote: See #137 <#137> I can push a bugfix release later today. Thanks all for the attention and patience. Sorry again for the delay! My future in-laws were in town (😁), and then my car decided to go bottoms up (I just sold it--phew!). — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#136?email_source=notifications&email_token=AACKZTDYZGJWIK5RAJP2VBDQR3IQDA5CNFSM4I7QGKY2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC5UFSI#issuecomment-549143241>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTCMQC2RRECJOVEKDJTQR3IQDANCNFSM4I7QGKYQ> .

[palkeo]

Yeah I think we were quite a few people blocked on that. Thanks for your time fixing it Erik! And congrats :)

[eriknw]

Okay, cytoolz 0.10.1 is up.

• This is the same as 0.10.0, but with *.c files from the latest Cython (0.29.14).
• When running python setup.py (or pip install cython), this will re-cythonize everything if Cython is available.
  • Note that pip instal cytoolz will fallback to building from *.c files if Cython is unavailable.
• pip install 'cytoolz[cython]' will now install Cython and always cythonize everything.

[flekschas]

Is it expected that 0.10.1 fails to compile with Python 3.6 while 0.10.0 does not?

We're getting a Cython.Compiler.Errors.CompileError: cytoolz/functoolz.pyx raised by cytoolz/functoolz.pyx:518:4: Special method __eq__ must be implemented via __richcmp__.

[eriknw]

Oh no! No, this is not expected. Can you provide more information?

Here are passing tests showing 0.10.1 (released) compiling without Cython for Python 3.4-3.8: https://travis-ci.org/pytoolz/cytoolz/builds/606940112

And these tests show virtually the same code compiling with Cython: https://travis-ci.org/pytoolz/cytoolz/builds/606936141

I also just compiled and successfully ran the tests locally with Python 3.6.

[flekschas]

That's odd. Maybe it's a clash with some other library in our system. Below is the full log of the crash when we use v0.10.1:

    ERROR: Command errored out with exit status 1:
     command: /opt/conda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-j_f1__2g/cytoolz/setup.py'"'"'; __file__='"
'"'/tmp/pip-install-j_f1__2g/cytoolz/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"'
);f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-j_f1__2g/cytoolz/pip-egg-info
         cwd: /tmp/pip-install-j_f1__2g/cytoolz/
    Complete output (44 lines):
    Compiling cytoolz/dicttoolz.pyx because it changed.
    Compiling cytoolz/functoolz.pyx because it changed.
    Compiling cytoolz/itertoolz.pyx because it changed.
    Compiling cytoolz/recipes.pyx because it changed.
    Compiling cytoolz/utils.pyx because it changed.
    [1/5] Cythonizing cytoolz/dicttoolz.pyx
    [2/5] Cythonizing cytoolz/functoolz.pyx

    Error compiling Cython file:
    ------------------------------------------------------------
    ...

        def __repr__(self):
            return '{.__class__.__name__}{!r}'.format(
                self, tuple(reversed((self.first, ) + self.funcs)))

        def __eq__(self, other):
       ^
    ------------------------------------------------------------

    cytoolz/functoolz.pyx:518:4: Special method __eq__ must be implemented via __richcmp__
    Error compiling Cython file:
    ------------------------------------------------------------
    ...
        def __eq__(self, other):
            if isinstance(other, Compose):
                return other.first == self.first and other.funcs == self.funcs
            return NotImplemented

        def __ne__(self, other):
       ^
    ------------------------------------------------------------

    cytoolz/functoolz.pyx:523:4: Special method __ne__ must be implemented via __richcmp__
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-j_f1__2g/cytoolz/setup.py", line 87, in <module>
        ext_modules = cythonize(ext_modules, force=not is_dev)
      File "/opt/conda/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 934, in cythonize
        cythonize_one(*args)
      File "/opt/conda/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1056, in cythonize_one
        raise CompileError(None, pyx_file)
    Cython.Compiler.Errors.CompileError: cytoolz/functoolz.pyx
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Some other version:

• python: v3.6.8
• pip: 19.3.1
• cython: v0.25.2

What other information would be useful?

[eriknw]

Hmm, that version of Cython is nearly 3 years old. It looks like we need a minimum Cython version of 0.27.0 (this is good to know--now I can update metadata!). I wasn't expecting this, although it has been a while since I tested different Cython versions. I can also update the build matrix on the release branch to test different Cython versions.

If cytoolzversion 0.10.0 works with Python 3.6, then I recommend using it (it will compile from the provided *.c files, so it won't try to use the old Cython hanging around your environment).

Thanks for the info and quick reply @flekschas!

[flekschas]

Thanks for your quick replies :) We'll update Cython (I guess we should have done that quite some time ago already haha) and give it another try.