Unable to build docker image with TA-Lib

[yanivakiva]

Describe the bug
Unable to build the docker image - not sure what changed, but now i'm getting these errors:

In file included from /usr/local/include/python3.10/Python.h:74,
30.39                        from talib/_ta_lib.c:16:
30.39       talib/_ta_lib.c: In function ‘__pyx_f_5numpy_PyDataType_SHAPE’:
30.39       talib/_ta_lib.c:10264:39: error: ‘PyArray_Descr’ {aka ‘struct _PyArray_Descr’} has no member named ‘subarray’
30.39       10264 |     __Pyx_INCREF(((PyObject*)__pyx_v_d->subarray->shape));
30.39             |                                       ^~
30.39       /usr/local/include/python3.10/object.h:112:41: note: in definition of macro ‘_PyObject_CAST’
30.39         112 | #define _PyObject_CAST(op) ((PyObject*)(op))
30.39             |                                         ^~
30.39       talib/_ta_lib.c:1653:27: note: in expansion of macro ‘Py_INCREF’
30.39        1653 |   #define __Pyx_INCREF(r) Py_INCREF(r)
30.39             |                           ^~~~~~~~~
30.39       talib/_ta_lib.c:10264:5: note: in expansion of macro ‘__Pyx_INCREF’
30.39       10264 |     __Pyx_INCREF(((PyObject*)__pyx_v_d->subarray->shape));
30.39             |     ^~~~~~~~~~~~
30.39       talib/_ta_lib.c:10265:36: error: ‘PyArray_Descr’ {aka ‘struct _PyArray_Descr’} has no member named ‘subarray’
30.39       10265 |     __pyx_r = ((PyObject*)__pyx_v_d->subarray->shape);
30.39             |                                    ^~
30.39       error: command '/usr/bin/gcc' failed with exit code 1
30.39       [end of output]

To Reproduce
Steps to reproduce the behavior:
Dockerfile

FROM python:3.10


RUN apt-get update \
    && apt-get install -y \
        build-essential \
        wget \
        libpcre3-dev \
        curl \
        python3-dev \
        git


RUN curl -sSL https://install.python-poetry.org | python3 -
RUN pip install poetry


ENV PATH="$PATH:/root/.poetry/bin"

WORKDIR /tmp
RUN wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz \
    && tar -xzf ta-lib-0.4.0-src.tar.gz \
    && cd ta-lib \
    && ./configure --prefix=/usr --build=aarch64-unknown-linux-gnu \
    && make \
    && make install

RUN pip install numpy==1.21.1

RUN pip install TA-Lib==0.4.29

Desktop:

• OS: Ubuntu 20.04

[s-hfarooq]

Had the same error. If I clone the repo and install using python setup.py install I don't get the error. Also needed to run pip install setuptools first

[tibitTW]

I met a similar error but on google colab, here's the link: https://colab.research.google.com/drive/1DzJbfYvfro4Iulb0PZLwrIegNKjrfrBG?usp=sharing

...
building 'talib._ta_lib' extension
  creating build/temp.linux-x86_64-cpython-310
  creating build/temp.linux-x86_64-cpython-310/talib
  x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include -I/usr/local/include -I/opt/include -I/opt/local/include -I/opt/homebrew/include -I/opt/homebrew/opt/ta-lib/include -I/tmp/pip-build-env-e5ra00kj/normal/local/lib/python3.10/dist-packages/numpy/_core/include -I/usr/include/python3.10 -c talib/_ta_lib.c -o build/temp.linux-x86_64-cpython-310/talib/_ta_lib.o
  In file included from /tmp/pip-build-env-e5ra00kj/normal/local/lib/python3.10/dist-packages/numpy/_core/include/numpy/ndarraytypes.h:1909,
                   from /tmp/pip-build-env-e5ra00kj/normal/local/lib/python3.10/dist-packages/numpy/_core/include/numpy/ndarrayobject.h:12,
                   from /tmp/pip-build-env-e5ra00kj/normal/local/lib/python3.10/dist-packages/numpy/_core/include/numpy/arrayobject.h:5,
                   from talib/_ta_lib.c:1235:
  /tmp/pip-build-env-e5ra00kj/normal/local/lib/python3.10/dist-packages/numpy/_core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
     17 | #warning "Using deprecated NumPy API, disable it with " \
        |  ^~~~~~~
  In file included from /usr/include/python3.10/Python.h:74,
                   from talib/_ta_lib.c:16:
  talib/_ta_lib.c: In function ‘__pyx_f_5numpy_PyDataType_SHAPE’:
  talib/_ta_lib.c:10268:39: error: ‘PyArray_Descr’ {aka ‘struct _PyArray_Descr’} has no member named ‘subarray’
  10268 |     __Pyx_INCREF(((PyObject*)__pyx_v_d->subarray->shape));
        |                                       ^~
  /usr/include/python3.10/object.h:112:41: note: in definition of macro ‘_PyObject_CAST’
    112 | #define _PyObject_CAST(op) ((PyObject*)(op))
        |                                         ^~
  talib/_ta_lib.c:1657:27: note: in expansion of macro ‘Py_INCREF’
   1657 |   #define __Pyx_INCREF(r) Py_INCREF(r)
        |                           ^~~~~~~~~
  talib/_ta_lib.c:10268:5: note: in expansion of macro ‘__Pyx_INCREF’
  10268 |     __Pyx_INCREF(((PyObject*)__pyx_v_d->subarray->shape));
        |     ^~~~~~~~~~~~
  talib/_ta_lib.c:10269:36: error: ‘PyArray_Descr’ {aka ‘struct _PyArray_Descr’} has no member named ‘subarray’
  10269 |     __pyx_r = ((PyObject*)__pyx_v_d->subarray->shape);
        |                                    ^~
  error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
  error: subprocess-exited-with-error
  
  × Building wheel for TA-Lib (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /usr/bin/python3 /usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpss6hjp6m
  cwd: /tmp/pip-install-j_7fber9/ta-lib_c0f5c13d1a544769979ccb300e217551
  Building wheel for TA-Lib (pyproject.toml) ... error
  ERROR: Failed building wheel for TA-Lib
Failed to build TA-Lib
ERROR: Could not build wheels for TA-Lib, which is required to install pyproject.toml-based projects

Reproduce Steps

!wget https://sourceforge.net/projects/ta-lib/files/ta-lib/0.4.0/ta-lib-0.4.0-src.tar.gz
!tar -xzf ta-lib-0.4.0-src.tar.gz
%cd ta-lib/
!./configure --prefix=/usr
!make
!sudo make install
%cd ..

!python -m pip install TA-Lib -v

[xmatthias]

What changed is (most likely) the release of numpy2 - causing some incompatibilities with ta-lib.

it should be installable via the following sequence:

pip install numpy==1.26.4
pip install TA-Lib -v --no-build-isolation

root cause:
during build, pip doesn't care what numpy version is installed unless --no-build-isolation is given.
That's due to a (long-standing) bug in pip - pypa/pip#9542 - which re-installes build-dependencies - ignoring pins and previously installed versions unless the version pin is part of the "install-requires" section.

A short-term fix to this is probably pinning numpy to <2.0.0 in pyproject.toml (in the build-requires section - can't remember the name rn).
Obviously longer term, we'd hope to have numpy 2.x supported, too.

[schrodervictor]

I'm also experiencing the same issue using v0.4.30 and ta-lib v0.4.0 on a Google Workbench instance. If installing manually with setup.py everything goes well.

During my research, I stumbled upon this issue: numpy/numpy#21865 (comment)

It seems that adding the necessary annotation #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION would circumvent the issue, but my knowledge of the bindings in this repo is very limited to know exactly where. I'll keep digging though.

[SarahWang0102]

• Same here. I had a image built with same Dockerfile and base image. it seems been broken today.
• Updated: xmatthias's fix saved my life, thanks!

[mrjbq7]

If this is the numpy2 issue, then it's fixed as of release 0.4.31 this morning.

If this is related to the NPY_1_7_API_VERSION thing, perhaps we can fix that too later.

[schrodervictor]

I'm having this issue also when using numpy v1.24.4, that's why I believe using the #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION annotation is the most robust solution. The library and the bindings work just fine if installing them manually, but the warning makes the installation through pip/conda fail.

[gnzsnz]

same issue here -> https://github.com/quantbelt/jupyter-quant/actions/runs/9546855109/job/26319548842

i'm pinning numpy, so it's not related to numpy v2

tested with v0.4.30 and ta-lib v0.4.0

[gnzsnz]

If this is the numpy2 issue, then it's fixed as of release 0.4.31 this morning.

I confirm that 0.4.31 fixes the issue for amd64 and arm64

[arbuzovv]

0.4.31 fixed problem

[tibitTW]

0.4.31 fixed here too