Unable to build on Alpine Linux with PyArrow 0.15.1

[ghost]

PyArrow dep was bumped to 0.15.1 recently. Unfortunately the installation of this version errors out on Alpine as below.

Expected results

Successful installation on Alpine of PyArrow, subsequently Superset..

Actual results

Have tried 2 routes: 1. a direct pip install, and 2. a manual build of PyArrow 0.15.1.

1. Direct pip3 install of pyarrow==0.15.1 fails as follows, even though numpy was installed:

  Configured for RELEASE build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})
  -- Build Type: RELEASE
  -- Build output directory: /tmp/pip-install-qe4_90uh/pyarrow/build/temp.linux-x86_64-3.7/release
  -- Found PythonInterp: /usr/local/bin/python (found version "3.7.5")
  -- Searching for Python libs in /usr/local/lib64;/usr/local/lib;/usr/local/lib/python3.7/config-3.7m-x86_64-linux-gnu
  -- Looking for python3.7m
  -- Found Python lib /usr/local/lib/libpython3.7m.so
  -- Found PythonLibs: /usr/local/lib/libpython3.7m.so
  CMake Error at cmake_modules/FindNumPy.cmake:62 (message):
    NumPy import failure:

    Traceback (most recent call last):

      File "<string>", line 1, in <module>

    ModuleNotFoundError: No module named 'numpy'

  Call Stack (most recent call first):
    CMakeLists.txt:189 (find_package)


  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-install-qe4_90uh/pyarrow/build/temp.linux-x86_64-3.7/CMakeFiles/CMakeOutput.log".
  See also "/tmp/pip-install-qe4_90uh/pyarrow/build/temp.linux-x86_64-3.7/CMakeFiles/CMakeError.log".
  error: command 'cmake' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for pyarrow

2. Building from the 0.15.1 tarball fails reproducibly with a flatbuffers_ep compile error, as per this Apache Arrow ticket.

Note that flatbuffers_ep is gone in Arrow since 0.15.1 was cut, and PyArrow builds fine on Alpine from the project's master.

How to reproduce

Simple build with pre-reqs:

FROM python:3.7-alpine

# Configure environment
RUN apk add autoconf \
    bash \
    bison \
    boost-dev \
    cmake \
    cython \
    flex \
    g++ \
    gcc \
    make

RUN pip3 install --no-cache-dir \
    six \
    cython \
    numpy

# confirm that numpy is installed
RUN python -c "import numpy; print(numpy.__version__)"
# outputs 1.17.4

RUN pip3 install pyarrow==0.15.1

Environment

• superset version: current master
• python version: 3.7.5
• node.js version: N/A
• npm version: N/A

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if there are any.
• I have reproduced the issue with at least the latest released version of superset.
• I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

Best route for Alpine at this point seems to be

• stick to 0.14.1, unless affected by the CVE's that triggered the PyArrow version bump
• move to next PyArrow release once available

Am I missing something? Any other thoughts / pointers?

[issue-label-bot]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.71. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[dpgaspar]

@uros-tm thank you so much for this info. Guess we'll have to wait for 0.15.2 to bump the dependency on superset

[jaksmid]

It seems the issue is tracked by the Arrow team and will be resolved in the 1.0 release.

See https://issues.apache.org/jira/browse/ARROW-7407

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.