Source wheel fails to build due to missing Cargo.toml

[ztsweet]

`
uname -a
inux bi-Precision-3630-Tower 5.15.0-50-generic #56~20.04.1-Ubuntu SMP Tue Sep 27 15:51:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

error

Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [5 lines of output]
💥 maturin failed
Caused by: Can't find /tmp/pip-install-z4jah88t/datafusion_f3b0577bcff64d52845e7c19eb5c3d8f/Cargo.toml (in /tmp/pip-install-z4jah88t/datafusion_f3b0577bcff64d52845e7c19eb5c3d8f)
Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/tmp/pip-modern-metadata-bh7yllpq', '--interpreter', '/home/fzt/miniconda3/envs/test/bin/python']' returned non-zero exit status 1.
Checking for Rust toolchain....
Running maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-bh7yllpq --interpreter /home/fzt/miniconda3/envs/test/bin/python
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
`

[andygrove]

Thanks for the report. I will try and figure out why this fails. FWIW, it works for me.

$ uname -a
Linux ripper 5.19.0-38-generic #39~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Mar 17 21:16:15 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

$ pip3 install datafusion==21.0.0
Collecting datafusion==21.0.0
  Downloading datafusion-21.0.0-cp37-abi3-manylinux_2_34_x86_64.whl (13.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.7/13.7 MB 10.7 MB/s eta 0:00:00
Requirement already satisfied: pyarrow>=6.0.1 in /home/andy/.local/lib/python3.10/site-packages (from datafusion==21.0.0) (11.0.0)
Requirement already satisfied: numpy>=1.16.6 in /home/andy/.local/lib/python3.10/site-packages (from pyarrow>=6.0.1->datafusion==21.0.0) (1.24.2)
Installing collected packages: datafusion
  Attempting uninstall: datafusion
    Found existing installation: datafusion 20.0.0
    Uninstalling datafusion-20.0.0:
      Successfully uninstalled datafusion-20.0.0
Successfully installed datafusion-21.0.0

[andygrove]

@ztsweet What version of Python is installed at /home/fzt/miniconda3/envs/test/bin/python?

[andygrove]

Also could you run with pip's --verbose flag?

[stromnov]

@andygrove

  Downloading datafusion-21.0.0-cp37-abi3-manylinux_2_34_x86_64.whl (13.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.7/13.7 MB 10.7 MB/s eta 0:00:00

It works because you did the installation from the pre-created wheel package, not from the source code.

[ztsweet]

@andygrove

$ python --version
Python 3.9.15

$ pip install datafusion==21.0.0 --verbose
Using pip 22.3.1 from /home/fzt/miniconda3/envs/test/lib/python3.9/site-packages/pip (python 3.9)
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Collecting datafusion==21.0.0
  Using cached https://mirrors.aliyun.com/pypi/packages/f2/46/63985a46fd8c186862cbb1c34b85505154815f095bb13e6528a66e18d261/datafusion-21.0.0.tar.gz (95 kB)
  Running command pip subprocess to install build dependencies
  Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
  Collecting maturin<0.15,>=0.11
    Using cached https://mirrors.aliyun.com/pypi/packages/dd/05/bcdd149b785de55baeee3494c672f39ee34b8a615cc20dbf2bd372454992/maturin-0.14.17-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl (10.5 MB)
  Collecting tomli>=1.1.0
    Using cached https://mirrors.aliyun.com/pypi/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl (12 kB)
  Installing collected packages: tomli, maturin
  Successfully installed maturin-0.14.17 tomli-2.0.1
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  💥 maturin failed
    Caused by: Can't find /tmp/pip-install-4gk7356m/datafusion_d1288c2759a14295893d4d14cf66befe/Cargo.toml (in /tmp/pip-install-4gk7356m/datafusion_d1288c2759a14295893d4d14cf66befe)
  Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/tmp/pip-modern-metadata-4mqh1imi', '--interpreter', '/home/fzt/miniconda3/envs/test/bin/python']' returned non-zero exit status 1.
  Checking for Rust toolchain....
  Running `maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-4mqh1imi --interpreter /home/fzt/miniconda3/envs/test/bin/python`
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/fzt/miniconda3/envs/test/bin/python /home/fzt/miniconda3/envs/test/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpujsq_7b9
  cwd: /tmp/pip-install-4gk7356m/datafusion_d1288c2759a14295893d4d14cf66befe
  Preparing metadata (pyproject.toml) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[andygrove]

I can reproduce the issue in the repo with these commands:

maturin sdist
pip --verbose install target/wheels/datafusion-21.0.0.tar.gz

Produces:

Using pip 23.0.1 from /home/andy/miniconda3/lib/python3.10/site-packages/pip (python 3.10)
Processing ./target/wheels/datafusion-21.0.0.tar.gz
  Running command pip subprocess to install build dependencies
  Collecting maturin<0.15,>=0.11
    Using cached maturin-0.14.17-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl (10.5 MB)
  Collecting tomli>=1.1.0
    Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
  Installing collected packages: tomli, maturin
  Successfully installed maturin-0.14.17 tomli-2.0.1
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  💥 maturin failed
    Caused by: Can't find /tmp/pip-req-build-kx7zxjs_/Cargo.toml (in /tmp/pip-req-build-kx7zxjs_)