Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bin sdist fails to build #1571

Closed
1 of 2 tasks
epage opened this issue Apr 19, 2023 · 2 comments · Fixed by #1577
Closed
1 of 2 tasks

bin sdist fails to build #1571

epage opened this issue Apr 19, 2023 · 2 comments · Fixed by #1577
Assignees
Labels
bug Something isn't working sdist Source distribution

Comments

@epage
Copy link

epage commented Apr 19, 2023

Bug Description

I've adopted maturin for distributing my pre-commit hooks so people can use the pre-built wheels. Unfortunately, I just got a bug report (crate-ci/typos#713) that the sdist fails to build because it can't find the Cargo.toml

Collecting typos==1.13.20
  Using cached typos-1.13.20.tar.gz (1.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [5 lines of output]
      💥 maturin failed
        Caused by: Can't find /tmp/pip-install-h_ghf2j0/typos_b9b297fa519b4611aa15b3189fe4a7b7/Cargo.toml (in /tmp/pip-install-h_ghf2j0/typos_b9b297fa519b4611aa15b3189fe4a7b7)
      Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/tmp/pip-modern-metadata-rwz40ctg', '--interpreter', '/home/epage/src/personal/typos/foo/bin/python3']' returned non-zero exit status 1.
      Checking for Rust toolchain....
      Running `maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-rwz40ctg --interpreter /home/epage/src/personal/typos/foo/bin/python3`
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

I can reproduce it locally but its not all clear what is going on behind the scenes that is causing this to fail. I would have assumed the wheels are created from the sdist but our CI job passes that creates all wheels and sdist

Doing a quick comparison

  • ruff (assuming sdist works)
    • /pyproject.toml
    • /crates/ruff-cli/Cargo.toml
    • Sets tool.maturin.manifest-path for this to work
  • typos 1.13.20 (sdist fails)
    • /pyproject.toml
    • /Cargo.toml
  • typos 1.14.7 (sdist fails)
    • /crates/typos-cli/pyproject.toml
    • /crates/typos-cli/Cargo.toml
    • Note: moved typos-cli into the crates folder so I could have a pure-python pyproject.toml in the root for pre-commit to use because it always installs the top-level package from source

When writing the reason why we switched from 1.13.20's layout to 1.14.7's, that means it did work at some point (crate-ci/typos#682), so I'm a bit confused as to why its not working for me right now or the person who opened the issue on my repo.

ruff sdist includes:

... ruff-0.0.261/local_dependencies ...
ruff-0.0.261/crates/ruff_cli/Cargo.toml
ruff-0.0.261/crates/ruff_cli/src/args.rs
ruff-0.0.261/crates/ruff_cli/src/bin/ruff.rs
ruff-0.0.261/crates/ruff_cli/src/cache.rs
ruff-0.0.261/crates/ruff_cli/src/commands/add_noqa.rs
ruff-0.0.261/crates/ruff_cli/src/commands/clean.rs
ruff-0.0.261/crates/ruff_cli/src/commands/config.rs
ruff-0.0.261/crates/ruff_cli/src/commands/linter.rs
ruff-0.0.261/crates/ruff_cli/src/commands/mod.rs
ruff-0.0.261/crates/ruff_cli/src/commands/rule.rs
ruff-0.0.261/crates/ruff_cli/src/commands/run.rs
ruff-0.0.261/crates/ruff_cli/src/commands/run_stdin.rs
ruff-0.0.261/crates/ruff_cli/src/commands/show_files.rs
ruff-0.0.261/crates/ruff_cli/src/commands/show_settings.rs
ruff-0.0.261/crates/ruff_cli/src/diagnostics.rs
ruff-0.0.261/crates/ruff_cli/src/lib.rs
ruff-0.0.261/crates/ruff_cli/src/panic.rs
ruff-0.0.261/crates/ruff_cli/src/printer.rs
ruff-0.0.261/crates/ruff_cli/src/resolve.rs
ruff-0.0.261/crates/ruff_cli/tests/black_compatibility_test.rs
ruff-0.0.261/crates/ruff_cli/tests/integration_test.rs
ruff-0.0.261/pyproject.toml
ruff-0.0.261/crates/ruff_cli/Cargo.lock
ruff-0.0.261/python/ruff/__init__.py
ruff-0.0.261/python/ruff/__main__.py
ruff-0.0.261/README.md
ruff-0.0.261/LICENSE
ruff-0.0.261/PKG-INFO

1.13.20 sdist includes:

typos-1.13.20/local_dependencies/typos-vars/Cargo.toml
typos-1.13.20/local_dependencies/typos-vars/src/lib.rs
typos-1.13.20/local_dependencies/typos-vars/src/vars_codegen.rs
typos-1.13.20/local_dependencies/typos-dict/Cargo.toml
typos-1.13.20/local_dependencies/typos-dict/src/dict_codegen.rs
typos-1.13.20/local_dependencies/typos-dict/src/lib.rs
typos-1.13.20/local_dependencies/typos/Cargo.toml
typos-1.13.20/local_dependencies/typos/src/check.rs
typos-1.13.20/local_dependencies/typos/src/dict.rs
typos-1.13.20/local_dependencies/typos/src/lib.rs
typos-1.13.20/local_dependencies/typos/src/tokens.rs
typos-1.13.20/local_dependencies/varcon-core/Cargo.toml
typos-1.13.20/local_dependencies/varcon-core/src/borrowed.rs
typos-1.13.20/local_dependencies/varcon-core/src/lib.rs
typos-1.13.20/local_dependencies/varcon-core/src/parser.rs
typos-1.13.20/local_dependencies/dictgen/Cargo.toml
typos-1.13.20/local_dependencies/dictgen/src/lib.rs
typos-1.13.20/local_dependencies/dictgen/src/map.rs
typos-1.13.20/local_dependencies/dictgen/src/table.rs
typos-1.13.20/local_dependencies/dictgen/src/trie.rs
typos-1.13.20/pyproject.toml
typos-1.13.20/rust_src/typos/Cargo.toml
typos-1.13.20/rust_src/typos/Cargo.lock
typos-1.13.20/rust_src/typos/LICENSE-APACHE
typos-1.13.20/rust_src/typos/LICENSE-MIT
typos-1.13.20/rust_src/typos/README.md
typos-1.13.20/rust_src/typos/benches/checks.rs
typos-1.13.20/rust_src/typos/benches/corrections.rs
typos-1.13.20/rust_src/typos/benches/data.rs
typos-1.13.20/rust_src/typos/benches/tokenize.rs
typos-1.13.20/rust_src/typos/src/bin/typos-cli/args.rs
typos-1.13.20/rust_src/typos/src/bin/typos-cli/main.rs
typos-1.13.20/rust_src/typos/src/bin/typos-cli/report.rs
typos-1.13.20/rust_src/typos/src/config.rs
typos-1.13.20/rust_src/typos/src/default_types.rs
typos-1.13.20/rust_src/typos/src/dict.rs
typos-1.13.20/rust_src/typos/src/file.rs
typos-1.13.20/rust_src/typos/src/file_type.rs
typos-1.13.20/rust_src/typos/src/lib.rs
typos-1.13.20/rust_src/typos/src/policy.rs
typos-1.13.20/rust_src/typos/src/report.rs
typos-1.13.20/Cargo.lock
typos-1.13.20/PKG-INFO

1.14.7 sdist includes:

typos-1.14.7/local_dependencies/typos-dict/Cargo.toml
typos-1.14.7/local_dependencies/typos-dict/src/dict_codegen.rs
typos-1.14.7/local_dependencies/typos-dict/src/lib.rs
typos-1.14.7/local_dependencies/typos-vars/Cargo.toml
typos-1.14.7/local_dependencies/typos-vars/src/lib.rs
typos-1.14.7/local_dependencies/typos-vars/src/vars_codegen.rs
typos-1.14.7/local_dependencies/dictgen/Cargo.toml
typos-1.14.7/local_dependencies/dictgen/src/lib.rs
typos-1.14.7/local_dependencies/dictgen/src/map.rs
typos-1.14.7/local_dependencies/dictgen/src/table.rs
typos-1.14.7/local_dependencies/dictgen/src/trie.rs
typos-1.14.7/local_dependencies/varcon-core/Cargo.toml
typos-1.14.7/local_dependencies/varcon-core/src/borrowed.rs
typos-1.14.7/local_dependencies/varcon-core/src/lib.rs
typos-1.14.7/local_dependencies/varcon-core/src/parser.rs
typos-1.14.7/local_dependencies/typos/Cargo.toml
typos-1.14.7/local_dependencies/typos/src/check.rs
typos-1.14.7/local_dependencies/typos/src/dict.rs
typos-1.14.7/local_dependencies/typos/src/lib.rs
typos-1.14.7/local_dependencies/typos/src/tokens.rs
typos-1.14.7/pyproject.toml
typos-1.14.7/rust_src/typos-cli/Cargo.toml
typos-1.14.7/rust_src/typos-cli/benches/checks.rs
typos-1.14.7/rust_src/typos-cli/benches/corrections.rs
typos-1.14.7/rust_src/typos-cli/benches/data.rs
typos-1.14.7/rust_src/typos-cli/benches/tokenize.rs
typos-1.14.7/rust_src/typos-cli/src/bin/typos-cli/args.rs
typos-1.14.7/rust_src/typos-cli/src/bin/typos-cli/main.rs
typos-1.14.7/rust_src/typos-cli/src/bin/typos-cli/report.rs
typos-1.14.7/rust_src/typos-cli/src/config.rs
typos-1.14.7/rust_src/typos-cli/src/default_types.rs
typos-1.14.7/rust_src/typos-cli/src/dict.rs
typos-1.14.7/rust_src/typos-cli/src/file.rs
typos-1.14.7/rust_src/typos-cli/src/file_type.rs
typos-1.14.7/rust_src/typos-cli/src/lib.rs
typos-1.14.7/rust_src/typos-cli/src/policy.rs
typos-1.14.7/rust_src/typos-cli/src/report.rs
typos-1.14.7/Cargo.lock
typos-1.14.7/PKG-INFO

Unsure why I have a Cargo.lock in the root but ruff does not

Your maturin version (maturin --version)

0.14.17

Your Python version (python -V)

Python 3.10.6

Your pip version (pip -V)

pip 22.0.2

What bindings you're using

bin

Does cargo build work?

  • Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

  • Yes

Steps to Reproduce

  1. Run pip install typos --no-binary typos
@messense
Copy link
Member

messense commented Apr 19, 2023

Thanks for the report, I'll take a look this weekend along with #1570.

In the meantime you can build and publish musllinux wheels for Alpine users so they don't need to compile from sdist.

For example:

jobs:
  musllinux:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        target: [x86_64, aarch64]
    steps:
      - uses: actions/checkout@v3
      - name: Build wheels
        uses: PyO3/maturin-action@v1
        with:
          target: ${{ matrix.target }}
          args: --release --out dist
          sccache: 'true'
          manylinux: musllinux_1_2
      - name: Upload wheels
        uses: actions/upload-artifact@v3
        with:
          name: wheels
          path: dist

@messense messense added the sdist Source distribution label Apr 19, 2023
@messense messense self-assigned this Apr 22, 2023
@messense
Copy link
Member

So the problem is that in workspace Cargo.toml, pyproject.toml isn't in the include list, so cargo package --list doesn't include it, add pyproject.toml to it should fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working sdist Source distribution
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants