Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

maturin 1.3 produces invalid source distributions for workspace packages #1837

Closed
2 tasks done
mhils opened this issue Nov 6, 2023 · 3 comments · Fixed by #1841
Closed
2 tasks done

maturin 1.3 produces invalid source distributions for workspace packages #1837

mhils opened this issue Nov 6, 2023 · 3 comments · Fixed by #1841
Labels
bug Something isn't working sdist Source distribution

Comments

@mhils
Copy link
Contributor

mhils commented Nov 6, 2023

Bug Description

Hi folks!

First off, thank y'all for building maturin. It's been super pleasant and smooth sailing so far. 🍰
We've hit a small roadbump with 1.3.0 and 1.3.1 which I wanted to report. :)

$ maturin build --release --compatibility manylinux2014 --zig --sdist
$ pip install target/wheels/*.tar.gz
Processing ./target/wheels/mitmproxy_rs-0.4.0.tar.gz
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [13 lines of output]
      error: failed to load manifest for workspace member `/tmp/pip-req-build-_e2p_y8n/mitmproxy-windows/redirector`
      
      Caused by:
        failed to read `/tmp/pip-req-build-_e2p_y8n/mitmproxy-windows/redirector/Cargo.toml`
      
      Caused by:
        No such file or directory (os error 2)
      💥 maturin failed
        Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?
        Caused by: `cargo metadata` exited with an error:
      Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/tmp/pip-modern-metadata-aoszm7an', '--interpreter', '/opt/hostedtoolcache/Python/3.11.6/x64/bin/python']' returned non-zero exit status 1.
      Checking for Rust toolchain....
      Running `maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-aoszm7an --interpreter /opt/hostedtoolcache/Python/3.11.6/x64/bin/python`
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Notice:  A new release of pip is available: 23.2.1 -> 23.3.1
Notice:  To update, run: pip install --upgrade pip
Error: Process completed with exit code 1.

Your maturin version (maturin --version)

1.3.1

Your Python version (python -V)

Python 3.12

Your pip version (pip -V)

pip 23.2.1

What bindings you're using

pyo3

Does cargo build work?

  • Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

  • Yes

Steps to Reproduce

  1. Update maturin from 1.2.3 to 1.3.1
  2. Build an sdist for a package in a workspace.

maturin 1.2.3:

$ ls -l mitmproxy_rs-0.3.11/
-rwxrwxrwx 1 user user  68K Sep 28 05:11 Cargo.lock
-rwxrwxrwx 1 user user  985 Jan  1  1970 Cargo.toml
-rwxrwxrwx 1 user user 3.7K Jan  1  1970 PKG-INFO
-rwxrwxrwx 1 user user 2.8K Sep 28 05:11 README.md
drwxrwxrwx 1 user user 4.0K Nov  6 11:37 local_dependencies
drwxrwxrwx 1 user user 4.0K Nov  6 11:37 mitmproxy_rs
-rwxrwxrwx 1 user user  827 Sep 28 05:11 pyproject.toml
drwxrwxrwx 1 user user 4.0K Nov  6 11:37 src
-rwxrwxrwx 1 user user  166 Sep 28 05:11 stubtest-allowlist.txt

maturin 1.3.1:

$ ls -l mitmproxy_rs-0.4.0/
-rwxrwxrwx 1 user user 6.1K Nov  4 11:12 CHANGELOG.md
-rwxrwxrwx 1 user user 2.0K Nov  4 11:12 CONTRIBUTING.md
-rwxrwxrwx 1 user user  68K Nov  4 11:12 Cargo.lock
-rwxrwxrwx 1 user user 2.2K Nov  4 11:12 Cargo.toml
-rwxrwxrwx 1 user user 1.1K Nov  4 11:12 LICENSE
-rwxrwxrwx 1 user user 3.7K Jan  1  1970 PKG-INFO
-rwxrwxrwx 1 user user 3.1K Nov  4 11:12 README.md
-rwxrwxrwx 1 user user 238K Nov  4 11:12 architecture.png
drwxrwxrwx 1 user user 4.0K Nov  6 11:37 benches
drwxrwxrwx 1 user user 4.0K Nov  6 11:37 mitmproxy-macos
drwxrwxrwx 1 user user 4.0K Nov  6 11:37 mitmproxy-rs
drwxrwxrwx 1 user user 4.0K Nov  6 11:37 mitmproxy-windows
drwxrwxrwx 1 user user 4.0K Nov  6 11:37 mitmproxy_rs
-rwxrwxrwx 1 user user  883 Jan  1  1970 pyproject.toml
drwxrwxrwx 1 user user 4.0K Nov  6 11:37 src

$ ls -l mitmproxy_rs-0.4.0/mitmproxy-windows/
-rwxrwxrwx 1 user user  555 Nov  4 11:12 README.md
drwxrwxrwx 1 user user 4.0K Nov  6 11:37 mitmproxy_windows
-rwxrwxrwx 1 user user  596 Nov  4 11:12 pyproject.toml

It looks like mitmproxy-windows/redirector is missing entirely from the sdist, even though it's a workspace member.

See mitmproxy/mitmproxy_rs#123 for a full reproducer. :)
@mhils mhils added the bug Something isn't working label Nov 6, 2023
@mhils mhils mentioned this issue Nov 6, 2023
Closed
@messense
Copy link
Member

messense commented Nov 7, 2023 •
edited
Loading

Thanks for the report, can you try build sdist from maturin main branch? This might be the same issue as #1803 which was fixed in #1811, actually I meant #1828.

@messense
Copy link
Member

messense commented Nov 8, 2023

Not sure why but windows redirector isn't included in cargo package --list

$ cargo package --list --allow-dirty
warning: manifest has no description.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
.cargo/config.toml
.github/FUNDING.yml
.github/dependabot.yml
.github/python-version.txt
.github/scripts/build-macos-redirector.sh
.github/scripts/pin-versions.py
.github/workflows/autofix.yml
.github/workflows/ci.yml
.github/workflows/docs.yml
.gitignore
CHANGELOG.md
CONTRIBUTING.md
Cargo.toml
Cargo.toml.orig
LICENSE
README.md
architecture.png
benches/.gitignore
benches/local.conf
benches/nonlocal.conf
benches/openvpnserv.exe
benches/plot.py
benches/process.rs
benches/py_echo_client.py
benches/py_echo_server.py
benches/wg_echo_client.py
benches/wg_echo_server.py
mitmproxy-macos/.swift-format
mitmproxy-macos/.swiftlint.yml
mitmproxy-macos/README.md
mitmproxy-macos/mitmproxy_macos/__init__.py
mitmproxy-macos/mitmproxy_macos/macos-certificate-truster.app/Contents/Info.plist
mitmproxy-macos/mitmproxy_macos/macos-certificate-truster.app/Contents/Resources/mitmproxy.icns
mitmproxy-macos/pyproject.toml
mitmproxy-macos/redirector/.gitignore
mitmproxy-macos/redirector/ExportOptions.plist
mitmproxy-macos/redirector/README.md
mitmproxy-macos/redirector/ipc/mitmproxy_ipc.pb.swift
mitmproxy-macos/redirector/ipc/utils.swift
mitmproxy-macos/redirector/macos-redirector/Assets.xcassets/AppIcon.appiconset/Contents.json
mitmproxy-macos/redirector/macos-redirector/Assets.xcassets/AppIcon.appiconset/logo-circle-1024.png
mitmproxy-macos/redirector/macos-redirector/Assets.xcassets/AppIcon.appiconset/logo-circle-128.png
mitmproxy-macos/redirector/macos-redirector/Assets.xcassets/AppIcon.appiconset/logo-circle-16.png
mitmproxy-macos/redirector/macos-redirector/Assets.xcassets/AppIcon.appiconset/logo-circle-256.png
mitmproxy-macos/redirector/macos-redirector/Assets.xcassets/AppIcon.appiconset/logo-circle-32.png
mitmproxy-macos/redirector/macos-redirector/Assets.xcassets/AppIcon.appiconset/logo-circle-512.png
mitmproxy-macos/redirector/macos-redirector/Assets.xcassets/AppIcon.appiconset/logo-circle-64.png
mitmproxy-macos/redirector/macos-redirector/Assets.xcassets/Contents.json
mitmproxy-macos/redirector/macos-redirector/app.swift
mitmproxy-macos/redirector/macos-redirector/macos_redirector.entitlements
mitmproxy-macos/redirector/macos-redirector.xcodeproj/project.pbxproj
mitmproxy-macos/redirector/macos-redirector.xcodeproj/project.xcworkspace/contents.xcworkspacedata
mitmproxy-macos/redirector/macos-redirector.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist
mitmproxy-macos/redirector/macos-redirector.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved
mitmproxy-macos/redirector/macos-redirector.xcodeproj/xcshareddata/xcschemes/macos-redirector.xcscheme
mitmproxy-macos/redirector/network-extension/FlowExtensions.swift
mitmproxy-macos/redirector/network-extension/Info.plist
mitmproxy-macos/redirector/network-extension/InterceptConf.swift
mitmproxy-macos/redirector/network-extension/ProcessInfoCache.swift
mitmproxy-macos/redirector/network-extension/TransparentProxyProvider.swift
mitmproxy-macos/redirector/network-extension/libproc-Bridging-Header.h
mitmproxy-macos/redirector/network-extension/main.swift
mitmproxy-macos/redirector/network-extension/network_extension.entitlements
mitmproxy-macos/version-info.toml
mitmproxy-windows/README.md
mitmproxy-windows/mitmproxy_windows/LICENSE
mitmproxy-windows/mitmproxy_windows/WINDIVERT_VERSION
mitmproxy-windows/mitmproxy_windows/WinDivert.dll
mitmproxy-windows/mitmproxy_windows/WinDivert.lib
mitmproxy-windows/mitmproxy_windows/WinDivert64.sys
mitmproxy-windows/mitmproxy_windows/__init__.py
mitmproxy-windows/mitmproxy_windows/editable.marker
mitmproxy-windows/pyproject.toml
src/intercept_conf.rs
src/ipc/mitmproxy_ipc.proto
src/ipc/mitmproxy_ipc.rs
src/ipc/mod.rs
src/lib.rs
src/macos.rs
src/messages.rs
src/network/mod.rs
src/network/task.rs
src/network/tests.rs
src/network/virtual_device.rs
src/packet_sources/macos.rs
src/packet_sources/mod.rs
src/packet_sources/windows.rs
src/packet_sources/wireguard.rs
src/processes.rs
src/shutdown.rs
src/util.rs
src/windows/icons.rs
src/windows/mod.rs
src/windows/network.rs
src/windows/processes.rs

@messense messense mentioned this issue Nov 8, 2023
Merged
@messense messense added the sdist Source distribution label Nov 8, 2023
@mhils
Copy link
Contributor Author

mhils commented Nov 8, 2023

Sorry for the slow turnaround and thank you for the nonetheless super quick fix. You folks are fantastic! 🤩🤩🤩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working sdist Source distribution
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix rewriting workspace Cargo.toml in sdist messense/maturin
2 participants
@mhils @messense