`uv python install 3.12` is not working on armv7 libc is identify as `gnu` instead of `gnueabihf` #6873

zarch · 2024-08-30T14:31:41Z

I tried uv 0.3.5 to install a specific version of python on my raspberry-pi:

❯ uname -rmo
6.1.21-v7l+ armv7l GNU/Linux

When I try to install python I got:

❯ uv python install 3.12
Searching for Python versions matching: Python 3.12
error: No download found for request: cpython-3.12-linux-armv7-gnu

Looking at the code it seems that all the version available are taken from uv-python/download-metadata.json, where at the moment we have:

{
  "cpython-3.12.5-linux-armv7-gnueabi": {
    "name": "cpython",
    "arch": "armv7",
    "os": "linux",
    "libc": "gnueabi",
    "major": 3,
    "minor": 12,
    "patch": 5,
    "url": "https://github.com/indygreg/python-build-standalone/releases/download/20240814/cpython-3.12.5%2B20240814-armv7-unknown-linux-gnueabi-install_only_stripped.tar.gz",
    "sha256": "7a584de9c2824f43d7a7b1c26eb61a18af770ebd603a74b45d57601ba62ba508"
  },
  "cpython-3.12.5-linux-armv7-gnueabihf": {
    "name": "cpython",
    "arch": "armv7",
    "os": "linux",
    "libc": "gnueabihf",
    "major": 3,
    "minor": 12,
    "patch": 5,
    "url": "https://github.com/indygreg/python-build-standalone/releases/download/20240814/cpython-3.12.5%2B20240814-armv7-unknown-linux-gnueabihf-install_only_stripped.tar.gz",
    "sha256": "a9992b30d7b3ecb558cd12fde919e3e2836f161f8f777afea31140d5fff6362e"
  }
}

Using other tool such as pystand they work as expected:

❯ uvx pystand show
3.9.19 @ 20240814 distribution="armv7-unknown-linux-gnueabihf"
3.10.14 @ 20240814 distribution="armv7-unknown-linux-gnueabihf"
3.11.9 @ 20240814 distribution="armv7-unknown-linux-gnueabihf"
3.12.5 @ 20240814 distribution="armv7-unknown-linux-gnueabihf"

Looking at the code from my understanding uv is trying to infer this information from ldd while pystand look at the platform in a simpler way:

# Default distributions for various platforms
DISTRIBUTIONS = {
    ('Linux', 'x86_64'): 'x86_64-unknown-linux-gnu',
    ('Linux', 'aarch64'): 'aarch64-unknown-linux-gnu',
    ('Linux', 'armv7l'): 'armv7-unknown-linux-gnueabihf',
    ('Linux', 'armv8l'): 'armv7-unknown-linux-gnueabihf',
    ('Darwin', 'x86_64'): 'x86_64-apple-darwin',
    ('Darwin', 'aarch64'): 'aarch64-apple-darwin',
    ('Windows', 'x86_64'): 'x86_64-pc-windows-msvc',
    ('Windows', 'i686'): 'i686-pc-windows-msvc',
}

Perhaps uv should use/integrate a similar approach?

The text was updated successfully, but these errors were encountered:

zarch · 2024-08-30T14:35:32Z

Or at least it would be nice to be able to define / force the right value by command line interface?

charliermarsh · 2024-08-30T14:36:03Z

@konstin -- Do you know if this is expected to work yet?

konstin · 2024-08-30T17:17:50Z

iirc we have to detect whether we're on a hardfloat abi or not and use guneabi or gnueabihf depending on that

zarch · 2024-09-02T09:19:34Z

If you like I can prepare a PR, but I'm not sure which is the best approach to detect if we are are using hardfloat abi or not.

Executing the code enabling the RUST_LOG=trace

❯ RUST_LOG=trace uv python install 3.12 --verbose
DEBUG uv 0.4.2
TRACE Checking lock for `/root/.local/share/uv/python` at `/root/.local/share/uv/python/.lock`
DEBUG Acquired lock for `/root/.local/share/uv/python`
Searching for Python versions matching: Python 3.12
TRACE ld path: /lib/ld-linux-armhf.so.3
TRACE stdout output from `ld`: ""
TRACE stderr output from `ld`: "Usage: ld.so [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...]\nYou have invoked `ld.so', the helper program for shared library executables.\nThis program usually lives in the file `/lib/ld.so', and special directives\nin executable files using ELF shared libraries tell the system's program\nloader to load the helper program from this file.  This helper program loads\nthe shared libraries needed by the program executable, prepares the program\nto run, and runs it.  You may invoke this helper program directly from the\ncommand line to load and run an ELF executable file; this is like executing\nthat file itself, but always uses this helper program from the file you\nspecified, instead of the helper program file specified in the executable\nfile you run.  This is mostly of use for maintainers to test new versions\nof this helper program; chances are you did not intend to run this program.\n\n  --list                list all dependencies and how they are resolved\n  --verify              verify that given object really is a dynamically linked\n\t\t\tobject we can handle\n  --inhibit-cache       Do not use /etc/ld.so.cache\n  --library-path PATH   use given PATH instead of content of the environment\n\t\t\tvariable LD_LIBRARY_PATH\n  --inhibit-rpath LIST  ignore RUNPATH and RPATH information in object names\n\t\t\tin LIST\n  --audit LIST          use objects named in LIST as auditors\n  --preload LIST        preload objects named in LIST\n"
TRACE Tried to find musl version by running `"/lib/ld-linux-armhf.so.3"`, but failed: Could not find musl version in output of: `/lib/ld-linux-armhf.so.3`
DEBUG Released lock at `/root/.local/share/uv/python/.lock`
error: No download found for request: cpython-3.12-linux-armv7-gnu

I see that, in debian, in the ld path it is already indicated the is hardfloat: TRACE ld path: /lib/ld-linux-armhf.so.3, so we could infer from the path, but I'm not sure it is general enough to cover other case and distributions. I tried few other distributions on my laptop:

fedora (rawhide): TRACE ld path: /lib64/ld-linux-x86-64.so.2
archlinux (stable): TRACE ld path: /lib64/ld-linux-x86-64.so.2
alpine(edge): TRACE ld path: /lib/ld-musl-x86-64.so.1

So it seems to me that this pattern is shared among linux distributions.
Perhaps, I can add a if condition within the detect_linux_libc() function that check if the path contains armhf string then it return gnueabihf, if contains just arm then return: gnueabi else continue with the other check.

Otherwise, we can simply implement the same logic used by pystand and see the platform architecture calling uname -m. Other approaches that you would like to follow?

orgua · 2024-09-25T15:11:46Z

I have the same trouble on a armv7 beaglebone black. Latest uv 0.4.16 won't install any python version, not just 3.12. Would love to be able to use that feature! Armv7 is slow enough and could use every improvement newer py-versions bring :)

kakkoyun · 2024-09-26T19:43:29Z

I’ve thought through a few different options for detecting whether to use gnueabi or gnueabihf. One possibility is analyzing an ELF binary using readelf (or Goblin) to check for Tag_ABI_VFP_args (see ARM documentation), but I decided to go with a simpler approach: checking /proc/cpuinfo (#7725). The idea is to look for the vfp flag, which indicates hardware floating-point support. There’s some more info on that here.

I have some Raspberry Pis lying around, but they need to be provisioned first. Could I ask for your help testing #7725, @orgua @zarch? I’d be happy to provide a pre-built version of uv from the PR if that’s easier for you!

konstin · 2024-10-01T15:35:55Z

(i tried testing this but my raspi is too new and docker run -it --rm -v $(pwd):/io --platform linux/arm/v7 ubuntu shows my native cpu)

kakkoyun · 2024-10-02T15:46:47Z

(i tried testing this but my raspi is too new and docker run -it --rm -v $(pwd):/io --platform linux/arm/v7 ubuntu shows my native cpu)

Thanks. I'm in the process of reviving some of my old pi's. I'll update the PR when I've something.

zarch · 2024-10-03T10:21:12Z

@kakkoyun Sorry for the delay but I have to free up some space to be able to compile uv on my rpi... :-D

The current branch is not working for me. I tried:

uv on  detect_hardfloat is 📦 v0.4.16 via 🐍 via 🦀 v1.81.0 took 3m19s 
❯ target/debug/uv python install 3.12
Searching for Python versions matching: Python 3.12                                                                  
error: Failed to parse Python installation key `cpython-3.12.6-linux-armv7-gnueabi`: invalid libc: Unknown libc envir
onment: gnueabi

I'm on:

uv on  detect_hardfloat is 📦 v0.4.16 via 🐍 via 🦀 v1.81.0 took 10s 
❯ git show
commit fde37914b7c01df2d97c747c9a6a28f1b3bd6dc1 (HEAD -> detect_hardfloat, origin/detect_hardfloat)
Author: Kemal Akkoyun <kakkoyun@gmail.com>
Date:   Fri Sep 27 15:02:32 2024 +0200
commit fde37914b7c01df2d97c747c9a6a28f1b3bd6dc1 (HEAD -> detect_hardfloat, origin/detect_hardfloat)               
Author: Kemal Akkoyun <kakkoyun@gmail.com>
Date:   Fri Sep 27 15:02:32 2024 +0200
[...]

Here my cpuinfo:

uv on  detect_hardfloat is 📦 v0.4.16 via 🐍 via 🦀 v1.81.0 took 6s 
❯ cat /proc/cpuinfo
processor       : 0
model name      : ARMv7 Processor rev 3 (v7l)
BogoMIPS        : 108.00
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 1
model name      : ARMv7 Processor rev 3 (v7l)
BogoMIPS        : 108.00
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 2
model name      : ARMv7 Processor rev 3 (v7l)
BogoMIPS        : 108.00
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 3
model name      : ARMv7 Processor rev 3 (v7l)
BogoMIPS        : 108.00
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

Hardware        : BCM2711
Revision        : d03115
Serial          : 100000008cd9439f
Model           : Raspberry Pi 4 Model B Rev 1.5

And here the command execution with the full log:

uv on  detect_hardfloat is 📦 v0.4.16 via 🐍 via 🦀    v1.81.0 
❯ RUST_LOG=trace cargo run python install 3.12 --verbose
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3.92s
     Running `target/debug/uv python install 3.12 --verbose`
DEBUG uv 0.4.16
TRACE Checking lock for `/home/zarch/.local/share/uv/python` at `/home/zarch/.local/share/uv/python/.lock`
DEBUG Acquired lock for `/home/zarch/.local/share/uv/python`
WARN Ignoring malformed managed Python entry:
    Failed to parse Python installation key `cpython-3.12.6-linux-armv7-gnueabi`: invalid libc: Unknown libc environment: gnueabi
Searching for Python versions matching: Python 3.12
TRACE ld path: /lib/ld-linux-armhf.so.3
TRACE stdout output from `ld`: ""
TRACE stderr output from `ld`: "Usage: ld.so [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...]\nYou have invoked `ld.so', the helper program for shared library executables.\nThis program usually lives in the file `/lib/ld.so', and special directives\nin executable files using ELF shared libraries tell the system's program\nloader to load the helper program from this file.  This helper program loads\nthe shared libraries needed by the program executable, prepares the program\nto run, and runs it.  You may invoke this helper program directly from the\ncommand line to load and run an ELF executable file; this is like executing\nthat file itself, but always uses this helper program from the file you\nspecified, instead of the helper program file specified in the executable\nfile you run.  This is mostly of use for maintainers to test new versions\nof this helper program; chances are you did not intend to run this program.\n\n  --list                list all dependencies and how they are resolved\n  --verify              verify that given object really is a dynamically linked\n\t\t\tobject we can handle\n  --inhibit-cache       Do not use /etc/ld.so.cache\n  --library-path PATH   use given PATH instead of content of the environment\n\t\t\tvariable LD_LIBRARY_PATH\n  --inhibit-rpath LIST  ignore RUNPATH and RPATH information in object names\n\t\t\tin LIST\n  --audit LIST          use objects named in LIST as auditors\n  --preload LIST        preload objects named in LIST\n"
TRACE Tried to find musl version by running `"/lib/ld-linux-armhf.so.3"`, but failed: Could not find musl version in output of: `/lib/ld-linux-armhf.so.3`
DEBUG Using request timeout of 30s
DEBUG Released lock at `/home/zarch/.local/share/uv/python/.lock`
error: Failed to parse Python installation key `cpython-3.12.6-linux-armv7-gnueabi`: invalid libc: Unknown libc environment: gnueabi

kakkoyun · 2024-10-04T12:30:44Z

@zarch Thanks a lot for testing it. These logs are helpful. I will give it another shot.

golgor · 2024-10-07T15:13:21Z

This is not recommended, but if it is blocking you a it is possible to install a Python version manually.

It is a bit cumbersome but this worked at least for the RPi 4 I'm using:

Download and compile the source (This can be done with Pyenv; pyenv install 3.12.7)
Find the path to the installed python and copy it to ~/.local/share/uv/python
Make sure that the file name is similar to the following: cpython-3.12.7-linux-armv7l-gnu

tree ~/.local/share/uv/python -L 2

.
└ cpython-3.12.7-linux-armv7l-gnu
  ├── bin
  ├── include
  ├── lib
  └── share

zanieb · 2024-10-07T15:31:00Z

You can also request a full download key, e.g., uv python install cpython-3.12.6-linux-armv7-gnueabihf, then rename the directory as described above

edit: Actually this fails to parse correctly because the error reported above. i'll fix that. #7975

zarch · 2024-10-29T23:28:22Z

Dear @kakkoyun and @zanieb

I've just tested the #8498 applied on #7725 and rebase it to the current main branch (commit: cd408cbc5), and it works on my pc.

How would you like to proceed?

uv on  detect_hardfloat [!⇕] is 📦 v0.4.28 via 🐍 via 🦀 v1.81.0 took 7s 
❯ cargo run python install 3.13
   Compiling uv-python v0.0.1 (/home/zarch/source/uv/crates/uv-python)
   Compiling uv-types v0.0.1 (/home/zarch/source/uv/crates/uv-types)
   Compiling uv-virtualenv v0.0.4 (/home/zarch/source/uv/crates/uv-virtualenv)
   Compiling uv-distribution v0.0.1 (/home/zarch/source/uv/crates/uv-distribution)
   Compiling uv-resolver v0.0.1 (/home/zarch/source/uv/crates/uv-resolver)
   Compiling uv-installer v0.0.1 (/home/zarch/source/uv/crates/uv-installer)
   Compiling uv-build-frontend v0.0.1 (/home/zarch/source/uv/crates/uv-build-frontend)
   Compiling uv-settings v0.0.1 (/home/zarch/source/uv/crates/uv-settings)
   Compiling uv-requirements v0.1.0 (/home/zarch/source/uv/crates/uv-requirements)
   Compiling uv-dispatch v0.0.1 (/home/zarch/source/uv/crates/uv-dispatch)
   Compiling uv-scripts v0.0.1 (/home/zarch/source/uv/crates/uv-scripts)
   Compiling uv-cli v0.0.1 (/home/zarch/source/uv/crates/uv-cli)
   Compiling uv-tool v0.0.1 (/home/zarch/source/uv/crates/uv-tool)
   Compiling uv v0.4.28 (/home/zarch/source/uv/crates/uv)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 01s
     Running `target/debug/uv python install 3.13`
Searching for Python versions matching: Python 3.13
Installed Python 3.13.0 in 9.96s
 + cpython-3.13.0-linux-armv7-gnueabihf

uv on  detect_hardfloat [!⇕] is 📦 v0.4.28 via 🐍 via 🦀 v1.81.0 took 3m11s 
❯ /home/zarch/.local/share/uv/python/cpython-3.13.0-linux-armv7-gnueabihf/bin/python3
Python 3.13.0 (main, Oct 16 2024, 02:57:06) [GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 1+1
2

zanieb · 2024-10-29T23:38:11Z

Sorry I got a bit confused that your branch was on top of the other pull request. I'll clean it up and merge, thanks!

zanieb added bug Something isn't working uv python Related to the uv python interface labels Sep 3, 2024

kakkoyun mentioned this issue Sep 26, 2024

uv python install: Detect hard-float support #7725

Closed

1 task

zarch mentioned this issue Oct 23, 2024

Fix hard and soft float libc detection for managed Python distributions on ARM #8498

Merged

zanieb closed this as completed Oct 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`uv python install 3.12` is not working on armv7 libc is identify as `gnu` instead of `gnueabihf` #6873

`uv python install 3.12` is not working on armv7 libc is identify as `gnu` instead of `gnueabihf` #6873

zarch commented Aug 30, 2024

zarch commented Aug 30, 2024

charliermarsh commented Aug 30, 2024

konstin commented Aug 30, 2024

zarch commented Sep 2, 2024

orgua commented Sep 25, 2024

kakkoyun commented Sep 26, 2024

konstin commented Oct 1, 2024 •

edited

Loading

kakkoyun commented Oct 2, 2024

zarch commented Oct 3, 2024

kakkoyun commented Oct 4, 2024

golgor commented Oct 7, 2024 •

edited

Loading

zanieb commented Oct 7, 2024 •

edited

Loading

zarch commented Oct 29, 2024

zanieb commented Oct 29, 2024

uv python install 3.12 is not working on armv7 libc is identify as gnu instead of gnueabihf #6873

uv python install 3.12 is not working on armv7 libc is identify as gnu instead of gnueabihf #6873

Comments

zarch commented Aug 30, 2024

zarch commented Aug 30, 2024

charliermarsh commented Aug 30, 2024

konstin commented Aug 30, 2024

zarch commented Sep 2, 2024

orgua commented Sep 25, 2024

kakkoyun commented Sep 26, 2024

konstin commented Oct 1, 2024 • edited Loading

kakkoyun commented Oct 2, 2024

zarch commented Oct 3, 2024

kakkoyun commented Oct 4, 2024

golgor commented Oct 7, 2024 • edited Loading

zanieb commented Oct 7, 2024 • edited Loading

zarch commented Oct 29, 2024

zanieb commented Oct 29, 2024

`uv python install 3.12` is not working on armv7 libc is identify as `gnu` instead of `gnueabihf` #6873

`uv python install 3.12` is not working on armv7 libc is identify as `gnu` instead of `gnueabihf` #6873

konstin commented Oct 1, 2024 •

edited

Loading

golgor commented Oct 7, 2024 •

edited

Loading

zanieb commented Oct 7, 2024 •

edited

Loading