Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pkgx install python@3.11 - missing destination operand #1100

Closed
abitrolly opened this issue Jan 26, 2025 · 24 comments
Closed

pkgx install python@3.11 - missing destination operand #1100

abitrolly opened this issue Jan 26, 2025 · 24 comments

Comments

@abitrolly
Copy link
Contributor

abitrolly commented Jan 26, 2025 •
edited
Loading 

# curl https://pkgx.sh | sh
# pkgx install python@3.11
install: missing destination file operand after 'python@3.11'
Try '/root/.pkgx/gnu.org/coreutils/v9.6.0/bin/install --help' for more information.

Installed pkgx with curl into a fresh Fedora container.

# pkgx --version
pkgx 2.1.2
@jhheider
Copy link
Contributor

Try '/root/.pkgx/gnu.org/coreutils/v9.6.0/bin/install --help' for more information.

it's running GNU install. you want pkgm install python@3.11.

@abitrolly
Copy link
Contributor Author

@jhheider

# pkgm install python@3.11
Error: "cmd not found: deno"

abitrolly referenced this issue in abitrolly/www Jan 26, 2025
@abitrolly
Use pkgm to install stuff
b988591 
See https://github.com/pkgxdev/pkgx/issues/1094
@abitrolly abitrolly mentioned this issue Jan 26, 2025
Closed
@jhheider
Copy link
Contributor

that seems like a pkgm bug. I'll move this over.

@jhheider jhheider transferred this issue from pkgxdev/pkgx Jan 26, 2025
@abitrolly
Copy link
Contributor Author

abitrolly commented Jan 26, 2025 •
edited
Loading

@jhheider it actually breaks after Ctrl-C.

[root@36fff2f4eb4c workdir]# pkgm install python@3.11
7.09s ❲⚯⚯⚯⚯⚯                                                   ❳ 4% 152.27 KiB/s 1.88/53.1 MiB^C
[root@36fff2f4eb4c workdir]# pkgm install python@3.11
Error: "cmd not found: deno"

@abitrolly
Copy link
Contributor Author

Trying to "fix" is by running with pkgx leads to more bugs.

# pkgx pkgm
Error: SqlInputError { error: Error { code: Unknown, extended_code: 1 }, msg: "table aliases already exists", sql: "\n    CREATE TABLE aliases (\n        project TEXT,\n        alias TEXT\n    );\n    CREATE INDEX idx_project ON provides(project);\n    CREATE INDEX idx_program ON provides(program);\n    CREATE INDEX idx_project_dependencies ON dependencies(project);\n    CREATE INDEX idx_project_companions ON companions(project);\n    CREATE INDEX idx_alias_project ON aliases(alias);\n    ", offset: 18 }

Not my day today.

@jhheider
Copy link
Contributor

that's the second db error with the new version. looks like a fix is coming: #1095

@abitrolly
Copy link
Contributor Author

Updated pkgx to 2.1.3, still no luck.

# pkgm install vale
Error: "cmd not found: deno"
# pkgx pkgm install vale
Error: CmdNotFound("pkgm")

@jhheider
Copy link
Contributor

It's possible you're in a bad state. I'd try:

pkgx --sync
pkgx +deno@latest

@abitrolly
Copy link
Contributor Author

@jhheider

# pkgx --sync
thread 'main' panicked at crates/cli/src/args.rs:76:22:
unknown argument --sync
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

@jhheider
Copy link
Contributor

Fun. Perhaps that wasn't ported forward. Well, you can obviously find and remove pantry.2.db. hopefully just the + command will do it, though.

@abitrolly
Copy link
Contributor Author

Well, you can obviously find and remove pantry.2.db.

Not so obvious.

# fd pantry.2.db /
#

@jhheider
Copy link
Contributor

Odd. 2.1.2 and above should be creating pantry.2.db. it should be next to pantry.db (in your cache dir for your system).

Does pkgx +deno@latest work?

@abitrolly
Copy link
Contributor Author

@jhheider

# pkgx +deno@latest
Error: invalid semver

@jhheider
Copy link
Contributor

So, it's in a bad state. Completely removing the cache dir should be:

# darwin
% rm -r ~/Library/Caches/pkgx
# linux
% rm -rf "${XDG_CACHE_HOME:-$HOME/.cache}/pkgx"

@abitrolly
Copy link
Contributor Author

@jhheider so, it is in --hidden dir. That's why fd didn't get anything.

# fd pantry.2.db --hidden /
/root/.cache/pkgx/pantry.2.db
`# rm /root/.cache/pkgx/pantry.2.db
rm: remove regular file '/root/.cache/pkgx/pantry.2.db'? y

There is still an issue, but that is probably cause by network disruption.

# pkgm install python@3.11
Error: reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("dist.pkgx.dev")), port: None, path: "/pantry.tgz", query: None, fragment: None }, source: hyper::Error(Connect, Ssl(Error { code: ErrorCode(5), cause: Some(Io(Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" })) }, X509VerifyResult { code: 0, error: "ok" })) }

Got back to

# pkgm install python@3.11
Error: "cmd not found: deno"

@jhheider
Copy link
Contributor

so, on a failed fetch, it's corrupting your db. so, you have two problems.

for your HTTPS issue, what does

% curl -LI https://dist.pkgx.dev/pantry.tgz

show?

for the latter, it sounds like we need to be a little more robust on the pkgx side. i'm going to, shamefacedly, move this ticket back again.

@jhheider jhheider transferred this issue from pkgxdev/pkgm Jan 27, 2025
@abitrolly
Copy link
Contributor Author

@jhheider it just hangs indefinitely.

# curl -LIvvvv https://dist.pkgx.dev/pantry.tgz
* Host dist.pkgx.dev:443 was resolved.
* IPv6: (none)
* IPv4: 13.32.99.66, 13.32.99.69, 13.32.99.2, 13.32.99.46
*   Trying 13.32.99.66:443...
* Connected to dist.pkgx.dev (13.32.99.66) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: none

@jhheider
Copy link
Contributor

very strange.

% curl -LIvvvv https://dist.pkgx.dev/pantry.tgz
* Host dist.pkgx.dev:443 was resolved.
* IPv6: (none)
* IPv4: 18.161.34.2, 18.161.34.78, 18.161.34.64, 18.161.34.29
*   Trying 18.161.34.2:443...
* Connected to dist.pkgx.dev (18.161.34.2) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=dist.tea.xyz
*  start date: Aug 27 00:00:00 2024 GMT
*  expire date: Sep 25 23:59:59 2025 GMT
*  subjectAltName: host "dist.pkgx.dev" matched cert's "dist.pkgx.dev"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://dist.pkgx.dev/pantry.tgz
* [HTTP/2] [1] [:method: HEAD]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: dist.pkgx.dev]
* [HTTP/2] [1] [:path: /pantry.tgz]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]
> HEAD /pantry.tgz HTTP/2
> Host: dist.pkgx.dev
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/2 200
HTTP/2 200
< content-type: application/x-tar
content-type: application/x-tar
< content-length: 464397
content-length: 464397
< x-amz-id-2: eCYYhVqb8fEPdL5I/Yx7vY6PpA81Tio7jFHQqltv1DT8QyEzfxl80RxnjuR7AKBKcbCm9Ra7oM9jtg9H33NPZnICJN23XKcozxu2GReDj8U=
x-amz-id-2: eCYYhVqb8fEPdL5I/Yx7vY6PpA81Tio7jFHQqltv1DT8QyEzfxl80RxnjuR7AKBKcbCm9Ra7oM9jtg9H33NPZnICJN23XKcozxu2GReDj8U=
< x-amz-request-id: S7WD4CQYZCN9D8J2
x-amz-request-id: S7WD4CQYZCN9D8J2
< date: Mon, 27 Jan 2025 17:38:43 GMT
date: Mon, 27 Jan 2025 17:38:43 GMT
< last-modified: Mon, 27 Jan 2025 17:37:25 GMT
last-modified: Mon, 27 Jan 2025 17:37:25 GMT
< etag: "5ca1274d2202e9d861b4957c3b282d50"
etag: "5ca1274d2202e9d861b4957c3b282d50"
< x-amz-server-side-encryption: AES256
x-amz-server-side-encryption: AES256
< x-amz-version-id: .CY1PporqX0KmE0LtNGiEsC.tl6kcduq
x-amz-version-id: .CY1PporqX0KmE0LtNGiEsC.tl6kcduq
< accept-ranges: bytes
accept-ranges: bytes
< server: AmazonS3
server: AmazonS3
< x-cache: Hit from cloudfront
x-cache: Hit from cloudfront
< via: 1.1 50e6d63b846cb6b459785b607dcb07c8.cloudfront.net (CloudFront)
via: 1.1 50e6d63b846cb6b459785b607dcb07c8.cloudfront.net (CloudFront)
< x-amz-cf-pop: BOS50-P2
x-amz-cf-pop: BOS50-P2
< x-amz-cf-id: tFuqAv9TXSyX1SvRibvdhadA6VJyHAJlt2xIgDdIBg-KDf5Lzy6vrA==
x-amz-cf-id: tFuqAv9TXSyX1SvRibvdhadA6VJyHAJlt2xIgDdIBg-KDf5Lzy6vrA==
< age: 2846
age: 2846
< vary: Origin
vary: Origin
<

* Connection #0 to host dist.pkgx.dev left intact

so, you definitely have some kind of connectivity issue. and it's to AWS CloudFront, sooooo... that's unusual. if that's in your container, does it ever complete on the host?

@abitrolly
Copy link
Contributor Author

abitrolly commented Jan 27, 2025 •
edited
Loading

From the host. I think my connection is being filtered by some guys who are not very honest with themselves. This goes endlessly.

$ curl -LIvvvv https://dist.pkgx.dev/pantry.tgz
21:55:10.929082 [0-x] == Info: [READ] client_reset, clear readers
21:55:11.979654 [0-0] == Info: Host dist.pkgx.dev:443 was resolved.
21:55:11.980371 [0-0] == Info: IPv6: (none)
21:55:11.981211 [0-0] == Info: IPv4: 13.32.99.66, 13.32.99.46, 13.32.99.2, 13.32.99.69
21:55:11.981795 [0-0] == Info: [HTTPS-CONNECT] added
21:55:11.982030 [0-0] == Info: [HTTPS-CONNECT] connect, init
21:55:11.982406 [0-0] == Info: [HTTPS-CONNECT] connect, check h21
21:55:11.982859 [0-0] == Info: [HAPPY-EYEBALLS] created ipv4 (timeout 149473ms)
21:55:11.983365 [0-0] == Info: [HAPPY-EYEBALLS] ipv4 starting (timeout=149473ms)
21:55:11.983967 [0-0] == Info:   Trying 13.32.99.66:443...
21:55:11.984339 [0-0] == Info: [TCP] cf_socket_open() -> 0, fd=4
21:55:11.984907 [0-0] == Info: [TCP] local address 192.168.100.27 port 44804...
21:55:11.985350 [0-0] == Info: [HAPPY-EYEBALLS] ipv4 connect -> 0, connected=0
21:55:11.985820 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
21:55:11.986224 [0-0] == Info: [TCP] adjust_pollset, !connected, POLLOUT fd=4
21:55:11.986715 [0-0] == Info: [HAPPY-EYEBALLS] adjust_pollset -> 1 socks
21:55:11.987044 [0-0] == Info: [HTTPS-CONNECT] adjust_pollset -> 1 socks
21:55:12.182616 [0-0] == Info: [HTTPS-CONNECT] connect, check h21
21:55:12.183280 [0-0] == Info: [TCP] not connected yet
21:55:12.183655 [0-0] == Info: [HAPPY-EYEBALLS] ipv4 connect -> 0, connected=0
21:55:12.184292 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
21:55:12.184850 [0-0] == Info: [TCP] adjust_pollset, !connected, POLLOUT fd=4
21:55:12.185299 [0-0] == Info: [HAPPY-EYEBALLS] adjust_pollset -> 1 socks
21:55:12.185741 [0-0] == Info: [HTTPS-CONNECT] adjust_pollset -> 1 socks
21:55:12.994492 [0-0] == Info: [HTTPS-CONNECT] connect, check h21
21:55:12.995005 [0-0] == Info: [TCP] connected
21:55:12.995266 [0-0] == Info: [HAPPY-EYEBALLS] ipv4 connect -> 0, connected=1
21:55:12.995790 [0-0] == Info: [HAPPY-EYEBALLS] Connected to dist.pkgx.dev (13.32.99.66) port 443
21:55:12.996530 [0-0] == Info: [SSL] cf_connect()
21:55:13.010119 [0-0] == Info: [SSL] No cached session ID for https://dist.pkgx.dev:443
21:55:13.011140 [0-0] == Info: ALPN: curl offers h2,http/1.1
21:55:13.012314 [0-0] => Send SSL data, 5 bytes (0x5)
0000: .....
21:55:13.013802 [0-0] == Info: TLSv1.3 (OUT), TLS handshake, Client hello (1):
21:55:13.017649 [0-0] => Send SSL data, 512 bytes (0x200)
0000: ......d.1w...d;6...[-u.. .Z..'...S.E.c p.d."..d?...I.b..r.Zw.ODD
0040: ........<.......,.0.........+./...$.(.k.#.'.g.....9.....3.....=.
0080: <.5./...w..............dist.pkgx.dev............................
00c0: .............h2.http/1.1.........1.....0........................
0100: .........................+........-.....3.&.$... H2.s.u.Z....Q..
0140: ..!..n...L?S..@.................................................
0180: ................................................................
01c0: ................................................................
21:55:13.027747 [0-0] == Info: [TCP] send(len=517) -> 517, err=0
21:55:13.029353 [0-0] == Info: [SSL] ossl_bio_cf_out_write(len=517) -> 517, err=0
21:55:13.030223 [0-0] == Info: [TCP] recv(len=5) -> -1, err=81
21:55:13.030447 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=5) -> -1, err=81
21:55:13.030767 [0-0] == Info: [SSL] populate_x509_store, path=none, blob=0
21:55:13.060659 [0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=2
21:55:13.060889 [0-0] == Info: [SSL] SSL_connect() -> want recv
21:55:13.061109 [0-0] == Info: [SSL] cf_connect() -> 0, done=0
21:55:13.061225 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
21:55:13.061542 [0-0] == Info: [SSL] adjust_pollset, POLLIN fd=4
21:55:13.061687 [0-0] == Info: [TCP] adjust_pollset, !active, POLLIN fd=4
21:55:13.061858 [0-0] == Info: [HTTPS-CONNECT] adjust_pollset -> 1 socks
21:55:14.062538 [0-0] == Info: [HTTPS-CONNECT] connect, check h21
21:55:14.062899 [0-0] == Info: [SSL] cf_connect()
21:55:14.063153 [0-0] == Info: [TCP] recv(len=5) -> -1, err=81
21:55:14.063612 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=5) -> -1, err=81
21:55:14.064094 [0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=2
21:55:14.064522 [0-0] == Info: [SSL] SSL_connect() -> want recv
21:55:14.064905 [0-0] == Info: [SSL] cf_connect() -> 0, done=0
21:55:14.065251 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
21:55:14.065659 [0-0] == Info: [SSL] adjust_pollset, POLLIN fd=4
21:55:14.065951 [0-0] == Info: [TCP] adjust_pollset, !active, POLLIN fd=4
21:55:14.066306 [0-0] == Info: [HTTPS-CONNECT] adjust_pollset -> 1 socks
21:55:15.067323 [0-0] == Info: [HTTPS-CONNECT] connect, check h21
21:55:15.068193 [0-0] == Info: [SSL] cf_connect()
21:55:15.068556 [0-0] == Info: [TCP] recv(len=5) -> -1, err=81
21:55:15.068928 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=5) -> -1, err=81
21:55:15.069389 [0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=2
21:55:15.069812 [0-0] == Info: [SSL] SSL_connect() -> want recv
21:55:15.070252 [0-0] == Info: [SSL] cf_connect() -> 0, done=0
21:55:15.070736 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
...

@jhheider
Copy link
Contributor

well, i don't see us solving that one ;)

but the db corruption should be prevented. or recovered. basically, you shouldn't have the weird failure the second time.

@mxcl
Copy link
Member

mxcl commented Jan 30, 2025 •
edited
Loading

This bug is: we install in-place and don’t recover or record partial installations. Should be an easy fix. Basically deno is partially installed and pkgx thinks it is fully installed so it adds the env for deno and then there is no deno executable actually there.

pkgx^1 handled this by untar’ing to a tmp directory then moving it, but I never liked that hence didn’t implement it for ^2 hoping I’d come up with a better solution.

Aside

pkgx^2 doesn’t support @latest syntax. Currently. I figure we probably should add it back, but it’s a bitch to do since
it requires update checks and installation to occur before we then resolve the graph again.

@abitrolly
Copy link
Contributor Author

@mxcl thanks for the update. I don't think I need @latest right now. Might be solved at the installation time, but the installed version will still be fixed one like @3.11.

@Dranero
Copy link

Dranero commented Feb 6, 2025

Just for reference: This bug also happend when I tried to install the huggingface-cli into the latest pkgx docker container. Switching the docker tag to v1 fixed it for me. Thanks for the sidenote about pkgx v1. That was the missing piece for me 😀

@abitrolly
Copy link
Contributor Author

pkgm works, and for pkgx

# curl https://pkgx.sh | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4805  100  4805    0     0   7715      0 --:--:-- --:--:-- --:--:--  7712
installing: /usr/local/bin/pkgx
######################################################################## 100.0%
pkgx 2.2.0
ensure you have the `pkgx` pre-requisites installed:
   yum --assumeyes install libatomic
now type: pkgx --help
[root@3c81bdce25d3 workdir]# pkgx install python@3.11
install: missing destination file operand after 'python@3.11'
Try '/root/.pkgx/gnu.org/coreutils/v9.6.0/bin/install --help' for more information.
# pkgm install python@3.11
[root@3c81bdce25d3 workdir]# python
Python 3.11.11 (main, Dec  3 2024, 18:52:07) [Clang 19.1.4 ] on linux

mxcl added a commit that referenced this issue Feb 8, 2025
@mxcl
Install to tmpdir then atomic move
c8a923a 
Refs #1100
@mxcl mxcl mentioned this issue Feb 8, 2025
Merged
mxcl added a commit that referenced this issue Feb 8, 2025
@mxcl
Install to tmpdir then atomic move
84c65c4 
Refs #1100
mxcl added a commit that referenced this issue Feb 8, 2025
@mxcl
Install to tmpdir then atomic move
7b5e998 
Refs #1100
mxcl added a commit that referenced this issue Feb 8, 2025
@mxcl
Install to tmpdir then atomic move
1fd2a4a 
Refs #1100
mxcl added a commit that referenced this issue Feb 8, 2025
@mxcl
Install to tmpdir then atomic move
2c7ad18 
Refs #1100
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mxcl @abitrolly @jhheider @Dranero