Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: proper archspec detection using archspec-rs #584

Merged
merged 1 commit into from
Mar 30, 2024

Conversation

baszalmstra
Copy link
Collaborator

@baszalmstra baszalmstra commented Mar 30, 2024

Implements proper archspec detection using https://github.com/prefix-dev/archspec-rs which is a port of https://github.com/archspec/archspec used by conda. This opens the doors to build and use architecture specific packages.

We loose the __archspec virtual package for wasm targets because the archspec standard does not describe these targets.

I set the default archspec for osx-arm64 to m1 because I think thats greatest common denominator for arm based chips on mac.

I also added a subcommand to rattler-bin to print the detected virtual packages.

On my windows machine:

> cargo run -- virtual-packages
__win=0=0
__cuda=12.3=0
__archspec=1=skylake

Under WSL:

> cargo run -- virtual-packages
__unix=0=0
__linux=5.15.146.1=0
__glibc=2.35=0
__cuda=12.3=0
__archspec=1=skylake

@wolfv Would you be able to run this on you mac and report the output?
@ruben-arts Can you try on your machines as well?

You can verify that this is the same as conda by creating an environment with conda and archspec and running conda info.

@baszalmstra baszalmstra requested a review from wolfv March 30, 2024 08:09
@baszalmstra
Copy link
Collaborator Author

baszalmstra commented Mar 30, 2024

On my Windows AMD machine (AMD Ryzen 9 5950X "zen3"):

> cargo run -- virtual-packages
__win=0=0
__archspec=1=x86_64_v3

WSL under windows AMD:

> cargo run -- virtual-packages
__unix=0=0
__linux=5.15.146.1=0
__glibc=2.35=0
__archspec=1=zen2

Interestingly WSL and normal windows report a different archspec (zen2 vs x86_64_v3), but conda info reports the same thing on this machine. Either way, it should be zen3 I think.. I guess this is an issue with the archspec-json files.

@ruben-arts
Copy link
Collaborator

ruben-arts commented Mar 30, 2024

On my Fedora Linux 37 Workstation, (CPU: 12th Gen Intel i7-12800H (20)) :

> cargo run -- virtual-packages
__unix=0=0
__linux=6.5.12=0
__glibc=2.36=0
__cuda=12.3=0
__archspec=1=skylake

On my windows machine (AMD Ryzen 7 7840HS) :

> pixi run cargo run -- virtual-packages
__win=0=0
__archspec=1=x86_64_v4

Same as conda 👍 except that conda also includes the conda version as a virtual package e.g.:

__conda=24.3.0=0

@wolfv
Copy link
Contributor

wolfv commented Mar 30, 2024

This is what I get :)

__unix=0=0
__osx=14.4=0
__archspec=1=m2

And with conda:

       virtual packages : __archspec=1=m2
                          __conda=24.1.2=0
                          __osx=14.4=0
                          __unix=0=0

@baszalmstra baszalmstra marked this pull request as ready for review March 30, 2024 12:35
@baszalmstra
Copy link
Collaborator Author

Nice! Then Ill mark this ready for review!

Copy link
Contributor

@wolfv wolfv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, just one question regarding Windows ARM64

crates/rattler_virtual_packages/src/lib.rs Show resolved Hide resolved
@baszalmstra baszalmstra merged commit 056f08b into conda:main Mar 30, 2024
16 checks passed
@baszalmstra baszalmstra mentioned this pull request Mar 30, 2024
@chawyehsu
Copy link

What I got on my Windows on ARM machine

__win=0=0
__archspec=1=aarch64
The repo is big with git-lfs files (the test-data) and took me a hour to finalize the clone to build and run it. I have no idea why, I'm probably encountering issues of git-lfs on Win on ARM.

baszalmstra added a commit that referenced this pull request Apr 2, 2024
## 🤖 New release
* `rattler`: 0.19.5 -> 0.19.6 (✓ API compatible changes)
* `rattler_conda_types`: 0.20.3 -> 0.20.4 (✓ API compatible changes)
* `rattler_lock`: 0.21.0 -> 0.22.0 (⚠️ API breaking changes)
* `rattler_virtual_packages`: 0.19.4 -> 0.19.5 (✓ API compatible
changes)
* `rattler_package_streaming`: 0.20.1 -> 0.20.2
* `rattler_repodata_gateway`: 0.19.4 -> 0.19.5
* `rattler_solve`: 0.20.3 -> 0.20.4
* `rattler_index`: 0.19.4 -> 0.19.5
* `rattler_shell`: 0.19.4 -> 0.19.5

### ⚠️ `rattler_lock` breaking changes

```
--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.30.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field PypiPackageData.editable in /tmp/.tmpcIdzpM/rattler/crates/rattler_lock/src/pypi.rs:40
```

<details><summary><i><b>Changelog</b></i></summary><p>

## `rattler`
<blockquote>

##
[0.19.6](rattler-v0.19.5...rattler-v0.19.6)
- 2024-03-30

### Other
- remove unused dependencies
([#585](#585))
</blockquote>

## `rattler_conda_types`
<blockquote>

##
[0.20.4](rattler_conda_types-v0.20.3...rattler_conda_types-v0.20.4)
- 2024-03-30

### Fixed
- matchspec empty namespace and channel cannonical name
([#582](#582))
</blockquote>

## `rattler_lock`
<blockquote>

##
[0.22.0](rattler_lock-v0.21.0...rattler_lock-v0.22.0)
- 2024-03-30

### Added
- editable pypi packages
([#581](#581))
</blockquote>

## `rattler_virtual_packages`
<blockquote>

##
[0.19.5](rattler_virtual_packages-v0.19.4...rattler_virtual_packages-v0.19.5)
- 2024-03-30

### Added
- proper archspec detection using archspec-rs
([#584](#584))
</blockquote>

## `rattler_package_streaming`
<blockquote>

##
[0.20.2](rattler_package_streaming-v0.20.1...rattler_package_streaming-v0.20.2)
- 2024-03-30

### Other
- updated the following local packages: rattler_conda_types
</blockquote>

## `rattler_repodata_gateway`
<blockquote>

##
[0.19.5](rattler_repodata_gateway-v0.19.4...rattler_repodata_gateway-v0.19.5)
- 2024-03-30

### Other
- updated the following local packages: rattler_conda_types
</blockquote>

## `rattler_solve`
<blockquote>

##
[0.20.4](rattler_solve-v0.20.3...rattler_solve-v0.20.4)
- 2024-03-30

### Other
- updated the following local packages: rattler_conda_types
</blockquote>

## `rattler_index`
<blockquote>

##
[0.19.5](rattler_index-v0.19.4...rattler_index-v0.19.5)
- 2024-03-30

### Other
- updated the following local packages: rattler_conda_types
</blockquote>

## `rattler_shell`
<blockquote>

##
[0.19.5](rattler_shell-v0.19.4...rattler_shell-v0.19.5)
- 2024-03-30

### Other
- updated the following local packages: rattler_conda_types
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/MarcoIeni/release-plz/).

Signed-off-by: Bas <4995967+baszalmstra@users.noreply.github.com>
Co-authored-by: Bas <4995967+baszalmstra@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants