Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only key by Cargo.toml and Cargo.lock files of workspace members #180

Merged
merged 2 commits into from
Dec 3, 2023

Conversation

max-heller
Copy link
Contributor

@max-heller max-heller commented Nov 25, 2023

Currently, we glob for all Cargo.toml and Cargo.lock files anywhere in a workspace, but this can pick up false positives (see #179). This PR switches to only using Cargo.toml files of workspace members (determined using cargo metadata --no-deps) and per-workspace Cargo.lock files.

Closes #179

Copy link
Owner

@Swatinem Swatinem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks good, in theory.
however previously we had some problems with cargo metadata building everything within the workspace eagerly (on cache "restore"), which negated the whole purpose of the cache.

I’m wondering if the --no-deps flag might change that behavior or not, or if a more uptodate version of cargo does not suffer from this. But I am very reluctant to try and find out. Let me run a couple of tests for this, as the problem with cargo metadata was reproducible on https://github.com/getsentry/relay, and I can try with that on monday.

src/config.ts Outdated Show resolved Hide resolved
@max-heller
Copy link
Contributor Author

however previously we had some problems with cargo metadata building everything within the workspace eagerly (on cache "restore"), which negated the whole purpose of the cache.

I’m wondering if the --no-deps flag might change that behavior or not

My guess is that it wouldn't be building anything, but downloading dependencies from the registry to analyze their manifests. The documentation for --no-deps says it "[doesn't] fetch dependencies", which sounds like what we want.

@max-heller
Copy link
Contributor Author

the problem with cargo metadata was reproducible on https://github.com/getsentry/relay, and I can try with that on monday

Did you get a chance to test this?

max-heller added a commit to max-heller/mdbook-pandoc that referenced this pull request Dec 1, 2023
The `rust-cache` action in the release workflow currently breaks because this repo's submodules contain a lot of `Cargo.toml` and `Cargo.lock` files. This will be fixed by Swatinem/rust-cache#180, but for now this PR disables cloning submodules during the release workflow.
@Swatinem
Copy link
Owner

Swatinem commented Dec 3, 2023

Sorry for the delay, I wasn’t able to reproduce the original problem with cargo metadata locally. Even back then it wasn’t so easy to reproduce I remember. Maybe in the meantime things might have changed. I will merge this and then do some tests in CI.

@Swatinem Swatinem merged commit d30f114 into Swatinem:master Dec 3, 2023
22 checks passed
@max-heller max-heller deleted the cargo-manifests branch December 3, 2023 11:01
max-heller added a commit to max-heller/mdbook-pandoc that referenced this pull request Jan 11, 2024
This includes Swatinem/rust-cache#180 and allows using the unmodified `cargo-dist` release workflow
github-merge-queue bot referenced this pull request in knope-dev/knope Jan 12, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [Swatinem/rust-cache](https://togithub.com/Swatinem/rust-cache) |
minor | `v2` -> `v2.7.2` |

---

### Release Notes

<details>
<summary>Swatinem/rust-cache (Swatinem/rust-cache)</summary>

###
[`v2.7.2`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.7.2)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.7.0...v2.7.2)

#### What's Changed

- Update action runtime to `node20` by
[@&#8203;rhysd](https://togithub.com/rhysd) in
[https://github.com/Swatinem/rust-cache/pull/175](https://togithub.com/Swatinem/rust-cache/pull/175)
- Only key by `Cargo.toml` and `Cargo.lock` files of workspace members
by [@&#8203;max-heller](https://togithub.com/max-heller) in
[https://github.com/Swatinem/rust-cache/pull/180](https://togithub.com/Swatinem/rust-cache/pull/180)

#### New Contributors

- [@&#8203;rhysd](https://togithub.com/rhysd) made their first
contribution in
[https://github.com/Swatinem/rust-cache/pull/175](https://togithub.com/Swatinem/rust-cache/pull/175)
- [@&#8203;max-heller](https://togithub.com/max-heller) made their first
contribution in
[https://github.com/Swatinem/rust-cache/pull/180](https://togithub.com/Swatinem/rust-cache/pull/180)

**Full Changelog**:
Swatinem/rust-cache@v2.7.1...v2.7.2

###
[`v2.7.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.7.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.2...v2.7.0)

##### What's Changed

- Fix save-if documentation in readme by
[@&#8203;rukai](https://togithub.com/rukai) in
[https://github.com/Swatinem/rust-cache/pull/166](https://togithub.com/Swatinem/rust-cache/pull/166)
- Support for `trybuild` and similar macro testing tools by
[@&#8203;neysofu](https://togithub.com/neysofu) in
[https://github.com/Swatinem/rust-cache/pull/168](https://togithub.com/Swatinem/rust-cache/pull/168)

##### New Contributors

- [@&#8203;rukai](https://togithub.com/rukai) made their first
contribution in
[https://github.com/Swatinem/rust-cache/pull/166](https://togithub.com/Swatinem/rust-cache/pull/166)
- [@&#8203;neysofu](https://togithub.com/neysofu) made their first
contribution in
[https://github.com/Swatinem/rust-cache/pull/168](https://togithub.com/Swatinem/rust-cache/pull/168)

**Full Changelog**:
Swatinem/rust-cache@v2.6.2...v2.7.0

###
[`v2.6.2`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.2)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.1...v2.6.2)

##### What's Changed

- dep: Use `smol-toml` instead of `toml` by
[@&#8203;NobodyXu](https://togithub.com/NobodyXu) in
[https://github.com/Swatinem/rust-cache/pull/164](https://togithub.com/Swatinem/rust-cache/pull/164)

**Full Changelog**:
Swatinem/rust-cache@v2...v2.6.2

###
[`v2.6.1`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.0...v2.6.1)

-   Fix hash contributions of `Cargo.lock`/`Cargo.toml` files.

###
[`v2.6.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.5.1...v2.6.0)

##### What's Changed

- Add "buildjet" as a second `cache-provider` backend
[@&#8203;joroshiba](https://togithub.com/joroshiba) in
[https://github.com/Swatinem/rust-cache/pull/154](https://togithub.com/Swatinem/rust-cache/pull/154)
-   Clean up sparse registry index.
-   Do not clean up src of `-sys` crates.
-   Remove `.cargo/credentials.toml` before saving.

##### New Contributors

- [@&#8203;joroshiba](https://togithub.com/joroshiba) made their first
contribution in
[https://github.com/Swatinem/rust-cache/pull/154](https://togithub.com/Swatinem/rust-cache/pull/154)

**Full Changelog**:
Swatinem/rust-cache@v2.5.1...v2.6.0

###
[`v2.5.1`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.5.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.5.0...v2.5.1)

-   Fix hash contribution of `Cargo.lock`.

###
[`v2.5.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.5.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.4.0...v2.5.0)

##### What's Changed

- feat: Rm workspace crates version before caching by
[@&#8203;NobodyXu](https://togithub.com/NobodyXu) in
[https://github.com/Swatinem/rust-cache/pull/147](https://togithub.com/Swatinem/rust-cache/pull/147)
- feat: Add hash of `.cargo/config.toml` to key by
[@&#8203;NobodyXu](https://togithub.com/NobodyXu) in
[https://github.com/Swatinem/rust-cache/pull/149](https://togithub.com/Swatinem/rust-cache/pull/149)

##### New Contributors

- [@&#8203;NobodyXu](https://togithub.com/NobodyXu) made their first
contribution in
[https://github.com/Swatinem/rust-cache/pull/147](https://togithub.com/Swatinem/rust-cache/pull/147)

**Full Changelog**:
Swatinem/rust-cache@v2.4.0...v2.5.0

###
[`v2.4.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.4.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.3.0...v2.4.0)

-   Fix cache key stability.
- Use 8 character hash components to reduce the key length, making it
more readable.

###
[`v2.3.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.3.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.2.1...v2.3.0)

- Add `cache-all-crates` option, which enables caching of crates
installed by workflows.
- Add installed packages to cache key, so changes to workflows that
install rust tools are detected and cached properly.
-   Fix cache restore failures due to upstream bug.
-   Fix `EISDIR` error due to globed directories.
- Update runtime `@actions/cache`, `@actions/io` and dev `typescript`
dependencies.
- Update `npm run prepare` so it creates distribution files with the
right line endings.

###
[`v2.2.1`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.2.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.2.0...v2.2.1)

- Update `@actions/cache` dependency to fix usage of `zstd` compression.

###
[`v2.2.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.2.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.1.0...v2.2.0)

- Add new `save-if` option to always restore, but only conditionally
save the cache.

###
[`v2.1.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.1.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.0.2...v2.1.0)

- Only hash `Cargo.{lock,toml}` files in the configured workspace
directories.

###
[`v2.0.2`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.0.2)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.0.1...v2.0.2)

-   Avoid calling cargo metadata on pre-cleanup.
-   Added `prefix-key`, `cache-directories` and `cache-targets` options.

###
[`v2.0.1`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.0.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2...v2.0.1)

- Primarily just updating dependencies to fix GitHub deprecation
notices.

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/knope-dev/knope).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjcuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
zharinov referenced this pull request in zharinov/verver Feb 28, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [Swatinem/rust-cache](https://togithub.com/Swatinem/rust-cache) |
action | minor | `v2.5.1` -> `v2.7.3` |

---

### Release Notes

<details>
<summary>Swatinem/rust-cache (Swatinem/rust-cache)</summary>

###
[`v2.7.3`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.7.3)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.7.2...v2.7.3)

- Work around upstream problem that causes cache saving to hang for
minutes.

**Full Changelog**:
Swatinem/rust-cache@v2.7.2...v2.7.3

###
[`v2.7.2`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.7.2)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.7.1...v2.7.2)

##### What's Changed

- Update action runtime to `node20` by
[@&#8203;rhysd](https://togithub.com/rhysd) in
[https://github.com/Swatinem/rust-cache/pull/175](https://togithub.com/Swatinem/rust-cache/pull/175)
- Only key by `Cargo.toml` and `Cargo.lock` files of workspace members
by [@&#8203;max-heller](https://togithub.com/max-heller) in
[https://github.com/Swatinem/rust-cache/pull/180](https://togithub.com/Swatinem/rust-cache/pull/180)

##### New Contributors

- [@&#8203;rhysd](https://togithub.com/rhysd) made their first
contribution in
[https://github.com/Swatinem/rust-cache/pull/175](https://togithub.com/Swatinem/rust-cache/pull/175)
- [@&#8203;max-heller](https://togithub.com/max-heller) made their first
contribution in
[https://github.com/Swatinem/rust-cache/pull/180](https://togithub.com/Swatinem/rust-cache/pull/180)

**Full Changelog**:
Swatinem/rust-cache@v2.7.1...v2.7.2

###
[`v2.7.1`](https://togithub.com/Swatinem/rust-cache/compare/v2.7.0...v2.7.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.7.0...v2.7.1)

###
[`v2.7.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.7.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.2...v2.7.0)

##### What's Changed

- Fix save-if documentation in readme by
[@&#8203;rukai](https://togithub.com/rukai) in
[https://github.com/Swatinem/rust-cache/pull/166](https://togithub.com/Swatinem/rust-cache/pull/166)
- Support for `trybuild` and similar macro testing tools by
[@&#8203;neysofu](https://togithub.com/neysofu) in
[https://github.com/Swatinem/rust-cache/pull/168](https://togithub.com/Swatinem/rust-cache/pull/168)

##### New Contributors

- [@&#8203;rukai](https://togithub.com/rukai) made their first
contribution in
[https://github.com/Swatinem/rust-cache/pull/166](https://togithub.com/Swatinem/rust-cache/pull/166)
- [@&#8203;neysofu](https://togithub.com/neysofu) made their first
contribution in
[https://github.com/Swatinem/rust-cache/pull/168](https://togithub.com/Swatinem/rust-cache/pull/168)

**Full Changelog**:
Swatinem/rust-cache@v2.6.2...v2.7.0

###
[`v2.6.2`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.2)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.1...v2.6.2)

##### What's Changed

- dep: Use `smol-toml` instead of `toml` by
[@&#8203;NobodyXu](https://togithub.com/NobodyXu) in
[https://github.com/Swatinem/rust-cache/pull/164](https://togithub.com/Swatinem/rust-cache/pull/164)

**Full Changelog**:
Swatinem/rust-cache@v2...v2.6.2

###
[`v2.6.1`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.0...v2.6.1)

-   Fix hash contributions of `Cargo.lock`/`Cargo.toml` files.

###
[`v2.6.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.5.1...v2.6.0)

##### What's Changed

- Add "buildjet" as a second `cache-provider` backend
[@&#8203;joroshiba](https://togithub.com/joroshiba) in
[https://github.com/Swatinem/rust-cache/pull/154](https://togithub.com/Swatinem/rust-cache/pull/154)
-   Clean up sparse registry index.
-   Do not clean up src of `-sys` crates.
-   Remove `.cargo/credentials.toml` before saving.

##### New Contributors

- [@&#8203;joroshiba](https://togithub.com/joroshiba) made their first
contribution in
[https://github.com/Swatinem/rust-cache/pull/154](https://togithub.com/Swatinem/rust-cache/pull/154)

**Full Changelog**:
Swatinem/rust-cache@v2.5.1...v2.6.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/zharinov/verver).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi4yNy4xIiwidXBkYXRlZEluVmVyIjoiMzcuMTI3LjAiLCJ0YXJnZXRCcmFuY2giOiJtYWluIn0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Post job cleanup errors with "argument list too long" in repository with many Cargo.toml/Cargo.lock files
2 participants