Fix races during parallel downloads #2903
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nirs
changed the title
|
Tested using a script creating 10 instances in parallel. Test scripts and logs: |
|
Another test, creating and starting 3 instances in parallel: |
We tested that modified last modified time on the server cause a redownload, but we did not test that after a redownload we update the cache, so the next attempt will used the cached download. Add a failing test verifying the issue, and improve test comments and configuration to make it more clear. Part-of: lima-vm#2902 Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Extract getCache() to handle getting and file from the cache, and fetch() for fetching a file from the remote to the cache. This will allow locking around the code checking and updating the cache, and much easier to maintain and understand. Signed-off-by: Nir Soffer <nsoffer@redhat.com>
To solve the races during concurrent downloads and avoid unneeded work and bandwidth, we allow one concurrent download of the same image. When a limactl process try to access the cache, it takes a lock on the file cache directory. If multiple processes try to get the lock in the same time, only one will take the lock, and the other will block. The process that took the lock tries to get the file from the cache. This is the fast path and the common case. This can fail if the file is not in the cache, the digest does not match, or the cached last modified time does not match the last modified returned from the server. If the process cannot get the file from the cache, it downloads the file from the remote server, and update the cached data and metadata files. Finally the process release the lock on the cache directory. Other limactl processes waiting on the lock wake up and take the lock. In the common case they will find the image in the cache and will release the lock quickly. Since we have exactly one concurrent download, updating the metadata files is trivial and we don't need the writeFirst() helper. Fixes: lima-vm#2902 Fixes: lima-vm#2732 Signed-off-by: Nir Soffer <nsoffer@redhat.com>
We can assert now that only one thread downloaded the file, and all other threads used the cache. Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Checking if an image is cached races with parallel downloads. Take the lock when validating the digest or the data file to ensure that we validate the cached when it is in consistent state. If an image is being downloaded, the check will block until the download completes. Signed-off-by: Nir Soffer <nsoffer@redhat.com>
tmeijn
pushed a commit
to tmeijn/dotfiles
that referenced
this pull request
Dec 13, 2024
⚠️ **CAUTION: this is a major update, indicating a breaking change!**⚠️ This MR contains the following updates: | Package | Update | Change | |---|---|---| | [lima-vm/lima](https://github.com/lima-vm/lima) | major | `v0.23.2` -> `v1.0.2` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>lima-vm/lima (lima-vm/lima)</summary> ### [`v1.0.2`](https://github.com/lima-vm/lima/releases/tag/v1.0.2) [Compare Source](lima-vm/lima@v1.0.1...v1.0.2) #### Changes - DNS: - Fixed the host resolver regression in v1.0.0 [#​2939](lima-vm/lima#2939) ([#​2964](lima-vm/lima#2964)) - `limactl create`: - Fixed races during parallel downloads ([#​2903](lima-vm/lima#2903), thanks to [@​nirs](https://github.com/nirs)) - Optimized qcow2-to-raw conversion for vz mode ([#​2933](lima-vm/lima#2933), thanks to [@​nirs](https://github.com/nirs)) - `limactl start-at-login`: - Fixed the support for Linux hosts (systemd) ([#​2943](lima-vm/lima#2943), thanks to [@​kachick](https://github.com/kachick)) - nerdctl: - Updated to [v2.0.1](https://github.com/containerd/nerdctl/releases/tag/v2.0.1) ([#​2966](lima-vm/lima#2966)) - Templates: - Updated to the latest revisions ([#​2936](lima-vm/lima#2936) [#​2953](lima-vm/lima#2953), thanks to [@​tcooper](https://github.com/tcooper)) - Web site: - Added an example of running Lima on GitHub Actions to run commands on non-Ubuntu ([#​2954](lima-vm/lima#2954)): https://lima-vm.io/docs/examples/gha/ - Project: - Invite Nir Soffer ([@​nirs](https://github.com/nirs)) as a Reviewer ([#​2916](lima-vm/lima#2916), thanks to [@​jandubois](https://github.com/jandubois)) Full changes: https://github.com/lima-vm/lima/milestone/51?closed=1 Thanks to [@​SpiffyEight77](https://github.com/SpiffyEight77) [@​alexandear](https://github.com/alexandear) [@​jandubois](https://github.com/jandubois) [@​kachick](https://github.com/kachick) [@​nirs](https://github.com/nirs) [@​norio-nomura](https://github.com/norio-nomura) [@​tamird](https://github.com/tamird) [@​tcooper](https://github.com/tcooper) #### Usage ```console [macOS]$ limactl create [macOS]$ limactl start ... INFO[0029] READY. Run `lima` to open the shell. [macOS]$ lima uname Linux ``` *** The binaries were built automatically on GitHub Actions. The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/12134682585 The sha256sum of the SHA256SUMS file itself is `02ef78494c498ca4180915ba78d5e2fc471ed401f63dfb2b5864c3711f3c0fb2` . *** Release manager: [@​AkihiroSuda](https://github.com/AkihiroSuda) ### [`v1.0.1`](https://github.com/lima-vm/lima/releases/tag/v1.0.1) [Compare Source](lima-vm/lima@v1.0.0...v1.0.1) Reverted the default port forwarder from gRPC to SSH for the stability reason ([#​2864](lima-vm/lima#2864)). This reversion fixes several regressions related to `docker run -p` in Lima v1.0.0 ([#​2859](lima-vm/lima#2859)). Although the gRPC forwarder is faster and has an advanced feature (UDP support), it turned out to be still immature. Set `LIMA_SSH_PORT_FORWARDER=false` to opt-in to the gRPC forwarder. See <https://lima-vm.io/docs/config/port/>. Full changes: https://github.com/lima-vm/lima/milestone/50?closed=1 Thanks to [@​alexandear](https://github.com/alexandear) [@​jandubois](https://github.com/jandubois) [@​norio-nomura](https://github.com/norio-nomura) #### Usage ```console [macOS]$ limactl create [macOS]$ limactl start ... INFO[0029] READY. Run `lima` to open the shell. [macOS]$ lima uname Linux ``` *** The binaries were built automatically on GitHub Actions. The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/11735352652 The sha256sum of the SHA256SUMS file itself is `f5c12d003e25dc46291803a8acae9e9d325a45eca0c1f9f40bd6852ec8ed9be1` . *** Release manager: [@​AkihiroSuda](https://github.com/AkihiroSuda) ### [`v1.0.0`](https://github.com/lima-vm/lima/releases/tag/v1.0.0) [Compare Source](lima-vm/lima@v0.23.2...v1.0.0) With the support from 110+ contributors in 3+ years, the Lima project has finally reached v1.0. 🎉 This release introduces several breaking changes, such as switching the default machine driver from QEMU to VZ for better filesystem performance. The `limactl` CLI is designed to print hints when the user hits those breaking changes. e.g., `limactl create template://experimental/vz` now fails with a hint that suggests using `limactl create --vm-type=vz template://default` instead. 🔴 = Major breaking changes 🟡 = Minor breaking changes - VZ: - Graduate VZ machine driver from experimental ([#​2758](lima-vm/lima#2758)) - 🔴 Use VZ by default for new instances on macOS >= 13.5 ([#​1951](lima-vm/lima#1951)) - Support nested virtualization on M3 ([#​2530](lima-vm/lima#2530), thanks to [@​abiosoft](https://github.com/abiosoft)) - Optimize qcow2-to-raw image conversion (lima-vm/go-qcow2reader@v0.1.2...v0.4.0 , thanks to [@​nirs](https://github.com/nirs)) - Support specifying a custom kernel ([#​2562](lima-vm/lima#2562), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - QEMU: - Graduate 9p mount driver from experimental ([#​2758](lima-vm/lima#2758)) - 🔴 Use 9p by default for most templates ([#​1953](lima-vm/lima#1953), [#​2822](lima-vm/lima#2822)) - riscv64: switch from u-boot to EDK2 ([#​2592](lima-vm/lima#2592)) - Network: - Graduate user-v2 network driver from experimental ([#​2758](lima-vm/lima#2758)) - Support UDP port forwarding ([#​2411](lima-vm/lima#2411), thanks to [@​balajiv113](https://github.com/balajiv113)) - 🔴 Strictly require `socket_vmnet` binary to be owned by root ([#​2734](lima-vm/lima#2734)) - SSH: - 🟡 Disable `ssh.loadDotSSHPubKeys` by default ([#​2706](lima-vm/lima#2706)) - YAML: - Support generating jsonschema ([#​2306](lima-vm/lima#2306), thanks to [@​afbjorklund](https://github.com/afbjorklund)) - Support specifying `param` for provisioning scripts ([#​2570](lima-vm/lima#2570), thanks to [@​jandubois](https://github.com/jandubois)) - Support specifying `minimumLimaVersion` and `vmOpts.qemu.minimumVersion` ([#​2659](lima-vm/lima#2659), thanks to [@​jandubois](https://github.com/jandubois)) - Support template expansion in mounts ([#​2588](lima-vm/lima#2588), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - `limactl` CLI: - Add `limactl tunnel` command so as to allow the host to join the guest network ([#​2710](lima-vm/lima#2710)) - Add `--log-format=json` ([#​2584](lima-vm/lima#2584), thanks to [@​nirs](https://github.com/nirs)) - `limactl prune`: Add `--keep-referred` ([#​2569](lima-vm/lima#2569), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - nerdctl: - Updated to [v2.0.0](https://github.com/containerd/nerdctl/releases/tag/v2.0.0) ([#​2178](lima-vm/lima#2178)) - rootless: allocate 1G subuids from 524288 (0x80000) for new users ([#​2725](lima-vm/lima#2725)) - Templates: - 🔴 `experimental/vz`: Merged into the `default` template ([#​2730](lima-vm/lima#2730), [#​2736](lima-vm/lima#2736)) - 🟡 `experimental/{riscv64, armv7l}`: Merged into the `default` template ([#​2730](lima-vm/lima#2730), [#​2736](lima-vm/lima#2736)) - 🔴 `vmnet`: Removed in favor of `limactl create --network=lima:shared template://default` ([#​2736](lima-vm/lima#2736)) - 🟡 `experimental/net-user-v2`: Removed in favor of `limactl create --network=lima:user-v2 template://default` ([#​2736](lima-vm/lima#2736)) - 🔴 `experimental/9p`: Removed in favor of `limactl create --mount-type=9p template://default` ([#​2736](lima-vm/lima#2736)) - 🟡 `experimental/virtiofs-linux`: Removed in favor of `limactl create --mount-type=virtiofs template://default` ([#​2736](lima-vm/lima#2736)) - 🔴 `alpine`: Renamed to `alpine-iso` ([#​2704](lima-vm/lima#2704)) - 🔴 `alpine-image`: Renamed to `alpine` ([#​2704](lima-vm/lima#2704)) - `archlinux`: Demoted from Tier 1 to Tier 2 ([#​2717](lima-vm/lima#2717), [#​2823](lima-vm/lima#2823)) - `default`, `ubuntu`, ...: Updated to Ubuntu 24.10. The older versions are available as `ubuntu-20.04`, `ubuntu-22.04`, and `ubuntu-24.04` ([#​2755](lima-vm/lima#2755), [#​2795](lima-vm/lima#2795)) - `fedora`: Updated to Fedora 41 ([#​2821](lima-vm/lima#2821), [#​2822](lima-vm/lima#2822), thanks to [@​subpop](https://github.com/subpop)) - `opensuse`: Renamed to `opensuse-leap`. Still aliased as `opensuse` ([#​2612](lima-vm/lima#2612), thanks to [@​afbjorklund](https://github.com/afbjorklund)) - `experimental/opensuse-tumbleweed`: Support aarch64 ([#​2613](lima-vm/lima#2613), thanks to [@​afbjorklund](https://github.com/afbjorklund)) - `hack/update-template.sh` is added for automating updates ([#​1347](lima-vm/lima#1347), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - Project: - Invite Norio Nomura ([@​norio-nomura](https://github.com/norio-nomura)) as a Reviewer ([#​2567](lima-vm/lima#2567)) Full changes: https://github.com/lima-vm/lima/milestone/47?closed=1 Thanks to [@​AdamKorcz](https://github.com/AdamKorcz) [@​Mr-Sunglasses](https://github.com/Mr-Sunglasses) [@​SmartManoj](https://github.com/SmartManoj) [@​YorikSar](https://github.com/YorikSar) [@​abiosoft](https://github.com/abiosoft) [@​afbjorklund](https://github.com/afbjorklund) [@​alexandear](https://github.com/alexandear) [@​balajiv113](https://github.com/balajiv113) [@​hasan4791](https://github.com/hasan4791) [@​jandubois](https://github.com/jandubois) [@​nirs](https://github.com/nirs) [@​norio-nomura](https://github.com/norio-nomura) [@​pvdvreede](https://github.com/pvdvreede) [@​subpop](https://github.com/subpop) [@​tsukasaI](https://github.com/tsukasaI) #### Usage ```console [macOS]$ limactl create [macOS]$ limactl start ... INFO[0029] READY. Run `lima` to open the shell. [macOS]$ lima uname Linux ``` *** The binaries were built automatically on GitHub Actions. The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/11695321667 The sha256sum of the SHA256SUMS file itself is `4bd200a163111fe78c6f3e6de405113d416053802fe1507597f9a42f89a98c90` . *** Release manager: [@​AkihiroSuda](https://github.com/AkihiroSuda) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix races during parallel downloads by holding a per file lock while checking or updating the cache.
Fixes #2902, #2732