Add --log-format json option #2584
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nirs
added a commit
to nirs/ramen
that referenced
this pull request
Sep 2, 2024
Testing lima new --log-format json option[1]. This allows cleaning up
drenv log by dropping the redundant "time" and "level", and avoiding
extra quoting inside other logging fields.
If we get additional fields (e.g. when downloading images) we log them
as a separate dict (similar to ramen log).
Example run:
% drenv start envs/vm.yaml -v
2024-09-03 00:40:17,217 INFO [vm] Starting environment
2024-09-03 00:40:17,226 DEBUG [main] Add executor profiles
2024-09-03 00:40:17,230 DEBUG [cluster] Running ['/Users/nsoffer/src/lima/_output/bin/limactl', 'list', '--format', '{{.Name}}', '--log-format', 'json']
2024-09-03 00:40:17,246 INFO [cluster] Starting lima cluster
2024-09-03 00:40:17,246 DEBUG [cluster] Running ['/Users/nsoffer/src/lima/_output/bin/limactl', 'list', '--format', '{{.Name}}', '--log-format', 'json']
2024-09-03 00:40:17,268 DEBUG [cluster] Running ['/Users/nsoffer/src/lima/_output/bin/limactl', 'create', '--name', 'cluster', '/var/folders/qs/_z10qgxn3gsfw8vmkr9b3s4w0000gn/T/drenv.lima.cluster.tmpbtq8413s', '--log-format', 'json']
2024-09-03 00:40:17,284 DEBUG [cluster] Terminal is not available, proceeding without opening an editor
2024-09-03 00:40:17,299 DEBUG [cluster] Attempting to download the image {'arch': 'aarch64', 'digest': '', 'location': 'https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-arm64.img'}
2024-09-03 00:40:28,724 DEBUG [cluster] Downloaded the image from "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-arm64.img"
2024-09-03 00:40:28,725 DEBUG [cluster] Converting "/Users/nsoffer/.lima/cluster/basedisk" (qcow2) to a raw disk "/Users/nsoffer/.lima/cluster/diffdisk"
2024-09-03 00:40:46,819 DEBUG [cluster] Expanding to 20GiB
2024-09-03 00:40:47,064 DEBUG [cluster] Attempting to download the nerdctl archive {'arch': 'aarch64', 'digest': 'sha256:77c747f09853ee3d229d77e8de0dd3c85622537d82be57433dc1fca4493bab95', 'location': 'https://github.com/containerd/nerdctl/releases/download/v1.7.6/nerdctl-full-1.7.6-linux-arm64.tar.gz'}
2024-09-03 00:40:55,915 DEBUG [cluster] Downloaded the nerdctl archive from "https://github.com/containerd/nerdctl/releases/download/v1.7.6/nerdctl-full-1.7.6-linux-arm64.tar.gz"
2024-09-03 00:40:55,915 DEBUG [cluster] Run `limactl start cluster` to start the instance.
2024-09-03 00:40:55,919 DEBUG [cluster] Running ['/Users/nsoffer/src/lima/_output/bin/limactl', 'start', 'cluster', '--log-format', 'json']
2024-09-03 00:40:55,961 DEBUG [cluster] Using the existing instance "cluster"
2024-09-03 00:40:55,964 DEBUG [cluster] Starting the instance "cluster" with VM driver "vz"
2024-09-03 00:40:56,914 DEBUG [cluster] [hostagent] GRPC port forwarding is experimental
2024-09-03 00:40:56,919 DEBUG [cluster] [hostagent] hostagent socket created at /Users/nsoffer/.lima/cluster/ha.sock
2024-09-03 00:40:56,929 DEBUG [cluster] [hostagent] Starting VZ (hint: to watch the boot progress, see "/Users/nsoffer/.lima/cluster/serial*.log")
[1] lima-vm/lima#2584
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
|
Merged #2582, please rebase |
|
@AkihiroSuda should be ready now. |
|
One test failed - is this a flaky test? I cannot rerun the failing test, I hope some can rerun it... |
jandubois
requested changes
Sep 3, 2024
jandubois
requested changes
Sep 3, 2024
There was a problem hiding this comment.
I no longer want to change the --log-format=text option, but this PR includes 2 commits that have already been merged to master. Please rebase.
I see failing VZ tests even on re-runs, but I don't think they are related to this PR, so just need more re-runs, but I'll wait until the commit history is fixed.
I already rebased on master, anything else to change? |
My apologies, I mistook "Show progress only when using text format" for a duplicate of "Show progress only when logging to terminal". We do request that commits are squashed though, so could you please do that: $ g log --oneline -4
5cbac54 (HEAD -> log-format) Add json log format
547e45d Fix another terminal check
49f0925 Show progress only when using text format
01eb321 (upstream/master) Merge pull request #2582 from nirs/progress-to-file |
Sorry I was not aware of that, pushing squashed version. |
@nirs I don't see the squashed version on GitHub; are you sure you did a force-push? |
Add an option to to format logs in json format. This can be useful for
programs running limactl that want to consume some content from the logs
in the program own logs.
Example:
% _output/bin/limactl -h
...
Flags:
--debug debug mode
-h, --help help for limactl
--log-format string Set the logging format [text, json] (default "text")
--log-level string Set the logging level [trace, debug, info, warn, error]
...
% _output/bin/limactl start --log-format json --plain --vm-type vz --tty=false
{"level":"info","msg":"Terminal is not available, proceeding without opening an editor","time":"2024-09-02T05:12:28+03:00"}
{"level":"info","msg":"Starting the instance \"default\" with VM driver \"vz\"","time":"2024-09-02T05:12:28+03:00"}
...
Related changes included:
- Show progress only when using text format. When we have json format
for the progress we can enable it.
- Fix terminal check for windows, was using os.Stdout when we log to
os.Stderr.
Discussed in lima-vm#2576
Fixes lima-vm#2583
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
|
@jandubois should be ready now. Latest push fixed a typo in a new comment. |
tmeijn
pushed a commit
to tmeijn/dotfiles
that referenced
this pull request
Dec 13, 2024
⚠️ **CAUTION: this is a major update, indicating a breaking change!**⚠️ This MR contains the following updates: | Package | Update | Change | |---|---|---| | [lima-vm/lima](https://github.com/lima-vm/lima) | major | `v0.23.2` -> `v1.0.2` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>lima-vm/lima (lima-vm/lima)</summary> ### [`v1.0.2`](https://github.com/lima-vm/lima/releases/tag/v1.0.2) [Compare Source](lima-vm/lima@v1.0.1...v1.0.2) #### Changes - DNS: - Fixed the host resolver regression in v1.0.0 [#​2939](lima-vm/lima#2939) ([#​2964](lima-vm/lima#2964)) - `limactl create`: - Fixed races during parallel downloads ([#​2903](lima-vm/lima#2903), thanks to [@​nirs](https://github.com/nirs)) - Optimized qcow2-to-raw conversion for vz mode ([#​2933](lima-vm/lima#2933), thanks to [@​nirs](https://github.com/nirs)) - `limactl start-at-login`: - Fixed the support for Linux hosts (systemd) ([#​2943](lima-vm/lima#2943), thanks to [@​kachick](https://github.com/kachick)) - nerdctl: - Updated to [v2.0.1](https://github.com/containerd/nerdctl/releases/tag/v2.0.1) ([#​2966](lima-vm/lima#2966)) - Templates: - Updated to the latest revisions ([#​2936](lima-vm/lima#2936) [#​2953](lima-vm/lima#2953), thanks to [@​tcooper](https://github.com/tcooper)) - Web site: - Added an example of running Lima on GitHub Actions to run commands on non-Ubuntu ([#​2954](lima-vm/lima#2954)): https://lima-vm.io/docs/examples/gha/ - Project: - Invite Nir Soffer ([@​nirs](https://github.com/nirs)) as a Reviewer ([#​2916](lima-vm/lima#2916), thanks to [@​jandubois](https://github.com/jandubois)) Full changes: https://github.com/lima-vm/lima/milestone/51?closed=1 Thanks to [@​SpiffyEight77](https://github.com/SpiffyEight77) [@​alexandear](https://github.com/alexandear) [@​jandubois](https://github.com/jandubois) [@​kachick](https://github.com/kachick) [@​nirs](https://github.com/nirs) [@​norio-nomura](https://github.com/norio-nomura) [@​tamird](https://github.com/tamird) [@​tcooper](https://github.com/tcooper) #### Usage ```console [macOS]$ limactl create [macOS]$ limactl start ... INFO[0029] READY. Run `lima` to open the shell. [macOS]$ lima uname Linux ``` *** The binaries were built automatically on GitHub Actions. The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/12134682585 The sha256sum of the SHA256SUMS file itself is `02ef78494c498ca4180915ba78d5e2fc471ed401f63dfb2b5864c3711f3c0fb2` . *** Release manager: [@​AkihiroSuda](https://github.com/AkihiroSuda) ### [`v1.0.1`](https://github.com/lima-vm/lima/releases/tag/v1.0.1) [Compare Source](lima-vm/lima@v1.0.0...v1.0.1) Reverted the default port forwarder from gRPC to SSH for the stability reason ([#​2864](lima-vm/lima#2864)). This reversion fixes several regressions related to `docker run -p` in Lima v1.0.0 ([#​2859](lima-vm/lima#2859)). Although the gRPC forwarder is faster and has an advanced feature (UDP support), it turned out to be still immature. Set `LIMA_SSH_PORT_FORWARDER=false` to opt-in to the gRPC forwarder. See <https://lima-vm.io/docs/config/port/>. Full changes: https://github.com/lima-vm/lima/milestone/50?closed=1 Thanks to [@​alexandear](https://github.com/alexandear) [@​jandubois](https://github.com/jandubois) [@​norio-nomura](https://github.com/norio-nomura) #### Usage ```console [macOS]$ limactl create [macOS]$ limactl start ... INFO[0029] READY. Run `lima` to open the shell. [macOS]$ lima uname Linux ``` *** The binaries were built automatically on GitHub Actions. The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/11735352652 The sha256sum of the SHA256SUMS file itself is `f5c12d003e25dc46291803a8acae9e9d325a45eca0c1f9f40bd6852ec8ed9be1` . *** Release manager: [@​AkihiroSuda](https://github.com/AkihiroSuda) ### [`v1.0.0`](https://github.com/lima-vm/lima/releases/tag/v1.0.0) [Compare Source](lima-vm/lima@v0.23.2...v1.0.0) With the support from 110+ contributors in 3+ years, the Lima project has finally reached v1.0. 🎉 This release introduces several breaking changes, such as switching the default machine driver from QEMU to VZ for better filesystem performance. The `limactl` CLI is designed to print hints when the user hits those breaking changes. e.g., `limactl create template://experimental/vz` now fails with a hint that suggests using `limactl create --vm-type=vz template://default` instead. 🔴 = Major breaking changes 🟡 = Minor breaking changes - VZ: - Graduate VZ machine driver from experimental ([#​2758](lima-vm/lima#2758)) - 🔴 Use VZ by default for new instances on macOS >= 13.5 ([#​1951](lima-vm/lima#1951)) - Support nested virtualization on M3 ([#​2530](lima-vm/lima#2530), thanks to [@​abiosoft](https://github.com/abiosoft)) - Optimize qcow2-to-raw image conversion (lima-vm/go-qcow2reader@v0.1.2...v0.4.0 , thanks to [@​nirs](https://github.com/nirs)) - Support specifying a custom kernel ([#​2562](lima-vm/lima#2562), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - QEMU: - Graduate 9p mount driver from experimental ([#​2758](lima-vm/lima#2758)) - 🔴 Use 9p by default for most templates ([#​1953](lima-vm/lima#1953), [#​2822](lima-vm/lima#2822)) - riscv64: switch from u-boot to EDK2 ([#​2592](lima-vm/lima#2592)) - Network: - Graduate user-v2 network driver from experimental ([#​2758](lima-vm/lima#2758)) - Support UDP port forwarding ([#​2411](lima-vm/lima#2411), thanks to [@​balajiv113](https://github.com/balajiv113)) - 🔴 Strictly require `socket_vmnet` binary to be owned by root ([#​2734](lima-vm/lima#2734)) - SSH: - 🟡 Disable `ssh.loadDotSSHPubKeys` by default ([#​2706](lima-vm/lima#2706)) - YAML: - Support generating jsonschema ([#​2306](lima-vm/lima#2306), thanks to [@​afbjorklund](https://github.com/afbjorklund)) - Support specifying `param` for provisioning scripts ([#​2570](lima-vm/lima#2570), thanks to [@​jandubois](https://github.com/jandubois)) - Support specifying `minimumLimaVersion` and `vmOpts.qemu.minimumVersion` ([#​2659](lima-vm/lima#2659), thanks to [@​jandubois](https://github.com/jandubois)) - Support template expansion in mounts ([#​2588](lima-vm/lima#2588), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - `limactl` CLI: - Add `limactl tunnel` command so as to allow the host to join the guest network ([#​2710](lima-vm/lima#2710)) - Add `--log-format=json` ([#​2584](lima-vm/lima#2584), thanks to [@​nirs](https://github.com/nirs)) - `limactl prune`: Add `--keep-referred` ([#​2569](lima-vm/lima#2569), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - nerdctl: - Updated to [v2.0.0](https://github.com/containerd/nerdctl/releases/tag/v2.0.0) ([#​2178](lima-vm/lima#2178)) - rootless: allocate 1G subuids from 524288 (0x80000) for new users ([#​2725](lima-vm/lima#2725)) - Templates: - 🔴 `experimental/vz`: Merged into the `default` template ([#​2730](lima-vm/lima#2730), [#​2736](lima-vm/lima#2736)) - 🟡 `experimental/{riscv64, armv7l}`: Merged into the `default` template ([#​2730](lima-vm/lima#2730), [#​2736](lima-vm/lima#2736)) - 🔴 `vmnet`: Removed in favor of `limactl create --network=lima:shared template://default` ([#​2736](lima-vm/lima#2736)) - 🟡 `experimental/net-user-v2`: Removed in favor of `limactl create --network=lima:user-v2 template://default` ([#​2736](lima-vm/lima#2736)) - 🔴 `experimental/9p`: Removed in favor of `limactl create --mount-type=9p template://default` ([#​2736](lima-vm/lima#2736)) - 🟡 `experimental/virtiofs-linux`: Removed in favor of `limactl create --mount-type=virtiofs template://default` ([#​2736](lima-vm/lima#2736)) - 🔴 `alpine`: Renamed to `alpine-iso` ([#​2704](lima-vm/lima#2704)) - 🔴 `alpine-image`: Renamed to `alpine` ([#​2704](lima-vm/lima#2704)) - `archlinux`: Demoted from Tier 1 to Tier 2 ([#​2717](lima-vm/lima#2717), [#​2823](lima-vm/lima#2823)) - `default`, `ubuntu`, ...: Updated to Ubuntu 24.10. The older versions are available as `ubuntu-20.04`, `ubuntu-22.04`, and `ubuntu-24.04` ([#​2755](lima-vm/lima#2755), [#​2795](lima-vm/lima#2795)) - `fedora`: Updated to Fedora 41 ([#​2821](lima-vm/lima#2821), [#​2822](lima-vm/lima#2822), thanks to [@​subpop](https://github.com/subpop)) - `opensuse`: Renamed to `opensuse-leap`. Still aliased as `opensuse` ([#​2612](lima-vm/lima#2612), thanks to [@​afbjorklund](https://github.com/afbjorklund)) - `experimental/opensuse-tumbleweed`: Support aarch64 ([#​2613](lima-vm/lima#2613), thanks to [@​afbjorklund](https://github.com/afbjorklund)) - `hack/update-template.sh` is added for automating updates ([#​1347](lima-vm/lima#1347), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - Project: - Invite Norio Nomura ([@​norio-nomura](https://github.com/norio-nomura)) as a Reviewer ([#​2567](lima-vm/lima#2567)) Full changes: https://github.com/lima-vm/lima/milestone/47?closed=1 Thanks to [@​AdamKorcz](https://github.com/AdamKorcz) [@​Mr-Sunglasses](https://github.com/Mr-Sunglasses) [@​SmartManoj](https://github.com/SmartManoj) [@​YorikSar](https://github.com/YorikSar) [@​abiosoft](https://github.com/abiosoft) [@​afbjorklund](https://github.com/afbjorklund) [@​alexandear](https://github.com/alexandear) [@​balajiv113](https://github.com/balajiv113) [@​hasan4791](https://github.com/hasan4791) [@​jandubois](https://github.com/jandubois) [@​nirs](https://github.com/nirs) [@​norio-nomura](https://github.com/norio-nomura) [@​pvdvreede](https://github.com/pvdvreede) [@​subpop](https://github.com/subpop) [@​tsukasaI](https://github.com/tsukasaI) #### Usage ```console [macOS]$ limactl create [macOS]$ limactl start ... INFO[0029] READY. Run `lima` to open the shell. [macOS]$ lima uname Linux ``` *** The binaries were built automatically on GitHub Actions. The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/11695321667 The sha256sum of the SHA256SUMS file itself is `4bd200a163111fe78c6f3e6de405113d416053802fe1507597f9a42f89a98c90` . *** Release manager: [@​AkihiroSuda](https://github.com/AkihiroSuda) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When using --log-format json we use machine friendly flogs to make limactl easier to consume by other programs.
Since we don't have a json format for the progress bar, the progress is also hidden when using json log format.
Here is an example usage of the new option in drenv:
nirs/ramen@b000119
Fixes #2583