Skip to content
This repository has been archived by the owner on May 12, 2021. It is now read-only.

yq: failed traversing anchors in version.yaml #2430

Closed
Pennyzct opened this issue Feb 2, 2020 · 4 comments
Closed

yq: failed traversing anchors in version.yaml #2430

Pennyzct opened this issue Feb 2, 2020 · 4 comments
Labels
bug Incorrect behaviour needs-review Needs to be assessed by the team.

Comments

@Pennyzct
Copy link
Contributor

Pennyzct commented Feb 2, 2020

Description of problem

Recently, initrd-related CI are failing frequently.
http://jenkins.katacontainers.io/job/kata-containers-runtime-ubuntu-18-04-PR-initrd/2843/

INFO: Image to generate: kata-containers-*default-initrd-name-*default-initrd-version-osbuilder-7526f49-agent-36b37f6.initrd
......
./rootfs-builder/rootfs.sh: line 280: /tmp/jenkins/workspace/kata-containers-runtime-ubuntu-18-04-PR-initrd/go/src/github.com/kata-containers/osbuilder/rootfs-builder/*default-initrd-name/config.sh: No such file or directory
Failed at 120: sudo -E AGENT_INIT="${AGENT_INIT}" AGENT_VERSION="${agent_commit}" GOPATH="$GOPATH" USE_DOCKER=true OS_VERSION=${os_version} ./rootfs-builder/rootfs.sh "${distro}"
Failed at 26: "${cidir}/install_kata_image.sh" "${tag}"

Debugging on the above output, the cause may come from that yq failed reading values from assets.initrd.architecture.$arch.name
Based on the latest yq spec, the read command could print out the anchors of a document and can also traverse them.
But when anchor refers to an scalar (string, integer etc), the latest yq could not traverse it.
e.g.

$ yq read versions.yaml "assets.initrd.architecture.ppc64le.name"
*default-initrd-name
@Pennyzct Pennyzct added bug Incorrect behaviour needs-review Needs to be assessed by the team. labels Feb 2, 2020
Pennyzct added a commit to Pennyzct/runtime that referenced this issue Feb 3, 2020
The latest yq might not "resolve" the anchor, if it refers to
an scalar (string, integer etc). But it's fine with the collections
(sequences, mappings).

Fixes: kata-containers#2430

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
@grahamwhaley
Copy link
Contributor

@grahamwhaley
Copy link
Contributor

For ref, some yq docs referring to the difference between fetching an anchor and resolving (following) an anchor ... https://mikefarah.gitbook.io/yq/commands/read#anchors-and-aliases
I think some of this got 'added' in v3 - so, maybe we need to adapt to a new call?

@grahamwhaley
Copy link
Contributor

I dug a fraction more (as this is something I don't know much about in yaml, and wanted to). I think the issue is when we have an anchor on something that has a value - I think that is called a 'scalar'?. So, with the yaml:

---
assets:
  initrd:
    architecture:
      aarch64:
        name: &default-initrd-name "alpine"
        version: &default-initrd-version "3.7"
      ppc64le:
        name: *default-initrd-name

    a2:
      aarch64: &default-initrd
        name: "alpine"
        version: "3.7"
      ppc64le: *default-initrd

and the code:

#!/bin/bash

set -x

yq2 read v2.yaml "assets.initrd.architecture.ppc64le.name"
yq3 read v2.yaml "assets.initrd.architecture.ppc64le.name"

yq2 read v2.yaml "assets.initrd.a2.ppc64le.name"
yq3 read v2.yaml "assets.initrd.a2.ppc64le.name"

I get:

 ./go.sh 
+ yq2 read v2.yaml assets.initrd.architecture.ppc64le.name
alpine
+ yq3 read v2.yaml assets.initrd.architecture.ppc64le.name
*default-initrd-name
+ yq2 read v2.yaml assets.initrd.a2.ppc64le.name
alpine
+ yq3 read v2.yaml assets.initrd.a2.ppc64le.name
alpine

Looking around, I think what we have is valid YAML anchors - so, I suspect this is a yq issue.
/cc @jodh-intel for yaml expertise.... and tenacity....

@jodh-intel
Copy link
Contributor

I've raised mikefarah/yq#340.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Incorrect behaviour needs-review Needs to be assessed by the team.
Projects
None yet
Development

No branches or pull requests

3 participants