CORS-3652 : RHCOS Stream Marketplace Images

[patrickdillon]

This is an implementation of openshift/enhancements#1655

• Adds a standalone go file ./hack/rhcos/populate-marketplace-imagestream.go which generates data/data/coreos/marketplace-rhcos.json
• Adds packages to populate the stream with free Azure marketplace images published by ARO
• Vendors local fork of coreos/stream-metadata-go with updated struct definitions for the marketplace (PR forthcoming)
• Updates existing installer RHCOS libraries to merge marketplace-rhcos.json with rhcos.json when:
  • openshift-install coreos print-stream-json
  • running ./hack/build-coreos-manifest.go (as part of the Dockerfile build) to generate the RHCOS stream configmap
  • selecting boot images for IPI installs with openshift-install create cluster
• Includes a PoC commit to demonstrate a CI-based IPI Azure install using the stream. I expect it may be better to break this work into a separate PR, but including the initial work here serves as a good test/verification.

openshift/enhancements#1655 is still a WIP and needs to be updated to reflect this work, but enough consensus has been reached that the work itself can help guide the enhancement.

How it works:

Users can run this standalone command (using authentication libs of the openshift-installer) to generate/update the marketplace stream:

STREAM_RELEASE_OVERRIDE=417.94.202407010000-0 go run -mod=vendor ./hack/rhcos/populate-marketplace-imagestream.go

By default, the command will look for (Azure unpaid) images in the current branch (discovered from the local rhcos.json). I added an override to force searching for a specific release. In this case, the override value is hacky with zero-padding because I could not determine the RHCOS build which was used to publish the ARO images (they truncate the timestamp).

The results of this command produce the marketplace-rhcos.json file, checked in here.

When using the print-stream-json command, the marketplace-rhcos.json results are merged with rhcos.json:

$ ./openshift-install coreos print-stream-json | jq '.architectures.x86_64."rhel-coreos-extensions"'
{
  "azure-disk": {
    "release": "418.94.202410090804-0",
    "url": "https://rhcos.blob.core.windows.net/imagebucket/rhcos-418.94.202410090804-0-azure.x86_64.vhd"
  },
  "marketplace": {
    "azure": {
      "no-purchase-plan": {
        "hyperVGen1": {
          "publisher": "azureopenshift",
          "offer": "aro4",
          "sku": "aro_417",
          "version": "417.94.20240701",
          "thirdParty": false
        },
        "hyperVGen2": {
          "publisher": "azureopenshift",
          "offer": "aro4",
          "sku": "417-v2",
          "version": "417.94.20240701",
          "thirdParty": false
        }
      }
    }
  }
}

Similarly, the configmap generated from ./hack/build-coreos-manifest.go contains the merged json.

Finally, the poc ipi azure install demonstrates the installation time gains we achieve when not needing to upload an image as part of the install process.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from patrickdillon. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[patrickdillon]

/cc @bryan-cox @bennerv
fyi @sdodson @jlebon @travier
I will update the enhancement ASAP.

[openshift-ci]

@patrickdillon: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-azure-default-config	6944b1b	link	false	/test e2e-azure-default-config
ci/prow/golint	6944b1b	link	true	/test golint
ci/prow/verify-vendor	6944b1b	link	true	/test verify-vendor
ci/prow/e2e-azurestack	6944b1b	link	false	/test e2e-azurestack

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.