ci/cli: add Regression tests

Run basic regression tests for older version of K3s/RKE2 on Rancher
Manager Prime version.

Signed-off-by: Loic Devulder <ldevulder@suse.com>

.github/workflows/SCHEDULING.md

@@ -12,6 +12,7 @@ We try to spread the tests as best as we can to avoid SPOT issue as well as not
 | CLI K3s Scalability | Sunday | 2am | us-central1-f |
 | CLI K3s SELinux | Sunday | 2am | us-central1-c |
 | CLI Multicluster | Sunday | 5am | us-central1-b |
+| CLI Regression | Saturday | 11am | us-central1-c |
 | CLI Rancher Manager Devel | Sunday | 8am | us-central1-c |
 | UI Rancher Manager Devel | Sunday | 12pm | us-central1-a |
 | CLI K3s Downgrade | Sunday | 2pm | us-central1-b |

.github/workflows/cli-regression-matrix.yaml

@@ -0,0 +1,49 @@
+# This workflow calls the master E2E workflow with custom variables
+name: CLI-Regression
+
+on:
+  workflow_dispatch:
+    inputs:
+      destroy_runner:
+        description: Destroy the auto-generated self-hosted runner
+        default: true
+        type: boolean
+      k8s_downstream_version:
+        description: Rancher cluster downstream version to use
+        default: '"v1.25.16+k3s4"'
+        type: string
+      k8s_upstream_version:
+        description: Rancher cluster upstream version to use
+        default: '"v1.28.11+k3s2"'
+        type: string
+      qase_run_id:
+        description: Qase run ID where the results will be reported
+        type: string
+  schedule:
+    # Every Saturday at 6am UTC (11am in us-central1)
+    - cron: '0 6 * * 6'
+
+jobs:
+  cli:
+    strategy:
+      fail-fast: false
+      max-parallel: 4
+      matrix:
+        k8s_downstream_version: ${{ fromJSON(format('[{0}]', inputs.k8s_downstream_version || '"v1.25.16+k3s4","v1.26.15+k3s1","v1.27.14+k3s1","v1.28.10+k3s1","v1.25.16+rke2r1","v1.26.15+rke2r1","v1.27.14+rke2r1","v1.28.10+rke2r1"')) }}
+        k8s_upstream_version: ${{ fromJSON(format('[{0}]', inputs.k8s_upstream_version || '"v1.28.11+k3s2"')) }}
+    uses: ./.github/workflows/master_e2e.yaml
+    secrets:
+      credentials: ${{ secrets.GCP_CREDENTIALS }}
+      pat_token: ${{ secrets.SELF_HOSTED_RUNNER_PAT_TOKEN }}
+      qase_api_token: ${{ secrets.QASE_API_TOKEN }}
+    with:
+      ca_type: selfsigned
+      destroy_runner: ${{ github.event_name == 'schedule' && true || inputs.destroy_runner }}
+      k8s_downstream_version: ${{ matrix.k8s_downstream_version }}
+      k8s_upstream_version: ${{ matrix.k8s_upstream_version }}
+      qase_run_id: ${{ github.event_name == 'schedule' && 'auto' || inputs.qase_run_id }}
+      rancher_version: prime/latest
+      reset: true
+      sequential: false
+      test_type: cli
+      zone: us-central1-c

README.md

@@ -22,6 +22,7 @@
 [![CLI-K3s-SELinux](https://github.com/rancher/elemental/actions/workflows/cli-k3s-selinux-matrix.yaml/badge.svg)](https://github.com/rancher/elemental/actions/workflows/cli-k3s-selinux-matrix.yaml)
 [![CLI-Multicluster](https://github.com/rancher/elemental/actions/workflows/cli-multicluster-matrix.yaml/badge.svg)](https://github.com/rancher/elemental/actions/workflows/cli-multicluster-matrix.yaml)
 [![CLI-Rancher-Manager-Head-2.9](https://github.com/rancher/elemental/actions/workflows/cli-rm-head-2.9-matrix.yaml/badge.svg)](https://github.com/rancher/elemental/actions/workflows/cli-rm-head-2.9-matrix.yaml)
+[![CLI-Regression](https://github.com/rancher/elemental/actions/workflows/cli-regression-matrix.yaml/badge.svg)](https://github.com/rancher/elemental/actions/workflows/cli-regression-matrix.yaml)
 [![UI-Rancher-Manager-Head-2.9](https://github.com/rancher/elemental/actions/workflows/ui-rm-head-2.9-matrix.yaml/badge.svg)](https://github.com/rancher/elemental/actions/workflows/ui-rm-head-2.9-matrix.yaml)
 
 ## Goal