test (e2e) : Use pinned version of crc while executing e2e tests (#4629)

[rohanKanojia]

Description

Fixes: #4629

⚠️ We need to test these changes by building a container image based on these changes and updating pipeline to use that specific container image

Instead of assuming e2e tests would pick up crc binary added by the pipeline, that may or may not work if some user has added a binary in any of the PATH dirs, explicitly specify crc installation dir during the test run

Type of change

• Bug fix (non-breaking change which fixes an issue)
• Feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change
• Chore (non-breaking change which doesn't affect codebase;
  test, version modification, documentation, etc.)

Proposed changes

• In test images run.sh script introduce a new variable crcBinaryDir that would contain path to directory that contains CRC binary
• Add another command line argument --crc-binary for explicitly specifying test arguments

Testing

⚠️ It's not tested yet.

Contribution Checklist

• I Keep It Small and Simple: The smaller the PR is, the easier it is to review and have it merged
• I have performed a self-review of my code
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Which platform have you tested the code changes on?
  • Linux
  • Windows
  • MacOS

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[praveenkumar]

Is this to do testing locally because on the pipeline side it suppose to build binary and put in the required path?

[rohanKanojia]

Is this to do testing locally because on the pipeline side it suppose to build binary and put in the required path?

@praveenkumar : No, this change is done to enforce the path where the pipeline has copied the binary built during the build phase. At the moment, we just assume that system would pick up crc binary from where pipeline has copied it. However, if some user uses the test instance and adds crc binary in $HOME/go/bin it was taking more precedence over the binary added by the pipeline. We were facing this issue in devtools-qe-incubator/deliverest#59 where pipeline was failing due to presence of an outdated binary in some other location that was taking more precedence over the location where pipeline copied the CRC binary. Once we deleted that binary, pipeline started passing.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: albfan
Once this PR has been reviewed and has the lgtm label, please assign cfergeau for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[albfan]

We should remove the vtest tags and update the existing vnext tags

[rohanKanojia]

@albfan : Thanks, I've deletedvtest-* tags from crcont/crc-e2e repository

[openshift-ci]

@rohanKanojia: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-crc	617b956	link	true	/test e2e-crc

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.