Run e2e and integration test on windows and macOS host

[prietyc123]

[kind/Feature]

Which functionality do you think we should add?

We can borrow macOS host on travis and windows from appveyor then only we can start our tests on these hosts.

Why is this needed?

Not yet implemented in CI.

• Integration tests
  • Mac OS Run e2e and integration test on MacOS using github actions #5301
  • Windows Run e2e and integration test on windows using windows server instance in IBM Cloud #5300
• e2e tests
  • Mac OS Run e2e and integration test on MacOS using github actions #5301
  • Windows Run e2e and integration test on windows using windows server instance in IBM Cloud #5300

[amitkrout]

@mohammedzee1000 Can we please add those inputs that we discussed yesterday and before. @prietyc123 FYI...

[mohammedzee1000]

We can use travis.rb which is the official cli for travis ci or we can directly use the API.

The main drawbacks of travis.rb is we cannot create job manually, just restarting existing ones and also while we can set envs, we cannot do it per build

So i propose:

• configure travis job which looks for env and if not present fails
• prow job starts and brings up cluster
• prow job passes on cluster url to travis job with env and restarts it

NOTE as said we will have to consider possible dirty reads with env and figure out how to work around that

[mohammedzee1000]

https://github.com/travis-ci/travis.rb
The problematic bit where dirty reads can occur is https://github.com/travis-ci/travis.rb#env and we should plan around that.

For example after cluster is brought up, the prow job goes into a loop, where it injects env say CLUSTER_URL=10;example.com (where 10 is PR no) and restarts the job, then waits for build to become a success. If any builds fail then it restarts the same for say 2 times after which it is assumed as failed forever (basically full retrigger required).

Travis job on its part reads CLUSTER_URL="10;example.com" reads pr no and if it does not match, it fails. Otherwise, it reads the cluster URL and stores it in different env and resumes tests

❯ travis env set FOO BAR
[+] setting environment variable $FOO
❯ travis env list
# environment variables for mohammedzee1000/odo
FOO=[secure]

TRAVIS_PULL_REQUEST gets the pull req no or false and TRAVIS_PULL_REQUEST_BRANCH for branch name

Steve said:
Prow provides these env vars, so you could get the PR number by:
pr_num="$( jq .refs.pulls[0].num <<<"${JOB_SPEC}" )" # recommended
pr_num="${PULL_NUMBER}" # may be deprecated
https://github.com/kubernetes/test-infra/blob/master/prow/jobs.md#job-environment-variables

@amitkrout @girishramnani wdyt

[mohammedzee1000]

travis-ci/travis.rb#717

[mohammedzee1000]

Steve said:
Prow provides these env vars, so you could get the PR number by:
pr_num="$( jq .refs.pulls[0].num <<<"${JOB_SPEC}" )" # recommended
pr_num="${PULL_NUMBER}" # may be deprecated
https://github.com/kubernetes/test-infra/blob/master/prow/jobs.md#job-environment-variables

[mohammedzee1000]

[kadel]

Would it be possible to run tests against some of the internal clusters (like ocp311.odo) instead of using OpenShift's Prow?

[dharmit]

@mohammedzee1000 how long do you think before get #4063 in? Only then we can start working on this, right?

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[prietyc123]

/remove-lifecycle stale

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[rm3l]

Closing this.

Related issues for MacOS (M1):

• POC - Try access to AWS M1 instances  #5929
• POC - Run tests on Mac M1 instance (AWS preview) #6012

/close

[openshift-ci]

@rm3l: Closing this issue.

In response to this:

Closing this.

Related issues for MacOS (M1):

• POC - Try access to AWS M1 instances  #5929
• POC - Run tests on Mac M1 instance (AWS preview) #6012

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.