Add scripts for e2e test on psi

[anandrkskd]

Signed-off-by: anandrkskd anandrkskd@gmail.com

What type of PR is this?
/kind feature

What does this PR do / why we need it:
This PR adds scripts required for automating windows and mac tests for PR

Which issue(s) this PR fixes:

Fixes - part of #4412
Fixes - part of #4408

PR acceptance criteria:

• Unit test

• Integration test

• Documentation

• I have read the test guidelines

How to test changes / Special notes to the reviewer:
N/A

[anandrkskd]

We are facing failures on PSI tests due to some known issues
Issue 1. #2877

[ssh:Windows 10] [2] + Failure [244.628 seconds]
[ssh:Windows 10] [2] odo java e2e tests
[ssh:Windows 10] [2] C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/e2escenarios/e2e_java_test.go:11
[ssh:Windows 10] [2]   odo component creation
[ssh:Windows 10] [2]   C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/e2escenarios/e2e_java_test.go:59
[ssh:Windows 10] [2]     Should be able to deploy a spring boot uberjar file using openjdk [It]
[ssh:Windows 10] [2]     C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/e2escenarios/e2e_java_test.go:96
[ssh:Windows 10] [2] 
[ssh:Windows 10] [2]     No future change is possible.  Bailing out early after 240.681s.
[ssh:Windows 10] [2]     Running odo.exe with args [odo push --context C:\Users\Admin\849480791]
[ssh:Windows 10] [2]     Expected
[ssh:Windows 10] [2]         <int>: 1
[ssh:Windows 10] [2]     to match exit code:
[ssh:Windows 10] [2]         <int>: 0
[ssh:Windows 10] [2] 
[ssh:Windows 10] [2]     C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/helper/helper_run.go:34
[ssh:Windows 10] [2] ------------------------------
[ssh:Windows 10] [2] Created dir: C:\Users\Admin\849480791
...
[ssh:Windows 10] [2] [odo] 
 X  Waiting for component to start [4m]
[ssh:Windows 10] [2] [odo]  X  waited 4m0s but couldn't find running pod matching selector: 'deploymentconfig=sb-jar-test-app'

Issue 2. #3256

[ssh:Windows 10] [2] odo devfile supported tests
[ssh:Windows 10] [2] C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/e2escenarios/e2e_devfile_test.go:24
[ssh:Windows 10] [2]   odo debug support for devfile components
[ssh:Windows 10] [2]   C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/e2escenarios/e2e_devfile_test.go:66
[ssh:Windows 10] [2]     Verify output debug information for java-openliberty debug works [It]
[ssh:Windows 10] [2]     C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/e2escenarios/e2e_devfile_test.go:73
[ssh:Windows 10] [2] 
[ssh:Windows 10] [2]     No future change is possible.  Bailing out early after 300.461s.
[ssh:Windows 10] [2]     Running odo.exe with args [odo push --context C:\Users\Admin\692823498/projectDir]
[ssh:Windows 10] [2]     Expected
[ssh:Windows 10] [2]         <int>: 1
[ssh:Windows 10] [2]     to match exit code:
[ssh:Windows 10] [2]         <int>: 0
[ssh:Windows 10] [2] 
[ssh:Windows 10] [2]     C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/helper/helper_run.go:34
[ssh:Windows 10] [2] ------------------------------
[ssh:Windows 10] [2] Created dir: C:\Users\Admin\692823498
[ssh:Windows 10] [2] Setting KUBECONFIG=C:\Users\Admin\692823498\config
[ssh:Windows 10] [2] Creating a new project: e2e-devfile-test73brp
[ssh:Windows 10] [2] Running odo.exe with args [odo project create e2e-devfile-test73brp -w -v4]

[ssh:Windows 10] [2] [odo] I0426 09:17:28.957046   11160 deployments.go:152] Waiting for deployment "tpqovd" rollout to finish: 0 of 1 updated replicas are available...
[ssh:Windows 10] [2] [odo] I0426 09:17:28.957046   11160 deployments.go:159] Waiting for deployment spec update to be observed...
[ssh:Windows 10] [2] [odo] 
 X  Failed to start component with name "tpqovd". Error: Failed to create the component: error while waiting for deployment rollout: timeout while waiting for tpqovd deployment roll out
[ssh:Windows 10] [2] [odo]  X  Waiting for component to start [5m]

[prietyc123]

@anandrkskd Thanks for the pr. AFAIU this pr will partially resolve #4412 and #4408 . However As per the pr subject this pr will close the issue #4412 and #4408 . I think if you want to mention a part of its fix you may write for example Fixes - part of #4412 in pr template.

[prietyc123]

INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]                              
INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]     Unexpected error:   
INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]         <*exec.Error | 0xc0000ffe20>: { 
INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]             Name: "oc",      
INFO[2021-04-30T11:23:13Z]                                              
INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]             Err: {           
INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]                 s: "executable file not found in $PATH", 
INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]             },               
INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]         }                    
INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]         exec: "oc": executable file not found in $PATH 
INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]     occurred             
INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]                              
INFO[2021-04-30T11:23:13Z] [ssh:Fedora 32]     /home/fedora/amqp_ci_rcv_odo-minikube-pr-build_PR_4658/repo/tests/helper/helper_run.go:27 

This has been fixed in #4677

/test psi-kubernetes-integration-e2e

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: anandrkskd
To complete the pull request process, please assign mohammedzee1000 after the PR has been reviewed.
You can assign the PR to them by writing /assign @mohammedzee1000 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• scripts/OWNERS
• tests/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[girishramnani]

is this still WIP @anandrkskd ?

[girishramnani]

why not run integration-devfile instead of each test separately?

[anandrkskd]

we are facing issues in which odo push is failing, this is mostly failing on integration,integration-devfile tests. so for now we are just testing those parts that doesn't require odo push in integration, integration-devfile suites.

[dharmit]

@anandrkskd how relevant is this going to be w.r.t IBM Cloud? Would the changes proposed in this PR be useful for it or would we have to do things differently through a different PR?

[openshift-ci]

@anandrkskd: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[rnapoles-rh]

@anandrkskd how relevant is this going to be w.r.t IBM Cloud? Would the changes proposed in this PR be useful for it or would we have to do things differently through a different PR?

@dharmit When we switch to use the IBM Cloud cluster we will submit another PR to change the target server and use the API key to login. We can not use the IBM Cloud yet as I just found out that the API token for the Cluster running in the IBM Cloud is expiring. Checking on that. But this does not affect this PR that is using the cluster running in PSI.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign mohammedzee1000 after the PR has been reviewed.
You can assign the PR to them by writing /assign @mohammedzee1000 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• scripts/OWNERS
• tests/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sonarqubecloud]

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities
0 Security Hotspots
21 Code Smells

No Coverage information
0.0% Duplication

[openshift-ci]

@anandrkskd: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/v4.7-integration-e2e	9a9bbda	link	/test v4.7-integration-e2e
ci/prow/psi-unit-test-windows	9a9bbda	link	/test psi-unit-test-windows
ci/prow/psi-unit-test-mac	9a9bbda	link	/test psi-unit-test-mac
ci/prow/psi-kubernetes-integration-e2e	9a9bbda	link	/test psi-kubernetes-integration-e2e
ci/prow/unit	9a9bbda	link	/test unit

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[anandrkskd]

Splitting the PR into two parts

• Script for automation (add script for tests on IBMC #4877)
• Fix tests related to platform(Tests/fix for test on ibmc #4873)
  Thus, closing this issue.
  /close

[openshift-ci]

@anandrkskd: Closed this PR.

In response to this:

Splitting the PR into two parts

• Script for automation (add script for tests on IBMC #4877)
• Fix tests related to platform(Tests/fix for test on ibmc #4873)
  Thus, closing this issue.
  /close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.