PR got merged even when TravisCI test were failing

[kadel]

on travis CI
https://travis-ci.com/openshift/odo/jobs/225078001

notice the projects names

Running odo with args [odo service create dh-postgresql-apb --plan dev -p postgresql_user=luke -p postgresql_password=secret -p postgresql_database=my_data -p postgresql_version=9.6 --app uphmxpo --project ajdvqxarqf]

[odo]  ✗  Component dh-postgresql-apb does not exist in application uphmxpo

full output

Creating a new project: ajdvqxarqf
Running odo with args [odo project create ajdvqxarqf -w -v4]
[odo] I0814 08:14:02.342870   23023 preference.go:116] The configFile is /home/travis/.odo/preference.yaml
[odo] I0814 08:14:02.343241   23023 occlient.go:478] Trying to connect to server 127.0.0.1:8443
[odo] I0814 08:14:02.346659   23023 occlient.go:485] Server https://127.0.0.1:8443 is up
[odo] I0814 08:14:02.461155   23023 occlient.go:408] isLoggedIn err:  <nil> 
[odo]  output: "developer"
[odo] I0814 08:14:02.461193   23023 context.go:371] Ignoring error as it usually means flag wasn't set: flag accessed but not defined: context
[odo] I0814 08:14:02.461203   23023 context.go:371] Ignoring error as it usually means flag wasn't set: flag accessed but not defined: context
[odo] I0814 08:14:02.461236   23023 context.go:371] Ignoring error as it usually means flag wasn't set: flag accessed but not defined: app
[odo] I0814 08:14:02.461243   23023 context.go:371] Ignoring error as it usually means flag wasn't set: flag accessed but not defined: project
[odo] I0814 08:14:02.471270   23023 context.go:371] Ignoring error as it usually means flag wasn't set: flag accessed but not defined: app
[odo] I0814 08:14:02.471297   23023 context.go:371] Ignoring error as it usually means flag wasn't set: flag accessed but not defined: output
[odo] I0814 08:14:02.471302   23023 context.go:371] Ignoring error as it usually means flag wasn't set: flag accessed but not defined: component
[odo]  •  Waiting for project to come up  ...
[odo]  ✓  Waiting for project to come up [1s]
[odo]  ✓  Project 'ajdvqxarqf' is ready for use
[odo]  ✓  New project created and now using project : ajdvqxarqf
[odo] I0814 08:14:03.650283   23023 odo.go:70] Could not get the latest release information in time. Never mind, exiting gracefully :)
Created dir: /tmp/659686430
Current working dir: /home/travis/gopath/src/github.com/openshift/odo/tests/integration
Setting current dir to: /tmp/659686430
Setting current dir to: /home/travis/gopath/src/github.com/openshift/odo/tests/integration
Running odo with args [odo service create dh-postgresql-apb --plan dev -p postgresql_user=luke -p postgresql_password=secret -p postgresql_database=my_data -p postgresql_version=9.6 --app uphmxpo --project ajdvqxarqf]
[odo]  •  Creating service  ...
[odo]  ✓  Creating service [183ms]
[odo]  ✓  Service 'dh-postgresql-apb' was created
[odo] Progress of the provisioning will not be reported and might take a long time.
[odo] You can see the current status by executing 'odo service list'
Running oc with args [oc get serviceinstance -o name -n ajdvqxarqf]
[oc] serviceinstance.servicecatalog.k8s.io/dh-postgresql-apb
Running odo with args [odo service list --app uphmxpo --project ajdvqxarqf]
[odo] NAME                  TYPE                  STATUS
[odo] dh-postgresql-apb     dh-postgresql-apb     Provisioning
Running odo with args [odo delete -f dh-postgresql-apb --app uphmxpo --project ajdvqxarqf]
[odo]  ✗  Component dh-postgresql-apb does not exist in application uphmxpo
Deleting project: ajdvqxarqf
Running odo with args [odo project delete ajdvqxarqf -f]
[odo] This project contains the following applications, which will be deleted
[odo] Application uphmxpo
[odo] This application has following service that will be deleted
[odo] service named dh-postgresql-apb of type dh-postgresql-apb
[odo]  •  Deleting project ajdvqxarqf  ...
[odo]  ✓  Deleting project ajdvqxarqf [8s]
[odo] Deleted project : ajdvqxarqf
Deleting dir: /tmp/659686430
Setting current dir to: /home/travis/gopath/src/github.com/openshift/odo/tests/integration
• Failure [15.788 seconds]
odoServiceE2e
/home/travis/gopath/src/github.com/openshift/odo/tests/integration/service_test.go:15
  When working from outside a component dir
  /home/travis/gopath/src/github.com/openshift/odo/tests/integration/service_test.go:105
    should be able to create, list and delete services without a context and using --app and --project flags instaed [It]
    /home/travis/gopath/src/github.com/openshift/odo/tests/integration/service_test.go:143
    No future change is possible.  Bailing out early after 0.723s.
    Running odo with args [odo delete -f dh-postgresql-apb --app uphmxpo --project ajdvqxarqf]
    Expected
        <int>: 1
    to match exit code:
        <int>: 0
    /home/travis/gopath/src/github.com/openshift/odo/tests/helper/helper_run.go:32

[amitkrout]

@kadel I could not find any abnormality in project name ajdvqxarqf. But the part that stuck to mind is service list STATUS

Running odo with args [odo service list --app uphmxpo --project ajdvqxarqf]
[odo] NAME                  TYPE                  STATUS
[odo] dh-postgresql-apb     dh-postgresql-apb     Provisioning

It seems the service is not created properly because the SATATUS is still showing Provisioning. Then immediately in the very next step deletion of service is invoked and causes the failure

Running odo with args [odo delete -f dh-postgresql-apb --app uphmxpo --project ajdvqxarqf]
[odo]  ✗  Component dh-postgresql-apb does not exist in application uphmxpo

Test is hitting the issue because helper.WaitForCmdOut is looking for the service name instead of service status.

Will send a pr

[kadel]

It seems the service is not created properly because the SATATUS is still showing Provisioning. Then immediately in the very next step deletion of service is invoked and causes the failure

Running odo with args [odo delete -f dh-postgresql-apb --app uphmxpo --project ajdvqxarqf]
[odo]  ✗  Component dh-postgresql-apb does not exist in application uphmxpo

what is odd is that the delete command uses ajdvqxarqf but the error message mentiones uphmxpo

[amitkrout]

It seems the service is not created properly because the SATATUS is still showing Provisioning. Then immediately in the very next step deletion of service is invoked and causes the failure

Running odo with args [odo delete -f dh-postgresql-apb --app uphmxpo --project ajdvqxarqf]
[odo]  ✗  Component dh-postgresql-apb does not exist in application uphmxpo

what is odd is that the delete command uses ajdvqxarqf but the error message mentiones uphmxpo

uphmxpo is the app name

[kadel]

uphmxpo is the app name

oh, my bad 😇, sorry

but still, the deletion should work regardless of the status

[amitkrout]

but still, the deletion should work regardless of the status

Ideally it should, but no idea why its not working. IMO the deletion code need some amount of investigation. I will create a separate followup issue.

[kadel]

This is blocking all PR right now :-(

[kadel]

▶ odo service list --app uphmxpo --project ajdvqxarqf
NAME                  TYPE                  STATUS
dh-postgresql-apb     dh-postgresql-apb     ProvisionedAndBound

github.com/openshift/odo  master ✔                                                                                                                                                                 18h10m
▶ odo delete -f dh-postgresql-apb --app uphmxpo --project ajdvqxarqf
 ✗  Component dh-postgresql-apb does not exist in application uphmxpo

[kadel]

LOL, the test is not deleting service, but it is trying to delete component !!!
How the hell this got in?

[kadel]

This got in with #2001

[amitkrout]

This got in with #2001

Ahh

[kadel]

@amitkrout Can you please look into how this happened. It looks like that travis tests are not set as required :-(

[amitkrout]

This is very unfortunate. Will make it required then. cc_ @mohammedzee1000

[amitkrout]

@amitkrout Can you please look into how this happened. It looks like that travis tests are not set as required :-(

I am surprise, why i was not ping by the bot in this pr

[mohammedzee1000]

Travis was removed from required due to incompatibility with prow. I will see if this was sorted. Otherwise, we cant make it as required.
I would say until then, make sure to exercise caution with satisfying the merge conditions

[amitkrout]

Travis was removed from required due to incompatibility with prow. I will see if this was sorted. Otherwise, we cant make it as required.

As per platform team they don't expect to support Travis soon

[amitkrout]

Travis was removed from required due to incompatibility with prow. I will see if this was sorted. Otherwise, we cant make it as required.

As per platform team they don't expect to support Travis soon
As per the slack thread discussion there is no ETA for travis support. So reducing the priority cc_

@kadel
/priority medium

[amitkrout]

Travis was removed from required due to incompatibility with prow. I will see if this was sorted. Otherwise, we cant make it as required.
I would say until then, make sure to exercise caution with satisfying the merge conditions

@girishramnani @dharmit @cdrage @mik-dass ^

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[kadel]

/remove-lifecycle rotten

This is starting to cause a lot of problems :-(

We can create a temporary workaround.
Let's create a job on OpenShift CI, that will check Travis-CI status for given PR.
The job should have all information that is needed (PR number, repo). It should be just one simple API call to github status API endpoint.

/priority high
/remove-priority medium

[amitkrout]

/remove-lifecycle rotten

This is starting to cause a lot of problems :-(

We can create a temporary workaround.
Let's create a job on OpenShift CI, that will check Travis-CI status for given PR.
The job should have all information that is needed (PR number, repo). It should be just one simple API call to github status API endpoint.

/priority high
/remove-priority medium

To get the travis login, status, PR number we need shared secret and ofcourse env variable which is not supported by the template based test. @prietyc123 is working on the migration part in #3270 which supports secret sharing and exposing of environment variable that we are looking for.

So keeping it as For consideration

[kadel]

Blocked on #3270

/status blocked

[kadel]

/remove-status blocked
#3270 was merged

[amitkrout]

Unfortunately GitHub checks (the new GH test API) are not visible to Prow, so the merge bot cannot be
configured to require them for merging. I think it should be possible to configure that the check is needed for 
merges in branch protection somehow, then the bot will not be able to merge unless that check is green.

I tried it. Seems it's working but there are consistent /retest trigger is happening. AFAIU /retest will only rerun Prow's failed jobs, so there are no actual unnecessary retests

[kadel]

@amitkrout not sure I understand :-(
So if you configure branch protection for the master branch to make Travis-CI tests required it works or Prow ends in endless /retest loop?

[amitkrout]

@amitkrout not sure I understand :-(
So if you configure branch protection for the master branch to make Travis-CI tests required it works. Prow ends in endless /retest loop?

yes, is works. Prow sense the travis failure but prow goes into endless /retest loop even after the successful prow CI run. Prow bot merges the pr only when travis job goes green after a job restart.