Replace assertions in Catalog-e2e to use Gomega's Matcher library #1502

harishsurf · 2020-05-06T14:17:36Z

Description of the change:
This PR replaces testify/require assertions with Gomega's assertions. This PR will be followed up with another PR that uses ginkgo's Container blocks to ensure BDD style structuring

Motivation for the change:

Reviewer Checklist

Implementation matches the proposed design, or proposal is updated to match implementation
Sufficient unit test coverage
Sufficient end-to-end test coverage
Docs updated or added to /docs
Commit messages sensible and descriptive

test/e2e/catalog_e2e_test.go

test/e2e/e2e_test.go

test/e2e/catalog_e2e_test.go

harishsurf · 2020-06-05T13:23:20Z

/retest

Bowenislandsong

small nit, not a necessary change. /lgtm

test/e2e/catalog_e2e_test.go

harishsurf · 2020-06-23T14:41:57Z

/retest

test/e2e/catalog_e2e_test.go

harishsurf · 2020-08-12T18:06:47Z

/test e2e-aws-olm

harishsurf · 2020-08-13T16:41:40Z

/retest

benluddy

/approve

benluddy · 2020-08-21T13:10:59Z

test/e2e/catalog_e2e_test.go


 // Ensure the timing
 catalogConnState := fetchedUpdatedCatalog.Status.GRPCConnectionState
 subUpdatedTime := subscription.Status.LastUpdated
 timeLapse := subUpdatedTime.Sub(catalogConnState.LastConnectTime.Time).Seconds()
- require.True(GinkgoT(), timeLapse < 60)
+ Expect(timeLapse).Should(BeNumerically("<", 60))


Non-blocking nit: There's a built-in Gomega matcher for time.Time: https://pkg.go.dev/github.com/onsi/gomega?tab=doc#BeTemporally.

But, I don't see if it could be used here since we are asserting that the var timeLapse has a value < 60. BeTemporarily expects 60 to be of type time.Time{}

I think Ben is suggesting something like Expect(catalogConnState.LastConnectTime.Time).Should(BeTemporally("<", subUpdatedTime.Add(60 * time.Second) @harishsurf

changed it to:
Expect(subUpdatedTime.Time).Should(BeTemporally("<", catalogConnState.LastConnectTime.Add(60*time.Second)))

benluddy · 2020-08-21T13:14:50Z

test/e2e/catalog_e2e_test.go

- if err != nil {
- return err
- }
+ Eventually(waitForScaleup, 5*time.Minute, 1*time.Second).Should(BeTrue())


Aren't 5 minutes and 1 second the defaults?

The default timeout is 1min

openshift-ci-robot · 2020-08-21T13:16:02Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: benluddy, harishsurf

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [benluddy]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

harishsurf · 2020-08-27T12:54:51Z

/retest

harishsurf · 2020-08-28T14:21:21Z

/retest

harishsurf · 2020-08-31T14:50:31Z

/retest

openshift-ci-robot · 2020-09-09T20:08:38Z

@harishsurf: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/e2e-upgrade	`dcf9d6d`	link	`/test e2e-upgrade`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

harishsurf · 2020-10-27T19:43:47Z

/test e2e-upgrade

of-deploy-bot · 2020-10-27T22:40:42Z

This PR failed 1 out of 1 times with 4 individual failed tests and 4 skipped tests. A test is considered flaky if failed on multiple commits.

totaltestcount: 1
failedtestcount: 1
flaketestcount: 4
skippedtestcount: 4
flaketests:

classname: End-to-end
name: 'Subscription creation using existing CSV'
counts: 3
details:

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/subscription_e2e_test.go:90

	Error Trace:	subscription_e2e_test.go:97
	            				runner.go:113
	            				runner.go:64
	            				it_node.go:26
	            				spec.go:215
	            				spec.go:138
	            				spec_runner.go:200
	            				spec_runner.go:170
	            				spec_runner.go:66
	            				suite.go:62
	            				ginkgo_dsl.go:226
	            				ginkgo_dsl.go:214
	            				e2e_test.go:54
	Error:      	Received unexpected error:
	            	E2E bug detected: configmaps "mock-ocs" already exists

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/github.com/stretchr/testify/require/require.go:1005

meandurationsec: 0.03929166666666666

classname: End-to-end
name: 'Install Plan with CRD schema change Test existing version is present
in new CRD (deprecated field)'
counts: 1
details:
- count: 1
  error: "\n\n/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:43\n\n\tError
  Trace:\tinstallplan_e2e_test.go:769\n\t \t\t\t\tvalue.go:460\n\t
  \ \t\t\t\tvalue.go:321\n\t \t\t\t\ttable_entry.go:37\n\t
  \ \t\t\t\trunner.go:113\n\t \t\t\t\trunner.go:64\n\t
  \ \t\t\t\tit_node.go:26\n\t \t\t\t\tspec.go:215\n\t
  \ \t\t\t\tspec.go:138\n\t \t\t\t\tspec_runner.go:200\n\t
  \ \t\t\t\tspec_runner.go:170\n\t \t\t\t\tspec_runner.go:66\n\t
  \ \t\t\t\tsuite.go:62\n\t \t\t\t\tginkgo_dsl.go:226\n\t
  \ \t\t\t\tginkgo_dsl.go:214\n\t \t\t\t\te2e_test.go:54\n\tError:
  \ \tNot equal: \n\t \texpected: "Complete"\n\t \tactual
  \ : "Failed"\n\t \t\n\t \tDiff:\n\t \t---
  Expected\n\t \t+++ Actual\n\t \t@@ -1,2 +1,2 @@\n\t
  \ \t-(v1alpha1.InstallPlanPhase) (len=8) "Complete"\n\t \t+(v1alpha1.InstallPlanPhase)
  (len=6) "Failed"\n\t \t \n\n/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/github.com/stretchr/testify/require/require.go:162"
  meandurationsec: 22.057747

classname: End-to-end
name: 'Operator when a subscription to a package exists should automatically
adopt components'
counts: 1
details:

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/operator_test.go:209
Timed out after 60.000s.
Expected
    <v1alpha1.SubscriptionState>: UpgradePending
to be equivalent to
    <string>: AtLatestKnown
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/operator_test.go:259

meandurationsec: 60.324765

classname: End-to-end
name: 'Installing bundles with new object types when a bundle with a pdb, priorityclass,
and VPA object is installed should create the additional bundle objects'
counts: 1
details:

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/bundle_e2e_test.go:124
Timed out after 60.000s.
expected no error getting pdb object associated with CSV
Expected success, but got an error:
    <*errors.StatusError | 0xc000b95b80>: {
        ErrStatus: {
            TypeMeta: {Kind: "", APIVersion: ""},
            ListMeta: {
                SelfLink: "",
                ResourceVersion: "",
                Continue: "",
                RemainingItemCount: nil,
            },
            Status: "Failure",
            Message: "poddisruptionbudgets.policy \"busybox-pdb\" not found",
            Reason: "NotFound",
            Details: {
                Name: "busybox-pdb",
                Group: "policy",
                Kind: "poddisruptionbudgets",
                UID: "",
                Causes: nil,
                RetryAfterSeconds: 0,
            },
            Code: 404,
        },
    }
    poddisruptionbudgets.policy "busybox-pdb" not found
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/bundle_e2e_test.go:144

meandurationsec: 77.473194
skippedtests:

classname: End-to-end
name: 'Subscriptions create required objects from Catalogs Given a Namespace
when a CatalogSource is created with a bundle that contains prometheus objects
creating a subscription using the CatalogSource should install the operator
successfully'
counts: 1
details: []
meandurationsec: 15.100081
classname: End-to-end
name: 'Subscription updates existing install plan'
counts: 1
details: []
meandurationsec: 0.705988
classname: End-to-end
name: 'Catalog represents a store of bundles which OLM can use to install Operators
image update'
counts: 1
details: []
meandurationsec: 0.295481
classname: End-to-end
name: 'Subscriptions create required objects from Catalogs Given a Namespace
when a CatalogSource is created with a bundle that contains prometheus objects
creating a subscription using the CatalogSource should have created the expected
prometheus objects'
counts: 1
details: []
meandurationsec: 15.286522

exdx · 2020-10-28T20:01:01Z

/lgtm

openshift-bot · 2020-10-28T22:01:52Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci-robot requested review from gallettilance and kevinrizza May 6, 2020 14:17

harishsurf changed the title ~~Replace assertions to use Gomega's Matcher library~~ Replace assertions in Catalog-e2e to use Gomega's Matcher library May 7, 2020

openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 14, 2020

harishsurf force-pushed the catalog-e2e branch from 3af91af to 01fe4bb Compare May 21, 2020 17:36

openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 21, 2020

benluddy reviewed May 29, 2020

View reviewed changes

harishsurf force-pushed the catalog-e2e branch 2 times, most recently from 8d5c0b2 to dd6bc94 Compare June 4, 2020 20:08

Bowenislandsong reviewed Jun 19, 2020

View reviewed changes

test/e2e/catalog_e2e_test.go Outdated Show resolved Hide resolved

harishsurf force-pushed the catalog-e2e branch from dd6bc94 to 0d48886 Compare June 22, 2020 14:01

benluddy reviewed Jun 24, 2020

View reviewed changes

test/e2e/catalog_e2e_test.go Outdated Show resolved Hide resolved

test/e2e/catalog_e2e_test.go Outdated Show resolved Hide resolved

test/e2e/catalog_e2e_test.go Outdated Show resolved Hide resolved

harishsurf force-pushed the catalog-e2e branch from 0d48886 to cfff446 Compare July 6, 2020 15:40

harishsurf force-pushed the catalog-e2e branch from cfff446 to 38ccdb2 Compare August 12, 2020 15:04

benluddy reviewed Aug 21, 2020

View reviewed changes

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 21, 2020

harishsurf force-pushed the catalog-e2e branch from 38ccdb2 to dcf9d6d Compare August 26, 2020 17:51

exdx self-assigned this Aug 31, 2020

Replace assertions to use Gomega's Matcher library

c09facf

harishsurf force-pushed the catalog-e2e branch from dcf9d6d to c09facf Compare October 27, 2020 21:46

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 28, 2020

openshift-merge-robot merged commit 8e5ad4d into operator-framework:master Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace assertions in Catalog-e2e to use Gomega's Matcher library #1502

Replace assertions in Catalog-e2e to use Gomega's Matcher library #1502

harishsurf commented May 6, 2020

harishsurf commented Jun 5, 2020

Bowenislandsong left a comment

harishsurf commented Jun 23, 2020

harishsurf commented Aug 12, 2020

harishsurf commented Aug 13, 2020

benluddy left a comment

benluddy Aug 21, 2020

harishsurf Aug 25, 2020

exdx Aug 25, 2020

harishsurf Aug 26, 2020

benluddy Aug 21, 2020

harishsurf Aug 25, 2020

openshift-ci-robot commented Aug 21, 2020

harishsurf commented Aug 27, 2020

harishsurf commented Aug 28, 2020

harishsurf commented Aug 31, 2020

openshift-ci-robot commented Sep 9, 2020

harishsurf commented Oct 27, 2020

of-deploy-bot commented Oct 27, 2020

exdx commented Oct 28, 2020

openshift-bot commented Oct 28, 2020

Replace assertions in Catalog-e2e to use Gomega's Matcher library #1502

Replace assertions in Catalog-e2e to use Gomega's Matcher library #1502

Conversation

harishsurf commented May 6, 2020

harishsurf commented Jun 5, 2020

Bowenislandsong left a comment

Choose a reason for hiding this comment

harishsurf commented Jun 23, 2020

harishsurf commented Aug 12, 2020

harishsurf commented Aug 13, 2020

benluddy left a comment

Choose a reason for hiding this comment

benluddy Aug 21, 2020

Choose a reason for hiding this comment

harishsurf Aug 25, 2020

Choose a reason for hiding this comment

exdx Aug 25, 2020

Choose a reason for hiding this comment

harishsurf Aug 26, 2020

Choose a reason for hiding this comment

benluddy Aug 21, 2020

Choose a reason for hiding this comment

harishsurf Aug 25, 2020

Choose a reason for hiding this comment

openshift-ci-robot commented Aug 21, 2020

harishsurf commented Aug 27, 2020

harishsurf commented Aug 28, 2020

harishsurf commented Aug 31, 2020

openshift-ci-robot commented Sep 9, 2020

harishsurf commented Oct 27, 2020

of-deploy-bot commented Oct 27, 2020

exdx commented Oct 28, 2020

openshift-bot commented Oct 28, 2020