Add E2E test to create pipeline run #2565

marniks7 · 2022-11-03T02:52:01Z

Changes

Add e2e test to create pipeline run that can be executed from any environment.
This is preparation for tests for workspaces and for yaml editor

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Docs included if any changes are user facing
Tests included if any functionality added or changed
Follows the commit message standard
Meets the Tekton contributor standards (including
functionality, content, code)
Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
Release notes block below has been updated with any user facing changes (new features, significant UI changes, API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
Release notes contains the string "action required" if the change requires additional action from users switching to the new release

tekton-robot · 2022-11-03T02:52:10Z

Hi @marniks7. Thanks for your PR.

I'm waiting for a tektoncd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

marniks7 · 2022-11-03T02:52:14Z

/kind misc

AlanGreene

Thanks for the PR @marniks7. Overall it looks good, we just need to avoid doing anything platform specific where possible. Let me know if you have any questions.

packages/e2e/Dockerfile

packages/e2e/cypress/e2e/run/create-pipelinerun.cy.js

test/e2e-tests.sh

AlanGreene · 2022-11-03T14:31:56Z

packages/e2e/cypress/e2e/run/create-pipelinerun.cy.js

+    cy.contains('button', 'Create').click();
+    cy.contains('simple-pipeline').first().click();
+
+    cy.get('header[class="tkn--pipeline-run-header"]', { timeout: 10000 })


I'm curious where the timeout values here came from, was it from experimentation?
We'll need to keep an eye on these to ensure we don't end up with a flaky test.

It was experimentation over many runs (but I have done that for pipeline with workspaces)

reworked

remove timeout for header (let's assume my system has been hanging)

decrease timeout for status from 15s to 10s

add retry = 3
https://cypress.io/blog/2020/12/03/retry-rerun-repeat/

Sorry I should have been clearer, my comment was just pointing out that we'll need to be aware of this and account for CI potentially being slower than a local cluster, not requesting a change in strategy.

I would prefer to increase the timeout for the status and remove the retries. Especially in CI where we create a new cluster and registry it's likely to take longer to run the first time due to pulling the image used in the pipeline. Subsequent attempts will most likely be faster as the image has already been pulled which is why the retries succeed. Using retries to accommodate this also has the side effect of creating multiple runs which we don't really need.

We should also consider cleanup after a successful run and delete it but that can be done later.

Please also squash your commits when done so this is ready to merge.

Ok,

timeout increased to 15s for status wait

on my pc and CI it takes 6s for everything (but image is pulled before that because image: busybox is used in static resources and those tests)

retries removed

@AlanGreene, note, I have changed permission for ~/.kube/config file to 644 otherwise permission denied on CI. I don't like that much for local runs, because it change permissions for common user kubeconfig file
Other option:

copy ~/.kube/config file to ~/.kube/tekton, change permissions for ~/.kube/tekton, add trap before docker run and remove file ~/.kube/tekton in the trap

packages/e2e/cypress/e2e/run/create-pipelinerun.cy.js

AlanGreene · 2022-11-04T12:38:50Z

/ok-to-test

Add e2e test to create pipeline run that can be executed from any environment

marniks7 · 2022-11-05T02:30:27Z

/test tekton-dashboard-unit-tests

tekton-robot · 2022-11-07T11:43:56Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AlanGreene

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [AlanGreene]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

AlanGreene · 2022-11-07T11:45:48Z

Thanks @marniks7
/lgtm

tekton-robot requested review from briangleeson and LyndseyBu November 3, 2022 02:52

tekton-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Nov 3, 2022

tekton-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Nov 3, 2022

tekton-robot added kind/misc Categorizes issue or PR as a miscellaneuous one. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 3, 2022

marniks7 force-pushed the e2e-test branch from e2c5eb8 to a587bb3 Compare November 3, 2022 03:04

marniks7 force-pushed the e2e-test branch from e2c5eb8 to a587bb3 Compare November 3, 2022 03:06

tekton-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 3, 2022

marniks7 force-pushed the e2e-test branch from a587bb3 to 9ba463e Compare November 3, 2022 03:17

tekton-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 3, 2022

marniks7 force-pushed the e2e-test branch from 9ba463e to 44767b4 Compare November 3, 2022 03:52

AlanGreene requested changes Nov 3, 2022

View reviewed changes

AlanGreene reviewed Nov 4, 2022

View reviewed changes

packages/e2e/cypress/e2e/run/create-pipelinerun.cy.js Outdated Show resolved Hide resolved

tekton-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 4, 2022

AlanGreene removed request for briangleeson and LyndseyBu November 4, 2022 12:39

marniks7 force-pushed the e2e-test branch from 4de573e to b8566ef Compare November 5, 2022 00:07

Add e2e test to create pipeline run

b436b9c

Add e2e test to create pipeline run that can be executed from any environment

marniks7 force-pushed the e2e-test branch from 8f42f20 to b436b9c Compare November 5, 2022 02:09

AlanGreene approved these changes Nov 7, 2022

View reviewed changes

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 7, 2022

tekton-robot assigned AlanGreene Nov 7, 2022

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 7, 2022

tekton-robot merged commit 8b7ccf2 into tektoncd:main Nov 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add E2E test to create pipeline run #2565

Add E2E test to create pipeline run #2565

marniks7 commented Nov 3, 2022 •

edited by AlanGreene

Loading

tekton-robot commented Nov 3, 2022

marniks7 commented Nov 3, 2022

AlanGreene left a comment

AlanGreene Nov 3, 2022

marniks7 Nov 4, 2022 •

edited

Loading

AlanGreene Nov 4, 2022

AlanGreene Nov 4, 2022

marniks7 Nov 5, 2022 •

edited

Loading

AlanGreene commented Nov 4, 2022

marniks7 commented Nov 5, 2022

tekton-robot commented Nov 7, 2022

AlanGreene commented Nov 7, 2022

Add E2E test to create pipeline run #2565

Add E2E test to create pipeline run #2565

Conversation

marniks7 commented Nov 3, 2022 • edited by AlanGreene Loading

Changes

Submitter Checklist

tekton-robot commented Nov 3, 2022

marniks7 commented Nov 3, 2022

AlanGreene left a comment

Choose a reason for hiding this comment

AlanGreene Nov 3, 2022

Choose a reason for hiding this comment

marniks7 Nov 4, 2022 • edited Loading

Choose a reason for hiding this comment

AlanGreene Nov 4, 2022

Choose a reason for hiding this comment

AlanGreene Nov 4, 2022

Choose a reason for hiding this comment

marniks7 Nov 5, 2022 • edited Loading

Choose a reason for hiding this comment

AlanGreene commented Nov 4, 2022

marniks7 commented Nov 5, 2022

tekton-robot commented Nov 7, 2022

AlanGreene commented Nov 7, 2022

marniks7 commented Nov 3, 2022 •

edited by AlanGreene

Loading

marniks7 Nov 4, 2022 •

edited

Loading

marniks7 Nov 5, 2022 •

edited

Loading