Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

E2e Test Fixes #25058

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

E2e Test Fixes #25058

wants to merge 11 commits into from

Conversation

kdumontnu
Copy link
Contributor

@kdumontnu kdumontnu commented Jun 2, 2023 •
edited
Loading

  • Run e2e tests in series so that DB + fixtures can reinitialize between each test
  • Run only a single test each test loop iteration 🤦‍♂️
  • Reenable firefox test (Closes Firefox E2E tests are failing with "Page closed" #21355)
  • Use locator instead of query for playwright functions

To Do:

  • Add more tests?
  • Bump playwright and review test setup

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jun 2, 2023
@pull-request-size pull-request-size bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 2, 2023
@silverwind
Copy link
Member

review test setup

should check if Firefox can be re-enabled or if it's still failing.

delvh
delvh reviewed Jun 2, 2023
View reviewed changes
tests/e2e/README.md Outdated Show resolved Hide resolved
@kdumontnu @delvh
Update tests/e2e/README.md
7182dc9 
Co-authored-by: delvh <dev.lh@web.de>
@pull-request-size pull-request-size bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 5, 2023
@kdumontnu
Copy link
Contributor Author

review test setup

should check if Firefox can be re-enabled or if it's still failing.

I've reenabled. I've been running it on my fork for a while and haven't seen it fail yet. 👍

As far as other tests go, do you have any recommendations for pieces of the front-end we know are fragile or we've run into issues repeatedly?

@kdumontnu kdumontnu marked this pull request as ready for review June 5, 2023 21:29
@kdumontnu kdumontnu changed the title Remove DISPLAY env setting and add debug option E2e Test Fixes Jun 5, 2023
@silverwind
Copy link
Member

silverwind commented Jun 8, 2023 •
edited
Loading

As far as other tests go, do you have any recommendations for pieces of the front-end we know are fragile or we've run into issues repeatedly?

Nothing really comes to mind, but if you are looking for something to test, I'd say the PR flow is something that would be nice to have, e.g.

  • Create branch
  • Edit a file in Monaco, commit it
  • Create PR
  • Review the PR on a second account
  • Merge the PR
  • Delete the branch

Part of this flow is already covered by backend integration tests, but it should really be moved to E2E instead and out of backend integration tests.

@silverwind
Copy link
Member

As far as other tests go, do you have any recommendations for pieces of the front-end we know are fragile or we've run into issues repeatedly?

Nothing really comes to mind, but if you are looking for something to test, I'd say the PR flow is something that would be nice to have, e.g.

* Create branch

* Edit a file in Monaco, commit it

* Create PR

* Review the PR on a second account

* Merge the PR

* Delete the branch

Part of this flow is already covered by backend integration tests, but it should really be moved to E2E instead and out of backend integration tests.

Maybe also start simpler:

  • Open issue
  • Comment on it
  • Close it

@kdumontnu kdumontnu requested a review from silverwind July 7, 2023 18:57
silverwind
silverwind reviewed Jul 7, 2023
View reviewed changes
tests/e2e/e2e_test.go
fmt.Printf("%v", stderr.String())
log.Fatal("Playwright Failed: %s", err)
} else {
fmt.Printf("%v", stdout.String())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surposed ths fmt.Printf passes the linter.

silverwind
silverwind reviewed Jul 7, 2023
View reviewed changes
tests/e2e/issue.test.e2e.js
@@ -0,0 +1,56 @@
// @ts-check
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this for?

silverwind
silverwind reviewed Jul 7, 2023
View reviewed changes
tests/e2e/utils_e2e.js
@@ -53,7 +47,7 @@ export async function save_visual(page) {
timeout: 20000,
mask: [
page.locator('.dashboard-navbar span>img.ui.avatar'),
page.locator('.ui.dropdown.jump.item span>img.ui.avatar'),
page.locator('.ui.dropdown.jump.item.tooltip span>img.ui.avatar'),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No better selector available? Maybe add a class name? This will break too easily.

silverwind
silverwind reviewed Jul 7, 2023
View reviewed changes
tests/e2e/example.test.e2e.js
await page.waitForLoadState('networkidle');
await page.locator('input#user_name').fill('user2');
await page.locator('input#password').fill('password');
await page.locator('form button.ui.green.button:visible').click();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also a rather suboptimal selector. Afraid we have to add some class names to make tests more robust against class name changes.

silverwind
silverwind reviewed Jul 7, 2023
View reviewed changes
tests/e2e/example.test.e2e.js
await page.locator('input#email').fill(`e2e-test-${workerInfo.workerIndex}@test.com`);
await page.locator('input#password').fill('test123');
await page.locator('input#retype').fill('test123');
await page.locator('form button.ui.green.button:visible').click();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@delvh delvh delvh left review comments

@silverwind silverwind silverwind left review comments

Assignees
No one assigned
Labels
lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Firefox E2E tests are failing with "Page closed"
4 participants
@kdumontnu @silverwind @delvh @GiteaBot