Convert end-to-end tests to Playwright #6976

jmcameron · 2023-03-15T19:36:26Z

Because Protractor is now obsolete for end-to-end tests, we need to convert the end-to-end tests to some other framework.
See Issue #6669 for details. Based on the feedback in that issue and some experimentation, I think we should try to convert the end-to-end tests to Playwright.

Trying the experimental version of the End-to-End tests with Playwright

To try the current playwright end-to-end tests, look at my branch: https://github.com/jmcameron/bhima/tree/playwright-test.
Update ('yarn')
You also need to install playwright via npm:
- npm install playwright
- Verify the two versions match:
  - grep '@playwright/test@' yarn.lock
  - npx playwright --version
Use the command: yarn test:e2e

Eslint update

This PR includes an optional update to support eslint-ing jsdoc blocks. We encourage you to add JSDoc blocks above all your function definitions in javascript files.
To enable it on your local installs and via VS Code, do this:
- cd client; ln -s ../.eslintrc.local.json .eslintrc.json; cd ..
- cd server; ln -s ../.eslintrc.local.json .eslintrc.json; cd ..
- cd test; ln -s ../.eslintrc.local.json .eslintrc.json; cd ..
The symbolic links created here are ignored by git so that commits will not save them. Therefore the using the ESlint extensions for JSDoc will only occur in your local setup, not on the master in github.
The JSDOc eslint extensions are in the file: .eslintrc.local.json
Note that these JSDoc extensions only apply to javascript files under the three directories (client, server, and test). Therefore they do not apply to top-level files, files within the sh or utiliities directories.
This work-around is because hound does not work well with the JSDoc extensions for eslint.

Log of updates (open to see list)

2023-08-23 Playwright end-to-end tests working in CI/semaphore
2023-08-17 Got most tests working
- Split new playwright end-to-end tests to stock tests and non-stock tests
  - 'npx playwright install`
- npm run test:e2e
- npm run test:e2e-stock
- Non-stock tests: 463 passed (26 skipped, none failed, 13.0min, 1 workers - NOT parallel)
- Stock tests: 81 passed (2 skipped, none failed, 2.6min, 1 workers - NOT parallel)
2023-08-03 Completed migration
- Completed migrating stock tests and remaining components
- Completed 100% of 144 migration tasks
- 546 passed (16 skipped, 10 failed, 15.0min, 1 workers - NOT parallel)
2023-07-21 PM Migrated journal tests
- Completed 135 of 144 (including 98% of test directories, 64/65)
- 466 passed (15 skipped, 8 failed, 12.3min, 1 workers - NOT parallel)
2023-07-21 Migrated reports tests
- Migrated 'report' tests (11 subdirectories, 23 files)
  - Note: Most stock exit reports skipped due to loading problem
- Migrated components: bhCronSelect, bhEntityGroupSelect, bhMultipleCashBoxSelect, bhPeriodSelection, bhReportPeriodSelect, bhYesNoRadio
- Completed 134 of 144 (including 97% of test directories, 63/65)
- 454 passed (15 skipped, 8 failed, 11.8min, 1 workers - NOT parallel)
2023-07-19 Migrated more tests
- Migrated tests: costCenter, user (6 files)
- Migrated components: bhAllocationBasisSelect (new), bhFiscalPeriodSelect, bhPercentageInput, bhServicesMultipleSelect
- Note: Although the costCenter tests were migrated, most are skipped due to a problem during center creation
- Completed 126 of 144 (including 94% of test directories, 61/65)
- 417 passed (12 skipped, 3 failed, 10.4min, 1 workers - NOT parallel)
2023-07-18 Migrated more tests
- Migrated tests: visits, vouchers, ward (with 3 subdirectories)
- Migrated components: bhCashBoxSelect, bhRoomSelect, bhWardSelect
- Completed 118 of 142 (including 87% of test directories, 56/64)
- 399 passed (5 skipped, 5 failed, 10.7min, 1 workers - NOT parallel)
2023-07-14 Migrated more tests
- Migrated tests: staffingIndice, subsidies, suppliers, surveyForm, transactionType, verificationLinks
- Migrated components: bhDataCollector, bhSurveyFormTypeSelect, bhSurveyListSelect
- Completed 112 of 142 (including 83% of test directories, 53/64)
- 361 passed (3 skipped, 5 failed, 8.7min, 1 workers - NOT parallel)
2023-07-13 Migrated more tests
- Migrated purchases, rubrics, rubricsConfig, services tests
- Migrated bhAddItem, bhProjectSelect, bhPurchaseStatusSelect components
- Created new bhCostCenterSelect component
- Completed 103 of 142 (including 72% of test directories, 46/64)
- 327 passed (3 skipped, 2 failed, 2.5min, 8 workers)
- Note: I ran the test with 1 worker (not in parallel). All tests pass but requires 8 minutes.
2023-07-12 Migrated more tests
- Migrated payrollConfig, payrollProcess, pinceList tests
- Migrated components: bhAccountConfigSelect, bhEmployeeConfigSelect, bhIprConfigSelect,
  bhPayrollPeriodSelect, bhPayrollStatusSelect, bhRubricConfigSelect, and bhWeekCofnigSelect
- Completed 94 of 141 (including 64% of test directories, 41/64)
- Fixed a couple of problems when running all tests in parallel
- 299 passed (2 skipped, 1 failed, 2.0min, 8 workers)
2023-07-10 Migrated more test
- Migrated iprTaxes, iprTaxesConfig, tags, weekendConfig and component bhIprScale
- Completed 84 of 141 (including 61%, 39/64 test directories)
- 282 passed (4 skipped, 4 failed, 2.0min, 8 workers)
2023-07-07 Migrate more tests
- migrated navigation, invoicingFees tests
  - Hand problems on one test in invoicingFees and it is currently skipped
- Implemented GridAction utility
- Completed 78 of 141 (including 53% of test directories)
- 255 passed (2 skipped, 6 failed, 1.9min, 8 workers)
2023-07-05 Migrate more tests
- Migrated indicators_files and inventory tests
- Migrated 5 components
- Modified some tests not to fail when run in full parallel mode
- Completed 75 of 141 (including 50% of test directories)
- 252 passed (1 skipped, 4 failed, 1.9min, 8 workers)
2023-07-03 Updates
- update dependencies
- Modified some tests not to fail when run in full parallel mode
- Completed 68 of 141 (including 47% of test directories)
- 235 passed (1 skipped, 1.7min, approx 2.3 secs/test, 8 workers)
2023-06-30 Migrated more tests
- Migrated holidays and component bhEmployeeSelect
- Completed 68 of 141 (including 47% of test directories)
- 233 passed (1 skipped, 2 failed, 1.9min)
- Failed test are due to interference between related tests in different directories.
  These tests all work when the directory containing them is run by itself.
2023-06-29 Migrated more tests
- Migrated fiscalYears, functions, grades and component bhSurveyFormSelect
- Completed 66 of 141 (including 44% of test directories)
- 227 passed (1 skipped, 3 failed, 1.9min)
2023-06-28 Migrated more tests and components
- Migrated employees tests and bhFonctionSelect + bhGradeSelect components
- Completed 61 of 141 (including 39% of test directories)
- 208 passed (1 skipped, 4 failed, 1.8min)
- Most failures were in the patients and employee tests. These tests need to be refactored
  in order to avoid conflicts when the tests are executed in different orders. Fixed a few tests
  by allowing several different counts.
2023-06-14 Migrated more tests
- Migrated f_employee_config tests and bhCheckTree.js component
- Completed 58 of 141 (including 39% of test directories)
- 200 passed (1 skipped) in 1.4 minutes (approx 2.4 secs/test, 8 workers, 2nd try)
2023-06-13 Migrated more tests
- Migrated tests for dataCollectionManagement and edit_journal
- Completed 56 of 141 (including 34% of test directories)
- 193 passed (1 skipped) in 1.4 minutes (approx 2.3 secs/test, 8 workers, 2nd try)
2023-06-08 Migrated another test
- Migrated tests for creditor_groups
- Completed 54 of 141 (including 36% of test directories)
- 182 passed (1 skipped) in 1.3 minutes (approx 2.3 secs/test, 8 workers)
2023-06-07 Migrated more tests
- Migrated tests for choiceListManagement, configurationAnalysisTools
- Migrated components bhAccountReferenceSelect, beAnalysisToolTypeSelect
- Completed 53 of 141 (including 33% of test directories)
- 178 passed (1 skipped) in 1.2 minutes (approx 2.5 secs/test, 8 workers)
- Tests are still a bit flakey. Getting all the tests to pass required 3 tries.
2023-06-06 Migrated more tests
- Migrated tests for exchange, errors
- Completed 49 of 141 (including 30% of test directories)
- 168 passed (1 skipped) in 1.2 minutes (approx 2.3 secs/test, 8 workers)
2023-06-05 Minor updates
- Minor updates to keep up with dependencies in master
- Changed the default time that NotifyService success/failure messages get shown from 3 seconds to 5 seconds to
  reduce the number timeout failures that the Playwright end-to-end tests are experiencing. It seems to help but
  has not eliminated all random failures
- Completed 47 of 141 (including 29% of test directories)
- 164 passed (1 skipped) in 1.2 minutes (approx 2.3 secs/test, 8 workers)
2023-06-02 Various updates
- Minor updates pushed to PR on May 30, 31, and June 1
- Updated to master and pushed a change to the PR supporting using JSDoc with eslint locally (see previous section)
2023-04-21 Finished patient tests
- Pushed changes to PR that finished the 'patient' tests (10 files total)
- Impelmented 3 more components (ChoiceListSelect, DiagnosisSelect, PatientGroupSelect)
- Completed 46 of 141 (including 29% of test directories)
- 164 passed (1 skipped) in 48.6 seconds (2.1 secs/test, 8 workers)
2023-04-20 Keep up with master
- Pushed changes to the PR to keep up with bump updates to the master
- Continue working on patient tests (not pushed to PR yet)
2023-04-19 Worked on new tests (patient)
- Nothing released yet
2023-04-18 Submitted a PR enable the already-completed Playwright end-to-end tests
- Primarily for testing and to see how it works for others
- When I've fixed to some of the repeatability problems, we get it working with github / bors
2023-04-14 Implemented cashbox and debtors tests
- Completed 41 of 141 (including 27% of test directories)
- One old test is breaking occasionally, skipping it until I can fix it
- 121 passed (2 skipped) in 48.6 seconds (2.5 secs/test, 8 workers)
2023-04-13 Implemented more tests
- Implemented account-config and accounts tests
- Completed 39 of 141 (deleted sub-listings of some of tests)
- 110 passed (1 skipped) in 41.9 seconds (2.6 secs/test, 8 workers)
2023-04-12 Implemented more tests
- Got accountReferenceType tests and the rest of account-references tests working
- Got TestUtils.uiSelect() working for 'exact' and 'fullWord' option.
- Completed 44 of 148 migration tasks
- 93 passed (1 skipped) in 34.4 seconds (2.7 secs/test, 8 workers)
2023-04-11 Got most of account-references test working
- These tests were difficult to debug due to (probable) bugs in Playwright when executing javacript in the client when clicking on buttons.
- Completed 42 of 148 migration tasks
- 87 passed (1 skipped) in 34.6 seconds (2.5 secs/test, 8 workers)
2023-04-10 Fixed an issue with TU.locator() chaining
- Fixed a problem with TU.locator() that was preventing chaining with it. Went back and fixed all prior tests.
- Completed 42 of 148 migration tasks
- 80 passed (1 skipped) in 30.5 seconds (2.6 secs/test, 8 workers)
2023-04-07 Working on account-reference tests
- Implemented the bhAccountReferenceType component
- Completed 42 of 148 migration tasks
- 80 passed (1 skipped) in 30.7 seconds (2.6 secs/test, 8 workers)
2023-04-06 Continuing to work on 'cash' tests
- Fixed timeout issue with cash.spec.ts. Migrated cash/registry.spec.js
- Implemented a couple more components
- Completed 48 of 148 migration tasks
- 77 passed (1 skipped) in 27.9 seconds (2.76 secs/test, 8 workers)
2023-04-05 Continuing working on 'cash' tests
- Got cash.spec.js tests working when run alone. But some of them are failing when run with the full test suite. I need to debug this problem.
- Migrated 11 more components
- Completed 36 of 148 tasks
- Currently 69 tests
2023-04-04 Continuing working on 'cash' tests
- Updated branch with recent updates to master
- Migrated bhCurrencyInput and bhCurrencySelect components
- Completed 24 of 146 conversion tasks
2023-04-03 Started on 'cash' tests
- Updated branch with recent updates to master
2023-03-31 Got account_statement tests working
- Implemented most of the remaining functions in GridUtils
- 59 passed (1 skipped) in 24.3 seconds (2.43 secs/test)
2023-03-30 Added depots tests
- Added depots.spec.js (and depots.page.js) including a couple of tests that were skipped in the past
- Implemented bhDepotSelect
- 55 passed (1 skipped) in 23.3 seconds (2.36 secs/test)
2023-03-29 Worked to speed up tests
- Figured out how to log in only once in each test file. This speeds things up from approximately 1 test per second to nearly 3 tests per second (for 45 tests).
- Updated prot2play script to migrate files better (including changes reducing the number of logins, etc)
2023-03-28 Worked on the last 'locations' test
- Completed 19 of 146 (45 individual tests running)
- The tests seem to be getting a bit flaky -- Need to track that down. It may be that we are creating/editing locations in two different test files. May have to make sure the names are unique and be more specific in selectors to avoid spill-over between tests files.
2023-03-27 Started script to do partial automatic conversion to playwright (/sh/prot2play).
- Competed 17 of 146 tests
2023-03-24
- Added checklist for converting shared utilities
- Started writing up conversion guide (in this comment, see below)
- Completed 14 of 141 conversion tasks
2023-03-23 Added progress check list of end-to-end tests in comments below
2023-03-08 I have started converting our end-to-end tests to Playwright.

Progress converting end-to-end tests to Playwright (64, open to see list)

Progress converting end-to-end shared utilities to Playwright (open to see list)

Progress converting end-to-end components to Playwright (74, open to see list)

Protractor => Playwright Conversion Guide

See Playwright conversion guide for details on converting element(), by.css(), etc.
At the top of the test file, insert
const { test, expect } = require('@playwright/test');
The tests run in English. Therefore, it may be necessary to change some French labels and text used to find elements to English counterparts.

Conversions

Protractor	Playwright
`FU = require('../shared/FormUtils')`	`TU = require('../shared/TestUtils');`
`const { expect } = require('chai');`	`const { test, expect } = require('@playwright/test');`	1
`describe(...)`	`test.describe(...)`
`before(() => helpers.navigate(path));`	`test.beforeEach(async ({ page }) => {...});`	2
`it('description', async () => <fn>);`	`test('description', async ({ page }) => <fn>`);	3
`element(...)`	`await TU.locator(...)`	4
`await element(by...).click()`	`await (await page.locator(..)).click();`	5
`element(by.css('<css selector>'))`	`await TU.locator('<css selector>')`

Playwright assertions differ from 'chai' assertions (eg, ..to.be...). For details see how the Playwright assertions functions differ.
Regarding test.beforeEach() : Although the Protractor configuration logged the user into the test BHIMA server at the start of each test file, Playwright is not configured to do that. Therefore we need to do that in the test.beforeEach() call. A login() function has been added to TestUtils to make this simple. Note that test.beforeEach() block can be inside the test.describe() block or outside of it. Note that TU.registerPage(page); (2nd line) must be called once before any TestUtils functions that use the page object.

    test.beforeEach(async ({ page }) => {
      TU.registerPage(page);  // Important
      await TU.login();
      await TU.navigate(<desired starting page URL>);
    });

If your test does not use page argument, it can be omitted. Often the necessary operations can be done by using TestUtils functions without explicitly referencing the page object. The page object is saved in TestUtils when the registerPage(p) function is called and is used internally for most TestUtils functions.
See Playwright conversion guide for details on converting element(), by.css(), etc.
Migrating .all() requires some recoding. In Protractor, .all() can take an argument; not in Playwright. Also .all() returns a single protractor object that can contain multiple atomic locators (which has a .count() function to get it's length). But in Playwright, .all() is a function of a locator and returns an (possibly empty) array of atomic locator objects (which has a .length property to get the number of locators it contains).
Warning: If a page contains a grid and you use TU.navigate() to got that page in your test, you cannot always be sure the grid will be loaded immediately after the navigation. It is good to use TU.waitForSelector() for something in the grid to make sure the grid is actually loaded before continuing. This may not be necessary when running the tests for only one directory, but is often required when running all the tests in parallel.

Closes #6669

The text was updated successfully, but these errors were encountered:

jmcameron · 2023-03-15T19:56:36Z

I've done some initial experiments with Playwright and think that will work for our end-to-end regtests. I've converted 6 regtests and I'm in the process of converting others. There will be some difficulties:

When I started, I had some difficulties with operations on locators producing "Target Closed" errors. So I was using deprecated functions (eg, deprecated page.click(selector) instead of non-deprecated page.locator(selector).click()). I was able to get the non-deprecated calls to work. I thought there was some issue with Playwright handling our older version of AngularJS. However, later I was able to get the regular non-deprecated functions working so I think we are okay on that issue (not sure what I did differently).
There are many end-to-end tests and related shared functions for interacting with the website. I was not able to make all the shared functions to work exactly like the old functions, so will be necessary to modify all regtests as they are converted to the new framework (Playwright). This means there is a lot of work ahead of us to migrate the end-to-end tests.

There is good news about using Playwright:

Playwright seems to be popular and is being actively maintained and developed
It is pretty fast (so far it is faster than one test per second - on average)
It does not depend on webdriver/chromedriver.
It works with Firefox too (although I'm currently only trying it with Chrome).
Even though the tests are done headless, playwright has the ability to trace and replay operations and let you see (visually) what is happening in each step of the regtests.

7215: Migrate end-to-end tests to Playwright r=jmcameron a=jmcameron This PR enables end-to-end tests that have been migrated from Protractor to Playwright. Note that only 23% of the end-to-end have been converted, but it seems appropriate to start using the tests that have been migrated now to avoid any regressions that they might catch. Note that the old Protractor tests have not been deleted yet, since the obsolete files may still be useful in the migration process. Also the original command `yarn test:ends` are unchanged so it remains broken. Use the new command `yarn test:e2e` instead. Note that a few of these tests still sporadically fail; I'm trying to resolve those problems. See [Issue 6976](#6976) for details about the migration. Note: This PR replaces [PR 7024](#7024). It is essentially a rebase to consolidate all the commit messages (via git diff vs master and patch, not via `git rebase`). ### Testing - Set your .env file for bhima_test - Run `yarn` to update dependencies (you may want to run `yarn upgrade` too) - Install playwright: - `npm install playwright` - Verify the two versions match: - `grep '`@playwright/test@'` yarn.lock` - `npx playwright --version` - Run the tests: `yarn test:e2e` Co-authored-by: Jonathan Cameron <jmcameron@gmail.com>

jmcameron · 2023-08-24T16:40:45Z

Closed by #7215

jmcameron mentioned this issue Mar 15, 2023

Next Generation of BHIMA End to End Testing #6669

Closed

jmcameron added the tests label Apr 7, 2023

jmcameron pinned this issue Apr 14, 2023

jmcameron mentioned this issue Apr 18, 2023

[OBSOLETE] Migrate end-to-end tests to Playwright #7024

Closed

jmcameron mentioned this issue Jul 19, 2023

Redo the Cost Center end-to-end tests #7153

Open

jmcameron mentioned this issue Aug 23, 2023

Migrate end-to-end tests to Playwright #7215

Merged

jmcameron mentioned this issue Aug 24, 2023

Clean out old Protractor E2E tests #7218

Closed

jmcameron closed this as completed Aug 24, 2023

jniles unpinned this issue Aug 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert end-to-end tests to Playwright #6976

Convert end-to-end tests to Playwright #6976

jmcameron commented Mar 15, 2023 •

edited

Loading

jmcameron commented Mar 15, 2023 •

edited

Loading

jmcameron commented Aug 24, 2023

Convert end-to-end tests to Playwright #6976

Convert end-to-end tests to Playwright #6976

Comments

jmcameron commented Mar 15, 2023 • edited Loading

jmcameron commented Mar 15, 2023 • edited Loading

jmcameron commented Aug 24, 2023

jmcameron commented Mar 15, 2023 •

edited

Loading

jmcameron commented Mar 15, 2023 •

edited

Loading