Copyright (C) 2017-2018 The Open Library Foundation
This software is distributed under the terms of the Apache License, Version 2.0. See the file "LICENSE" for more information.
- About FOLIO UI Tests
- Install node packages first
- Run the main tests with default user/password/URL/debug-level
- Run individual tests contained in ui-testing
- Run tests from a UI module
- Optional: environment variables to modify tests
- Alternative for module tests: use command line arguments
- Choose the source of UI module tests to run
- Develop tests for a UI module
- Manage ui-testing versions
- Additional information
The tests are using the NightmareJS browser automation library, and Mocha for the tests itself.
Some tests are contained within ui-testing itself while others live in the individual UI modules, and can be executed from there by ui-testing.
You may need to install supporting libraries for Electron(Nightmare uses electron to execute browser tests).
Detailed instructions can be found here: Running nightmare on AWS
To immediately find out if you need to install dependencies execute:
$ cd node_modules/nigthmare/node_modules/electron/dist
$ ldd electron | grep 'not found'
$ yarn install
$ yarn test
$ ./node_modules/.bin/mocha test/checkout.js # runs the checkout test that visits multiple apps
or (if defined in package.json)
$ yarn test-checkout
to list pre-defined tests
$ yarn run
$ yarn test-module -- -o --run=users # runs all tests in test.js from the users module
$ yarn test-module -- -o --run=users:new_user # runs the new-user tests from the users module
to view options for running UI module tests
$ yarn test-module
set username/password to something different from the default:
export FOLIO_UI_USERNAME=diku_admin
export FOLIO_UI_PASSWORD=admin
run against a different Stripes service:
export FOLIO_UI_URL=http://localhost:3000
or
# Wait for up to 10 seconds for each action, not the 30s default
$ FOLIO_UI_WAIT_TIMEOUT=10000 yarn test
# headless browser test / command line
$ FOLIO_UI_DEBUG=0 yarn test
# with browser window on a desktop machine
$ FOLIO_UI_DEBUG=1 yarn test
# with browser window + debug console
$ FOLIO_UI_DEBUG=2 yarn test
Nightmare debug options:
$ DEBUG=nightmare yarn test
$ DEBUG=nightmare* yarn test
$ DEBUG=nightmare:actions* yarn test
$ DEBUG=nightmare:*,electron:* yarn test
nightmare:actions* logs each browser-driving action as it happens, so it's easier to see what's happened when something goes wrong.
All options in one:
$ FOLIO_UI_URL="http://folio-uidemo.aws.indexdata.com" DEBUG=nightmare FOLIO_UI_DEBUG=2 yarn test
See folio-ui.config.js for current environment variables.
run a module's test in browser against localhost:3000
$ yarn test-module -- -o --show --h=localhost --run=checkout:error_messages
A command line argument overrides the corresponding environment variable.
To see all command line options:
$ yarn test-module
When running a UI module's own test suites from ui-testing, there are three potential sources for the actual version of the module and its tests:
- FOLIO's release repository: npm-folio at repository.folio.org. This is appropriate for testing a stable FOLIO platform, built with npm released versions of UI modules. The package.json of ui-testing controls which releases are installed, and those releases should match the releases in the live service.
- FOLIO's continuous integration repository: npm-folioci at repository.folio.org. This is appropriate for regression tests and continuous integration. This would also suit developers who are programming test suites or UI modules, and want to test very the latest developments.
- A local checkout of the UI module, brought into ui-testing by 'yarn link' or similar. This could be for developers programming test suites for a UI module and wanting to execute the test scripts with local changes.
Install tests from release repository
npm config set @folio:registry https://repository.folio.org/repository/npm-folio/
rm yarn.lock
yarn install
Then run the tests against a service built on releases, for example the folio-staging service
export FOLIO_UI_URL=http://folio-staging.aws.indexdata.com/
yarn test-module -- -o --run=users
or run the same with command line arguments
yarn test-module -- -o --host=staging --run=users
Install tests from continuous integration repository
npm config set @folio:registry https://repository.folio.org/repository/npm-folioci/
rm yarn.lock
yarn install
Then run the tests against the folio-testing service, also based on the continuous integration repository:
export FOLIO_UI_URL=http://folio-testing.aws.indexdata.com/ # The current default URL for testing
yarn test-module -- -o --run=users
or run the same with command line arguments
yarn test-module -- -o --host=testing --run=users
In order for ui-testing to be able to run a UI module's own tests individually or as part of an overall test suite, the UI module should contain scripts with Nightmare tests that take one argument holding all of the test context (see below).
While test scripts could be placed anywhere in the module's source structure, ui-testing will be looking for a main test script in, say, the Users module, at /ui-users/test/ui-testing/test.js
This test script would probably contain references to all the individual tests that the module developers intend to get included in the overall test suite.
This is an example of a minimal test that logs in and evaluates if a module named 'app' opens:
module.exports.test = function(uiTestCtx) {
describe('Module test: app:minimal', function() {
const { config, helpers: { login, openApp, logout }, meta: { testVersion } } = uiTestCtx;
const nightmare = new Nightmare(config.nightmare);
this.timeout(Number(config.test_timeout));
describe('Login > Open module "App" > Logout', () => {
before( done => {
login(nightmare, config, done); // logs in with the default admin credentials
})
after( done => {
logout(nightmare, config, done);
})
it('should open module "App" and find version tag ', done => {
nightmare
.use(openApp(nightmare, config, done, 'app', testVersion))
.then(result => result )
})
})
})
}This script might be invoked from test.js:
const minimal = require('./minimal.js');
const extensive = require('./extensive.js');
module.exports.test = function(uiTestCtx) {
minimal.test(uiTestCtx);
extensive.test(uiTestCtx);
}The test context passed to the module's test from ui-testing has following content at the time of writing:
{
config : (see folio-ui.config.js)
helpers: {
login: function for logging into Stripes
logout: function for logging out of Stripes
openApp: function for opening a specified module's page
getUsers: function that returns an array of currently listed users
createInventory: function that creates inventory, holdings and item records
namegen: function for generating user names and addresses
}
meta: {
testVersion: the npm version of the module that the test is pulled from
}
}
NOTE: This is the first version of the context and it is subject to change.
The xnightmare.js file extends nightmare by adding actions that use XPath as a node selector. So far there are only two actions contained in this file:
Does the same as .click but takes an XPath instead of a CSS selector as an argument.
This will extract and return the textContent of an XPath node. The returned value will be passed to the next action in the chain (most likely .then).
This script creates random user data (100 possibilities). Returns: id, firstname, lastname, email, barcode, password
const Nightmare = require('nightmare')
const assert = require('assert')
const config = require('../folio-ui.config.js')
const names = require('../namegen.js')
const user = names.namegen()
...
it('should create a user: ' + user.id + '/' + user.password, done => {
nightmare
.type('#adduser_username',user.id)
.type('#pw',user.password)
.click('#useractiveYesRB')
.type('#adduser_firstname',user.firstname)
.type('#adduser_lastname',user.lastname)
.type('#adduser_email', user.email)
.type('#adduser_barcode',user.barcode)
.click('#clickable-createnewuser')
.wait('#clickable-newuser')
.wait(parseInt(process.env.FOLIO_UI_DEBUG) ? parseInt(config.debug_sleep) : 0) // debugging
.then(result => { done() })
.catch(done)
})openApp is a helper function that will open the page of a given UI module. If the test script passes it's version to openApp, then openApp will log the version of the test as well as the version of the module under test.
The UI module can find 'openApp' in 'helpers' and its own version in 'meta'.
For example:
const { config, helpers: { namegen, openApp }, meta: { testVersion } } = uiTestContext;
...
...
nightmare
.use(openApp(nightmare, config, done, 'checkout', testVersion ))
Output in the test log:
Module test: checkout:error_messages.
Open app > Trigger error messages > Logout
Test suite @folio/checkout:1.0.10020
Live module @folio/checkout:1.0.10019 (http://folio-testing.aws.indexdata.com)
Sometimes developers might update the UI and the tests scripts at the same time, for instance to ensure that tests still pass after changing the structure or functionality of the UI.
For that they would conventionally do this:
* check out ui-testing and install it from the npm-folioci repository
* check out a Stripes platform to run the UI module in locally
* check out the UI module in question
* yarn link the UI module into both ui-testing and the local Stripes platform.
Test scripts are both easier to write and read if unique IDs have been assigned to crucial elements of the UI.
Stripes will assign certain IDs out of the box that the test script can use:
The module's menu bar button: #clickable-users-module
The module's page: #users-module-display
The login button: #clickable-login
The logout button: #clickable-logout
Beyond that the UI module developer should assign IDs to actionable UI elements at least.
Following conventions are suggested:
Controls/links that can be clicked: #clickable-add-user
Elements for entering data: #input-user-name
Lists of data: #list-users
Section of a page: #section-loans-history
Examples of identifiers in a UI:
<Row id="section-patron" ...
<Field id="input-patron-identifier" ...
<Button id="clickable-find-patron" ...
<MultiColumnList id="list-patrons" ...
<Row id="section-item" ...
<Field id="input-item-barcode" ...
<Button id="clickable-add-item" ...
<MultiColumnList id="list-items-checked-out" ...
<Button id="clickable-done" ...
and usage in a test script:
it('should show error when scanning item before patron card', done => {
nightmare
.wait('#clickable-checkout-module')
.click('#clickable-checkout-module')
.wait('#input-item-barcode')
.insert('#input-item-barcode',"some-item-barcode")
.wait('#clickable-add-item')
.click('#clickable-add-item')
.wait('#section-patron div[class^="textfieldError"]')
.evaluate(function() {
var errorText = document.querySelector('#section-patron div[class^="textfieldError"]').innerText;
if (!errorText.startsWith("Please fill")) {
throw new Error("Error message not found for item entered before patron found");
}
})
.wait(parseInt(process.env.FOLIO_UI_DEBUG) ? parseInt(config.debug_sleep) : 555) // debugging
.then(result => { done() })
.catch(done)
})We maintain versions of ui-testing itself for the purpose of testing specific builds of FOLIO platforms based entirely on NPM released components.
When a stable FOLIO platform build is made that passes the ui-testing test suite, we lock down that version of the test suite like this:
$ npm config set @folio:registry https://repository.folio.org/repository/npm-folio/
$ rm yarn.lock
$ yarn install
Run the tests against the platform build to verify that they pass. If they do not, we are not yet ready to tag a version of ui-testing for the platform build. If they do, proceed with the versioning:
Add the newly generated yarn.lock to git and commit it with a commit message referring to the version of the platform build, for instance:
$ git add yarn.lock
$ git commit -m "Add yarn.lock v5.0.0"
Tag the commit with a label that indicates the platform build it pertains to, for instance:
$ git tag "v5.0.0"
$ git push
$ git push origin tag "v5.0.0"
Once the version is tagged, the yarn.lock should be removed again. The regular test builds for testing the continuous integration platform should always pick the latest from the CI repository, and should thus not be locked down by yarn.lock. Delete the yarn.lock file from ui-testing, git commit and git push.
See stripes-core and other modules.
See project UITEST at the FOLIO issue tracker.
Other FOLIO Developer documentation is at dev.folio.org