Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add instructions for image-based tests to the contributors guide #6073

Merged
merged 9 commits into from
Feb 25, 2019
Merged

Add instructions for image-based tests to the contributors guide #6073

Changes from 7 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
bfc55fa
Add instructions for image-based tests to the contributors guide
benmccann Feb 17, 2019
9fc7b57
Code review updates
benmccann Feb 20, 2019
27ae034
Minor wording changes
benmccann Feb 21, 2019
3c37d0e
Remove unnecessary step
benmccann Feb 21, 2019
06bf41e
Update file name
benmccann Feb 22, 2019
c9f3242
Add note about tooltip hover
benmccann Feb 22, 2019
d3f4261
Document debug option
benmccann Feb 24, 2019
38f1663
Minor updates
benmccann Feb 25, 2019
cdb1b9a
Expand path
benmccann Feb 25, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/developers/contributing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,26 @@ The following commands are now available from the repository root:

More information can be found in [gulpfile.js](https://github.com/chartjs/Chart.js/blob/master/gulpfile.js).

### Image-Based Tests

Some display-related functionality is difficult to test via typical Jasmine units. For this reason, we introduced image-based tests ([#3988](https://github.com/chartjs/Chart.js/pull/3988) and [#5777](https://github.com/chartjs/Chart.js/pull/5777)) to assert that a chart is drawn pixel-for-pixel matching an expected image.

Generated charts in image-based tests should be **as minimal as possible** and focus only on the tested feature to prevent failure if another feature breaks (e.g. disable the title and legend when testing scales).

You can create a new image-based test by following the steps below:
- Create a JS file ([example](https://github.com/chartjs/Chart.js/blob/f7b671006a86201808402c3b6fe2054fe834fd4a/test/fixtures/controller.bubble/radius-scriptable.js)) or JSON file ([example](https://github.com/chartjs/Chart.js/blob/4b421a50bfa17f73ac7aa8db7d077e674dbc148d/test/fixtures/plugin.filler/fill-line-dataset.json)) that defines chart config and generation options.
- Add this file in `test/fixtures/{spec.name}/{feature-name}.json`.
- Add a [describe line](https://github.com//chartjs/Chart.js/blob/4b421a50bfa17f73ac7aa8db7d077e674dbc148d/test/specs/plugin.filler.tests.js#L10) to the beginning of `specs/{spec.name}.js` if it doesn't exist yet.
benmccann marked this conversation as resolved.
Show resolved Hide resolved
- Run `gulp unittest --watch --inputs=test/specs/{spec.name}.tests.js`.
- Click the *"Debug"* button (top/right): a test should fail with the associated canvas visible.
- Right click on the chart and *"Save image as..."* `test/fixtures/{spec.name}/{feature-name}.png` making sure not to activate the tooltip or any hover functionality
- Refresh the browser page (`CTRL+R`): test should now pass
- Verify test relevancy by changing the feature values *slightly* in the JSON file.

Tests should pass in both browsers. In general, we've hidden all text in image tests since it's quite difficult to get them passing between different browsers. As a result, it is recommended to hide all scales in image-based tests. It is also recommended to disable animations. If tests still do not pass, adjust [`tolerance` and/or `threshold`](https://github.com/chartjs/Chart.js/blob/1ca0ffb5d5b6c2072176fd36fa85a58c483aa434/test/jasmine.matchers.js) at the beginning of the JSON file keeping them **as low as possible**.

When a test fails, the expected and actual images are shown. If you'd like to see the images even when the tests pass, set `"debug": true` in the JSON file.

## Bugs and Issues

Please report these on the GitHub page - at <a href="https://github.com/chartjs/Chart.js" target="_blank">github.com/chartjs/Chart.js</a>. Please do not use issues for support requests. For help using Chart.js, please take a look at the [`chartjs`](https://stackoverflow.com/questions/tagged/chartjs) tag on Stack Overflow.
Expand Down