Skip to content

Commit

Permalink
test(code): introduce component and visual testing to stacks (#1194)
Browse files Browse the repository at this point in the history
Co-authored-by: Dan Cormier <dcormier@stackoverflow.com>
Co-authored-by: Ben Kelly <b-kelly@users.noreply.github.com>
  • Loading branch information
3 people authored Dec 15, 2022
1 parent cd0ec76 commit a6c43aa
Show file tree
Hide file tree
Showing 32 changed files with 5,797 additions and 2,591 deletions.
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
screenshots/** filter=lfs diff=lfs merge=lfs -text
46 changes: 46 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: CI

on:
push:
branches: [develop]
pull_request:
branches: [develop]

jobs:
build-and-test:
name: Build and Test
runs-on: ubuntu-latest
steps:
- name: ⬇️ Checkout
uses: actions/checkout@v3
with:
lfs: true

- name: ⎔ Setup node
uses: actions/setup-node@v3
with:
node-version: 'lts/*'

- name: 🏗 Install and Build
run: |
npm ci
npx playwright install --with-deps
npm run build
- name: ▶️ Lint and Test
run: |
npm run lint
npm run test -- --config web-test-runner.config.ci.mjs
- name: ⬆️ Upload Visual Regression Test Results
uses: actions/upload-artifact@v3
if: failure()
with:
name: visual-regression-test-results
path: screenshots


# cancel the jobs if another workflow is kicked off for the same branch
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
8 changes: 5 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -78,11 +78,13 @@ dist/

# Testing Assets #
###############
/backstop_data/bitmaps_test/
/backstop_data/bitmaps_reference/
/backstop_data/html_report/
/screenshots/**/failed/**

# GitHub Actions #
###############
/.github/main.workflow

# Editors #
###########
.vscode/
/.idea
42 changes: 40 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,47 @@ npm run lint:format
```

## Testing Stacks
Stacks has implemented visual regression testing with [Backstop](https://github.com/garris/BackstopJS). To test if your new feature introduces visual regressions, run `npm run test` in a new Terminal window while Stacks is running. After the tests have run, a new browser window with any regressions will show. If the regressions are desired, you can run `npm run approve` to establish the new baseline.

Individual routes to test are found in [backstop.json](/backstop.json)
Run all test suites by running:
```sh
npm test
```
### Unit/Component Tests

Unit/Component tests are written with [DOM Testing Library](https://testing-library.com/docs/dom-testing-library/intro).
Please follow the library's principles and documentation to write tests.

Stacks uses [Web Test Runner](https://modern-web.dev/docs/test-runner/overview/) and [Playwright](https://modern-web.dev/docs/test-runner/browser-launchers/playwright/) to run tests in a real browser context.

Execute the unit/component tests suite by running:
```sh
npm run test:unit
```
or if you prefer watch mode run:
```sh
npm run test:unit:watch
```

### Visual Regression Tests

**Prerequisite:**
In order to pull and upload baseline images you need to have `git lfs` installed in your local machine. Please follow [this guide](https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage) to install that.


This [Web Test Runner plugin](https://www.npmjs.com/package/@web/test-runner-visual-regression) is used to run visual regression tests. [DOM Testing Library](https://testing-library.com/docs/dom-testing-library/intro).
Visual regression tests end with this suffix `*.visual.test.ts`.

Execute the visual regression tests suite by running:
```sh
npm run test:visual
```

Update the visual baseline via:
```sh
npm run test:visual:update
```

Failing tests (including diffs) can be found under `screenshots/[browser]/failed/` folders.

## Bugs and feature requests
Have a bug or feature request? First search existing or closed issues to make sure the issue hasn’t been noted yet. If not, review our [issue guidelines](/CONTRIBUTING.md#open-an-issue) for submitting [a bug report](/CONTRIBUTING.md#reporting-bugs) or [feature request](/CONTRIBUTING.md#feature-requests).
Expand Down
Loading

0 comments on commit a6c43aa

Please sign in to comment.