Skip to content

Commit

Permalink
Execute E2E tests more frequently
Browse files Browse the repository at this point in the history
So far, we've been running E2E tests only when a PR was changing feature flag,
contained `e2e` in the name of the feature branch or got merged to `main`. This
prevented us from noticing changes braking E2E tests fast enough - we sometimes
were noticing such changes only after they got already merged to `main`.
In this commit we modify CI config in a way that most of the E2E tests will be
run on every PR update. We also allow for specifying tests which shouldn't be
run this often - such tests should be marked with `@slow` tag in the test name
(read more here: https://playwright.dev/docs/test-annotations#tag-tests). We
may want to use this feature when we'll have tests that require spending some
funds.
  • Loading branch information
michalinacienciala committed Jun 23, 2023
1 parent 312dcc7 commit 168d945
Showing 1 changed file with 12 additions and 6 deletions.
18 changes: 12 additions & 6 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,10 +114,6 @@ jobs:
path-filter:
- '.env.defaults'
e2e-tests:
if: |
github.ref == 'refs/heads/main'
|| contains(github.head_ref, 'e2e')
|| needs.detect-if-flag-changed.outputs.path-filter == 'true'
needs: [build, detect-if-flag-changed]
timeout-minutes: 60
runs-on: ubuntu-latest
Expand All @@ -139,10 +135,20 @@ jobs:
name: extension-builds-${{ github.event.number || github.event.head_commit.id }}
- name: Extract extension
run: unzip -o chrome.zip -d dist/chrome
- name: Run Playwright tests
run: xvfb-run npx playwright test
# Some tests that we have configured in the `e2e-tests` folder may require
# spending funds. Although they're desined to not spend much, with
# frequent execution that can accumulate. We don't want to execute such
# tests on every PR update. We'll tag those tests with the `@slow` tag.
- name: Run free Playwright tests
run: xvfb-run npx playwright test --grep-invert @slow
#env:
# DEBUG: pw:api*
- name: Run costing Playwright tests
if: |
github.ref == 'refs/heads/main'
|| contains(github.head_ref, 'e2e')
|| needs.detect-if-flag-changed.outputs.path-filter == 'true'
run: xvfb-run npx playwright test --grep @slow
- uses: actions/upload-artifact@v3
if: failure()
with:
Expand Down

0 comments on commit 168d945

Please sign in to comment.