Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement E2E / acceptance tests #118

Closed
helen opened this issue Aug 25, 2021 · 0 comments · Fixed by #117
Closed

Implement E2E / acceptance tests #118

helen opened this issue Aug 25, 2021 · 0 comments · Fixed by #117
Assignees
Labels
type:enhancement New feature or request.
Milestone

Comments

@helen
Copy link
Contributor

helen commented Aug 25, 2021

Is your enhancement related to a problem? Please describe.
There are zero tests in this plugin, which is a problem. There is also a broader problem in our plugins that do have acceptance tests where they pass locally but fail in CI, typically due to timeouts that have proven difficult to debug. This issue is to focus on automated interaction tests - E2E and/or acceptance tests (I am not clear on where the lines really are between these things or if people even agree on the lines).

Describe the solution you'd like
An E2E/acceptance test setup that is reusable across 10up plugins and preferably by extension any WordPress plugin, that can run in CI such as GitHub Actions with consistent and expected results. We're targeting this plugin first because it is quite stable, relatively small in scope, includes various elements from an editor block to taxonomy handling to RSS feeds that can all benefit from tests, and does not have any existing tests.

Designs
Not visual designs, but I think the ideal likely includes a separately maintained package of WP utilities, much like the @wordpress/e2e-test-utils.

Describe alternatives you've considered
You might reading the above wondering why we don't just use that package and the related e2e package and setup from core/Gutenberg. This is being tried in #116 but we are running into the same old problems from other plugins, where Puppeteer as a browser simulator just cannot accomplish what we need without a lot of what amounts of manual intervention in terms of how tests are written (waiting, timeouts, etc.). It is a little disappointing that it seems we can't piggyback off of what is some really great work, but I think fundamentally our goals are a bit different, so targeting something that is running in a real browser environment with more robust debugging and interactive tools such as Cypress (see #117) or Playwright seems like a better course here.

Additional context
We have chatted about this a bit with some of the maintainers of core and Gutenberg JS, in particular some of those who are focused on tests. There is no worrying about aligning on tool choice.

@helen helen added the type:enhancement New feature or request. label Aug 25, 2021
This was referenced Aug 25, 2021
@jeffpaul jeffpaul added this to the 1.3.0 milestone Aug 30, 2021
@jeffpaul jeffpaul linked a pull request Dec 10, 2021 that will close this issue
@cadic cadic modified the milestones: 1.3.0, 1.2.1 Dec 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:enhancement New feature or request.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants