You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your enhancement related to a problem? Please describe.
There are zero tests in this plugin, which is a problem. There is also a broader problem in our plugins that do have acceptance tests where they pass locally but fail in CI, typically due to timeouts that have proven difficult to debug. This issue is to focus on automated interaction tests - E2E and/or acceptance tests (I am not clear on where the lines really are between these things or if people even agree on the lines).
Describe the solution you'd like
An E2E/acceptance test setup that is reusable across 10up plugins and preferably by extension any WordPress plugin, that can run in CI such as GitHub Actions with consistent and expected results. We're targeting this plugin first because it is quite stable, relatively small in scope, includes various elements from an editor block to taxonomy handling to RSS feeds that can all benefit from tests, and does not have any existing tests.
Designs
Not visual designs, but I think the ideal likely includes a separately maintained package of WP utilities, much like the @wordpress/e2e-test-utils.
Describe alternatives you've considered
You might reading the above wondering why we don't just use that package and the related e2e package and setup from core/Gutenberg. This is being tried in #116 but we are running into the same old problems from other plugins, where Puppeteer as a browser simulator just cannot accomplish what we need without a lot of what amounts of manual intervention in terms of how tests are written (waiting, timeouts, etc.). It is a little disappointing that it seems we can't piggyback off of what is some really great work, but I think fundamentally our goals are a bit different, so targeting something that is running in a real browser environment with more robust debugging and interactive tools such as Cypress (see #117) or Playwright seems like a better course here.
Additional context
We have chatted about this a bit with some of the maintainers of core and Gutenberg JS, in particular some of those who are focused on tests. There is no worrying about aligning on tool choice.
The text was updated successfully, but these errors were encountered:
Is your enhancement related to a problem? Please describe.
There are zero tests in this plugin, which is a problem. There is also a broader problem in our plugins that do have acceptance tests where they pass locally but fail in CI, typically due to timeouts that have proven difficult to debug. This issue is to focus on automated interaction tests - E2E and/or acceptance tests (I am not clear on where the lines really are between these things or if people even agree on the lines).
Describe the solution you'd like
An E2E/acceptance test setup that is reusable across 10up plugins and preferably by extension any WordPress plugin, that can run in CI such as GitHub Actions with consistent and expected results. We're targeting this plugin first because it is quite stable, relatively small in scope, includes various elements from an editor block to taxonomy handling to RSS feeds that can all benefit from tests, and does not have any existing tests.
Designs
Not visual designs, but I think the ideal likely includes a separately maintained package of WP utilities, much like the @wordpress/e2e-test-utils.
Describe alternatives you've considered
You might reading the above wondering why we don't just use that package and the related e2e package and setup from core/Gutenberg. This is being tried in #116 but we are running into the same old problems from other plugins, where Puppeteer as a browser simulator just cannot accomplish what we need without a lot of what amounts of manual intervention in terms of how tests are written (waiting, timeouts, etc.). It is a little disappointing that it seems we can't piggyback off of what is some really great work, but I think fundamentally our goals are a bit different, so targeting something that is running in a real browser environment with more robust debugging and interactive tools such as Cypress (see #117) or Playwright seems like a better course here.
Additional context
We have chatted about this a bit with some of the maintainers of core and Gutenberg JS, in particular some of those who are focused on tests. There is no worrying about aligning on tool choice.
The text was updated successfully, but these errors were encountered: