test: Migrate more runner UI specs to app #20118
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…into runner-e2e-round-2
…n attempt to reduce test time.
…html Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
…rs specs to appropriately reduce AUT refreshes.
…into runner-e2e-round-2 # Conflicts: # packages/example/bin/build.js
…press into runner-e2e-round-3
…into runner-e2e-round-3 # Conflicts: # packages/app/cypress/e2e/runner/reporter.errors.cy.ts # packages/app/cypress/e2e/runner/support/verify-failures.ts
|
Thanks for taking the time to open a PR!
|
lmiller1990
previously approved these changes
Feb 17, 2022
| if (Cypress.config().isInteractive) { | ||
| return cy.log('percy: skipping snapshot in interactive mode') | ||
| return applySnapshotMutations({ | ||
| ...snapshotMutationOptions, |
There was a problem hiding this comment.
Why are we doing the snapshot mutations if we're skipping the snapshots?
There was a problem hiding this comment.
Is it so we can get an idea of what will be snapshot when looking at the command log?
There was a problem hiding this comment.
That is exactly right.
ryanthemanuel
previously approved these changes
Feb 17, 2022
lmiller1990
previously approved these changes
Feb 17, 2022
brian-mann
dismissed stale reviews from lmiller1990 and ryanthemanuel
via
139ca03
brian-mann
approved these changes
Feb 18, 2022
lmiller1990
requested review from
lmiller1990
and removed request for
ryanthemanuel
lmiller1990
approved these changes
Feb 18, 2022
ryanthemanuel
approved these changes
Feb 18, 2022
tgriesser
pushed a commit
that referenced
this pull request
Feb 22, 2022
parent 6279a7d author Brian Mann <brian.mann86@gmail.com> 1644881743 -0500 committer Tim Griesser <tgriesser10@gmail.com> 1645497717 -0500 chore: post merge cleanup add visual tests for new errors, update formatting + copy of existing errors to bring into parity with new conventions fix require, lazy load if necessary export what is called directly to avoid circular ref proxy that's unstubbable export additional properties on error, update refs, fix tests fix missing ref rename clone -> cloneErr use proper method call fix: types for config changes fix: clean up a few test failures fix: additional cleanup of errors following merge fix: cleanup a few more tests/snapshot errors feat: reconfigure project from testing type card (#20109) fix: remove inner scroll from BaseError (#20104) feat: package manager in update modal (#20123) fix: Backing out driver change for type (#20155) * fix: Backing out driver change for type * Adding comment for posterity * Fixing test feat: detect component testing env (#20002) Co-authored-by: Zachary Williams <ZachJW34@gmail.com> Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com> fix: use github.com qualified path to deal with yarn cache bug (#20158) feat: Update version checking logic to query the manifest and ensure ctx.util.fetch can handle a proxy (#20107) Co-authored-by: Zach Bloomquist <git@chary.us> fix: cleanup integration folder after moving specs to e2e (#20149) Co-authored-by: Barthélémy Ledoux <bart@cypress.io> Co-authored-by: ElevateBart <ledouxb@gmail.com> fix: Remove mention of Cypress Studio from Experimental Project Settings (#20154) fix: remove phantom migration watchers (#20180) fix: migration ui differences (#20134) Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com> Co-authored-by: Barthélémy Ledoux <bart@cypress.io> test: use async copy for scaffolding project (#20181) fix: migration on windows (#20063) Co-authored-by: Zach Bloomquist <git@chary.us> Co-authored-by: Ryan Manuel <ryanm@cypress.io> test: clean up scaffoldMigrationProject (#20203) fix: rename folder instead of skip on migration (#20145) Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com> chore: ignore checksums when installing Chrome in Windows CI (#20210) feat(unify): inline specs list tweaks (#20157) test: fix some windows tests (#20225) test(unify): Settings E2E Tests (#20126) Co-authored-by: Zachary Williams <ZachJW34@gmail.com> fix: migration hover/focus states (#20243) Co-authored-by: ElevateBart <ledouxb@gmail.com> fix: refactor function getFilesByGlob to use concat instead of isArray (#20245) * fix: avoid using Array.isArray when possible * fix: add one more refactoring Co-authored-by: Zachary Williams <ZachJW34@gmail.com> Co-authored-by: Zachary Williams <ZachJW34@gmail.com> test: Migrate more runner UI specs to app (#20118) * Adding remaining reporter error tests. * Removing existing integration spec file * Removing existing runner fixtures for error specs * Fixing ts lint * Testing with less tests kept in memory * Testing some cypress-in-cypress settings to improve run-mode performance. * Reverting changes to doc_url specs * PR Updates. Inverting default value for 'open in IDE' validation in an attempt to reduce test time. * Cleaning up ported code * Update system-tests/projects/runner-e2e-specs/cypress/fixtures/index.html Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> * Trying build workaround * Bumping up parallelized builds for app integration tests * Removing suspicious tests to validate CI build * Removing more suspicious tests to test CI * Trying to not scaffold every navigation * Bumping resources for run-app-integration job. Updating reporter.errors specs to appropriately reduce AUT refreshes. * Couple API tweaks * test: Migrating runner hooks specs to app * Removing unused helper from runner package; cleaning up a few things * Updating yarn.lock * fix: Migrating runner sessions ui tests to app * test: Migrating runner's runner.ui tests to app * test: Migrating runner retries ui tests to app * Cleaning up a few things for consistency * Fixing ts errors * Removing tests for runner header that have existing coverage in app * Working around detached elements in CI * Still working around detached elements in CI * Updating sessions.ui test to click more carefully. Adding bit more coverage around runner header and snapshot pinning. * Update packages/app/cypress/e2e/runner/retries.ui.cy.ts Co-authored-by: Ryan Manuel <ryanm@cypress.io> * Migrating issue specs. Updating TODOs. Hopefully addressing some flake. * Stopping icon spin animation for percy snapshots * WIP * customPercyCommand.ts * WIP: working on custom percy snapshots for hiding elements other than the reporter [skip ci] * WIP: Adding naive mutation observer for reseting DOM mutations after snapshots [skip ci] Got tired of tripping over types, I can add them when we're feature complete * Implementing scoped snapshots throughout app/runner tests, let's see how percy output looks * Adding more snapshots to runner tests. * Using spec link as click target * Stop taking snapshots of failed tests * Scoping selector playground snapshots (though they're still unstyled) * Addressing flake in virtualized spec list * Drying up runner snapshotting * Logging single percy snapshot that reflects proper viewport and overrides * Updating describes to better match previous snapshot names * Removing added snapshots to get more accurate better diff presentation in Percy * Removing new runner.ui snapshots as well * Adding a few PR recommendations. Updating one last spec name that was corrected for grammar and no longer matched. Updating/removing relevant TODOs. * Updating to use anticipated default width for reporter * Updating tests for consistency with previous snapshots * Tying original command into promise chain * Let's try that again * Making percy command enqueuing explicit to test result * Scoping down on some variables. Using defaults for sessions tests. * Updating some overrides to see if visibility is handled inconsistently * Whitespace vanquished, working session test updates back in for long command logs * Viewport height is not reflected in snapshots, let's try forcing the height on the panel * Forcing the height issue * Let's try a minHeight on the percy command * Cleaning up alterations made during percy exploration * Trying to get a cleaner percy diff after merge * Fix Percy to use 10.0-release branch as baseline image comparison * add comments about temporary percy env vars * Adding some doc for changes made to customPercyCommand Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: Ryan Manuel <ryanm@cypress.io> Co-authored-by: Brian Mann <brian.mann86@gmail.com> Co-authored-by: Emily Rohrbough <emilyrohrbough@yahoo.com> feat(unify): show spec runner header in run mode (#20189) Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: Zachary Williams <ZachJW34@gmail.com> Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com> fix: connect to dashboard page when there are no existing projects (#20188) Co-authored-by: Barthélémy Ledoux <bart@cypress.io> chore: fix system-test-firefox screenshots_spec flake (#20260) fix all the tests that escape "doesn\'t" to use string templates fix test assertion fix test importing wrong strip_indent fix snapshot remove accidental system test it.only properly skip describe suites + tests and add TODO comments export errors.theme, update imports not to require from deep inside packages export stripAnsi onto the old errors object fix yarn.lock chore: remove system test it.only and prevent CI from ever running onlys (#20276) * remove accidental system test it.only * prevent system tests from running in CI if set to run exclusively with it.only(...) chore: fix server performance flake (#20280) * chore: fix server performance flake * rerun in circle Co-authored-by: Brian Mann <brian.mann86@gmail.com> fix invalid exports remove fixtures after shutting down the project add terminals app watch command updated snapshots, added tests to cover --spec CLI arg + custom specPattern config - removed unused system project update deep import of @packages/errors fix root system test & launchpad choose-a-browser spec fix: skip test that is no longer correct in 10.x fix: serialize details when serializing the error remove plugins file not found error/test update CONFIG_FILE_REQUIRE_ERROR working through mapping error cases from develop Fixing CONFIG_VALIDATION_ERROR / SETUP_NODE_EVENTS_IS_NOT_FUNCTION errors fix remaining test failures fix spec failures Attempt to fix awesome-typescript-loader failures fix snapshot feat(unify): Login error states (#20204) feat: Updating reporter panel's default width to match new designs (#20231) * feat: Updating default reporter width to 450px * Updating component tests to account for new default panel 2 width * Giving the cypress-in-cypress tests a little more room to breath * Updating trigger test coordinate assertions to be have consistent results regardless of AUT scaling * Updating scale assertion now that available width has changed
tgriesser
added a commit
that referenced
this pull request
Feb 22, 2022
* 10.0-release: (29 commits) feat: Updating reporter panel's default width to match new designs (#20231) feat(unify): Login error states (#20204) chore: fix server performance flake (#20280) chore: remove system test it.only and prevent CI from ever running onlys (#20276) chore: fix system-test-firefox screenshots_spec flake (#20260) fix: connect to dashboard page when there are no existing projects (#20188) feat(unify): show spec runner header in run mode (#20189) test: Migrate more runner UI specs to app (#20118) fix: refactor function getFilesByGlob to use concat instead of isArray (#20245) fix: migration hover/focus states (#20243) test(unify): Settings E2E Tests (#20126) test: fix some windows tests (#20225) feat(unify): inline specs list tweaks (#20157) chore: ignore checksums when installing Chrome in Windows CI (#20210) fix: rename folder instead of skip on migration (#20145) test: clean up scaffoldMigrationProject (#20203) fix: migration on windows (#20063) test: use async copy for scaffolding project (#20181) fix: migration ui differences (#20134) fix: remove phantom migration watchers (#20180) ...
tgriesser
added a commit
that referenced
this pull request
Feb 22, 2022
* 10.0-release: (29 commits) feat: Updating reporter panel's default width to match new designs (#20231) feat(unify): Login error states (#20204) chore: fix server performance flake (#20280) chore: remove system test it.only and prevent CI from ever running onlys (#20276) chore: fix system-test-firefox screenshots_spec flake (#20260) fix: connect to dashboard page when there are no existing projects (#20188) feat(unify): show spec runner header in run mode (#20189) test: Migrate more runner UI specs to app (#20118) fix: refactor function getFilesByGlob to use concat instead of isArray (#20245) fix: migration hover/focus states (#20243) test(unify): Settings E2E Tests (#20126) test: fix some windows tests (#20225) feat(unify): inline specs list tweaks (#20157) chore: ignore checksums when installing Chrome in Windows CI (#20210) fix: rename folder instead of skip on migration (#20145) test: clean up scaffoldMigrationProject (#20203) fix: migration on windows (#20063) test: use async copy for scaffolding project (#20181) fix: migration ui differences (#20134) fix: remove phantom migration watchers (#20180) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #19219
This PR converts the following runner tests to app e2e tests:
The conversion for these was straightforward. The tests that will remain in the runner package include:
*.mochaEvents.spec.jsrunIsolatedCypressto capture emitted mocha events. I will log a follow-up issue to track the changes necessary to migrate these.studio.*.spec.jsIn an effort to get representative Percy snapshots of these tests, the custom percy command we utilize was also enhanced. New features include:
User facing changelog
Additional details
How has the user experience changed?
PR Tasks
cypress-documentation?type definitions?cypress.schema.json?