tests: Add visual regression tests #27619
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gatsbot
bot
added
the
status: triage needed
ascorbic
force-pushed
the
tests/add-visual-tests
branch
3 times, most recently
from
ad9f73a to
12d1cb1
Compare
ascorbic
force-pushed
the
tests/add-visual-tests
branch
from
12d1cb1 to
f137bee
Compare
ascorbic
force-pushed
the
tests/add-visual-tests
branch
from
4954a4f to
e04aa5f
Compare
ascorbic
force-pushed
the
tests/add-visual-tests
branch
from
e04aa5f to
a601c97
Compare
ascorbic
force-pushed
the
tests/add-visual-tests
branch
from
53ec2f0 to
147efe4
Compare
LekoArts
added
topic: automation
laurieontech
approved these changes
Oct 26, 2020
| This test suite uses [cypress-image-snapshot](https://github.com/jaredpalmer/cypress-image-snapshot) | ||
| to compare screenshots of pages or elements with a saved snapshot. | ||
|
|
||
| To add a test, add a page to `src/pages`, then add a test to `cypress/integration`, or add to an existing spec. |
There was a problem hiding this comment.
Maybe a quick note about running the suite using yarn test. It's the same as other test suites, but we have those instructions in most READMEs.
ascorbic
added a commit
that referenced
this pull request
Oct 29, 2020
* Add static image plugin * Parse static image files * Fix package.json * Better static analysis * Use jsx utils package * Fix typings * Typecheck fix * Fix repo fields * Helpful warning * Re-enable duotone * Update readme * Improve typings * wip fixed image fields for tracedSVG and webP * handle fluid images as well * Use require rather than JSON import * Watch files for changes, and use relative paths * Improve types * Add type * Update yarn.lock * Add lots of comments and remove unused stuff * Create and watch our own nodes * Update readme * Update deps * Only watch in develop * Rename to gatsby-plugin-image * Rearrange, ready for merger * Import @wardpeet 's gatsby-image-netxtgen * Update typings and further merge packages * Build babel plugin * More merging * Add server/browser static image variants * include webpack changes * change error message to use gatsby-plugin-image * Fix compat compilation * Fix SSR * Merge readmes * Build browser bundle * Export correctly from browser entry point * Remove hook import * Apply suggestions from code review Co-authored-by: LB <laurie@gatsbyjs.com> * Changes from review * add path prefix, static image do e2e test * Add prepare script * Apply README suggestions from code review Co-authored-by: LB <laurie@gatsbyjs.com> * Fix image component in e2e test * Better error messages * Prefer svg over base64. Warn for unsupported art direction * Fix type * Use "layout" instead of fixed/fluid * add static images to production runtime site * add path prefix tests * Reorg types. Change private prop name * initial gatsby static image tests * add test suite to circle ci * add test suite to circle ci * add test suite to circle ci * remove unused imports, use valid fluid setting * Use new fluid props syntax * Better var name * update tests * remove tests that no longer match the expected DOM * More compat-fixes * Change classname to match old version * Compat improvements * Update tests to match new classname * v0.0.1 * Add readme caveat * Update version in packages * Remove forcewrapper * wip * have static image use gatsbyImageProps function * Upgrade sharp * Fix static image type * First steps to update API * Add descriptions * TypeScript fixes * Compile ts * Add @gillkyle's calculateImageSizes util funciton * remove unused file * unbump sharp until sharp PR is merged * comment out import * update snapshots * change JSX to take image prop * static image uses new image props * Rename all the things * Turns out it wasn't all of them * Update schema * fix intrinsic issue 1px problem, still doesn't seem to reach the correct max size * Fix layout * Update readme * Fix dep * Shouldn;t be in this * Fix sharp function name * Version * What is this snapshot out of date? * Add a helper funciton * Make backgroundColor work * update layout for images * no longer using base64 so remove that test * placeholder not getting passed from static, so this is still busted * Fix passing of props * Add TODO comments * Export batch queue function * Handle the default sizes elsewhere, as it depends on layout * feat(gatsby-plugin-sharp): create image sizes helper (#27554) * wip tests and tweaks to utility * more wip tests * remove file reading from helper * fixing fluid calculations and adding more tests * tweak a little more * remove extraneous export * check that user specified dimensions are positive * remove unnecessary code from review suggestions * add warning, simplify case of no fluid dimensions * add files to warnings and clean up logic * Split into two functions and remove 3x default * Rename warn function and use reporter Co-authored-by: Matt Kane <matt@gatsbyjs.com> * Use passed-in reporter * Use shared function to handle fit calculation * Return presentationW/H and aspect ratio from sizes function * Round sizes * Use new resizing functions * default layout wasn't passed to createImageSizes * add fixed width default and round calculated height * reversed logic for aspect ratio calculations * Correct description for outputPixelDensities resolver * Remove log * Handle null image in SSR * Version fix * fix(gatsby-plugin-utils): skip libcheck during typegen * Remove unused func * Remove dominantColor from fixed and fluid * sizing code for KG to drive refactor * remove isSmaller * fix tests * Fix sizes and srcset generation * Add object-fit * rename placeholder function for consistency * Update readme * fix path prefix images * fix sizing when larger image dimensions are requested, fix warning as well * rename file to match constrained naming * Changes from review * Add alpha warning * Change from review * fix placeholder opacity * fix placeholder bug * only change placeholder size for background color div * tests: Add visual regression tests (#27619) * tests: Add visual regression tests * Add config * Wait for images to load * Do multiple window sizes * Adjust settings for comparisons * Fix wrapper * Add 1024 window * Save reports * Update fixed image too large snapshot * Disable video * Update readme * Update element id * Force 1x pixel density * Update readme * Fix e2e test * dont flash placeholder image when removing opacity * Fix dependency * add analagous tests for static image * Add width to constrained images and update snapshots * Use float Co-authored-by: Kyle Gill <kylerobertgill@gmail.com> Co-authored-by: Laurie <laurie@gatsbyjs.com> Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>
ascorbic
added a commit
that referenced
this pull request
Nov 10, 2020
* Add static image plugin * Parse static image files * Fix package.json * Better static analysis * Use jsx utils package * Fix typings * Typecheck fix * Fix repo fields * Helpful warning * Re-enable duotone * Update readme * Improve typings * wip fixed image fields for tracedSVG and webP * handle fluid images as well * Use require rather than JSON import * Watch files for changes, and use relative paths * Improve types * Add type * Update yarn.lock * Add lots of comments and remove unused stuff * Create and watch our own nodes * Update readme * Update deps * Only watch in develop * Rename to gatsby-plugin-image * Rearrange, ready for merger * Import @wardpeet 's gatsby-image-netxtgen * Update typings and further merge packages * Build babel plugin * More merging * Add server/browser static image variants * include webpack changes * change error message to use gatsby-plugin-image * Fix compat compilation * Fix SSR * Merge readmes * Build browser bundle * Export correctly from browser entry point * Remove hook import * Apply suggestions from code review Co-authored-by: LB <laurie@gatsbyjs.com> * Changes from review * add path prefix, static image do e2e test * Add prepare script * Apply README suggestions from code review Co-authored-by: LB <laurie@gatsbyjs.com> * Fix image component in e2e test * Better error messages * Prefer svg over base64. Warn for unsupported art direction * Fix type * Use "layout" instead of fixed/fluid * add static images to production runtime site * add path prefix tests * Reorg types. Change private prop name * initial gatsby static image tests * add test suite to circle ci * add test suite to circle ci * add test suite to circle ci * remove unused imports, use valid fluid setting * Use new fluid props syntax * Better var name * update tests * remove tests that no longer match the expected DOM * More compat-fixes * Change classname to match old version * Compat improvements * Update tests to match new classname * v0.0.1 * Add readme caveat * Update version in packages * Remove forcewrapper * wip * have static image use gatsbyImageProps function * Upgrade sharp * Fix static image type * First steps to update API * Add descriptions * TypeScript fixes * Compile ts * Add @gillkyle's calculateImageSizes util funciton * remove unused file * unbump sharp until sharp PR is merged * comment out import * update snapshots * change JSX to take image prop * static image uses new image props * Rename all the things * Turns out it wasn't all of them * Update schema * fix intrinsic issue 1px problem, still doesn't seem to reach the correct max size * Fix layout * Update readme * Fix dep * Shouldn;t be in this * Fix sharp function name * Version * What is this snapshot out of date? * Add a helper funciton * Make backgroundColor work * update layout for images * no longer using base64 so remove that test * placeholder not getting passed from static, so this is still busted * Fix passing of props * Add TODO comments * Export batch queue function * Handle the default sizes elsewhere, as it depends on layout * feat(gatsby-plugin-sharp): create image sizes helper (#27554) * wip tests and tweaks to utility * more wip tests * remove file reading from helper * fixing fluid calculations and adding more tests * tweak a little more * remove extraneous export * check that user specified dimensions are positive * remove unnecessary code from review suggestions * add warning, simplify case of no fluid dimensions * add files to warnings and clean up logic * Split into two functions and remove 3x default * Rename warn function and use reporter Co-authored-by: Matt Kane <matt@gatsbyjs.com> * Use passed-in reporter * Use shared function to handle fit calculation * Return presentationW/H and aspect ratio from sizes function * Round sizes * Use new resizing functions * default layout wasn't passed to createImageSizes * add fixed width default and round calculated height * reversed logic for aspect ratio calculations * Correct description for outputPixelDensities resolver * Remove log * Handle null image in SSR * Version fix * fix(gatsby-plugin-utils): skip libcheck during typegen * Remove unused func * Remove dominantColor from fixed and fluid * sizing code for KG to drive refactor * remove isSmaller * fix tests * Fix sizes and srcset generation * Add object-fit * rename placeholder function for consistency * Update readme * fix path prefix images * fix sizing when larger image dimensions are requested, fix warning as well * rename file to match constrained naming * Changes from review * Add alpha warning * Change from review * fix placeholder opacity * fix placeholder bug * only change placeholder size for background color div * tests: Add visual regression tests (#27619) * tests: Add visual regression tests * Add config * Wait for images to load * Do multiple window sizes * Adjust settings for comparisons * Fix wrapper * Add 1024 window * Save reports * Update fixed image too large snapshot * Disable video * Update readme * Update element id * Force 1x pixel density * Update readme * Fix e2e test * dont flash placeholder image when removing opacity * Fix dependency * Continue moving options into options objects * Update API * Default to blurred * Fix test * Fix default * Change from review * Updates to fixed sizing * Update snapshots * fix readme, clean up unused variable * fix lint * weird readme lint issue Co-authored-by: Kyle Gill <kylerobertgill@gmail.com> Co-authored-by: Laurie <laurie@gatsbyjs.com> Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This test suite uses cypress-image-snapshot to compare screenshots of pages or elements with a saved snapshot. The initial use is for gatsby-plugin-image, where we are using it to check the display of various types of image, but this suite could be used for any other situations where we might want to check for visual regressions that can't be tested by regular Cypress DOM tests.
This PR includes a small number of image tests, but we will add more, covering all sorts of image. They currently test on three screen resolutions to show resizing behaviour. See the main test file to see how they are implemented.
When the tests fail, a comparison file is uploaded to the test artifacts, which can be viewed in the Artifacts tab in CircleCI. This is an example image of the "failure" when I merged in the fixed image size fixes from Friday without updating the snapshots:
